Solved NAT Forwarding

Below is the UI for the router we use for in-housing hosting.
tplink-virtualservers-UI.png


We have multiple web services running on multiple servers and the PF & Haproxy have been well-configured to redirect packets to the appropriate servers.

Before now, the router has been behaving well. By behaving well, I mean all the HTTPXXX entries [in the above image] were in the "on" status and all services were running. Out of the blue, the whole network went down. We fiddled with PF rules for a week. Part of it has to do with us trying additional features to ALTQ, such as FIBs and miniupnpd. And now that everything is up and running again, the router is now acting up.

According to the above diagram, only websites using the IP address with the last octet 220 are reachable. The 443 port nat forwarding is also set for that IP address [though not shown above]. As at now, we can only choose one at a time - disable 220 and enable 219, etc. As I said before, they were all enabled at one time. The router also gives other troubles, such as disabling all and never able to enable any or add new entries for that port. Of course, this would happen to any port number with multiple IP addresses.

What are our options?
1) PFSense mini-itx: If we place the router in a bridge mode, can we use any mini-itx running PFSense to connect to our NBN/VDSL or do we have to look for a hardware with such feature. In this case, PPPoE authentication will be handed over to the device connected to the router. Any recommendation for PFSense hardware?
2) Run other web services at other port numbers: We are worried that some services will require further tweaking; not only the Apache/HAproxy/NGINX would need to be tweeked. We tried Domain forwarding on our DNS settings but would not work.
3) Bundle PFSense to a decommisioned laptop; attach an additional NIC (external) to the laptop, set up router in the bridge mode and PFSense; and finally connect the laptop to another router {for wifi and wired connections} via the external NIC.
4)......

On the other hand, we are also now running a DHCP server in one of the FreeBSD jails via a dedicated interface (ix0). While it allocates IP addresses, the gateway with host [on another interface -igb0] do not provide Internet connection to the DHCP clients. PF keeps blocking incoming packet no matter what changes we made. We tried "route-to" based on a few examples [1, 2] on the Internet but no luck. We tried if we could route packets via the host interface (igb0) and another interface in it (ix1) that is connected to the router using the router's DHCP but no luck too.

References:
1) https://www.openbsd.org/faq/pf/pools.html#outgoing
2) https://calomel.org/pf_config.html [Example: Two external ISP connections using route-to and round-robin]
 
have multiple public IPs to serve every httpd server from inside.
That's what we have but not working.
And for "Use httpd reverse-proxy," I thought that was what Haproxy was also doing for us.

I however can think further of this your recommendation. And that is getting all web services on one host though for load balancing they can still be spread across multiple servers/ips/hosts. One thing that comes straight to my mind is that Letsencrypt certificate is finicky about that. Applications also have various requirements and that may require them running on distinct hosts. If we were using commercial certificate, that may not be a problem.
 
Thanks; the URL points to a short tutorial for Ubuntu. Notwithstanding, we have a couple of Java applications using the same configuration. In fact, monit with mmonit is one of them.

It works well for an application server (tomcat) running at its own port. We don't have the reported problem with such applications/services. We have already got such service running at a different port though HTTPS. We can't do that for all services. Even if we do, we would have to reconfigure Haproxy/Apache/Nginx to run at different ports for each application/service.
 
3) Bundle PFSense to a decommisioned laptop; attach an additional NIC (external) to the laptop, set up router in the bridge mode and PFSense; and finally connect the laptop to another router {for wifi and wired connections} via the external NIC.
The option 3 was chosen and now fully deployed. This option required purchasing a 4-port+4-POEport Gigabyte switch. The router, on the other hand, now no longer does the authentication.

We were earlier worried that the laptop may not be able to do the PPPoe authentication via Pfsense owing to the lack of a modem module. The reason is that we had tried running net/rp-pppoe from the FreeBSD repo but did not work. The maintainer of the port had already warned users about it and would not provide support for it.

Pfsense looks crisp. It is a collection of some managed and unmanaged ports that have been closely integrated. We now have several layers of (PF) firewalls and are dealing with new issues that come with Pfsense and multi-layer security.
 
Back
Top