IPFW Private VPN + Firewall on a VPS

Angelo Klin

Angelo Klin

Hello All,

With all the fuzz and issues with security and privacy these days I decided to give it a go with a VPN, mostly for the fun and challenge.

I am partially done with a scenario that sounds very typical these days, although it is not necessary plain vanilla.

The overall idea is to have a private VPN using a cloud server. Implies:
  • Not using any servers at HOME (except services for the home LAN, provided by DD-WRT). The Router is, for all intents and purposes, only a client for the remote VPN server.
  • The Virtual Private Server (VPS) runs SSHd for administration and OpenVPN server to clients (would accept road runner mobile as well).
  • NOTE: The VPS offers only ONE IPv4 interface. Communication from HOME to VPS is over the Internet. No second interface or backdoor.

What I have done so far:
  • Installed FreeBSD 11.1-RELEASE-p10 on the VPS
    • Runs SSHd for remote administration
    • Runs NTPd for time sync
    • (To Do) DNS server (Unbound/BIND/DNSMasq)
  • Installed OpenVPN on the VPS
    • Runs as OpenVPN server
  • Enabled IPFW + in-kernel NAT on the VPS
    • IPFW is wide open (here is my problem, please see below)
    • NAT is only replacing the tunnel 10.8.0.0/24 to/from Internet
  • Using DD-WRT on a Netgear R7000 at Hom
    • Runs as OpenVPN client
    • Provides the usual stuff for Home (DHCPd, DNSMasq, Wireless and others)
What is missing:
  • As hinted above the ipfw is wide open.
  • I have tried some suggested scripts from the documentation, articles and forums.
  • I used syslogd (net.inet.ip.fw.verbose*) to monitor the packet flow, but I am failing to grasp how the packets are navigating inside the kernel between ipfw/nat, to see the ipfw script as a programming logic.
  • I also tried to consider the tunnel (tun0) as a secondary interface, but there is always something missing, given the actual packets are flowing via the physical interface.
  • I created three FreeBSD VMs with VMWare, but I could not quite get a perfect scenario to clone the existing configuration.
  • I tried tcpdump/tshark/ntop in the "Lab", but it was not conclusive, as it does not match the real configuration.
Mild frustration, likely to be shared by others:
  • Documentation is scarce and kind of outdated;
  • Surprisingly enough I could not find a single example using this architecture;
  • The focus of articles/videos is usually:
    • Installing FreeBSD OR Configuring Services OR OpenVPN OR bare minimum IPFW/PF
    • There is not an "AND" related article/video
  • PF examples are OpenBSD related or specific;
    • Not clear how updated/outdated PF is on FreeBSD;
  • Linux examples with IPTables are of no help;
Considerations for the firewall rules:
  • NAT for the VPN tunnelled IPs
  • Stateful
  • There are requests from the LAN (most traffic)
    • OpenVPN to the VPS
    • SSH to the VPS
    • All the internet use to the Internet (i.e. DNS, Mail, Web, NTP)
  • There are requests from the VPS itself
    • Reply to OpenVPN and SSH
    • NTP / DNS to the Internet
  • Again, just ONE valid IP address
  • The [FONT=Courier New]tun0[/FONT] interface
Here is the architecture:[FONT=Courier New]
Code: 
  +----------+ +-------+ +--------+ +----+
  | Computer | | Phone | | Tablet | | Tv |
  +-----+----+ +---+---+ +----+---+ +--+-+
        |          |          |        |
        +----------+-----+----+--------+
                         |
                    +----@----+
                    | Router  | @ 192.168.1.0/24              HOME ^
 - - - - - - - - - -| R7000   |- - - - - - - - - - - - - - - - - - -
    +-(ADSL PPPoE)->+ -dd-wrt | o 10.8.0.2 (VPN tunnel)   INTERNET v
    |               +---o-*---+ * 200.1.1.100 (Valid IP)
  +-+---+               | |
  | ISP |               | |
  +-----+               | |
 - - - - - - - - - - - -|-|- - - - - - - - - - - - - - - - - - - - -
                        | | NOTE: VPS has only ONE interface
(VPN tunnel) 10.8.0.1 o | +----------+
(Valid IP) 250.1.1.50 * | |          |
     +------------------o-*-+      +-+--------+
     | Cloud VPS            |      | Internet |
     | FreeBSD 11.1-REL-p10 |      +----------+
     | -SSHd / NTPd         |
     | -OpenVPN Server      |
     | -IPFW + NAT (kernel) |
     +----------------------+
[/FONT]

Some configuration:
[FONT=Courier New]
Code: 
net.inet.ip.forwarding: 1
# some articles say to set one_pass to 0 to allow NAT to reinject the packet, but it is working nevertheless.
net.inet.ip.fw.one_pass: 1
[/FONT]

[FONT=Courier New]/etc/rc.conf[/FONT]
[FONT=Courier New]
Code: 
# . . .
gateway_enable="YES"
#
firewall_enable="YES"
firewall_type="open"
firewall_nat_enable="YES"
firewall_nat_interface="vtnet0"
#
openvpn_enable="YES"
openvpn_configfile="/usr/local/etc/openvpn/server.conf"
openvpn_dir="/usr/local/etc/openvpn"
[/FONT]

[FONT=Courier New] [/FONT]
PS: I am okay with some expert taking the idea to publish an article, make a video or present in a BSD conference, as long as we all get a working, easy to use model.

Thanks and Regards
 
FreeBSD uses an older set version of PF. The Book of PF gives examples from both versions, the current one in OpenBSD, and the one used in FreeBSD.

According to pf(4), the PF version in stable or production FreeBSD, is the same used in OpenBSD 4.5.

IPFW has a few canned firewalls that can be used, which are listed in /etc/rc.firewall
Code: 
firewall_enable="YES" 
firewall_type="client" # simple workstation

I've used a custom PF ruleset in combination with a defined IPFW ruleset before, and it caused no problems.

Linux's IPtables is irrelevant here.
 
I have similar scenario with FreeBSD with OpenVPN server with TAP interface and several branch offices with DD-WRT. You don't need to use NAT at all.

So what exact is your question?
 
sidetone said:
FreeBSD uses an older set version of PF. The Book of PF gives examples from both versions, the current one in OpenBSD, and the one used in FreeBSD.

According to pf(4), the PF version in stable or production FreeBSD, is the same used in OpenBSD 4.5.

IPFW has a few canned firewalls that can be used, which are listed in /etc/rc.firewall
Code: 
firewall_enable="YES"
firewall_type="client" # simple workstation

I've used a custom PF ruleset in combination with a defined IPFW ruleset before, and it caused no problems.

Linux's IPtables is irrelevant here.
Click to expand...

Hello sidetone,

Thanks for the update on the status of pf(4).

I tried other options of firewall type but not much success. I have to use cron to reset the setup, in case the new configuration locks me out.

I mention Iptables as its function. I even found a paper for a transpiler using meta definition of rules. It was more theoretical with just partial implementation. But I agree that does not help.

Best regards
 
VladiBG said:
I have similar scenario with FreeBSD with OpenVPN server with TAP interface and several branch offices with DD-WRT. You don't need to use NAT at all.

So what exact is your question?
Click to expand...

Hello VladiBG,

Thanks for your reply.

I guess I focus on explaining the scenario and the specific, and forgot about an specific point. I have seen some replies that asks what is the person’s intent, so I tried to what I want to accomplish.

To put in a question format I would frame as: “Could someone please suggest a rules set for the firewall (preferably ipfw, but exclusively) that protects and works in the described scenario? A template would be helpful as well assuming the inner workings are there. I can customise the ports and other particular aspects. Thanks”

Best regards
 
First you need to bring up your VPN connection without using any Firewall on both sides. You don't need to use NAT. You need only a routing. You can go for dynamic routing or static or use client config dir (ccd) to push the routes for the OpenVPN clients.

After that you can deploy the Firewall to restrict the network flows.

So in your case of the OpenVPN clients that are connecting from the Internet to the OpenVPN server at 250.1.1.50 You need to push the following route so they can reach 192.168.1.0/24 netwotk

At the OpenVPN server you need to have in rc.conf

Code: 
static_routes="internalnet1"

route_internalnet1="-net 192.168.1.0/24 10.8.0.2"

So the server will know when it need to talk to 192.168.1.0/24 to use the gateway address of 10.8.0.2.

For the roadwarriors clients use /usr/local/etc/openvpn/ccd/username.conf with the following sintax
push "route 192.168.1.0 255.255.255.0"

This will add the route in the routing table of the connecting client. Keep in mind that you need to run as administrator the OpenVPN client so it can modify the routing table of the Windows clients.
 
VladiBG said:
First you need to bring up your VPN connection without using any Firewall on both sides. You don't need to use NAT. You need only a routing. You can go for dynamic routing or static or use client config dir (ccd) to push the routes for the OpenVPN clients.

After that you can deploy the Firewall to restrict the network flows.

So in your case of the OpenVPN clients that are connecting from the Internet to the OpenVPN server at 250.1.1.50 You need to push the following route so they can reach 192.168.1.0/24 netwotk

At the OpenVPN server you need to have in rc.conf

Code: 
static_routes="internalnet1"

route_internalnet1="-net 192.168.1.0/24 10.8.0.2"

So the server will know when it need to talk to 192.168.1.0/24 to use the gateway address of 10.8.0.2.

For the roadwarriors clients use /usr/local/etc/openvpn/ccd/username.conf with the following sintax
push "route 192.168.1.0 255.255.255.0"

This will add the route in the routing table of the connecting client. Keep in mind that you need to run as administrator the OpenVPN client so it can modify the routing table of the Windows clients.
Click to expand...

Hello VladiBG,

Thanks again for your reply.
If I got it right, your explanation addresses the scenarios where a person in a hotel or at home reaches to the office.
My scenario is a domestic one. I have a normal home environment with an IP given via ADSL from the ISP, and a router does the middle man's job. The next step is to use a private VPN hosted in a server on the cloud.
I am in Australia, but to use more US brands, imagine that instead of having a NordVPN/ExpressVPN service, I am providing my own hosting with FreeBSD at a Digital Ocean droplet. These are just names for sake of the example, I don't use any of those.
As I stated, the OpenVPN setup is working and so the IPFW and the NAT (I believe I do need NAT to translate the 10.8.0.0/24 from the tunnel to go out on the Internet). I do NOT want a server from the Internet to get inside my server or home. My current issue is that the Rules Set for the firewall are wide open.

Thanks again for your help. Any other information is welcome.
Best regards.
 
If you want to provide internet connection for the VPN clients then Yes you will need NAT to translate 10.8.0.0/24 to the external_ip address of your server at 250.1.1.50. If you are going to use your local Internet at your Home that is coming from your ADSL with PPPoE then you do not need NAT. So after you have your VPN server UP and Running then you can step ahead and deploy the Firewall that will filter the connections from external interface to the tun interface at your server.

In IPFW default policy is to deny so after you enable it you need only to allow the ports of your services like UPD 1194 and TCP 22 then to pass/skip all traffic inspection of your tun interface.
 
Angelo Klin said:
Hello VladiBG,

Thanks again for your reply.
If I got it right, your explanation addresses the scenarios where a person in a hotel or at home reaches to the office.
Angelo Klin said:
Hello VladiBG,

Thanks again for your reply.
If I got it right, your explanation addresses the scenarios where a person in a hotel or at home reaches to the office.
My scenario is a domestic one. I have a normal home environment with an IP given via ADSL from the ISP, and a router does the middle man's job. The next step is to use a private VPN hosted in a server on the cloud.
I am in Australia, but to use more US brands, imagine that instead of having a NordVPN/ExpressVPN service, I am providing my own hosting with FreeBSD at a Digital Ocean droplet. These are just names for sake of the example, I don't use any of those.
As I stated, the OpenVPN setup is working and so the IPFW and the NAT (I believe I do need NAT to translate the 10.8.0.0/24 from the tunnel to go out on the Internet). I do NOT want a server from the Internet to get inside my server or home. My current issue is that the Rules Set for the firewall are wide open.

Thanks again for your help. Any other information is welcome.
Best regards.
Click to expand...

My scenario is a domestic one. I have a normal home environment with an IP given via ADSL from the ISP, and a router does the middle man's job. The next step is to use a private VPN hosted in a server on the cloud.
I am in Australia, but to use more US brands, imagine that instead of having a NordVPN/ExpressVPN service, I am providing my own hosting with FreeBSD at a Digital Ocean droplet. These are just names for sake of the example, I don't use any of those.
As I stated, the OpenVPN setup is working and so the IPFW and the NAT (I believe I do need NAT to translate the 10.8.0.0/24 from the tunnel to go out on the Internet). I do NOT want a server from the Internet to get inside my server or home. My current issue is that the Rules Set for the firewall are wide open.

Thanks again for your help. Any other information is welcome.
Best regards.
Click to expand...
VladiBG said:
If you want to provide internet connection for the VPN clients then Yes you will need NAT to translate 10.8.0.0/24 to the external_ip address of your server at 250.1.1.50. If you are going to use your local Internet at your Home that is coming from your ADSL with PPPoE then you do not need NAT. So after you have your VPN server UP and Running then you can step ahead and deploy the Firewall that will filter the connections from external interface to the tun interface at your server.

In IPFW default policy is to deny so after you enable it you need only to allow the ports of your services like UPD 1194 and TCP 22 then to pass/skip all traffic inspection of your tun interface.
Click to expand...

Hello VladiBG,
Thanks for further explaining it.

I have used some of the suggested code from the documentation and variations of some threads in the forum and articles on the Internet. No complete success so far (some partial success).
Overall I know what I want and how to do it (Big picture). I am failing on the specifics, so after about two weeks I decided to resort to some help. As I described, looks like it is or is becoming a common place for people to use such design. I this case I prefer to do it with FreeBSD instead of any other alternative.

Appreciated
 
There's a good example configuration for IPFW on which you can step and modify to your needs. Read /etc/rc.firewall there's several predefined modes which you can select using "firewall_type" in rc.conf

here's a example

Code: 
firewall_enable="YES"
firewall_type="workstation"
firewall_allowservices="any"
firewall_myservices="22/tcp 80/tcp"

When you understand better how IPFW is working and need something "EXTRA" to add then you can change it with your custom build rules using something like that:

Code: 
firewall_enable="YES"
firewall_logging="YES"
firewall_script="/usr/local/etc/ipfw.rules"

Edit:
workstation type is for example purpose only!
 
VladiBG said:
There's a good example configuration for IPFW on which you can step and modify to your needs. Read /etc/rc.firewall there's several predefined modes which you can select using "firewall_type" in rc.conf

here's a example

Code: 
firewall_enable="YES"
firewall_type="workstation"
firewall_allowservices="any"
firewall_myservices="22/tcp 80/tcp"

When you understand better how IPFW is working and need something "EXTRA" to add then you can change it with your custom build rules using something like that:

Code: 
firewall_enable="YES"
firewall_logging="YES"
firewall_script="/usr/local/etc/ipfw.rules"

Edit:
workstation type is for example purpose only!
Click to expand...

Hello VladiBG,
Thanks for your last instructions. I have already past those steps. My issues are with the actual rules set for the given scenario.

Best regards
 
Ho Angelo,

I agree to You that when it comes to a specific and deliberate design, there are not so many tutorials on how to design the fw rules. Probably this is because there are so many possible layouts, that there is no genuine approach. Also, one may need to know what shall actually be protected from what, in order to then design the fw around this aim.

I am using ipfw in a rather complex setup with multiple VPN, and my layout is quite different to Yours, so this cannot be used 1:1 for You. Nevertheless, most of Your issues may probably be addressed in my design. My approach was to separate the flows entirely, in a way like this:

Code: 
f=/sbin/ipfw
ipif=fxp0
dialupif1=tun3
vpnif1=tun0

$f add 9999 skipto 30000 all from any to any out

$f add 10000 skipto 10900 all from any to any not via $ipif
# add here all rules concerning stuff incoming on $ipif
$f add 11000 skipto 11900 all from any to any not via $dialupif1
# add here all rules concerning stuff incoming on $dialupif1
$f add 12000 skipto 12900 all from any to any not via $vpnif1
# add here all rules concerning stuff incoming on $vpnif1
# ... and so on

$f add 30000 skipto 30900 all from any to any not xmit $ipif
# add here all rules concerning stuff outgoing on $ipif
$f add 31000 skipto 31900 all from any to any not xmit $dialupif1
# add here all rules concerning stuff outgoing on $dialupif1
$f add 32000 skipto 32900 all from any to any not xmit $vpnif1
# add here all rules concerning stuff outgoing on $vpnif1
# ... and so on

The disadvantage is that for a single flow one usually needs to add four rules (incoming and outgoing for originate and answer). This may also be a performance penalty, but the advantage is that the design is manageable and extendable.
If a NAT is needed, that can easily be included within the respective section(s).
If stateful behaviour is desired, this can be considered individually for each flow.
Important: this design requires net.inet.ip.fw.one_pass to be switched OFF.

Now, concerning Your specific issues:
First of all, I seem to understand Your design, but I do not really understand the benefit, i.e. what shall actually be achieved by this.

Then:
Angelo Klin said:
  • I used syslogd (net.inet.ip.fw.verbose*) to monitor the packet flow, but I am failing to grasp how the packets are navigating inside the kernel between ipfw/nat, to see the ipfw script as a programming logic.
Click to expand...

This appears to be quite simple: When a nat rule appears, the matching packets are moved through the nat and then continue to be processed by the following rule (if net.inet.ip.fw.one_pass is OFF) with their modified properties. (Caveat: my experience is only with the external user-space NAT)

  • I also tried to consider the tunnel (tun0) as a secondary interface, but there is always something missing, given the actual packets are flowing via the physical interface.
Click to expand...

This is just what I am doing (as shown above), and what I would recommend.
Lets take a closer look, as far as I undertand Your design: The originate packet (at Your home) is crypted into another packet to achieve the VPN flow. This VPN flow (with the VPN IP addresses) appears incoming on your Cloud machine physical interface. Then the VPN is decoded, i.e. the inner packet is unpacked, and that original packet then appears incoming on your tun interface. Then the kernel's routing is executed on the packet, and consequently the packet appears again as outgoing on your physical interface.
For the answering packets from the Internet it runs the other way round.

For the local traffic (Home to/from Cloud-server, Cloud-server to/from Internet) only certain parts of this do happen.

This interesting question here is where to place the NAT. This depends on the layout of the routing, and may still require some thoughts.

  • Documentation is scarce and kind of outdated;
  • Surprisingly enough I could not find a single example using this architecture;
  • The focus of articles/videos is usually:
    • Installing FreeBSD OR Configuring Services OR OpenVPN OR bare minimum IPFW/PF
    • There is not an "AND" related article/video
Click to expand...

Thats why I created my design (as above) from scratch. (Caveat: I have not received proper discussion if such a design is generally feasible or bears it's own risks. I am open to comments and criticism.)
 
PMc said:
Ho Angelo,

I agree to You that when it comes to a specific and deliberate design, there are not so many tutorials on how to design the fw rules. Probably this is because there are so many possible layouts, that there is no genuine approach. Also, one may need to know what shall actually be protected from what, in order to then design the fw around this aim.

I am using ipfw in a rather complex setup with multiple VPN, and my layout is quite different to Yours, so this cannot be used 1:1 for You. Nevertheless, most of Your issues may probably be addressed in my design. My approach was to separate the flows entirely, in a way like this:

Code: 
f=/sbin/ipfw
ipif=fxp0
dialupif1=tun3
vpnif1=tun0

$f add 9999 skipto 30000 all from any to any out

$f add 10000 skipto 10900 all from any to any not via $ipif
# add here all rules concerning stuff incoming on $ipif
$f add 11000 skipto 11900 all from any to any not via $dialupif1
# add here all rules concerning stuff incoming on $dialupif1
$f add 12000 skipto 12900 all from any to any not via $vpnif1
# add here all rules concerning stuff incoming on $vpnif1
# ... and so on

$f add 30000 skipto 30900 all from any to any not xmit $ipif
# add here all rules concerning stuff outgoing on $ipif
$f add 31000 skipto 31900 all from any to any not xmit $dialupif1
# add here all rules concerning stuff outgoing on $dialupif1
$f add 32000 skipto 32900 all from any to any not xmit $vpnif1
# add here all rules concerning stuff outgoing on $vpnif1
# ... and so on

The disadvantage is that for a single flow one usually needs to add four rules (incoming and outgoing for originate and answer). This may also be a performance penalty, but the advantage is that the design is manageable and extendable.
If a NAT is needed, that can easily be included within the respective section(s).
If stateful behaviour is desired, this can be considered individually for each flow.
Important: this design requires net.inet.ip.fw.one_pass to be switched OFF.

Now, concerning Your specific issues:
First of all, I seem to understand Your design, but I do not really understand the benefit, i.e. what shall actually be achieved by this.

Then:


This appears to be quite simple: When a nat rule appears, the matching packets are moved through the nat and then continue to be processed by the following rule (if net.inet.ip.fw.one_pass is OFF) with their modified properties. (Caveat: my experience is only with the external user-space NAT)



This is just what I am doing (as shown above), and what I would recommend.
Lets take a closer look, as far as I undertand Your design: The originate packet (at Your home) is crypted into another packet to achieve the VPN flow. This VPN flow (with the VPN IP addresses) appears incoming on your Cloud machine physical interface. Then the VPN is decoded, i.e. the inner packet is unpacked, and that original packet then appears incoming on your tun interface. Then the kernel's routing is executed on the packet, and consequently the packet appears again as outgoing on your physical interface.
For the answering packets from the Internet it runs the other way round.

For the local traffic (Home to/from Cloud-server, Cloud-server to/from Internet) only certain parts of this do happen.

This interesting question here is where to place the NAT. This depends on the layout of the routing, and may still require some thoughts.



Thats why I created my design (as above) from scratch. (Caveat: I have not received proper discussion if such a design is generally feasible or bears it's own risks. I am open to comments and criticism.)
Click to expand...

Hello PMc,
Thanks for your reply.

I understand that there is no "one size fits all" solution. Maybe the best way to put is to have a sort of template.
As I put it, and you got the flow quite right, my current issue is to understand and work with the internal flow of the packages.
It is not clear to me from the documentation that using kernel NAT and net.inet.ip.fw.one_pass=0 where the NATed package will come back in the list. Is it at the top or from the next rule. You said it goes to the next line, but you are using the external NAT (maybe it works the same).
I got some more understanding with the documentation from Guide FreeBSD 11.0 Install Guide. This text and code were used in the official documentation (Handbook) and I saw variations in the forums as well. It does have a structure and works as a template. But it is not working in my case for some reason. Perhaps the one interface or the VPN are tripping the logic. Hence my cry for help. :)

I am trying both a lab at home with three VMs and with my actual setup to see the flow of the packages using tcpdump/tshark/ntop.

To address your point of "why?":
- In the US people are now very concerned with the issue of NetNeutrality, which is somewhat associated with privacy, as ISP will be allowed to sniff your habits and add cost or offer more services from that particular usage. Not that they don't sniff it already. So in a domestic context is to enhance privacy/security. This is why VPN for the masses is a booming service (NornVPN, ExpressVPN, and so on). Can you trust these guys either?
- A while back, before NetFlix was available in Australia, people used VPNs to have a US IP address so they could see US content. Some people still do, even having NetFlix now available here, as the content is still distinct given international copyright laws and distribution contracts between studios.
- Once I have this figured, the next step would be to transfer my personal domain to a self-managed model for web and email. Given the annual cost of web hosting vs cost of a virtual private server.
- But the main reason is mostly the challenge. :) And I have some friends that are very keen on using my recipe.

Cheers
 
Ho Angelo!

Angelo Klin said:
To address your point of "why?":
- In the US people are now very concerned with the issue of NetNeutrality, which is somewhat associated with privacy, as ISP will be allowed to sniff your habits and add cost or offer more services from that particular usage. Not that they don't sniff it already. So in a domestic context is to enhance privacy/security. This is why VPN for the masses is a booming service (NornVPN, ExpressVPN, and so on). Can you trust these guys either?
- A while back, before NetFlix was available in Australia, people used VPNs to have a US IP address so they could see US content. Some people still do, even having NetFlix now available here, as the content is still distinct given international copyright laws and distribution contracts between studios.
Click to expand...

Ah, now the story figures! :)
I was wondering as You seem to camouflage the path from Your home-base to the Internet, but then have the same data visible on the path from your cloud-site to the Internet. But, as You say, the cloud-provider can be chosen, and it can be in a different country, so there is an advantage.

"NornVPN, ExpressVPN"
I had a look into these. They don't even reveal their own identity. Why should one do business with somebody rather anonymous, even more a business that requires some trustworthiness? Nothing they tell needs to be actually true - the only thing I would rely on is that they can bill a credit card. I think this is a business of making money from the fears of people.

I understand that there is no "one size fits all" solution. Maybe the best way to put is to have a sort of template.
As I put it, and you got the flow quite right, my current issue is to understand and work with the internal flow of the packages.
It is not clear to me from the documentation that using kernel NAT and net.inet.ip.fw.one_pass=0 where the NATed package will come back in the list. Is it at the top or from the next rule. You said it goes to the next line, but you are using the external NAT (maybe it works the same).
Click to expand...

I would suppose it works the same. At the time I built my thing there was no in-kernel NAT yet. And over time it has grown to a hairy beast, and I am reluctant to tackle it and give it a full overhaul.

I got some more understanding with the documentation from Guide FreeBSD 11.0 Install Guide.
Click to expand...

Ups. That example appears to be very strange: they are doing keep-state on a skipto rule!

The functionality is: if you place a keep-state somewhere, then the 5-tuple (fromaddr, fromport, toaddr, toport, protocol) of a matching packet will be stored. When following packets with the same 5-tuple (in one direction or the other) meet a check-state rule, then the action from the original keep-state rule is executed on them.
So, if that action is a skipto, then the skipto will be executed again!

The bottomline of this is: if you use keep-state, then use it only on an "allow" rule, because everything else will make things very complicated and will usually not do what you want.

Lets have a closer look at that description:

The packet finally comes to rule 125 and matches. It's outbound through the NIC facing the public Internet. The packet still has its source IP address as a private LAN IP address. On the match to this rule, two actions take place. The keep-state option will post this rule into the keep-state dynamic rules table and the specified action is executed. The action is part of the info posted to the dynamic table. In this case it's "skipto rule 500". Rule 500 NATs the packet IP address and out it goes. Remember this, this is very important. This packet makes its way to the destination and returns and enters the top of the rule set. This time it does match rule 100 and has its destination IP address mapped back to its corresponding LAN IP address. It then is processed by the check-state rule, it's found in the table as an existing session conversation and is released to the LAN.

Wrong! At that point the stored action from the dynamic table ("skipto 500") is executed, and so processing does continue there. Rule 500 will not match, because the packet is not outbould, and then in rule 510 the packet is actually released to the LAN.

So that code will for now work as desired, but as soon as you need to add further functionality near rule 500, you will experience really weird behaviour and not instantly see why.
This is bad and dangerous coding - probably the author did not even see the problem, and the code works by accident.

It goes to the LAN PC that sent it and a new packet is sent requesting another segment of the data from the remote server. This time it gets checked by the check-state rule and its outbound entry is found. The associated action skipto 500 is executed. The packet jumps to rule 500, gets NATed and is released to exit out the external NIC.

So, now it does...
 
PMc said:
Ho Angelo!



Ah, now the story figures! :)
I was wondering as You seem to camouflage the path from Your home-base to the Internet, but then have the same data visible on the path from your cloud-site to the Internet. But, as You say, the cloud-provider can be chosen, and it can be in a different country, so there is an advantage.

"NornVPN, ExpressVPN"
I had a look into these. They don't even reveal their own identity. Why should one do business with somebody rather anonymous, even more a business that requires some trustworthiness? Nothing they tell needs to be actually true - the only thing I would rely on is that they can bill a credit card. I think this is a business of making money from the fears of people.



I would suppose it works the same. At the time I built my thing there was no in-kernel NAT yet. And over time it has grown to a hairy beast, and I am reluctant to tackle it and give it a full overhaul.



Ups. That example appears to be very strange: they are doing keep-state on a skipto rule!

The functionality is: if you place a keep-state somewhere, then the 5-tuple (fromaddr, fromport, toaddr, toport, protocol) of a matching packet will be stored. When following packets with the same 5-tuple (in one direction or the other) meet a check-state rule, then the action from the original keep-state rule is executed on them.
So, if that action is a skipto, then the skipto will be executed again!

The bottomline of this is: if you use keep-state, then use it only on an "allow" rule, because everything else will make things very complicated and will usually not do what you want.

Lets have a closer look at that description:

The packet finally comes to rule 125 and matches. It's outbound through the NIC facing the public Internet. The packet still has its source IP address as a private LAN IP address. On the match to this rule, two actions take place. The keep-state option will post this rule into the keep-state dynamic rules table and the specified action is executed. The action is part of the info posted to the dynamic table. In this case it's "skipto rule 500". Rule 500 NATs the packet IP address and out it goes. Remember this, this is very important. This packet makes its way to the destination and returns and enters the top of the rule set. This time it does match rule 100 and has its destination IP address mapped back to its corresponding LAN IP address. It then is processed by the check-state rule, it's found in the table as an existing session conversation and is released to the LAN.

Wrong! At that point the stored action from the dynamic table ("skipto 500") is executed, and so processing does continue there. Rule 500 will not match, because the packet is not outbould, and then in rule 510 the packet is actually released to the LAN.

So that code will for now work as desired, but as soon as you need to add further functionality near rule 500, you will experience really weird behaviour and not instantly see why.
This is bad and dangerous coding - probably the author did not even see the problem, and the code works by accident.

It goes to the LAN PC that sent it and a new packet is sent requesting another segment of the data from the remote server. This time it gets checked by the check-state rule and its outbound entry is found. The associated action skipto 500 is executed. The packet jumps to rule 500, gets NATed and is released to exit out the external NIC.

So, now it does...
Click to expand...

Hello PMc,
Thanks for your comment.
First the Forum went out, then I went out for a few days.

I see your point and it does make sense.
I have to put some hours with focus and testing, which I didn't have much recently.
I tried to have a look at the source code, but I think it is overkill for you to make it work. The documentation could/should be more educational rather than just a reference, or it would be nice to have both.

Cheers
 
I thought I might write a little paper on how to do this thing better - and I found that it is really difficult. In fact, one just cannot combine NAT and stateful rules in a straight-forward way, and I see only two options. Either doing something weird like the "skipto" rules in the example above, or -what I chose to do- catch the packets on the incoming side: A forwarded packet passes through ipfw two times: once when it is incoming from the LAN side on its way to the kernel routing, and the second time when it is afterwards outgoing to the WAN side. We cannot place a stateful allow rule before the NAT rule is reached, but we can do this on the incoming packets that are already destined to the WAN, and that will al well make the stateful mechanism work.
But both approaches need careful thinking and good planning.
 
You must log in or register to reply here.
Back
Top