PF Tuning

M

muzinim

What PF tuning parameters are recommended for firewall deployments in a large environment? I have an HP DL380/G7 running FreeBSD 8.2-RELEASE-p3/AMD64. The system has ten NIC's, 4 x HP NC382i, 2 x Intel PRO/1000, and four Intel PRO/10GbE ports. The following tuning parameters are in /etc/sysctl.conf. The system passes traffic with no delays or lost connections with the firewall rules disabled but connectivity through the firewall is sporadic with the firewall rules enabled. This sporadic behavior was seen with TCP, UDP, and ICMP. Traffic would ingress one interface but the traffic would not appear on the egress interface. The pf statistics showed state table searches of 25,104 per second.

Code: 
hw.syscons.kbd_reboot=0
debug.debugger_on_panic=0
hw.intr_storm_threshold=9000
kern.ipc.nmbjumbo9=100000
kern.ipc.nmbjumbo16=32000
# The next parameter is set in /boot/loader.conf
# kern.ipc.nmbclusters=190000
kern.ipc.nmbjumbop=500000
kern.ipc.maxsockbuf=1048576
vm.kstack_cache_size=512
# kern.ipc.maxsockets=80000
 
I have examined sections of the pf.conf file and it appears to be correct. I am using Firewall Builder to manage and publish rules. Each rule does have the quick option enabled which I would think helps efficiency.
 
I think I have determined the problem. The states hard limit was set to 10,000. When the firewall was in production I ran a pfctl -s info and it showed state table current entries of 9,245. My suspicion is when the current entires rose about the 10,000 hard limit, the firewall stopped accepting new connections. That would explain hosts being able to make connections through the firewall sporadically. Does this seem like a reasonable assumption?
 
muzinim said:
I think i have determined the problem. The states hard limit was set to 10,000. When the firewall was in production i ran a pfctl -s info and it showed state table current entries of 9,245. My suspicion is when the current entires rose about the 10,000 hard limit, the firewall stopped accepting new connections. That would explain hosts being able to make connections through the firewall sporadically. Does this seem like a reasonable assumption?
Click to expand...

Yes it does.

In such environments I always have used commercial vendor appliances like Nokia or Cisco. Their throughput and max connections numbers are usually reliable and tested.

BTW It would be very interesting if you show some CPU stats. I am really curious...
 
gkontos said:
Yes it does.
Click to expand...

Yes, that's sure.

In such environments I always have used commercial vendor appliances like Nokia or Cisco. Their throughput and max connections numbers are usually reliable and tested.
Click to expand...

They are also very expensive :)

Well at work, last year, we replaced one cisco cs7204 router and one PIX firewall by one PC running PF on OpenBSD (doing routing and firewalling). The fw protects the internal network from Internet.

The cs7204 is not very powerful and the result with OpenBSD/PF is far better (and it's free!).

In peak, states number is around 300k with packets/s ~50k/s in, 40k/s out. Throughput ~500Mbit/s and the cpu at 50% in system (because OpenBSD is not really SMP we run a mono-processor kernel, I expect FreeBSD would be better on that point).

We don't know exactly where is the limit, we have not reached it yet. We also have to improve the rules set because it is badly written.

We use intel network cards (fiber and copper), as far I can see the driver is critical. There were problems in OpenBSD < 5.0 on em(4) and the performances were not optimal causing packets loss in high load.

I've not tested FreeBSD because there are few missing features we need (pflow, carp looks buggy on FreeBSD, missing md5 tcp signature in OpenBGPd) and OpenBSD does the job quite well.

Regards.
 
plamaiziere said:
They are also very expensive :)
Click to expand...

Not really, you can get 2XASA 5520 around $10K. Of course you get to pay some annual fees as well for support and maintenance. But they are really good firewalls for Internet facing sites.

Checkpoints are expensive compared to Cisco but they are also rock solid.

plamaiziere said:
Well at work, last year, we replaced one cisco cs7204 router and one PIX firewall by one PC running PF on OpenBSD (doing routing and firewalling). The fw protects the internal network from Internet.

The cs7204 is not very powerful and the result with OpenBSD/PF is far better (and it's free!).
Click to expand...

Ok, but you have to realize that PIX firewalls are really old. During their era they were also very good and stable.

plamaiziere said:
In peak, states number is around 300k with packets/s ~50k/s in, 40k/s out. Throughput ~500Mbit/s and the cpu at 50% in system (because OpenBSD is not really SMP we run a mono-processor kernel, I expect FreeBSD would be better on that point).

We don't know exactly where is the limit, we have not reached it yet. We also have to improve the rules set because it is badly written.

We use intel network cards (fiber and copper), as far I can see the driver is critical. There were problems in OpenBSD < 5.0 on em(4) and the performances were not optimal causing packets loss in high load.

I've not tested FreeBSD because there are few missing features we need (pflow, carp looks buggy on FreeBSD, missing md5 tcp signature in OpenBGPd) and OpenBSD does the job quite well.

Regards.
Click to expand...

Firewall rule management and HA are for me the number one priority. I have worked in a place where we had reached 65000 ACES in the FWSMs. It took some really heavy work to consolidate all this to under 40000.

Some checkpoints had also 10000 rules!

Of course all those are distribution firewalls with many interfaces.

Best Regards,
Geoge
 
You must log in or register to reply here.
Back
Top