pf slow transfer after upgrade to 9.0

[da1]

Hi guys,

The situation I'm seeing is something I cannot understand and therefore I need some help. We have two PF, redundant, firewalls in our DC. The backup firewall was upgraded to 9.0-STABLE r236672. We have a backup machine there and we are sending zfs snapshots from our office through the backup firewall. The office is limited to 2MB up/down speed and in the DC we have 10MB up/down.

The problem is that when I'm sending a zfs stream to the backup machine (located behind the firewall, evidently) I do not get speeds greater than 800 KB. The situation is even stranger when I cannot get past 800KB if I copy directly to the backup firewall too.

Then, it gets worse. If I copy directly to the primary firewall, I get the full speed we have in the office (2MB) but if I go through the primary firewall to the backup machine, I'm again limited to max 800KB.

I tried scp and ssh (both with -c arcfour), zfs send|receive and ftp. I tried all of these to the backup machine and also to another machine which is located in another rack (different switch, etc)

Both machines are Dell PowerEdge 1850 and we have no bandwidth limiting of any sorts.

Here is the relevant part of pf.conf for both firewalls:

int_if="em0"
ext_if="em1"

internal_net="int_net_ip/24"
external_addr="ext_net_ip"

icmp_types="echoreq"

table <external_net> { .....}

set timeout { adaptive.start 0, adaptive.end 0 }
set limit { states 200000, frags 100000 }

set loginterface $int_if
set optimization conservative
set block-policy drop
set require-order yes
set debug urgent

# filtere nicht das loopback-interface
set skip on lo0

# Normalization
scrub in on $ext_if

# gateway
nat pass on $ext_if inet from $internal_net to any -> ($ext_if:0)


# redirect 2248 from buero to backup.local (ip48)
rdr pass on $ext_if proto tcp from "office_ip: to $ip port 2248 -> $internal_ip port 22

block in log all
pass out all

block in  log quick inet6 all
block out log quick inet6 all

# andtispoof
antispoof quick for { lo0 $int_if $ext_if }

I checked the mbufs and are way below the limit.

At the moment I have no other idea how to debug this. Any help is appreciated.


LE: /boot/loader.conf has no pf/carp related stuff and in /etc/sysctl.conf we have:

net.inet.carp.preempt=1
net.inet.carp.allow=1
net.inet.carp.log=1
net.inet.ip.forwarding=1

kern.ipc.somaxconn=32768

net.inet.tcp.keepidle=10000
net.inet.tcp.keepintvl=5000

 

[jnbek]

Aye, this is of great interest to me as I am wanting to upgrade my FreeBSD 8-STABLE/PF based Router/Firewall machine to 9-STABLE... Every time I go to update RELENG in my supfile, I see something like this that makes me stop cold. Please document here any fixes you make with regard to this da1, I appreciate it in advance

 

[da1]

Small update. If using SSH to any firewalls and from there SSH again to the backup machine, I get full speed.