Everyday problem with dummynet QFQ queue

Ramzes · Dec 27, 2011

I have a mid-load FreeBSD 8.2-STABLE server with two 100mbit interfaces and based on ZFS raidz1 4 disks pool on root partition.
I need to shape outgoing file flow on both interfaces to prevent web-site from lags. So i'm using ipfw to do this:

Code:

# First flow shape
$fwcmd pipe 1 config bw 80Mbit/s queue 20
$fwcmd sched 1 config type QFQ pipe 1

$fwcmd add queue 10 ip from $PLAYIP to any
$fwcmd queue 10 config weight 10 pipe 1 mask dst-ip 0x000000ff

# Second flow shape
$fwcmd pipe 2 config bw 90Mbit/s queue 20 #mask dst-ip 0x0000ffff burst 1048576
$fwcmd add pipe 2 ip from $PLAY2IP to any

# Policy-based routing
$fwcmd add 2100 fwd $PLAY2GW ip from $PLAY2IP to any

I have two IP's on first interface, and one on second:

Code:

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:25:90:04:73:4c
        inet $WEBIP netmask 0xfffffff8 broadcast 212.*.*.*
        inet $PLAYIP netmask 0xfffffff8 broadcast 212.*.*.*
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
        ether 00:25:90:04:73:4d
        inet $PLAY2IP netmask 0xfffffff8 broadcast 212.*.*.*
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active

This config works but for some time. Everyday I'm used to reboot firewall (/etc/rc.d/ipfw restart), because of failures. Every reboot follows with this messages in /var/log/messages:

Code:

Dec 25 16:29:46 mixupload kernel: Bump qfq weight to 1 (was 0)
Dec 25 16:29:46 mixupload kernel: Bump qfq maxlen to 1500 (was 0)
Dec 25 16:29:46 mixupload kernel: Bump flowset buckets to 64 (was 0)
Dec 25 16:29:46 mixupload kernel: Bump qfq maxlen to 1500 (was 0)
Dec 25 16:29:46 mixupload kernel: Bump qfq weight to 1 (was 0)
Dec 25 16:29:46 mixupload kernel: Bump qfq maxlen to 1500 (was 0)

And everyday it fails again. It happens between 12.00~14.00. You can see it on traffic monitoring tool of my ISP:

When lime graph falls below orange line - ip is not responding to users.
What could be wrong? This strange fails freaks me out.
Hope for your support

serik · Jan 22, 2014

Hi, I have a similar problem. QFQ works well some time and then stops all traffic until I flush and restart ipfw rules. Bandwith ~ 100 Mbit/s. FreeBSD 9.0-RELEASE-p5
Config QFQ:

Code:

ipfw pipe 1 config bw 100Mbit/s
ipfw pipe 2 config bw 100Mbit/s
ipfw sched 1 config type QFQ
ipfw sched 2 config type QFQ
ipfw queue 1 config sched 1 mask dst-ip 0xffffffff
ipfw queue 2 config sched 2 mask src-ip 0xffffffff
ipfw add queue 1 ip from any to 10.0.0.0/8 out
ipfw add queue 2 ip from 10.0.0.0/8 to any in

Please, help me resolve this problem.

Everyday problem with dummynet QFQ queue

Ramzes

serik