CARP + Jumbo Frames (can't set mtu)

P

Pfarthing6

I'm getting really crappy performance over my carp interface and I think the culprit is mismatched mtu between carp0 and lagg0.

If I do a transfer test using lagg0 directly, I get upwards of 70MB/s. If I do the same with carp0, I get about 1MB/s transfer rate.

Here's my config info:
Code: 
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            10.1.1.254         UGS         0   203099    em0
10.1.0.0/16        link#1             U           0  2155921    em0
10.1.101.1         link#1             UHS         0        0    lo0
10.10.100.0/24     link#9             U           0  8303485  lagg0
10.10.100.100      link#10            UH          0        0  carp0
10.10.100.101      link#9             UHS         0        0    lo0
127.0.0.1          link#8             UH          0       72    lo0

### carp0 info ###
[root@nas1 /zroot]# route get 10.10.100.100
   route to: 10.10.100.100
destination: 10.10.100.100
  interface: carp0
      flags: <UP,HOST,DONE,LOCAL>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1500         1         0

[root@nas1 /zroot]# ifconfig carp0
carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
        inet 10.10.100.100 netmask 0xffffff00
        carp: MASTER vhid 1 advbase 1 advskew 0


### lagg0 info ###
[root@nas1 /zroot]# route get 10.10.100.101
   route to: 10.10.100.101
destination: 10.10.100.101
  interface: lo0
      flags: <UP,HOST,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0     16384         1         0

[root@nas1 /zroot]# ifconfig lagg0
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:03:ba:95:c2:0a
        inet 10.10.100.101 netmask 0xffffff00 broadcast 10.10.100.255
        media: Ethernet autoselect
        status: active
        laggproto lacp
        laggport: cas1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: cas0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

[root@nas1 /zroot]# ifconfig cas0
cas0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:03:ba:95:c2:0a
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

[root@nas1 /zroot]# ifconfig cas1
cas1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:03:ba:95:c2:0a
        media: Ethernet autoselect (1000baseT <full-duplex,master>)
        status: active

What's interesting is that the mtu reported by route for lagg0 is different then that reported by ifconfig, 1500 vs 16384. Which is right I wonder? I'm assuming the 16384 value is correct, which is why the performance is so sucky when using carp0, when there shouldn't be any difference at all (is that right?).

I didn't set the mtu on any manually, but the switch has Jumbo Frames enabled, so I'm assuming that this was detected on the interfaces at boot or create time.

So, I would like to use Jumbo Frames, but need to get the the mtu settings to line up. Any pointers on how to do this?

Tried [CMD=]ifconfig <iface> mtu <val>[/CMD] on both carp0 and lagg0 and just get errors.

Thanks!
 
Well, the performance problem seems to have abated after a couple reboots. I now get the same performance (~40MB/s to-from, not bad for a single disk) whether I use carp0 over lagg0 or just lagg0. So I don't know what that deal was, but it is working as it should it seems.

I still can't set MTU on either though and I still have Jumbo Frames enabled on my switch, but it's probably not being taken advantage of.

Is there a way to set it on these virtual interfaces?
 
If you check the output of $ netstat -rn you will see that your 10.10.10.101 route is a localhost route, which uses an mtu of 16384. Your other routes will be non-local, and use the mtu of the interface.

To set the MTU for virtual interfaces, you must first set it on the physical interfaces. Virtual interfaces can have smaller MTUs than the physical interface, but not larger.
 
You must log in or register to reply here.
Back
Top