Data going to em1 is putting bandwidth on em0

Dex

Dex

I have a server with em0 (192.168.0.100) and em1 (192.168.0.101) and two Internet IPs, x.x.x.184 and x.x.x.185 respectively. Whenever I connect and start a transfer on the .185 IP, bandwidth is shown for em0. It's the same load on em0 for the .184 IP. How do I use the two Internet connections without having everything get routed into the first connection?
 
I thought I should add that I followed this guide with the same results: http://www.daemonforums.org/showthread.php?t=4610

Some output:
Code: 
# cat /etc/rc.local
setfib 0 route delete default
setfib 0 route add default 192.168.0.1
setfib 1 route delete default
setfib 1 route add default 192.168.0.1

ipfw -f flush
ipfw add allow ip from any to any via lo0
ipfw add setfib 1 ip from any to any via em0
ipfw add setfib 0 ip from any to any via em1
ipfw add allow ip from any to any

# netstat -ar
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS        35   187569    em0
localhost          link#5             UH          0   164286    lo0
192.168.0.0        link#1             U           0        0    em0
192.168.0.100      link#1             UHS         0        0    lo0
192.168.0.101      link#2             UHS         0        0    lo0

# cat /etc/rc.conf
ifconfig_em0="inet 192.168.0.100 netmask 255.255.255.0"
ifconfig_em1="inet 192.168.0.101 netmask 255.255.255.0"

# ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:50:56:01:14:d9
        inet 192.168.0.100 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:50:56:01:11:93
        inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
ipfw0: flags=8800<SIMPLEX,MULTICAST> metric 0 mtu 65536
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
 
Try using non-overlapping netmasks (or separate networks) on em0 and em1. With overlapping netmasks, traffic tends to stick to one route.

Try e.g. 192.168.0.0/24 and 192.168.1.0/24 networks, or 192.168.0.100/30 and 192.168.0.104/30 IP/netmasks (or even /32 netmasks if there are no attached networks behind them), to compartmentalize the NICs within their own networks.
 
Well, the server is connected through a virtual (VM) network/router so I need to connect to 192.168.0.1 and the two 192.168.0.100 192.168.0.101 IPs are assigned to the Internet. That leaves me with using a different submask, right? What should I use instead of 255.255.255.0?

I misunderstood, trying it now.

It seems only /24 subnet works for the rc.conf entries.

And:

Code: 
# setfib 2 route add default 192.168.1.0
route: writing to routing socket: Network is unreachable
add net default: gateway 192.168.1.0: Network is unreachable

It seems that it isn't possible for this way to have outgoing connections use both interfaces, being limited to one gateway/subnet. Some clarification would be appreciated.
 
I think the problem is with IPFW. I never managed to set up two routes with ipfw, so I switched to pf, and it works. In my case I have two Internet connections with different gateways and one LAN connection. It should work with one gateway too.

I wanted some users to go by first table, rest should go by second table, and it works.

Code: 
rc.local
/usr/sbin/setfib 0 route delete default
/usr/sbin/setfib 0 route add    default 192.168.20.1
/usr/sbin/setfib 1 route delete default
/usr/sbin/setfib 1 route add    default 10.0.0.1

Code: 
pf.conf
table <white> {172.21.0.190, 172.21.0.235}
pass all
pass in from 172.21.0.0/24 to any rtable 1
pass in from <white> to any rtable 0
 
I set the pf.conf and turned off ipfw. I did /etc/rc.d/pf onestart && /etc/rc.d/netif restart && /etc/rc.d/local restart. No go. I'm sure my configuration is wrong as I don't really know what I'm doing, please help.

Code: 
# cat /etc/rc.local
setfib 0 route delete default
setfib 0 route add default 192.168.0.1
setfib 1 route delete default
setfib 1 route add default 192.168.0.1

# cat /etc/rc.conf
firewall_type="open"
pf_rules="/etc/pf.conf"
nfs_client_enable="YES"
mysql_enable="YES"
ifconfig_em0="inet 192.168.0.100/24"
ifconfig_em1="inet 192.168.0.101/24"

# cat /etc/pf.conf
table <white> {192.168.0.100, 192.168.0.101}
pass all
pass in from 192.168.0.0/24 to any rtable 1
pass in from <white> to any rtable 0

# setfib 1 netstat -r
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS         1        0    em0
localhost          link#5             UH          0        0    lo0
192.168.0.0        link#1             U           0        0    em0


# setfib 2 netstat -r
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
localhost          link#5             UH          0        0    lo0
192.168.0.0        link#1             U           0        0    em0
 
With your pf.conf, your 192.168.0.100, 192.168.0.101 are going by first route table.

First, fibs start from 0, so ...
[CMD=]netstat -r[/CMD]
is for first fib
[CMD=]setfib 1 netsat -r[/CMD]
is for second fib

In pf.conf, I think you should have:

Code: 
pass all
pass in from 192.168.0.100 to any rtable 1
pass in from 192.168.0.101 to any rtable 0
 
It didn't seem to change anything

Code: 
# netstat -r && setfib 1 netstat -r
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS        46    23682    em0
localhost          link#5             UH          0     5195    lo0
192.168.0.0        link#1             U           0        0    em0
192.168.0.100      link#1             UHS         0        0    lo0
192.168.0.101      link#2             UHS         0        0    lo0

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS         0        1    em0
localhost          link#5             UH          0        0    lo0
192.168.0.0        link#1             U           0        0    em0

Is ALTQ required?
 
If you paste it in console, what you got?
Code: 
setfib 0 route delete default
setfib 0 route add default 192.168.0.1
setfib 1 route delete default
setfib 1 route add default 192.168.0.1
It looks like in all cases it is using em0. Second fib should have em1

Are em0 and em1 LAN or they are Internet connections?


What gateways have
two Internet IPs, x.x.x.184 and x.x.x.185 respectively
Click to expand...

This gatways should be assigned to your fibs.

Sorry for my english, writting in train is a pain.
 
Code: 
# setfib 0 route delete default
delete net default
# setfib 0 route add default 192.168.0.1
add net default: gateway 192.168.0.1
# setfib 1 route delete default
delete net default
# setfib 1 route add default 192.168.0.1
add net default: gateway 192.168.0.1
# netstat -r
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS        44     4043    em0
localhost          link#5             UH          0 13279023    lo0
192.168.0.0        link#1             U           0        0    em0
192.168.0.100      link#1             UHS         0      488    lo0
192.168.0.101      link#2             UHS         0        0    lo0

# setfib 1 netstat -r
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS         0        0    em0
localhost          link#5             UH          0        0    lo0
192.168.0.0        link#1             U           0        0    em0
192.168.0.100      192.168.0.1        UGHS        0        2    em0
192.168.0.101      192.168.0.1        UGHS        0        0    em0

Both IPs (184, 185) are on the same gateway. em0 and em1 are virtual LAN
 
Thanks for the help, however after reading it and attempting to understand it, I can't figure out what I'm supposed to do for my situation.
 
You must log in or register to reply here.
Back
Top