Routing Across Subnets via 1 FreeBSD Installation?

msullivan

msullivan

Hi all!

I'm having trouble wrapping my head around what to do to make this happen, so I'm hoping that any of you with more experience might have an idea.

Backstory: I'm working on a simulation related to some undergraduate research work. For this simulation I've got three VMs inside of VirtualBox; all three of these are connected to the same virtual network switch, which we'll call "BLUE_SWITCH".

I have the three VMs on completely different subnets:
172.16.1.1/24 (VM_1)
192.168.1.1/24 (VM_2)
10.10.1.1/24 (VM_3)

In my perfect world, I would like to create a FreeBSD install (VM_BSD) as the default gateway for all three of those subnets. Then, if VM_1 wanted to contact VM_2, the VM_BSD would forward those packets appropriately.

A) Can it be done at all? I'm assuming yes, but it's probably wizardry.
B) If it can be done, can someone provide a bit of config or an example?

The preceding are the parameters which I'm able to work with; while it would be much easier to just throw them on the same subnet and call it a day, this simulation simply can't operate like that (this small issue is part of a much larger project).

Any help would be very appreciated! I'm more of a software guy, so I'm admittedly out of my league here and anything is appreciated :)
 
a) yes
b) nothing more would be needed then a
Code: 
gateway_enable="YES"
in /etc/rc.conf.
 
Sorry to be a n00bsauce here, but in addition to enabling gateway mode how would I tell FreeBSD what networks it was working with? Aliases in each range?

Code: 
ifconfig_em0_alias0="172.16.1.1/24"
ifconfig_em0_alias1="192.168.1.1/24"
ifconfig_em0_alias2="10.10.1.1/24"

And finally, would I be needing to set static routes anywhere at all?
 
msullivan said:
Aliases in each range?
Click to expand...
Or multiple interfaces, each in its own subnet.

And finally, would I be needing to set static routes anywhere at all?
Click to expand...
Nope. All "leaves" have a default gateway pointing to the FreeBSD box. The FreeBSD box itself doesn't need any static routes as all the networks are "directly connected".
 
Well, I had some time to give this a try today. We're close, but not quite working.

On the FreeBSD box I am doing:

Code: 
ifconfig_em0_alias0="172.16.1.254 netmask 255.255.255.0"
ifconfig_em0_alias1="192.168.1.254 netmask 255.255.255.0"
ifconfig_em0_alias2="10.10.1.254 netmask 255.255.255.0"
VM_1 is set to be 172.16.1.1
VM_2 is set to be 192.168.1.1
VM_3 is set to be 10.10.1.1

Let's say I'm on VM_1. I can ping 172.16.1.254, 192.168.1.254 and 10.10.1.254 just fine presently (all gateway addresses). However, if I try to ping VM_2 at 192.168.1.1 I get no response. Traceroutes also show my packet making the first step to the router with no issues, but after that it doesn't get forwarded anywhere and dies.

Any thoughts? Thanks!
 
Oops, sorry for the double-post. It's also relevant to note that I've played with the subnet masks, changing them between "255.255.255.255", "255.255.255.0", or just leaving them off entirely. No dice when doing that either.
 
Sounds like it's not forwarding anything.
Did you add
Code: 
gateway_enable="YES"
to /etc/rc.conf?

Don't forget to reboot or enable it by hand:
Code: 
sysctl net.inet.ip.forwarding=1

Also turn off any firewalls if they're enabled. Get this working first.
 
Code: 
gateway_enable="YES"
is the first line in rc.conf; confirmed the setting is working by issuing:
Code: 
# sysctl net.inet.ip.forwarding
sysctl net.inet.ip.forwarding: 1 ( <-- returned 'yes' )

No other firewalls are running anywhere in the infrastructure; the FreeBSD server is a clean install with only sshd running, nothing else.
 
Ok, can you post the output of
$ netstat -rn

That should show the routing table.
 
Sure thing.
Code: 
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
34.6.92.0/24       link#1             U           0        3    em0
34.6.92.254        link#1             UHS         0        0    lo0
35.6.92.0/24       link#1             U           0        0    em0
35.6.92.254        link#1             UHS         0        0    lo0
127.0.0.1          link#3             UH          0      117    lo0
172.16.4.0/29      link#2             U           0        0    em1
172.16.4.2         link#2             UHS         0        0    lo0
199.100.16.0/24    link#1             U           0        0    em0
199.100.16.100     link#1             UHS         0        0    lo0
199.100.16.254     link#1             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UH          lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff01:3::/32                       fe80::1%lo0                   U           lo0
ff02::%lo0/32                     fe80::1%lo0                   U           lo0
 
Oh, I should note that my example from the original posts is simplified; the real address ranges I'm using are:
34.6.92.0/24
35.6.92.0/24
199.100.16.0/24

This network does not have any internet access or internet-viewing gateways, so the IP address scheme does not conflict with anything in the real world.
 
msullivan said:
Code: 
34.6.92.0/24       link#1             U           0        3    em0
34.6.92.254        link#1             UHS         0        0    lo0
35.6.92.0/24       link#1             U           0        0    em0
35.6.92.254        link#1             UHS         0        0    lo0
Click to expand...
This doesn't look right. The address should be bound to em0, not lo0. Similarly for the other addresses.

Or do you have your VMs bound to lo0?
 
Well for what it's worth, I quickly installed FreeBSD onto a real, physical box, then attempted to add an alias in the same way. After a netstat -nr, the IP alias I gave it reported lo0 with its mask on em0, just like in my VMs - so I'm thinking that's normal and default behavior.

My rc.conf parameters are as posted a few pages up (syntax of "ifconfig_em0_aliasX=BLAH"). Does that syntax look correct to you? From all the postings I've found it appears to be correct.

This is a doosie, but I really need to make it work, so I'm hoping for the best :D
 
wblock said:
This might be made clearer by posting the whole /etc/rc.conf.
Click to expand...

Fair enough. Nothing too special.

Code: 
gateway_enable="YES"
hostname="proxy-int.svc1.mytestnetwork.net"
ifconfig_em0="inet 199.100.16.100 netmask 255.255.255.0"
ifconfig_em1="inet 172.16.4.2/29"
sshd_enable="YES"
squid_enable="YES"
gateway_enable="YES"
ifconfig_em0_alias0="199.100.16.254 netmask 255.255.255.0"
ifconfig_em0_alias1="35.6.92.254 netmask 255.255.255.0"
ifconfig_em0_alias2="34.6.92.254 netmask 255.255.255.0"

Just for kicks, here's an ifconfig as well:
Code: 
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 08:00:27:d2:f4:bf
        inet 199.100.16.100 netmask 0xffffff00 broadcast 199.100.16.255
        inet 199.100.16.254 netmask 0xffffff00 broadcast 199.100.16.255
        inet 35.6.92.254 netmask 0xffffff00 broadcast 35.6.92.255
        inet 34.6.92.254 netmask 0xffffff00 broadcast 34.6.92.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 08:00:27:f3:96:b7
        inet 172.16.4.2 netmask 0xfffffff8 broadcast 172.16.4.7
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
 
msullivan said:
Code: 
gateway_enable="YES"
hostname="proxy-int.svc1.mytestnetwork.net"
[B]ifconfig_em0="inet 199.100.16.100 netmask 255.255.255.0"[/B]
ifconfig_em1="inet 172.16.4.2/29"
sshd_enable="YES"
squid_enable="YES"
gateway_enable="YES"
[B]ifconfig_em0_alias0="199.100.16.254 netmask 255.255.255.0"[/B]
ifconfig_em0_alias1="35.6.92.254 netmask 255.255.255.0"
ifconfig_em0_alias2="34.6.92.254 netmask 255.255.255.0"
Click to expand...

The main IP address and that alias are in the same net. Just make the main address 199.100.16.254 and don't add an alias, or use a different network for the main address.

It looks wrong without a defaultrouter entry, but I guess you don't need one if it's all internal traffic.
 
wblock said:
The main IP address and that alias are in the same net. Just make the main address 199.100.16.254 and don't add an alias, or use a different network for the main address.

It looks wrong without a defaultrouter entry, but I guess you don't need one if it's all internal traffic.
Click to expand...

That alias was leftover from some other testing, and although it is a good catch, removing it doesn't seem to affect anything one way or the other. Same result... I can always ping from a VM to a gateway, but from VM to VM still fails after the packet makes it to the FreeBSD installation.
 
Are the VMs using NAT or bridged networking? NAT will break pings. If the gateway is also a VM, or the VM host machine, there's "internal network" and "host-only networking" modes also.
 
I'm just wondering that does IP forwarding work using only one interface? For example on Linux forwarding works only between separate interfaces. Maybe you need to create three separate networks inside VirtualBox and use three interfaces on the FreeBSD system.
 
wblock said:
Are the VMs using NAT or bridged networking? NAT will break pings. If the gateway is also a VM, or the VM host machine, there's "internal network" and "host-only networking" modes also.
Click to expand...

Yes, the FreeBSD installation is also a VM. All four are connected using the "host-only network" option presently, but "internal network" has also been tried with the same results.
 
toddnni said:
I'm just wondering that does IP forwarding work using only one interface? For example on Linux forwarding works only between separate interfaces. Maybe you need to create three separate networks inside VirtualBox and use three interfaces on the FreeBSD system.
Click to expand...

I had considered that, but eventually this *might* need to be scaled up to even more subnets (it depends on how the first set of tests goes). If that happens, we'll be adding about 8 more VMs with separate subnets, and VirtualBox only supports 8 networking devices.

There's got to be some way... :(
 
msullivan said:
I had considered that, but eventually this *might* need to be scaled up to even more subnets (it depends on how the first set of tests goes). If that happens, we'll be adding about 8 more VMs with separate subnets, and VirtualBox only supports 8 networking devices.

There's got to be some way... :(
Click to expand...

I don't know if performance is important in this simulation, but you could try daisy-chain FreeBSD routers. Unfortunately you need to set routes between routers then.
 
Just on a side note,
Code: 
gateway_enable="YES"
hostname="proxy-int.svc1.mytestnetwork.net"
[B]ifconfig_em0="inet 199.100.16.100 netmask 255.255.255.0"[/B]
ifconfig_em1="inet 172.16.4.2/29"
sshd_enable="YES"
squid_enable="YES"
gateway_enable="YES"
[B]ifconfig_em0_alias0="199.100.16.254 netmask 255.255.255.[color="DarkRed"]255[/color]"[/B]
ifconfig_em0_alias1="35.6.92.254 netmask 255.255.255.0"
ifconfig_em0_alias2="34.6.92.254 netmask 255.255.255.0"
Should work as well. Whenever you have an alias ip that is in the same subnet you want to set the netmask to 255.255.255.255
 
You must log in or register to reply here.
Back
Top