Struggling with jail networking

[dison4linux]

I'm not able to wrap my head around what's going on with the networking in a jail.

I created a new jail 'jail1' and configured it with an IP in the same subnet as the bare metal OS.

root@jail1:~ # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 00:50:56:9a:6d:19
    inet 10.0.75.201 netmask 0xffffffff broadcast 10.0.75.201
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>

The trouble I'm having is that the same IP also shows up in the bare metal OS, and if I SSH to it I end up in the bare metal OS.

[root@freebsd-root1 /home/someone]# ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 00:50:56:9a:6d:19
    inet 10.0.75.102 netmask 0xffffff00 broadcast 10.0.75.255
    inet 10.0.75.201 netmask 0xffffffff broadcast 10.0.75.201
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

How can I configure the jail so that it has its own IP that is unrelated to the bare metal OS?

 

[lme@]

You need to change the sshd config on the host machine to listen only on host IP address(es). Look for "ListenAddress" in sshd_config().

 

[asteriskRoss]

You need to change the sshd config on the host machine to listen only on host IP address(es). Look for "ListenAddress" in sshd_config().

Futher to lme@'s post, if you are running sshd(1) on the host and in the jail, you need to modify /etc/ssh/sshd_config for each so that ListenAddress is the host's IP in the configuration file on the host and the jail's IP for configuration file in the jail.

The trouble I'm having is that the same IP also shows up in the bare metal OS, and if I SSH to it I end up in the bare metal OS.

That is totally normal. Jail IP addresses are added as aliases on the host.

 

[kpa]

That is totally normal. Jail IP addresses are added as aliases on the host.

This is true but the alias method is not the only way to dedicate an IP address for a jail, the address could be from a separate interface such as a tap(4) interface. Jails have nothing built in that forces the IP address to be an alias address.

 

[dison4linux]

Can you explain the tap option a bit? I don't know about the first few replies. If I have to change the listen address for SSHD then I'd have to change the listen address for any other services that happen to be running both on the host and the jails.
I've setup jails with the FreeNAS GUI before and those jails act more like what you'd expect with separate virtual machines running on a virtualization platform like ESXi.

 

[kpa]

What I mean is that the tap(4) interface is just another network interface as any other interface, physical or virtual. As long as it's configured properly and has an IP address bound to it a jail can use the IP address, the network stack on the host handles the rest including routing and (if configured) NAT.

 

[kpa]

That is understood, but it is interesting that tap(4) is used with VMs while the alias method is mainly used by jails. So what are the pros ans cons and why do VMs not use the alias method?

At least with VirtualBox the NAT networking and bridge networks use a "hack mode" where the packets are injected directly on the interface chosen and the virtual machine does not get its own network interface that you could see on the host. The host-only networking on the other hand does create a separate interface., vboxnet0 usually.

I would guess the virtualization softwares such as VirtualBox have chosen that path because it makes dead simple to get started with virtualization, no need to set up NAT on the host or anything equally complicated.

 

[kpa]

The only specialty of tap(4) is that it simulates ethernet hardware and can be used to create a tunnel (VPN or otherwise) that has broadcast semantics just like a real ethernet connection has. That's the reason it can be also briged easily with a real interface. It doesn't have to be connected to anything though and that's how you can use it in place of a cloned lo(4) for jails.