jails Interface does not return to host after kill jail

P

pehcorreias

Hello darlings!

Basically, i did created a JAIL, with this configuration:

Code: 
main {
    host.hostname = "main";
    exec.clean;
    exec.start =  "/bin/sh /etc/rc";
    exec.stop  = "/bin/sh /etc/rc.shutdown";
    exec.prestart += "kldload -n ipfw_nat";
    exec.prestart += "kldload -n ipsec";
    allow.raw_sockets =1;
    allow.chflags = 1;
    mount.devfs;
    devfs_ruleset = 15;
    vnet new;
    vnet.interface += "eth2.1600";
    exec.poststop += "ifconfig eth2.1600 -vnet 1";
    path ="/usr/jails/main";
}

eth2.1600 is a VLAN, with 1600 being the tag. When i start the jail, eth2.1600 disappears from the host, its ok.
But, when i stop the jail, eth2.1600 doesn't come back to the host

When i run the command


ifconfig eth2.1602 create


Receive:
ifconfig: interface eth2.1602 already exists

And also


ifconfig eth2.1602 destroy


Receive:
ifconfig: interface eth2.1602 does not exist

I thought it was the jail in infinite death, but NO, when i run
jls -d
it doesn't show this jail.

This problem ocurred in other environments, but the interface ALWAYS returned to the host after a few minutes or hours...

But this interface doesn't come back for some days
 
Perhaps, you need to pull the interface from the jail before the jail stops.

Code: 
exec.prestop += "ifconfig eth2.1600 -vnet main";
 
I have this exact same problem in 14.0-RELEASE-p5.

I have a second physical interface named ix1 that is sent to the jail with vnet.interface = ix1;

When I manually stop the jail, it ends up in a dying state. The interface never gets released back to the host. I have to resort to rebooting the host.
Here is my jail configuration:

Code: 
myjail {
  host.hostname = "${name}";
  path = "/usr/local/jails/containers/${name}";

  vnet;
  vnet.interface = ix1;

  exec.clean;
  exec.system_user = "root";
  exec.jail_user = "root";

  exec.start += "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.consolelog = "/var/log/jail_console_${name}.log";
  mount.devfs;

  exec.created += "/sbin/zfs jail ${name} pool/bhyve";
  exec.release += "/sbin/zfs unjail ${name} pool/bhyve";

  devfs_ruleset = 25;
  allow.vmm;
  allow.mount;
  allow.mount.zfs;
  allow.raw_sockets;
  enforce_statfs = 1;
}
 
Okay.
1. If the interfaces are not all properly disposed of or returned back to where they belong, the jail will stay in dying state forever. That is normal.
2. The interface given as vnet.interface should be handled automatically. man jail:
The interface will automatically be released when the jail is removed.
Click to expand...
If additional interfaces are moved into the jail, they must be moved back explicitely.
3. I found a bug/regression in 13.3, where under certain circumstances, interfaces are swallowed by the jail and not given back. PR 276862 . Jail will then not die either. But, if I read the commitlogs correctly, this should concern 13.3, not 14.0.

Mybe there is more to this. You are invited to do further investigations, relevant info is in the PR.
 
Indeed it should be, but it isn't. You mention additional interfaces within the jail that may cause issues, which is a nice clue here. In my jail, I am using vm-bhyve, which does additional stuff with interfaces. Here is what ifconfig shows within the jail:
Code: 
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4a538b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,HWSTATS,MEXTPG>
        ether [redacted]
        inet [redacted] netmask 0xffffff00 broadcast 172.16.28.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vm-public: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=0
        ether [redacted]
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 8 priority 128 path cost 2000000
        member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 20000
        groups: bridge vm-switch viid-4c918@
        nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: vmnet/myvm/0/public
        options=80000<LINKSTATE>
        ether [redacted]
        groups: tap vm-port
        media: Ethernet 1000baseT <full-duplex>
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        Opened by PID 3479

vm-public and tap0 are created by vm-bhyve. Is there anything I should or can do within the jail configuration to get the ix1 to be released?
 
Oha, a bhyve inside the jail. Thats cute, I never did that. That probably does lots of things.

What You certainly can do is make sure that all of these are properly removed again by the respective service stop routines (or whatever you use to terminate the applications inside the jail)
 
You must log in or register to reply here.
Back
Top