direct jail 2 jail networking

J

jaapa

I am trying to have a jail <-> bridge <-> jail networking setup, and I thought I had something that should (in theory) work.

Below you can find my 2 jail configs, however it does not seem to work at all.

Code: 
mount.devfs;

$p2pbridge = bridge100;

jail1 {
    # Not sureif this is neccesary
    exec.prestart = "";
    exec.start = "";
    exec.poststop = "";

    exec.start = "/bin/sh /etc/rc";
    exec.consolelog = "/var/log/jail_console_${name}.log"; 
    
    $id = "4";
    $epair = "epair${id}";

    # dependencies
    depend nginx;
    
    # Networking
    # This is just normal networking to get to the internet, ${bridge} is connected to the physical interface
    exec.prestart += "ifconfig ${epair} create";
    exec.prestart += "ifconfig ${bridge} addm ${epair}a";
    exec.prestart += "dhclient ${epair}a";
    exec.start += "ifconfig    ${epair}b ether 02:cb:94:c4:54:9b";
    exec.start += "dhclient ${epair}b";
    exec.poststop += "ifconfig ${epair}a destroy";

    #input network
    exec.prestart += "ifconfig ${p2pbridge} create";
    exec.prestart += "ifconfig epair100 create";
    exec.prestart += "ifconfig ${p2pbridge} addm epair100b";
    exec.start += "ifconfig    epair100a ether 02:cb:94:c4:54:9b";
    exec.poststop += "ifconfig epair100a destroy";
    exec.poststop += "ifconfig ${p2pbridge} destroy";

    host.hostname = jail1.local;
    vnet;
    vnet.interface = "${epair}b", "epair100a"; # Connect both interfaces, one going to the world and one for p2p between jails

    path="/jail/jail1";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.clean;
    
    # rest
    allow.raw_sockets;
    allow.sysvipc;
    mount.procfs;
    devfs_ruleset=11;
}
Code: 
mount.devfs;

.include jail1.conf;

test {
    # Not sureif this is neccesary
    exec.prestart = "";
    exec.start = "";
    exec.poststop = "";

    exec.start = "/bin/sh /etc/rc";
    exec.consolelog = "/var/log/jail_console_${name}.log";
    
    $id = "5";
    $epair = "epair${id}";

    # dependencies
    depend jail1;
    
    # Networking
    exec.prestart += "ifconfig ${epair} create";
    exec.prestart += "ifconfig ${p2pbridge} addm ${epair}a";
    exec.start += "ifconfig    ${epair}b ether 02:cb:94:c4:54:0b";
    exec.poststop += "ifconfig ${epair}a destroy";
    host.hostname = jail2.local;
    
    vnet;
    vnet.interface = "${epair}b"; # Other end of the connection, this should be connected to the p2p bridge

    path="/jail/jail2";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.clean;
    
    # rest
    allow.raw_sockets;
    allow.sysvipc;
    mount.procfs;
    devfs_ruleset=11;
}
While playing around, I have discovered that if I connect the jails directly with an epair, it does fully work. However, I would like to have multiple jails communicate to jail1, not just one. (and without having to create an epair for every connected jail). Therefore I thought a bridge would be perfect, it should be exactly how my existing jail communicate with the rest of my network, right?

However, I seem to be doing something wrong, and I don't know what. Could anyone help me figure this out?
 
I did direct jail-to-jail connections by simply putting a ng_eiface(4)() into each of them and connecting them. But I am using netgraph, so I am not into your epair stuff. Anyway, it should be similarly easy.
For network concerns, your vnet jails behave almost the same as freestanding machines, and you can treat them as such. There shouldn't be a difference whether you connect jail-to-host or jail-to-jail.
 
What's the output of ifconfig bridge100 when both jails are running?
 
This is executed on the host:
Code: 
# ifconfig bridge100
bridge100: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=0
    ether 58:9c:fc:10:ff:a2
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0
    member: epair5a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
           ifmaxaddr 0 port 22 priority 128 path cost 2000
    member: epair100b flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
           ifmaxaddr 0 port 17 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>

As you can see, both ends are connected to the bridge, which is what I would expect to produce a functioning connection
 
Yes, it was just to verify the jails are indeed actually connected to the bridge.

jaapa said:
Code: 
exec.start += "ifconfig    ${epair}b ether 02:cb:94:c4:54:9b";
<...>
exec.start += "ifconfig epair100a ether 02:cb:94:c4:54:9b";
Click to expand...
epair4b and epair100a have the same MAC address.
 
Thank you for the tip! Sadly this didn't solve it.

I did just have the idea that maybe every node in the chain should have an ip address, since in my normal setup they do have a dhcp client setup for them, and in the case of a direct epair it also only worked once I gave both ends an ip!

I wish I realized this yesterday already, but I would still like to thank you for the support!
 
jaapa said:
Code: 
# ifconfig bridge100
bridge100: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
Click to expand...

What about issuing

ifconfig bridge100 up

I think bridge interface and all involved epair interfaces should be UP and RUNNING, I did something similar and it did not work until I issued 'ifconfig up' for all relevant interfaces.
 
You must log in or register to reply here.
Back
Top