You cannot bridge your jails with your external interface on your VPS. You do need a bridge (to attach your jails to) but it'll be disconnected from your external interface, so this "network" will be entirely local. Then you set up NAT and such so your jails can connect to the outside world.
Not sure I'm following here, there's probably something I'm missing, but what do you mean you cannot bridge a jail to an external interface?
sh:
-> cat /etc/rc.conf
(...)
vlans_igb0="10 40 50 60 70 80"
cloned_interfaces="bridge10 bridge40 bridge50 bridge60 bridge70 bridge80"
ifconfig_bridge10="SYNCDHCP addm igb0.10 stp igb0.10 up"
ifconfig_bridge10_ipv6="inet6 auto_linklocal accept_rtadv"
ifconfig_bridge40="addm igb0.40 stp igb0.40 up"
ifconfig_bridge50="addm igb0.50 stp igb0.50 up"
(etc. for other bridges)
ifconfig_igb0_10="up"
ifconfig_igb0_40="up"
(etc. for other VLAN interfaces)
ifconfig_igb0="up"
With that I get my VLAN interfaces added to their corresponding bridges, with
bridge10 being the one the jail host itself networks on. One thing that is
very important here,
balanga, is that
any interface that goes on a bridge must itself
not have an IP configuration, as otherwise the setup will not work. Notice that I want my jail host to network on my VLAN 10, so I assign an IP configuration to
bridge10,
not to the
igb0.10 corresponding VLAN interface; additionally, I have no need for my jail host to network on my VLAN 40, so I don't assign any IP configuration to
bridge40, and that itself
does not prevent jails attached to that bridge to network themselves on that VLAN.
As a side note, that configuration works whether you're using VLANs or not, or a physical interface such as
igb or a virtual one such as
vtnet, etc.; in all of those cases, the resulting configuration would always look very similar to the above, e.g.:
Code:
cloned_interfaces="yourbridge"
ifconfig_yourbridge="SYNCDHCP addm vtnet0 stp vtnet0 up"
ifconfig_vtnet0="up"
And, with that, you'd then create your jails via your jail manager (and I seriously
do not recommend iocage at this point in time, something like Bastille would be a much better choice), and instruct it to a attach them to your bridge(s) of choice. For example, if I want a jail to be on my VLAN 10, then I attach it to my
bridge10 interface. How that is accomplished is heavily dependent on the inner workings of the chosen jail manager; but, in any case, what they essentially do upon booting up the jail is create an
epair(4) interface, with its regular
a &
b terminals, and attach terminal
a to the chosen bridge (basically an
ifconfig yourbridge addm instruction behind the scenes) and assign terminal
b to the jail.
How you configure the IP information for that jail with its own epair terminal is also dependent on the jail manager UI (configuration syntax, command line flags, etc.), but, behind the scenes it all essentially boils down to a collection of
jail(8) configuration options that get recorded in a
jail.conf file somewhere on your filesystem (e.g. for Bastille that's usually at
/usr/local/bastille/jails/$jailName/jail.conf and for iocage it should be at
/var/run/iocage-$jailName.conf), and they look more or less like this:
Code:
$jailName {
# networkng unrelated settings:
enforce_statfs = 2;
devfs_ruleset = 13;
exec.clean;
exec.consolelog = /usr/local/bastille/logs/$jailName_console.log;
exec.start = '/bin/sh /etc/rc';
exec.stop = '/bin/sh /etc/rc.shutdown';
host.hostname = $jailName;
mount.devfs;
mount.fstab = /usr/local/bastille/jails/$jailName/fstab;
path = /usr/local/bastille/jails/$jailName/root;
securelevel = 2;
osrelease = 14.4-RELEASE-p7;
# networking settings:
vnet;
vnet.interface = $epairBName;
exec.prestart += "epair0=\$(ifconfig epair create) && ifconfig \${epair0} up name $epairAName && ifconfig \${epair0%a}b up name $epairBName";
exec.prestart += "ifconfig $yourBridge addm $epairAName";
exec.prestart += "ifconfig $epairAName ether $macA";
exec.prestart += "ifconfig $epairBName ether $macB";
exec.prestart += "ifconfig $epairAName description \"vnet0 host interface for Bastille jail $jailName\"";
exec.poststop += "ifconfig $epairAName destroy";
}
Some jail managers may use some of those options, some others may not; for example, not all jail managers will necessarily assign descriptions or pre-computed MAC addresses to the epair terminals, but that's pretty much frosting on the cake. In any case, all of the relevant options can be read in
jail(8), with the
jail.conf(5) man page also being a very handy read when learning how FreeBSD jails work and how to handcraft a
jail.conf file should you want to do some manual experimentation.
A few things to note here:
- My jail hosts and all of the jails they host get their IPs from a local DHCP server, with a DHCP jail requiring the
vnet option shown above.
- You can still deploy
vnet jails without DHCP and utilize manual IP configurations for each of them, and you'd do that with ifconfig instructions inside the jail's /etc/rc.conf file very similar to those that you'd use on the host (the rc.conf(5) man page is also a very handy read). The jail managers usually do this for you, but this is essentially what the result would look like:
sh:
# For a DHCP jail
ifconfig_$epairBName_name="vnet0"
ifconfig_vnet0="SYNCDHCP"
ifconfig_vnet0_ipv6="inet6 -ifdisabled accept_rtadv"
ifconfig_vnet0_descr="jail interface for $yourBridge"
# For a static IP jail
ifconfig_$epairBName_name="vnet0"
ifconfig_vnet0="$yourIP/$yourNetmask up"
ifconfig_vnet0_ipv6="inet6 -ifdisabled accept_rtadv"
ifconfig_vnet0_descr="jail interface for $yourBridge"
- In summary,
vnet is very useful for a multitude of cases because it gives a jail its own, full-blown private networking stack, rather than sharing networking with the jail host.
- If you use DHCP, then traffic has to be able to flow back and forth between the jail and the DHCP server via the assigned networking path, e.g. epair (jail/host) -> bridge (host) -> exit interface (host) -> any intermediary steps -> DHCP server -> response sent back through the same networking path.
- If you don't use DHCP, and whether you use
vnet or not, then obviously you need a fair amount of networking knowledge to produce a working configuration, both for the jails and their host.
- Whether a jail reaches the outside world or not, and how, depends to a very large degree on your IP configuration, filtering, etc.
- My setup doesn't use NAT at all, as I rely on my local router for DHCP, VLANs, DNS, firewalling, etc. Most tutorials you find out there assume you're going to be doing NAT and IP filtering on the jail host, and I want to stress and emphasize that that's only one specific scenario, not a requirement for jails to have a working network configuration.
But, in any case, the grand conclusion is that networking jails to the outside world via bridging is, indeed, a very well supported scenario that I (and certainly tons and tons of other people) have been using successfully for years now. "outside" in this context, of course, can mean many different things, with the simplest one of all being the bridge you attach a jail to, which you can verify with the help of a packet sniffer such as
tcpdump(1); whether or not "outside" extends farther than that is usually heavily dependent on the details of your configuration, e.g. the details of the designated forwarder interface of your bridge (i.e. my
igb0.$tag VLAN interfaces in my own setup, or the
vtnet0 interface in the example above, etc.), whether that interface itself has access to the world outside of your jail host, whether the latter is a private network with egress firewalling, etc., etc.
How any of this applies to your "VPS" scenario, that I don't know, but hopefully I've explained some key concepts here that might help you in your journey, if you're still pursuing it, of course.
github.com
