Thanks
Phishfry and
Zirias.
Can I ask why you don't use separate Ethernet adapters for your VNET Jails? Passthru.
Instead of bridge and epairs just slap a 2/4 port ethernet card in and tie it into your network.
Yeah; using a dedicated network port for VNET jails is also recommended in
FreeBSD Mastery: Jails by Michael W. Lucas, so this did cross my mind 'cause of that. But I didn't know about the ease of configuration aspect.
I'm using a ThinkCentre M920q Tiny with its one onboard I219-LM port right now (also ruling out SR-IOV, I guess).
I remember seeing that there's an option to add an I350-T4 using a Lenovo-specific bracket. That is something I could try to pick up, although I don't really need more total bandwidth than one gigabit connections' worth.
You have switches correct?
Right now I only have an unmanaged switch at my desk, so LACP wouldn't be an option 'til I bought a new switch. The switch is also uplinked to my main switch using a single gigabit link, so that'd be a bottleneck even if I added LACP at my desk.
Same with bhyve. Passthru interfaces are my method.
It makes for a wiring mess but you get full bandwidth compared to a bridge.
'makes sense. The bandwidth part's not so much of a concern here since most of the traffic for this computer's bottlenecked by my Internet service.
I do like building from scratch jail.conf. It really isn't hard. I don't remember using the jib script though.
I feel the same way. I never felt motivated to learn about the jail tools in ports, because the tools in base (like
jail(8)) seemed high-level enough to me; they seem more high-level than the stuff in base for bhyve (for bhyve, I definitely use
sysutils/vm-bhyve).
I wouldn't have ever known about the
jib script if not for
FreeBSD Mastery: Jails by Michael W. Lucas.
I do like
jib since it let me get a bridged VNET jail going much more quickly than I could have without it. But it seems to have a drawback; it doesn't work with jails that have a hyphen in their name. And I'd really like to use a hyphen in this particular jail's name.
I mean I look at this and laugh. Mine is like 4 lines.
The amount of contortions required for bridging is astounding.
Your config' does look great. And without
jib I totally agree about the bridging configuration. I looked at it a few times over the years and thought, "uhh... no thanks." Plus I'd already invested a lot in to understanding the old-school, pre-VNET jail networking scheme and it worked for all of my uses, until recently.
I prefer to bundle all available physical interfaces with
lagg(4) (using LACP, of course the switch must support that). For things that must stay separated,
vlan(4)s on top. For virtual machines and VNET jails, use
bridge(4)s on these vlans.
Result: Full bandwidth, but used with full flexibility.
Thanks; I like this configuration a lot and used it elsewhere once. There's something fascinating about this LACP + VLANs set up. I guess what surprises me the most about it is that it actually
works.
I find that this computer's traffic tends to be bursty though. Rarely does it have multiple sustained, simultaneous flows that'd distribute well with LACP. And though LACP is undeniably
fun the truth is that I don't really need more bandwidth in this case.
The real unspoken hero of jails is disk speed.
Compared to bhyves 50%+ hit it really is refreshing to see your NVMe pulling same speed as on host.
Yes; I'm sure you're right about this! I had been running X clients in the bhyve VM and displaying them on my local X server. I found that the same X clients "felt" much snappier when running inside of the VNET jail, as opposed to bhyve.
I guess that's what happens with several complex layers are yanked out. Not to mention the sudden access to all CPU threads and memory. I love jails.
Here's the configuration I'm using now:
/etc/rc.conf
Code:
# See https://www.amoradi.org/20210908201936.html
cloned_interfaces="bridge0"
ifconfig_bridge0_name="em0bridge"
create_args_bridge0="addm em0"
jail_enable="YES"
/etc/jail.conf
Code:
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
host.hostname = $name;
path = /j/$name;
rgregg0a {
vnet;
vnet.interface="e0b_rgregg0a";
exec.prestart+="/usr/local/bin/jib addm rgregg0a em0";
exec.poststop+="/usr/local/bin/jib destroy rgregg0a";
}
