jail/VIMAGE stability

rootbert

rootbert

I have used jails for quite a long time now, but vimage only for test environments and my personal stuff. On my production machines, I have never used vimage. However, as I have read the vimage stack is blazingly fast with almost no overhead, I would really like to go into production now, all systems with 12.1-p3. Here and there I read about crashing hosts, and most of the time I had no information about what FreeBSD version used. I would like some feedback here, who is using it in production, are there any pitfalls in certain configuration, how stable is it really? thanks!
For your info: I have never had problems with my test environments and my personal infrastructure (however, changing production config for companies is always a little scary) but the findings on the web about crashes made me a little defensive here. My intended use: separation of services - databases, web- and application servers, openvpn/wireguard and running PF on the host + inside jails.
 
There is a reason why they waited until 12.0 to include VIMAGE in the GENERIC kernel. Many would've liked to see it shipped sooner ;-). So yes, you can assume its stable for production use.
 
pyret said:
I am not aware of pf or ipfilter being virtualized. They are still listed as outstanding items on VIMAGE/TODO even though VIMAGE is enabled by default in the kernel now. But maybe the page hasn't been updated and those items have been fixed.
Click to expand...
Correct that the page hasn't been updated, IPFW & PF should be fine in VNET jails.
 
I used VIMAGE on 11.x and (very rarely) the host panicked when taking a jail down. This never happened to me on 12.1 so far, so I guess it's not an issue any more. What still happens (although rarely as well) is that the epair interface is "half destroyed" after stopping a jail .. the end for inside the jail vanishes, the outer end is still there and has to be destroyed manually so I can create a new epair device to restart the jail. All this experience is only from a private server installation.
 
thanks for your response. yeah, for me jails/vimage is just awesome, I quite often forget about the quirks of localhost/bind ip addresses that non-vimage jails bring with them when I configure some services, so vimage jails feel more natural to me. I can live with some issues when starting and stopping the jails as long as it does not affect the host or other jails. I also read that only ipfw worked inside vimage jails, however, my experience was that pf works fine (you just need to have the same firewall tech on host and jail). I think the FreeBSD wiki is in a sad state, there are loads of outdated pages. I have a wiki account and wanted to update some pages, but it turned out that it is a lot of work if you are not involved in the subprojects directly - e.g. doing research from the commit mails and mailing lists just to update the VIMAGE/TODO page is so much work if you don't follow those mails on a regular basis so I unfortunately gave up.
 
All three firewalls work in VNET jails. One of last year's GSoC projects was to write tests for all three and that shook out some of the remaining issues.

pf was the first of the firewalls with automated tests, and those rely heavily on VNET.

In general VNET should be stable in 12.1. Once everything is up and running you're very unlikely to see issues. Most of the bugs we ran into (and fixed before enabling it by default) were related to stopping jails.
 
I changed to VIMAGE in December, 'til now no issues whatsover. The networking appears to be more clean design than jails without VIMAGE - at least mine go cleanly down and up with the same number - which they often didn't do without some delay before.
This is running 11.3 and interconnects done with Netgraph. It is small machine and bulked full with stuff and often extremely overloaded - which is usually a good test for robustness. If You also need feedback for high-performance/high-throughput scenarios, then ask somebody else. ;)

IPFW is working within VIMAGE. What is NOT working is logging from the IPFW - these logs appear in the main system (where they are useless). I do not know if/inhowfar this was, is, or will be fixed. (There are suboptimal workarounds.)
 
You must log in or register to reply here.
Back
Top