iocage - RACCT/RCTL question

fred974

fred974

Hi,

Each time I stop my sysutils/iocage, I get the following message:
Code: 
* Stopping 1b92750b-badc-11e6-9724-f04da20055fe (projects.trinitech.co.uk)
  + Running pre-stop         OK
  + Stopping services        OK
  + Removing jail process    OK
  + Running post-stop        OK
rctl: RACCT/RCTL present, but disabled; enable using kern.racct.enable=1 tunable
Should I be concern about this message?
Should I add the value in tunable?

Thank you
 
I get the same


Code: 
root@bhyve:~ # iocage stop git-new
* Stopping 05ae7700-9d7e-11e6-aaff-0cc47a68b3d8 (git-new)
  + Running pre-stop         OK
  + Stopping services        OK
  + Removing jail process    OK
  + Running post-stop        OK
rctl: RACCT/RCTL present, but disabled; enable using kern.racct.enable=1 tunable

I have not dug dipper into this but in my experience didn't affect the work of my jail instances. What scare's the shit out of me is the fact iocage Google group mailing list is dead silent (I am subscribed to it) and that iocage is finally forked.

https://forums.freebsd.org/threads/58818/

However the link

https://news.ycombinator.com/item?id=12946860

indicates that iocage is full of bugs. Yet majority of iocell (which is still not in ports) commits are related to branding. Even worse the adoptive parent of iocell seems not to care much about being able to import existing iocage jail instances into the iocell. He talks about rapid automated provisioning as the only way to migrate from old iocage provisioned jail instances to iocell instances. That might work for him but is not good on my pocket book.
If I do rapid provisioning I am only sure of one thing. The server will no longer run FreeBSD and use ZFS. I will provision those services back to old trusted OpenBSD where people don't abandon their pet project every year or so.
 
The iocage master branch hasn't received any bugfixes or upgrades since a long time. The 'problem' with the development branch (from which iocell was forked) and the reason why iocage jails can't be used with iocell is this commit: https://github.com/iocage/iocage/commit/13de231ef783977b4e353b96e9fee6454d03c01e
The configuration has been changed to use UCL config files instead of ZFS properties - as this was one of the major advantages for me, i'm also looking for an alternative...
 
  • Thanks
Reactions: Oko
fred974 Oko When it comes to the rctl message, as far as I am aware it doesn't mean there is any wrong with your jails. That seems to be just an informative message implemented in very annoing way. I am still familiarizing myself with iocell code I've inherited from iocage and fighting more important bugs, but when I'll have more time and more assurance it is safe to do so, I am planning to get rid of that thing.
 
Oko It is not that I don't care about providing a clear migration path from existing iocage installations to iocell ones or that I care only about 'branding' iocell. You have to be aware that I am only a one man, who took the burden of fixing iocage's existing issues and turning it into something usable and useful. There are only so many things I can do on my own with my spare time and it means sometimes the commits in master branch are going to be less revolutionary (development happens in develop branch, I am only merging to master, when there's a major bugfix that's tested to be working well) or cosmetic. It also means, that I have to prioritize some things over other, and this is why iocell google groups doesn't exist yet, readthedocs page doesn't exist and so on - for now bug free jails manager is more important to me. Other things, as iocell matures and workload gets smaller and/or shared between other contributors, will come. I would really appreciate some help with it, don't be afraid and get your hands dirty! ;)
 
  • Thanks
Reactions: Oko
robak said:
Oko It is not that I don't care about providing a clear migration path from existing iocage installations to iocell ones or that I care only about 'branding' iocell. You have to be aware that I am only a one man, who took the burden of fixing iocage's existing issues and turning it into something usable and useful. There are only so many things I can do on my own with my free time and it means sometimes the commits in master branch are going to be less revolutionary (development happens in develop branch, I am only merging to master, when there's a major bugfix that's tested to be working well) or cosmetic. It also means, that I have to prioritize some thing over others, and this is why iocell google groups don't exist yet, readthedocs page doesn't exist and so on - for now bug free Jails manager is more important to me, and other things, as iocell matures and workload gets smaller and/or shared between other contributors, will come. I would really appreciate some help with it, don't be afraid and get your hands dirty! ;)
Click to expand...

robak please don't be apologetic. You have no reason to explain yourself to anybody and particularly not to me, an open source consumer circa early 90s. Developers like you pick to work in their limited free time on things which interests them and are useful to themselves. There is nothing selfish about that fact. You are not getting paid iocell and nobody has any right to demand anything from you. sko posted a link which clearly explains why iocage and iocell jails are incompatible. I have to look into it little bit deeper in my limited free time:) I fully support your decision to make a small evolutionary changes. Personally I fell very much against aggressively adding new features. The only feature I was personally missing on iocage is the ability to send snapshot to the remote backup machine. I have a very crude script which does that on my severs and even starts the jail on the new machine effectively creating hot spare (failover) copies but it would be nice to be consistent along the lines of iocage snapshot script.

I am a bit upset at the FreeBSD foundation. Unlike OpenBSD which has benevolent dictator Theo de Raadt, FreeBSD has core. The way that I see the role of the core is to prioritize and support important projects funneling the funds from FreeBSD foundation to interested developers. Over and over I see that politics is often on the way of technological progress in FreeBSD camp which is not as great as FreeBSD fun boys would like to believe. FreeBSD currently has a solid technological edge over "competitors" in several areas largely due to the work which originated somewhere else (ZFS comes to mind). Jail is one of the ideas which actually came out of FreeBSD camp long before Solaris zones and few centuries before Linux guys contemplated containers. I am appalled that such a potent symbiosis of ZFS and Jail technologies can basically fail between the cracks just because original iocage developer got hired by ix systems to rewrite the tool in better suited language (Go). The core should have reacted immediately and fully fund the developer which would have saved iocage from rotting into oblivion. You stepped up but the same thing will happen tomorrow if you have to concentrate more on your day job which provides for your family. This is, IMHO, much better handled in OpenBSD camp for example. They appear to work on fewer things and in much slower pace but when they get things right the really get them right.
 
Oko I kindly disagree here. Both iocage and iocell are third party pieces of software that are wrapping existing and (more or less) complete OS features, like Jails and ZFS. The FreeBSD Foundation (despite not being the governing body for FreeBSD project, as many people think, but instead a supportive one) just as FreeBSD project itself, is not responsible in any way for such projects, it would be an overkill, given there's around 25k of ports available for FreeBSD. Keep in mind, that just as iocell/iocage is not a part of the OS (whereas ZFS and Jails are, and they are fully supported and cared about) the same way Linux Foundation is not responsible for Docker or Kubernetes, the Linux frameworks binding their unionfs, cgroups, kernel namespaces and whatnot. Beside, who's to decide that it should be iocage, iocell or something else, that should become 'official' in some way? There are other *very* interesting containers framework projects for FreeBSD, just to name Tredly (https://github.com/tredly) or JetPack (https://github.com/3ofcoins/jetpack) that may quickly surpass iocell/iocage in functionality, stability and popularity, and what then? Would we want to FreeBSD Foundation to fund less featured and popular project instead? ;)
 
  • Thanks
Reactions: Oko
robak: can you elaborate why the switch to UCL config files was made? In terms of migratability and also backup/restore I really loved the fact that configuration was essentially tied to the jail via its dataset properties. zfs send|receive was literally all you need to migrate or backup a jail and its configuration.

I'm currently hacking together a way of using netgraph with iocage and if this turns out to be usable (and after a lot of polishing ;)), I'd really like to commit these additions back to iocage/iocell. For faster PoC/prototyping I used the iocage develop branch just before the aforementioned commit as a basis, so I can test this in my existing environment.
OTOH, if UCL is the way to go for iocage/iocell in the long run, then I wouldn't want to introduce yet another incompatible variant of a jail manager. But I'm really struggling to understand why the switch to config-files was done/necessary, as it doesn't seem like an improvement or introducing any advantages.

From glancing over the changes, it seems to me the actual options weren't changed, so it should be easily possible to read the zfs properties from the dataset and put them into the UCL file, or even use them in parallel and introduce some way of 'synchronizing' them, e.g. at jail startup, and create the UCL file from zfs properties if it doesn't exist. I'm really a fan of the idea of UCL when it comes to unifying the configuration of services that are already configured via config files (often with very weird or even annoying syntax); but for something that should IMHO be easily to migrate between systems, the zfs properties approach just feels much more appropriate.
 
sko I have no idea, I haven't made that change ;) To be honest, I've not yet looked at it, for now I have to accept it as it is. When it comes to principles, I'd also like no dependencies and if there'll be time where there's nothing else to do, I might look into possibility of reverting this change, if no internal functionality is goint to be lost. However, that's a song of the future for me, but, again and again, patches are welcome!
 
Well, it turns out I'm obviously bad at botanics; especially when it comes to recognizing a tree within a forest....
While sifting through the changes for quite some time and figuring out an elegant and non-intrusive way of merging UCL-config and zfs-properties, I was constantly annoyed about the humongous pathnames when opening the config file that reside in the dataset of the jail. I believe the neighbors could very clearly hear my hand hitting my forehead at the moment this finally ...

So it seems my point about zfs properties is completely invalid after all :p


I just compared the options set by iocage as zfs properties and what iocell writes in the ucl config. In total there were 11 options added and 1 removed. After merging these and moving the datasets over to iocells tree I was able to run and manage an iocage-created jail with iocell.
So it seems like zfs rename/clone/send|receive the jails dataset from iocage to iocell, writing out the properties to the config file and merging the new options + updating a few (all easily accomplished with a handful lines of awk) is all that's needed to use an existing iocage jail with iocell...
I'll hack together a shellscript to automate this task and test it with some jails on another machine. If it works I'll either add this function to iocell or just write a migration-script.
 
I've wrote a working prototype for the jail migration over the weekend. Migrations with some freshly set up jails on my testing server at home went fine, so today I was testing it on the jails in my devel/staging environment at work. All devel/staging jails migrated fine and are currently managed by iocell :)
I triggered a failure on a thick jail early on at home, where iocell couldn't find/restructure some datasets on first startup and complained about unsupported properties, but I couldn't reproduce it. Even here none of the thick jails (which are kind of legacy anyways, everything newer is a basejail) triggered this error again. As the affected jail just worked fine after the first "failed" startup and I couldn't reproduce it within ~35 thickjail migrations, I just have to put this aside and hope to fall over it again and maybe get it to be reproducible.

iocell already has some migration mechanics in place, e.g. the zfs dataset structure gets updated on first start of a migrated jail and iocell list also generates a default config file when it discovers a new jail in /iocell/jails/; although this seems to be a side-effect of calling uclcmd. Still investigating on this. OTOH " iocell reset", which shuld set the jails config to defaults, fails to generate a config if the file isn't already there. I'll have a look at this and fix it beforehand, this way I can give myself an overview what mechanisms are already there and maybe could leverage iocell reset for the initial config generation after migration instead of using a side-effect of iocell list which might go away sometime.
I started to build an ioc-migrate script from my prototype and added the "iocell migrate" option. It basically works, but I feel it needs a lot of polishing and some more safety-nets to get up to the standards. Also it lacks proper integration and usage of the already present functions, variables etc in iocell.
I hope to find enough time this week to push a first working (and presentable) implementation before the holidays. (This will be my first commit to a public repo - I hope I won't screw up :().
 
  • Thanks
Reactions: Oko
To all iocell and iocage-devel users - I've found and (hopefully!) fixed another important bug in iocell, inherited from iocage-devel codebase. The information and fix is available here: https://github.com/bartekrutkowski/iocell/issues/5
I would love to get some more testing than I can conduct myself, before I merge it into develop/master branches and update the port to fixed iocell version.
 
  • Thanks
Reactions: Oko
robak: that was me who opened the issue on github ;)

I tried setting aside some time at work to wade trough the rather verbose routines for generating the config and everything it calls and that depends on it, and I may understand the initial intent behind the way it was split up (to use the same routines for generating the whole config or just updating/setting single properties), although the most costly (in terms of loops and if-statements parsed) parts are only called when generating the config from scratch when creating or migrating a jail and I still don't understand why they were made that complicated.
I don't know if uclcmd didn't had the "merge" option at the time the changes to iocage were made (and therefore the loops were necessary), but I started to simplify and slightly restructure the _dump_config and _configure_jail functions to reduce the chain of functions being called or calling one another on the one hand, and on the other hand ending up with a more general function for generating/changing/setting the UCL-configs by using uclcmd merge.


Dumping a pre-generated list of UCL-configuration (from zfs or the /iocell/.default file) to uclcmd merge instead of looping over every property/value pair and/or combing through a list of switches for every pre-defined property speeds up jail creation by several seconds! Especially for automated mass-deployment of jails this would result in a huge performance gain (which was one of the initial intents of the switch to UCL if I read the commit text correctly).
But before ripping out this constructs of 'case' and 'if' statements I just want to be really sure they weren't placed there for a very special reason I haven't discovered yet. I still have the impression that nobody would go through the struggle of writing it that complex if it weren't somehow necessary...
Also I'd rather set all available options in the .default file instead of hard-coding all properties within the functions, thus keeping them as general/universal as possible. This way new or even user-defined properties could be introduced relatively easy, maybe without even changing the code that sets/gets the properties.
 
  • Thanks
Reactions: Oko
I've only picked up work on this 2 days ago. I had to sort out some personal stuff after returning from family visits over christmas/new year and also just went back to work this week, so I'm a bit packed with work right now, but I hope I'll find a few hours towards the end of the week to get back up to speed.
The parsing from/to UCL and conversion from zfs properties is pretty much complete (leveraging uclcmd merge whenever possible), with the main code now being relatively general and the rubbing and chicken-waving for zfs/ucl/jail put in its own functions. Migrating from zfs and modifying stopped jails pretty much works (have to test for some edge-cases though), but the parts for modifying running jails (with jail expecting some weird/special property notations) are not completed yet and some loose ends and cleanups are still left from the restructuring/refactoring which have to be dealt with.

I've kept the upcoming weekend completely unoccupied to really catch up and make up for the unintended delay, so I'm confident to put everything through some final tests in the staging environment here at the beginning of next week and sending out the patches for review.
 
I think I've found another bug - can you guys compare your iocage and iocell installations and report back if 'iocage rcboot' works for you and if 'iocell rcboot' works too? If either doesn't, can you paste your iocage/iocell version and the output you're getting from running these commands?
I have a solution already, but I need to know if it is not my personal setup that's causing this.
 
It seems that iocage has now resurfaced in Python. Shame to have to install a bunch of dependencies to use it but probably makes the codebase a lot easier to work with.

Regarding the use of config files instead of properties. I made the same decision when writing vm-bhyve (although I'm just using rc style as it's easy to parse and easy for people to edit manually). My decision was partly due to wanting to support UFS/iSCSI/NFS/whatever as well as just ZFS, but to me the whole idea seemed nowhere near as good as people seem to think -

  • Using ZFS properties requires running commands to change any settings. With a config file you can just open it and edit the whole thing.
  • As already mentioned, it's very slow. With large numbers of jails people were finding it take a huge amount of time to list guests.
  • It's not ideal to modify the dataset every time you want to change a minor guest setting.
  • Having dozens of custom properties makes a bit of a mess of the ZFS property list. (Most people probably aren't as anal about things like this as me but properties don't seem the place to store huge numbers of config settings). Additionally it got ugly if you needed multiple values for a setting.

The only real good point was the properties being included with send/recv, which is a moot point if you're storing the config on the dataset itself. I thought iocage didn't do this because the config file would be visible to the guest, but having used it, it seems they create a jail-uuid and jail-uuid/root dataset, so they could easily have stored a config file under the jail's dataset, but above the root.

The chyves fork of iohyve changed to files not long after it forked.
 
You must log in or register to reply here.
Back
Top