rcd(8) - new service manager daemon

The difference is that CPAN went down (as in they turned OFF CPAN) and then you literally could not use entire perl scripts because you had no way of getting the dependent perl modules that were dependent in those scripts. Not to mention that in (NON-INTERNET) connected environments there was no way to get all of the various perl script depend modules - because you needed access to CPAN.

You honestly can't say that about sh(1) right?
You can do a lot with Perl Core without any Internet connection. If you need more there is p5-xxx packages.
 
The parallel boot will be coming "for free".

The increasing complexity and variety of user's boot scenarios will require drawing a proper dependency graph. Once you have one that works it is trivial to do it in parallel.

WRT boot messages - I'd like to keep them. Just sync the printing.
 
sh(1) doesn't change much either.
There are good reasons why you don't want to use shell scripting for this. See: https://skarnet.org/software/execline/dieshdiedie.html

In the s6 suite of tools this issue was solved with execline. If you have something legacy you don't want to re-write using shell scripts (or whatever else) you can simply pass it straight through using execline scripting. The way systemd "solved" the "hacky shell scripting" problem was using unit files which are even worse than shell scripts since it's hard to express complex logic since they aren't a real scripting language and instead are more akin to .ini files.

Lua and perl don't really matter but both are more general purpose languages that pull in their own stuff to make them work so now you have yet another thing in base that can go wrong. By comparison execline is smaller and more suited for this task. But given the choice between perl and lua the decision to use lua is probably the better choice. Other things I'm already using want lua so I personally need it anyway.

Perl is not bad despite its reputation and everyone deciding they suddenly hated it sometime around 1999-2000. There is a good reason why it exists and why people heavily used it over writing shell scripts. It's more portable and less prone to failing spectacularly. It's also good job security if you're the type of person into writing complex one-liners with it.

Perl and lua are certainly much better to have in the base system than something like python. Python is a glutton and it likes to break things every time a new version comes out. God help you if you want to use libraries.

A supervision suite is not bad thing in fact it solves a lot of problems within a UNIX-like OS's user space. But it brings a lot of complexity of its own and there are hard problems you have to solve if you're going to commit to having it. For example, how will daemons communicate? Now you need a standard way to do IPC that can not fail. You need to re-think how logging works especially if you're going to do complex starting/restarting of services at the same time. You have to make sure logs are ordered correctly and the logging mechanism is bullet proof so it won't ever crash and lose logs. You have to manage a complex web of dependencies. You now have to maintain and build some kind of database to ensure things start in the correct order and it quickly gets complex when you have things that require other things to already be running. You have to ensure the state is the same each time your complex tree of dependencies and daemons are started (one reason why using shell scripts is a bad idea). Most of these problems either require designing the service management with a lot of forethought or forcing new ways of doing things into the rest of the system.

Most popular supervision suites have been designed very badly hence why we're now in a situation where Linux boxes require rebooting when things in userspace are updated and the PID1 is now so large. Even the things that inspired systemd like launchd have a whole host of problems (e.g. use of XML). So people are right to be paranoid when they see such complexity being adopted by other OSs. The past has shown it's very hard to get this stuff right.

The faster boot times is a myth in my experience. At least in the Linux world. systemd is not only much harder to debug it's much slower at start up on every system I've used it on compared to things like runit, OpenRC, sysvinit and all of the others. As I said before "parrell start up" is mostly not real. Since you have to wait on daemons to "settle" anyway before they can actually communicate with each other and go on to do real work. Servers and laptops are rarely rebooted as much as people like to pretend anyway. A laptop is typically suspended to RAM/disk or the majority of the boot time is spent everywhere that isn't init/service manager firing up daemons at boot. Desktops and servers are typically running 24/7 and the few seconds at boot do not matter anyway. If your system is being rebooted all of the time you have other problems you should address. Similarly, auto restarting some daemon that's constantly crashing or otherwise acting up is not the proper way to address that problem. You should instead reconfigure/debug it or use something else. If it's that misbehaved then having some complex service manager restarting it multiple times an hour is just kicking the can down the road.

A lot of the supposed "benefits" provided by something like systemd is just marketing and bad engineering.
 
At least in the Linux world. systemd is not only much harder to debug it's much slower at start up on every system I've used it on compared to things like runit, OpenRC, sysvinit and all of the others.
No disagreement here. One of my biggest "wtf did you do this" with systemd is journald. If you've ever had a failed boot you get a string of digits, no text. Then you discover the logging is through journald but it's binary. So one can't cat | grep | awk like you can do with dmesg and syslog. The few times I've had this happen I luckily had another system that I could "hey duck duck go wtf does this systemd error message mean".

A lot of the supposed "benefits" provided by something like systemd is just marketing and bad engineering.
Ha. Thanks for making an old man LOL.

One thing I see with things like Solaris SMF/rcd is the management of things started. "Do I daemonize this thing or not" Current startup stuff (generically rc stuff) a lot of the responsibility is in the specific service script itself.
Normally that works fine, but corner cases make it fall apart.
So something that does a better job of managing things, in a consistent manner, should not be considered a bad thing. My understanding of Solaris/SMF is that consistent management is a big plus.

FreeBSD:
Base vs ports. Trivial (not easy) to change from one rc system to another.
Ports/Packages. Not at all trivial, not at all easy to change. Typically carry at least 2 sets of patches on service files, based on version, which leads to maintainers having to understand a bunch more. At some point one can assume convergence, but FreeBSD will still carry patches for a while. Chances of pushing specific patches back upstream are "You can ask but we don't need to pull"

I don't know what the right thing is anymore. I've lived through probably 3 or 5 different init systems between *BSD and *niux system, and imagine at least 10 more before I stop being able to type out rants.
 
this is completely irrelevant. if you don't enable it in rc.conf it doesn't matter if you have 3 or 3000 scripts in [/usr/local]/etc/rc.d
Every rc script you have in /etc/rc.d or /usr/local/etc/rc.d is being parsed and executed during startup, no matter if you have foo_enable="YES" in rc.conf or not.

I think you have a lot of misconceptions about how rcNG works and what rcd actually is and where it plugs into the whole system...
Very likely. But I'm trying to project future use of this new rcd startup subsystem, I can foresee lots of new problems added and very few resolved (if any?).
 
One thing I see with things like Solaris SMF/rcd is the management of things started. "Do I daemonize this thing or not" Current startup stuff (generically rc stuff) a lot of the responsibility is in the specific service script itself.
Normally that works fine, but corner cases make it fall apart.
So something that does a better job of managing things, in a consistent manner, should not be considered a bad thing. My understanding of Solaris/SMF is that consistent management is a big plus.
This is the only problem worth solving, but there's no need to introduce UCL or whatever new files for that. From what I can see, rcd already can parse rc scripts, just treat them as first class - parse them, build tree, run sequencially or in parallel, keep and monitor PIDs. This approch is good both for ports maintainers and for users. No changes are visible on user-level.
 
"parallel start up" works like this: We make a new daemon called daemon C. Daemon C comes up first and pretends to be daemon A, B, D, E, F, G etc. It fakes state that makes everything think daemon A is both up and that you can communicate with it. It holds whatever daemon B sends at start up in a queue and tells daemon B everything is a-okay and/or to wait. Once daemon A is really up our new daemon C spams it with all the crap in the queue from daemon B. daemon A can finally really respond to daemon B. daemon A and B go on about their merry way.
Nonsense the directed acyclic graph forbids this.
 
This is the only problem worth solving, but there's no need to introduce UCL or whatever new files for that. From what I can see, rcd already can parse rc scripts, just treat them as first class - parse them, build tree, run sequencially or in parallel, keep and monitor PIDs. This approch is good both for ports maintainers and for users. No changes are visible on user-level.
Working with POSIX shell is hell, do no recommend.
 
Working with POSIX shell is hell, do no recommend.
Shell scripts should best written in POSIX shell for maximum portabilities.
If it doesn't fit you, you'd better choosing different language that fits your needs.

rc.d scripts has shebang line at their top. So you can specify any interpreter there for ports (and any in-base scripting languages like flua even for /etc/rc.d/) if the language is sanely pulled in as RUN_DEPENDS.
The requirement for the language itself is that character # shall be considered as the top of comments in each lines. And need to implement the equivalent of /etc/rc.subr yourself. Not theoretically impossible.
 
Shell scripts should best written in POSIX shell for maximum portabilities.
I part this, because turing complete means things can get very ugly. For the other part POSIX shell is so unclear about what is what, FreeBSD manpages have little to no documentation, at least if not reading everything, which explains what the set flags do.
 
Working with POSIX shell is hell, do no recommend.
man sh says it's mostly POSIX.1 plus some BSD extensions.

Shell can be good or bad, but it is still widely used and there are people (old admins mostly) who understand it. I remember times when training courses on Shell programming were popular. I still keep a couple of OReilly books.
 
I watched the video an looked into the code, Lua is inevitable.



We have Shell and Perl, that's sufficient. Lua is ok for bootloader where resource is scarce, nowhere else.


Again, I read and watched the video, I looked into the code. I think I got he's idea quite well. FreeBSD will have systemd.


I'm quite old to know how this all ends, see my comment above. To add to that, we will have lots of broken ports which will be deprecated and removed becauase noone will want (have time) to fix them.

Clearly you didn't because all of what you said is factually incorrect. There's more clarity in the mailing list thread I referenced.

Also, Perl was removed from base years ago.
 
perl fell out of favor around that time because of worries about where the perl language was headed in the future. Before the whole perl 6/Raku disaster it could be depended upon to be on just about any UNIX-like OS running on servers and desktops. Hence why it was so widely used during the early days of the web and why so many people opted to write scripts in perl instead of shell scripts. Since perl was much easier to make truly portable. Although using perl did come with its own set of problems (mainly: Where is perl installed on this system vs. these other systems running a different OS?).

Anyway, the perl guys shot themselves in the foot after perl 5. People got terrified that their old stuff would no longer work on future versions. It took them years to finally decide what they were attempting with perl 6 was a horrible idea so they renamed the new version Raku. But by then it was too late and a lot of people had opted for Python instead. Which was gaining popularity at the same time popularity of perl was declining. perl went from the standard scripting language everyone suggested a newbie to programming learn to something considered old hat. Hence why PHP took over the web quickly and why Python took over general purpose scripting for most everything else. In truth, we would have probably been much better off if everyone just stuck to perl. But complex one-liners are scary I guess and as we've seen over the past 25 odd years any pretense that good practice was going to be followed by the industry went out of the window. So now you get mountains of incompatible python, javascript, PHP and other "frameworks"/libs all wanting to pull in GBs upon GBs of stuff just to do basic things like generate an HTML document and fetch updates for parts of the DOM in the background. Hell we now have pseudo-websites pretending to be full blown system applications that want to pull in an entire browser engine for each and every application. Then the same companies/people pushing this "portable" soup of garbage claim we can make it secure if we just stuff it into yet another glorified chroot. The madness has even taken over systems programming now with new C-like languages wanting to have their own package managers that pull in ungoldy amounts of random source code at compile time (and sometimes runtime).

At least when we were all using perl the syntax was similar enough that it was a step towards understanding C. Now I have no idea how any children are expected to understand C code. Much less what C code compiles down to. To the new generation of programmers the OS might as well be a black box. Just like in my generation the CPU was already a "black box" that you weren't supposed to understand because assembly was "too hard" compared to the generation before mine that got to cut their teeth on stuff like m68k asm.

There is no real reason to write yet another rant about the situation with PID1 on Linux. They won't listen anyway. The average user of Linux these days is not the sharpest spoon in the shed to say the least. If you dare to not follow their dogma they prefer to come up with reasons to ban you from the discussion since they can't argue on technical merits. They hold all the moderator positions at places where discussion happens so there is no point in attempting to save them or their new version of Windows. If you want a good laugh check out what's happened on the LKML lately (I suggest starting with Linus's latest commits).

As I keep saying: I welcome something akin to s6 in the base system. But I don't have much faith in anyone attempting to re-invent the wheel yet again when a good service manager and a suite of tools built around it already exists. Whatever standards are set for scripting services will eventually be the only option. Since that's the only way things can be. Once it spreads into base it will eventually take over ports. As we've seen in the Linux world all alt-init/service mangers were quickly excluded from most distros because no one is continuing to provide shell scripts for package. Hence why something like s6 is not used very widely. To use it you must be competant enough to write you own scripts in execline for each and every daemon/package you want to use on your system. Maybe if you're lucky there will be an overlay (Gentoo) where someone else has done part of that work. But most likely whomever is maintaining that overlay isn't interested in everything you're interested in. So instead you mostly get systemd unit files and shims. Hence why Linux is a bit of a lost cause now. IBM's plan to own userspace has worked so now even the "non-systemd" distros are just running 80+% of systemd stuff via shims and giving you a different PID1. I suppose this is better than running a complex PID1 consisting of about 1 million lines of code with complex code paths. At least something like runit or s6 is small enough that one person can understand it and you don't have to worry about it crashing randomly taking the entire system down with kernel panic or some "unintentional" bug exposing a back door into every server and desktop/laptop out there running mainstream Linux (totally a conspriacy theory I'm sure ;)).

Well, I must have missed when that happened. ;-) I have Perl installed on all my machines. Anyway, Shell is pretty much eloquent for this purpose.

Friend I'm not sure why you didn't read what I linked above that explains in great detail why shell scripts are horrible for this purpose. Yes we made do with them for many decades. No they were never a good idea or suited for this type of thing.

The main issue with shell scripts is parsing and then second is portability. Even if you write to POSIX standards shell scripts are never going to be truly portable. That's how it has always been and that's how it always will be. Shell scripts are also horribly insecure and the performance is terrible. Again see: https://skarnet.org/software/execline/dieshdiedie.html

Perhaps you don't like Lua but again out of all the programming languages in common use that could have been selected it's most likely the best choice. It's small, it's written in ANSI C and finally it's widely already used as a scripting language in many other applications. This makes it a great canidate to be included in the base system. Finally, it's very unlikely it will change much in the future or suffer the problems that have plauged other scripting languages like perl and python (among many others). Learning Lua is worth your time and you will find as much value out of knowing it as you do knowing how to write shell scripts. If you really insist on sticking to shell scripts for firing up daemons then you can still do it via Lua. Which is most likely how most things will be supported (just like s6 and execline) until all scripts in base are re-written directly in Lua. After that it won't be long until most ports follow. At which point you'll be on your own if you inisit on continuing to use shell scripts for these purposes. The new service manager's way of doing things will become the default. It always happens.

This will be a positive change as long as rcd ends up being good and doesn't go down the path of so many badly written init/service manager schemes from the past. Only time will tell.

Bit of a Rant follows:

There are actually much more concerning things happening in userspace to worry about. The fact of the matter is very soon FreeBSD (and all the other BSDs) are going to be put in a position where they'll either have to start maintaining a lot of software themselves that they've been pulling from "upstream" (read: IBM/Linux) or they'll be forced to adapt to using what "upstream" is shitting out as of late. This means things like having to drop support for X11 in favor of wayland or having to emulate stuff like logind (or write shims/wrappers around Linux's versions of these tools). Thankfully, on the X11 front the OpenBSD guys will probably be maintaining Xenocara for a long time to come. So we're okay there unless FreeBSD's developers decide to go all-in on Wayland and suffer from the same problems that are happening with mainstream Linux. But if we don't go all-in on Wayland we're going to be in a position soon where we'll be forced to start maintaining stuff like GTK, Qt, KDE and Gnome on our own. I personally think we should just stop putting in the effort to port things like GTK, KDE and Gnome while going back to maintaining gtk2 (maybe gtk3) and Qt 5/6. There is good effort happening on the gtk2 front lately. It seems more people are waking up to the madness that is Gnome/gtk4 and finally some people are getting behind keeping stuff like gtk2 going and adding new features to it. Thankfully, most things written around gtk still compile just fine with gtk2. Same is true of most stuff written for Qt.

I would personally enjoy seeing more effort going behind Arcan and its A12 protocol. Which most people don't seem to even know exists. Despite the fact that it already supports all X11 and Wayland applications out of the box and brings many new exciting features to the table. Like true support for networked GUI applications. I have Arcan running on a bunch of desktop systems on my LAN and it can do amazing things. For example, I can fire up a Neo-Geo emulator, load a ROM in it then drag+drop the window to any other machine on the LAN. It sends state over the wire fast and everything just works. I can even transfer the full state over to another machines CPU/GPU/IO/Input devices or I can continue running it on the machine where I first launched it and only send over video/audio/input events. It just works either way. It's really exciting software that solves a lot of problems people claim are either impossible or too hard to solve. The guy has been quietly working on Arcan for years with a small group of developers yet you'll never hear a peep about it on the major discussion platforms where the Linux people are constantly arguing over X11 vs. Wayland with the same tired arguments and talking points we've seen mulled over for 18 years now.

I would really like to see a major shake up in the FreeBSD base system where a lot of stuff currently installed by default was kicked out and users told they have 1-2 years to prepare for it. There is a lot of cruft in the base system that only exists for old systems running old configurations. It's why FreeBSD requires so much configuration up front to get it going correctly on most systems. It's why a lot of the current defaults are not secure. Much of this stuff should be kicked out to ports and things like the firewall should be cut down to one default option (I vote for pf) and anyone with a "legacy" system that wants something different should spend the time to get their configuration going with the port instead of expecting that stuff to continue to live in base. But I know this is not going to happen any time soon.

Believe it or not in many ways OpenBSD is better for most users on the desktop (and even laptops) because they keep their base system small, they offer a secure X11 implementation at install time that just works at first boot and they don't have problems like 3 different firewalls in their base system. Sure you won't get things like 32-bit support or wine (well wine is in their ports tree now as of a couple of weeks ago) but you will get a well documented coherant system with good base system that's well tested. Since most developers of OpenBSD are actually using it as their day-to-day desktop. Unlike Net/FreeBSD where most developers seem to be using it as a secondary system where they use something else for their desktop OS (I see a lot of Apple laptops at conventions, just sayin').

All that aside in FreeBSD we're already relying on a lot of things we shouldn't be. The Nvidia drivers make us beholden to Nvidia to provide binary blobs. The entire graphics stack/DRM is ported from Linux's kernel now (same is true of OpenBSD as well). A lot of wireless chipsets are badly supported by native drivers or require using the Linux drivers through a wrapper. I could go on but you get the idea. A lot of crucial things in kernel/base depend heavily on an "upstream" beyond our control that's quickly diverging from the BSDs and the developers of such things are pretty open about the fact that they don't care about supporting anything but what IBM pays them to support (which means the Linux kernel in the vast majority of cases).

Another big issue is even the Linux kernel is quickly dropping support for "legacy" hardware. By "legacy" they mean pretty much everything made before 2010. In a year or two running the Linux kernel on a lot of systems currently using it is going to become impossible due to their latest round of removing a lot of drivers due to hypothetical exploits found using LLM tools which weren't even tested (or attempted to be fixed assuming they are actually exploitable) before getting removed). So an influx of people looking for a different UNIX OS for "legacy" hardware is likely coming in the next few years. If FreeBSD decides to follow suit this will leave a lot of folks with no viable OS for those systems. Aside from maybe OpenBSD and NetBSD. With the latter while a nice community they often go years without testing their automated builds are actually working on systems they claim to support. As anyone that's tried to install NetBSD on a Dreamcast or comparable system has found out first hand.

My greater point here is there are much bigger issues than worrying about if something like a new service manager is base is going to turn into systemd in all but name.

Someone like me will be okay in the future because 1) I'm willing to accept running something like OpenBSD and doing without some things I get in Linux/FreeBSD and 2) I'm okay building things from source, setting compile time options and running "legacy" software even on a system with network access. Since I actually have a dedicated firewall and most of the hypothetical exploits don't scare me/don't affect me. But I do worry about what's going to happen to the vast majority of users that aren't willing to do these things (or learn how to do these things). I'm also worried about the fact that it's becoming very hard to find people that can even understand how something like the BSD/Linux kernels work. Many new programmers can't read C or refuse to learn C.

Many younger people don't even grasp the concept of how the FHS work. It has really gotten bad. I've been working with some recent graduates this past week that claim to be good programmers but they can't even do something as basic as write a web page in HTML directly or know how the man command works. I'm serious. But they'll argue for hours among themselves online about how something like Rust is better than writing things in C and tell you to "get with the times". Most of them can't even write those rants themselves they just prompt and LLM to do it for them and they're totally lost without a connection to the internet. I assigned some basic tasks to one of them on a machine without network access a few weeks ago. We do not allow cell phones in that room (and they can't talk to the cell network in there anyway). I gave them somthing incredibly basic that any child of the 1980s/1990s should have been able to do on their own. They had full access to the man pages and there were multiple books on the shelf they could have read that would have walked them through the task I assigned. When I came back to check on them a few hours later they hadn't done anything and blamed their lack of work on not having access to google. They obviously didn't last the week and I'm not sure how they even got to me in the first place. This person had a degree, multiple certs. and somehow managed to pass all the tests before they got hired. They are not the first one I've had be totally useless. I don't mind training on the job (in fact, I prefer it) but I can't train someone that can't even navigate the file system or read man pages. I don't have time for that. They knew less than my Grandmother and I still have to maintain my Grandmother's desktop PC and all her other crap (Ring cameras, cell phone, tablets etc.).

The users are getting dumber but our programmers are getting much dumber in my expereince. At least those calling themselves terms like programmer/coder. This isn't unique to CS I'm seeing the same problem happening across all fields I interact with on a weekly basis. The worst thing that ever happened to CS (and the internet as a whole) was "geek chic" becoming a thing. People want to appear like they're into technology now instead of like the good old days when they used to laugh at the geeks in the computer lab. They want to make the $120k+ a year salary they heard about. But they don't want to put in the butt time to actually learn how to do this stuff as my grumpy old professor used to call it. Worse yet, I'd argue many coming into the field never had the ability to learn this stuff in the first place. But a lot of them were pushed along ("no child left behind") to fill quotas and/or to prevent making the insituations they attended look bad. I know a lot of them straight up cheated their entire way through. Which is why we don't hire graduates coming from a handful of universities anymore (mostly from overseas) since it's obvious to everyone those "accredited institutions" are simply diploma mills.

I'll even go futher and say this: The people you actually want in this field are smart enough to see the writing on the wall and know it's a bad investment in their future to join the field in the 2020s. We're probably living through the last years of real CS now. At least for the vast majority of people. Most tasks will be fully automated soon and only a very select number of people will actually understand the software running the world or be given access to those systems. The rest of us are going to get dumb terminals and we're going to rent time on the real system living in a datacenter someplace. The era of commodity hardware is coming to an end. I hope you saved all your old hardware while you still could. Soon things like consumer GPUs for playing video games locally are no longer going to be produced for the average consumer. You won't be able to buy the ones going into the datacenters either because the hardware will be considered a state secret and access to it will be extremely limited. As it stands they're already black boxes at this point anyway. They did an end-run around x86 being a relatively open platform by pushing stuff over to the blackbox GPU. IBM and friends aren't going to make the same mistake they made with x86 twice.
 
Back
Top