Linus Torvalds Doesn't Recommend Using ZFS On Linux

Status
Not open for further replies.
I heard that one driving force behind systemd were embedded users. Your car audio f.e. boots up when your key comes into range so it can be "instantly on" when you open the door 3 seconds later. There, every millisecond counts.
 
I heard that one driving force behind systemd were embedded users

Its strange because I would imagine very few services would need to be started up in this use-case so perhaps no-init system (or a trivial script) would be preferred instead of even systemd.

If I had to guess. I always had the feeling that systemd made integration with GUIs much easier than freeform scripts (which is becoming very important for the modern Linux user). I.e it is much easier to change a Windows 3.1 style .ini file than to munge a shell script.
Whilst the "programmer" in me actually prefers systemd, the "UNIX aficionado" in me is deadly against it.
 
Being against improving init to gain parallelization is just as dogmatic as being against ZFS in principle because it's not the license you want.

Please don't confuse "improving init" to be "systemd is improved init"
 
ZFS is massive so I can understand why they want to pass on some of the maintenance burden to an external project

Can't blame them for that I suppose, work is work. Though it could end up being more work to keep it going with FreeBSD if the GNUtards decide to so some Linux-y things with it, pretty likely I'd say. Could be ugly.

I think the climate for FreeBSD is pro Linux right now, but I'd rather they not get too involved with it or influenced by it. Though I do need that compatibility layer. There is a specialty Linux app I use that will never be available in ports.
I heard that one driving force behind systemd were embedded users.

I have an AMLogic TV box that runs CoreELEC (Linux). I have to admit it cold boots impressively fast. Though I don't know how much I can attribute that to the skill of the CoreELEC developers (which is impressive) or to Linux and systemd itself.
 
I agree with this, even if ZoL doesn't screw up. Mostly because FreeBSD will become a second class citizen again. It will always be the one catching up.
ZoL will probably also do something weird like tie ZFS to systemd/Linux cgroups making it a big faff to maintain a portable patchkit.

I personally would prefer an "out-of-date" but rock solid "native" implementation of ZFS.

But ultimately it is up to the FreeBSD developers. ZFS is massive so I can understand why they want to pass on some of the maintenance burden to an external project.

It's not about being 2nd class. FreeBSD have reputation as a conservative OS with ultra stability, reliability and security in mind.... unlike Linux.

I think it's more sensible for FreeBSD ZFS developers to examine ZoL codes before pulling the codes into FreeBSD ZFS. I'm more interested in ZFS encryption but I wouldn't want them to pull in everything blindly because one day ZoL will really screw up and tarnish Linux reputation... not FreeBSD. I know as a developer, tweaking one thing can break 2 or 3 other things and the list goes on. Linux developers really likes to tweak things more than necessary.
 
He did not even recognized the difference between Oracle ZFS and OpenZFS ...
I'm not 100% certain, but I believe he was referring to the licensing. OpenZFS is released under the CDDL, a GPL incompatible licence, as with OpenSolaris, so it can't be combined with the GPL v2 Linux kernel. I believe that only Oracle could change the licence?
 
OpenZFS is released under the CDDL, a GPL incompatible licence, as with OpenSolaris, so it can't be combined with the GPL v2 Linux kernel. I believe that only Oracle could change the licence?
Bad reasoning ... CDDL is perfectly ok with GPL, its the GPL that is not compatible with CDDL.
 
1) Don’t conflate Linux developers and ZFS on Linux developers.

2) the OpenZFS plan going forward is to have FreeBSD, MacOS, Linux, and any other supported platforms to all be hosted in the same OpenZFS repository (renamed from ZFS on Linux.) This will include test suites running against FreeBSD such that any breakage is detected early (when it is comparatively much easier to address), and will make it much easier for different platforms to have the same (or at least much more similar) feature sets.

I’ve seen zero reason to be concerned with ZoL in general (as a long time user on both platforms, and OpenSolaris before that) ... yes, there have been some small bumps, typically due to variances across different Linux distributions handling of kernel upgrades, but those are exactly the portions of the code base that FreeBSD’s ZoL won’t be utilizing.
 
I'm somehow caught in a quantum state of believing and not believing they had no test suite on Linux for this. But well, gcc and cygnus comes to mind again.
 
I'm somehow caught in a quantum state of believing and not believing they had no test suite on Linux for this. But well, gcc and cygnus comes to mind again.

I did not say there was "no test suite for Linux", I'm saying they will be adding a FreeBSD buildbot to perform automated checks. Or were you referring to the kernel upgrade issues? That's a thorny one to try to have perfect testing for, since so much depends on the history of the system and when the operator chooses to apply updates. But that's not a FreeBSD discussion, so I will leave it at that.

The whole presentation is worth reviewing or watching. Slide 14 has the bulk of the information:
  • Lots of work to reorganize ZoL code to abstract out the Linux-specific bits
  • FreeBSD uses a different compiler (llvm, not gcc)
  • Trying to do things “the right way”, fix linuxism’s that have crept in
    • E.g. renaming functions/types that conflict with OS-provided ones rather than abusing the preprocessor
  • WIP code is functional and runs test suite on FreeBSD
  • Work being lead by iXsystems engineers
    • 60+ PR’s already landed, ~10 currently open
    • ~50,000 lines of code yet to land (mostly FreeBSD-specific)
  • Lots of upstreaming and code review still needed
  • FreeBSD buildbot integration in progress
 
From my limited understanding the big complaint with init is that it can't load processes in parallel. Seems to me like a small justification for replacing it with something largely more complicated, more monolithic, and more prone to issues (aside from its poorly written code). So what's the big advantage, some milliseconds saved at boot?
This complaint about the traditional init is old, and correct. There have been research/hacking papers presented at conferences as early as the late 90s or early 2000s that demonstrated that a parallel init can be much faster. Often the saving is 3x or something like that. Although sometimes the difference sometimes ends up being minor, because with spinning disks, system bootup may be disk limited, with the disk nearly continuously busy with random seeks. There was an interesting study (I think a PhD thesis) about how to improve that by grouping files on disk by access order during boot, which also demonstrated great improvement. And in those days, it often took 5 minutes for a system to boot (in particular a server with many services), so a speedup by a factor of 3 is VERY significant.

Today with SSDs (very little seek time), the landscape has changed: During booting, a system is no longer purely random IO limited. Yet, many init tasks today end up being network limited, and those latencies still exist. On the other hand, machines today hav multiple cores, so having parallel tasks during init is both less important and more important than it used to be.

So the important thing is: faster init is not "milliseconds" or "seconds", it can amount to several minutes, and it can be the long pole in the tent for recovery time. This stuff matters in many environments.

There are way more use cases for fast booting than just embedded computing. One particularly near to my heart is kernel development: for lack of good debugging techniques, one often ends up crashing the kernel, collecting a dump, rebooting, analyzing the dump, and trying again. In the old days (say ~98), that was often a 5-10 minute cycle. In one particular example, a colleague and me got that down to 27 seconds without hardware changes, and the difference in productivity is very significant. Another important reason for fast boot times is asynchronous servers: If a web page takes 30 seconds to load, many users (in particular internal users) will be mildly upset, but not walk away; if a web page takes 10 minutes to load, users will change their workflow, and productivity is disrupted. Having really fast boot times often makes it unnecessary to have redundant servers or reliable hardware (for example for planned maintenance), if instead one can crash the server and it comes back up really fast. On particular example of great cost savings: handling power outages. It used to be that serious computers had both UPSes (to handle short power outages) and diesel generators, which kick in after a minute of outage. Today, with fast boot times and better generators, we can often get rid of the UPSes: if the power fails, the server just crashes, within 10 seconds the diesel is up, and another 30 seconds later the server is back online. And the cost saving of that is large, since UPSes (with their short-lived batteries and regular battery maintenance) are expensive. Obviously, this requires looking at the whole system; having a UEFI BIOS that takes 3 minutes to initialize the motherboard, or needing to run fsck for 5 minutes does not work in such an environment.

Disclaimer: I have no idea whether fast boot times are actually what drove Lennart to work on systemd; I've never asked him for his motivation.
 
I find a lot of init system comparisons are flat wrong. So many people think all BSDs use the system V init instead of rc. 1998 was a long time ago. This is a worthwhile topic for another thread.
 
Being against improving init to gain parallelization is just as dogmatic as being against ZFS in principle because it's not the license you want.

Please don't confuse "improving init" to be "systemd is improved init"
How is parallelization improving init? It brings with it its own set of issues and complications. Speed is not everything, but stability and being able to boot consistently is.
You might have wanted to parallel run init services back in the 90s or early 2000s but now, with SSDs and NVMe SDs, who cares.
 
Thank you for the link. I think this is a very informative article.

It shows the deep-seated spite ingrained in a lot of (if not most) Linux kernel developers for any license that isn't GPL. It shows why FreeBSD being too chummy with Linux is like playing with a hand-grenade and a rusted pin.
I guess this is all fostered by Torvalds who has always been a "my way or the highway" authoritarian "leader".
 
There are arguments that Linux will mess things up, and how Linus' opinions are dubious, counter to that Linus rejects ZFS. He's kind of doing FreeBSD a favor.
===
One problem with any init is when a program like ntpd stalls because it can't find an internet connection.
 
That article was pretty interesting. There's a lot more to OpenZFS and operation with various systems than I thought. Though why the change of name to ZFS on Linux, why not just stick with OpenZFS?

Having a late model laptop that runs 12 logical cores with NVMe drives I can say it does everything ridiculously fast even though it runs Windows. So I don't know, is the old sequential loading init system going to be a problem or even a consideration for FreeBSD down the line?

My last FreeBSD system seemed to boot pretty quick and that was an old system I retired recently. The new system I'm putting together for FreeBSD will have all the latest stuff like 16 logical cores and NVMe drives. Will be interesting to see how that does in terms of boot speeds.

I have a lot of trepidation in seeing any changes to the init system since I'm used to and comfortable with what FreeBSD has been using forever. If they do go to a parallel loading init system down the road I just hope they do it right. Ideally it should be transparent to me in terms of configuration and administration, you know principle of least astonishment and all.
 
The dog barks but the caravan goes © proverb
On the one hand seems that ZFS interferes with some plans RedHat-IBM own filesystems , because Linus earn their money from RedHat. What says Linus Torvalds - that says RedHat-IBM.
On the other hand Canonical has own plans for ZFS - ZFS native root includes to Ubuntu 19.10 and how I understand ZFS native root will be include to Ubuntu 20.04 LTS.
I think that life will take its toll and ZFS in the near future will be aviability natively root install in different Linux distro.
In my opinion Linus Torvalds - a bloated dummy, bloated by RedHat and crazy Richard Stallman and hysteria 1990s with FreeSoftware, which stretched out the lucky ticket during a lawsuit against BSD in the 1990s.
 
Having a late model laptop that runs 12 logical cores with NVMe drives I can say it does everything ridiculously fast even though it runs Windows. So I don't know, is the old sequential loading init system going to be a problem or even a consideration for FreeBSD down the line?
Sorry to be blunt, but your experience on a laptop is irrelevant. A very large fraction of all computers in the world are either embedded or servers. Actual human interface machines are a small minority. Is faster booting relevant? Yes for embedded systems, where you want to come up fast (for example because you are acquiring data, or a human is waiting to press buttons on his dishwasher), and you don't have the luxury of running redundant servers (so while one server boots, the other is still serving). It is also highly relevant for servers, because (a) it greatly improves the effective uptime of servers, (b) it may allow one to get rid of the need for redundant backup server of battery-powered UPSes.

On the one hand seems that ZFS interferes with some plans RedHat-IBM own filesystems , because Linus earn their money from RedHat. What says Linus Torvalds - that says RedHat-IBM.
I would not assume that Linus says things because his employer wants him to do that. He's always been quite independent of where his paycheck comes from. Whic doesn't mean that he's always right, nor that he is easy to interact with.

In my opinion Linus Torvalds - a bloated dummy, ...
I strongly disagree. He's smart, and knowledgable about many things. But that doesn't imply that his public pronouncements are always correct, because he can be quite misguided. And the way he interacts with people in the Linux and kernel world doesn't always reflect his private style.
 
How is parallelization improving init? [..] who cares.

Everything operating at scale, including but not being limited to virtualization, containers, (micro)services, infrastructure and paid compute. Shaving off a full second on the init translates to getting back years of wasted compute on a daily basis. Servers these days are virtual --like apps-- and paying customers want to be able to (re)configure and start them quickly.

That said, if your only concern is your laptop I don't think parallel service startup is all that important.
 
Everything operating at scale, including but not being limited to virtualization, containers, (micro)services, infrastructure and paid compute. Shaving off a full second on the init translates to getting back years of wasted compute on a daily basis. Servers these days are virtual --like apps-- and paying customers want to be able to (re)configure and start them quickly.

That said, if your only concern is your laptop I don't think parallel service startup is all that important.

And so you have OSs rebooting every second of every day inside these VMs? Because that's what it would take to get years back from a second saved rebooting. This seems like a spurious argument to me. Paying customers paying per second for a server is unheard of (at least by me).

Don't get me wrong, parallelization of an init has some merit, some times, BUT it has a big down side. It requires synchronization. If the market Linux chases needs this, then more power to them. My argument is, and always will be, that for the likes of FreeBSD they should avoid it at all costs.
 
Status
Not open for further replies.
Back
Top