ZFS Porting ZFS

B

balanga

I assume that because FreeBSD uses ZFS, the source must be available and could be recompiled if someone wanted to do so. So, if someone wanted to port it to a new platform, the source would be readily available and it would mainly mean changing the parts that are FreeBSD specific...
 
You should accept right now, though, that porting ZFS to another platform is a monumental task that no one person could possibly do without a lot of persistence and dedication. The FreeBSD developers started porting ZFS ten years ago, and it took several years before support for it was no longer considered "experimental."

Filesystems are handled at the kernel level, so you'd have to have a pretty good understanding of how that new platform's kernel works in order for ZFS to do anything.
 
Look at efforts like IBM/Oracle/Linux BtrFS, or Stony Brook's BetrFS, or Lustre, or any other kernel-based file system to get a flavor for how much work it would be: Many many man-years. Even ReiserFS (which was particularly mean and lean in staff, until Hans Reiser vanished into the penal system) had a staff of a dozen engineers. The major commercial file systems out there are developed and supported by groups of many dozen to a few hundred people (and that includes Linux' file systems, remember that RedHat, IBM, HP, Oracle, ... all have significant Linux staff).

Porting ZFS to a new OS would probably take a half dozen or dozen very experienced people a year or two.

But then, what OS would you want to port it to? FreeBSD already has it. Several other BSD variants (MacOS, NetBSD, ...) have ports in various states of support and disrepair. Given the similarity of the BSD kernels, porting to other BSD flavors (such as OpenBSD) would be relatively easy (much easier than the estimate above), but the OpenBSD community has rejected that idea, last I checked. Linux obviously has it, as does Solaris. HP-UX, OpenVMS and AIX are no longer relevant to new development. While zOS = MVS does not have it, you can run Linux with ZFS on the same hardware (and the same argument can be made for AS-400, HP-3000, Tandem, and similar platforms). The only platform that's left is Windows, and there development would have to be done by the good people in Redmond and Kirkland; they seem to think that it is not a good investment (otherwise they would have started).

Many people in storage and file system land have dreamed of "one file system to rule the world". It hasn't worked yet, and I doubt it ever will. With the change of workloads towards much of the storage being used in cloud and big data installations, traditional single-host file systems are becoming much less relevant, and are no longer a particularly good place to invest in.
 
ralphbsz said:
Look at efforts like IBM/Oracle/Linux BtrFS, or Stony Brook's BetrFS, or Lustre, or any other kernel-based file system to get a flavor for how much work it would be: Many many man-years. Even ReiserFS (which was particularly mean and lean in staff, until Hans Reiser vanished into the penal system) had a staff of a dozen engineers. The major commercial file systems out there are developed and supported by groups of many dozen to a few hundred people (and that includes Linux' file systems, remember that RedHat, IBM, HP, Oracle, ... all have significant Linux staff).

Porting ZFS to a new OS would probably take a half dozen or dozen very experienced people a year or two.

But then, what OS would you want to port it to? FreeBSD already has it. Several other BSD variants (MacOS, NetBSD, ...) have ports in various states of support and disrepair. Given the similarity of the BSD kernels, porting to other BSD flavors (such as OpenBSD) would be relatively easy (much easier than the estimate above), but the OpenBSD community has rejected that idea, last I checked. Linux obviously has it, as does Solaris. HP-UX, OpenVMS and AIX are no longer relevant to new development. While zOS = MVS does not have it, you can run Linux with ZFS on the same hardware (and the same argument can be made for AS-400, HP-3000, Tandem, and similar platforms). The only platform that's left is Windows, and there development would have to be done by the good people in Redmond and Kirkland; they seem to think that it is not a good investment (otherwise they would have started).

Many people in storage and file system land have dreamed of "one file system to rule the world". It hasn't worked yet, and I doubt it ever will. With the change of workloads towards much of the storage being used in cloud and big data installations, traditional single-host file systems are becoming much less relevant, and are no longer a particularly good place to invest in.
Click to expand...

This is eye watering, I knew that it took a ton of resources to develop a filesystem but this is serious business. One wonders how a single person could then pull off the HAMMER filesystem.
 
Hammer has probably 5 to 6 man-years of development into it at this point, mostly by one guy. An aspect that made it easier: He didn't have to be on-disk format compatible with existing content or existing OSes. But Hammer is not ready for high-volume commercially supportable production in the same way that ZFS is. That is not meant as criticism; it simply underscores the point that getting a file system to that level of feature checklist and maturity takes a lot of resources. Rather on the contrary: Matt Dillon is a hero to get his OS and his FS to this point; but he doesn't provide competition to the likes of ext4, ZFS, GPFS, XFS, VxFS, Lustre, (and ReiserFS when it still had a development group).
 
ANOKNUSA said:
You should accept right now, though, that porting ZFS to another platform is a monumental task that no one person could possibly do without a lot of persistence and dedication. The FreeBSD developers started porting ZFS ten years ago, and it took several years before support for it was no longer considered "experimental."

Filesystems are handled at the kernel level, so you'd have to have a pretty good understanding of how that new platform's kernel works in order for ZFS to do anything.
Click to expand...

ZFSv6 was ported to FreeBSD 7.x by a single developer in under a week, thanks to the power of GEOM. The number of developers working on ZFS on FreeBSD is probably still in the single digits, and we still get features pretty much as soon as they hit OpenZFS repos, and they submit many fixes and features upstream as well.

It really depends on the storage stack of the OS it's being ported to. If it's well laid out, layered, and abstracted like FreeBSD, then it's a fairly simple process. If it's a mess like on Linux, then it will take longer. Just compare the process for getting ZFS working on those two OSes. :)

ZFS is a complex storage beast, but not it's not super duper hard to port to new systems.
 
pheonix: Yeah, I shouldn't have used such absolute language. Poor choice on my part. But I stick by the "serious persistence and dedication" part. ;) Of course it's possible to port low-level stuff from one operating system to another, and the variables that determine how difficult and time-consuming that will be different from one piece of software to the next. But porting something like a filesystem requires more knowledge, more skill, more effort, more care, perhaps more raw talent, and certainly more long-term dedication than porting something more high-level like, say, Docker, which is something that's also attracted a fair bit of attention but hasn't worked out nearly as well as ZFS did.

This is just a hunch, but I would bet that one major reason ZFS got ported to FreeBSD so quickly (aside from how awesome GEOM is), and the various kinks in porting it got worked out enough to give us what we have today, is that ZFS is an outstanding collection of software. It is something that is interesting enough to attract a programmer with exceptional skill and was valuable enough to that programmer for them to stick with it for years after the initial porting was done. Programmers with less experience and skill could also probably do it, and possibly even do it quickly. But I'd wager it would take considerably more time and effort for a programmer of average skill to do it right.
 
You must log in or register to reply here.
Back
Top