FreeBSD proliferation

[gofer_touch]

A question that has had be quite interested for some time, but one that I could never truly find a satisfactory answer for, was why has FreeBSD been so successful in its proliferation compared to the other BSDs?

I know that OpenBSD code has managed to to get in some pretty important corners of the web, networking infrastructure and so on. I know that NetBSD has in the past been used in scientific computing and if I recall correctly it is the basis for Apple's time capsule. DragonflyBSD has a number of very interesting technologies, but I haven't really seen wider adoption.

It is only FreeBSD that seems to have been the basis for products that are at the core of some extremely valuable companies. The list is long. What are some reasons for this besides the usual stuff like the BSD license and a relatively large and stable developer base...which other BSD's could be said to also benefit from via the wider BSD community?

 

[sidetone]

Something that stopped me from using NetBSD is my network card driver crashed boot up, until I would physically remove the card. FreeBSD has companies investing in making sure theirs, and a few other network cards work.

I read a book on OpenBSD, it said it is for people who know what they're doing, and that their community doesn't assist people who have questions on how to set up their operating system.

I tried Dragonfly, and it's differences in configuration from BSD weren't documented enough. Either I set up my network configuration wrong, or it's network abilities aren't stable. It worked, but it kept disconnecting, but I've had problems like that in FreeBSD too.

There is something I like from other BSD's plus Minix, but none of them have it all together. Eventually they will, and other BSDs and UNIXes can benefit from it too.

 

[Beastie7]

A question that has had be quite interested for some time, but one that I could never truly find a satisfactory answer for, was why has FreeBSD been so successful in its proliferation compared to the other BSDs?

I know that OpenBSD code has managed to to get in some pretty important corners of the web, networking infrastructure and so on. I know that NetBSD has in the past been used in scientific computing and if I recall correctly it is the basis for Apple's time capsule. DragonflyBSD has a number of very interesting technologies, but I haven't really seen wider adoption.

It is only FreeBSD that seems to have been the basis for products that are at the core of some extremely valuable companies. The list is long. What are some reasons for this besides the usual stuff like the BSD license and a relatively large and stable developer base...which other BSD's could be said to also benefit from via the wider BSD community?

You might want to check out George Neville-Neils "not a Linux distro" video. He addresses questions like that and explains why.

 

[gofer_touch]

You might want to check out George Neville-Neils "not a Linux distro" video. He addresses questions like that and explains why.

Quite a nice video! George Neville-Neil seems to be quite a guy. It did answer a lot of questions that had. FreeBSD is pretty awesome.

 

[Beastie7]

Yeah, he's a cool dude. I'm a huge fan of his work on networking.

 

[NewGuy]

While I agree FreeBSD is probably the most widely used on the desktop and server, I would like to point out that pieces of other BSDs are perhaps more widely spread. For example, OpenSSH is used virtually everywhere (Linux, OS X, other BSDs and it's even spreading to Windows). OpenSSH grew out of OpenBSD, though it's now more or less independent. The same applies for technologies like LibreSSL which started in OpenBSD and will probably soon be used by just about every BSD and Linux project.

So while OpenBSD as a whole might see less wide spread use than FreeBSD, I would argue pieces of it grow break off and become more popular than just about any single OS.

As to why FreeBSD is relatively so popular, I think it comes down to its wide focus. FreeBSD is a pretty good general purpose operating system and does well in a lot of areas (file systems, security, portability, documentation). Other BSDs might be better in one specific area (NetBSD is super portable, OpenBSD is very well documented and secure), but on balance I think FreeBSD offers the best combination of features, documentation, security, portability and performance.

 

[Oko]

NetBSD hopes for wider adoption have been dished around 2004-2005. Namely it was wildly speculated that NetBSD will be picked up as a based for new Google mobile OS. Unfortunately Google surprised everyone by acquiring Android, Inc. and betting its future on at that time inferior embedded OS (Linux) ridden with copy left license. After that the only serious consumer of NetBSD was Japanese company Wasabi Systems which went out of business 2009 but not before doing one great thing for humanity. Namely they released WAPBL Journaling file system (in traditional sense of that word) which is specifically tailored for embedded devices.
These days there are few smaller Japanise printer manufacturers who use NetBSD as the OS for their printer controllers. Other than that NetBSD is practically hobby project typically used for retro hardware. NetBSD annual budget is less than 30K which is far less than what other BSDs spend for electric bills alone.

DragonFly is an interesting story IMHO. Namely they have business friendly license and the most advanced file system in existence which happens to be light weight and ideally suited for desktops. I will speculate that Apple which has serious problem with the file system could port HAMMER1 or HAMMER2 to OS X. As many of you know HFS is not upper/lower case aware + plus that quick dirty hack which enables the file system which is designed for big-endian (PowerPC) to run on little-endian (Intel). HFS also suffers serious penalty performance on multi users environments as it is really designed to read and write one file at the time. HAMMER which doesn't have ZFS licensing problem could be ideal FS for Apple. Maybe they are waiting to see what happens with HAMMER2. That would be really quantum leap comparing to what other proprietary desktop vendors have to offer.

 

[fernandel]

Oko, here is the link for you:
http://www.netbsd.org/gallery/products.html

 

[Oko]

Oko, here is the link for you:
http://www.netbsd.org/gallery/products.html

Thanks for the link. I am aware of that link. I am not sure what was the purpose of pointing it out. If you look for example "Sortware products based on NetBSD" you will see half of that things are obsolete. It doesn't change my personal assessment of the state of NetBSD project. NetBSD is in sorry state more so than any other BSD

 

[Oko]

I think FreeBSD is the best of the BSDs at the minute due to its documentation and hardware support

I am not arguing with people's opinions and what they thing it is the best. However you are wrong about hardware support and documentation. OpenBSD has better documentation (man pages) than FreeBSD and equal or better hardware support with the exception of hardware RAID cards were FreeBSD has the edge. OpenBSD also works on far more architectures (over 20) than FreeBSD which until recently (significant work on ARM platform) was more or less (i386/amd64 OS only). FreeBSD focus on i386/amd64 was the original reason that many of old UNIX people like myself preferred NetBSD which worked on UNIX hardware in early nineteens.

as it stands, no other BSD works on such a wide variety of hardware so well. And while I love OpenBSD, the giant locked kernel inhibits multicore performance necessary for a lot of modern tasks and I think most of its security comes from its shipment in a locked down state.

While you are right that giant lock inhibits performance you are wrong about the role of giant lock in the security. It is true that systems without giant lock are more tricky to secure but that is not the reason that OpenBSD is so secure. There was a wonderful post by a French system admin about a year ago comparing Open and Free security. There has being a significant recent effort on removing Giant lock from OpenBSD kernel, making network truly multi-core aware and also PF. While OpenBSD might never get to the FreeBSD level of performance (FreeBSD was always about optimization) many people myself included think that taking 3% or even 5% performance penalty (I am talking about commodity hardware not high end network storage gear where Free might have significantly better performance) is worth of all that extra security. People who are casual desktop computer users are unlike to notice any serious difference between Free, Open or for that matter any other BSDs (Net, DragonFly). I would argue that from the point of casual desktop computer user lack of advanced file system like HAMMER or ZFS is by the far the most important reason why would one chose not to run OpenBSD. Finally if I need to use empirical method to compare the speed of BSDs, DF wins hands down

 

[sidetone]

Namely they have business friendly license

Not really, because of DragonFlyBSD's choice of GCC.

I may switch to DFBSD one day, but I'm less inclined to for their willingness to use GCC and other non-BSD-friendly tools in their packaging

I agree with this, but I'd use it anyways if their network drivers and configurations were reliable.

DFBSD continues to be a neat project, but I don't see eye to eye with the developers on their priorities or choices.

There's something I like from other UNIX distributions, (such as NetBSD's clean code, or DragonFLY's or Minix's hybrid kernel) but at the moment, the distributions don't work for me. Those benefits from other distributions aren't a necessity, while FreeBSD simply works better.

 

[Oko]

When I say FreeBSD has support for the widest range of hardware, I do assert my stance for x86 at least, I should have properly qualified that statement. While it does not have as many platforms, and the non-x86 ones are subpar at best, it has the widest GPU support and commercial/proprietary driver support. This is because of Nvidia's support for FreeBSD, and the presence of various proprietary drivers, some of which overlap on the other BSDs, but not to the same range. Therefore, it is most accessible to new users and easier for most users to get going and running as they like.

We can go back an forth all you want with this. The best graphics support has DragonFly thanks to hard work of French developer Francois Tigeot

closely followed by OpenBSD. FreeBSD until recently supported only VESA on modern video hardware (sorry I don't count NVidia binary blob). While I will concede that at work I have to run NVidia binary crap to be able to use Tesla K80 (CUDA on RedHat) I get paid for it. When I don't pay to use computers I would not touch a computer with NVidia binary blob even with a broomstick. FreeBSD has at best limited (an example is LSI Logic's MegaRAID controlling software) at worse non-existing vendor support (no MATLAB, Mathematica, Maple is just non-starter for an academic desktop at work). Other BSDs have zero vendors support.

I may switch to DFBSD one day, but I'm less inclined to for their willingness to use GCC and other non-BSD-friendly tools in their packaging, their outright 'we're not supporting non-x86 stance' whereas the other BSDs at least make an attempt to at some level, and its lack of support for Nvidia dealbreaks it for me, as I have a fanatical hatred for AMD due to shitty R&D and driver support and while Intel is pretty good, I still prefer the flexibility of a dedicated graphics card.

HAMMER is 64bit only file system so there is no point in supporting i386. You might as well use FreeBSD 4.8. DragonFly BSD has no packagings system. They use DPorts which are just FreeBSD adopted port three. There is a non-trivial number of FreeBSD ports which don't compile on DragonFly (collectd was a deal breaker for me). NVidia is close hardware. If you like that hardware you have to chose supported platform. FreeBSD happens to be at the moment one of those supported platforms (up to certain extend since NVIdia doesn't support CUDA programming on FreeBSD). I am not arguing about AMD. What is wrong with Intel graphics?

The performance between Free and OpenBSD on non-multithreaded applications is likely to be very similar, but anytime you need a kernel routine or syscall there's going to be a performance penalty due to giant locking. I'm not saying OpenBSD isn't practical as a desktop, but it isn't ideal for my use case, but for a server or network appliance, its worth a look. For network intensive, i'd probably use FreeBSD though.

Far enough. I will give you another reason to stick with FreeBSD. Diversity is the great enemy of the productivity in the server room. If you are running FreeBSD as a storage OS where Open has nothing to offer at the moment for more serious enterprise deployment you are more likely to stick with Free for other roles. Personally obsolete PF, lack of IPSec (in the base system), and portable OpenSSH version as oppose to native is just too much for me to swallow. I could care if FreeBSD/IPFW outperforms Open on 20 gigabit network. I like most other people don't have such network. I have mostly 1 Gigabit slowly migrating to 10 Gigabit network. In such environment Open preforms reasonably well.

DFBSD continues to be a neat project, but I don't see eye to eye with the developers on their priorities or choices.

Fair enough. Personally I am scared how few developer work on DF. If you check their mailing lists you will see far greater community enthusiasm prior to HAMMER and early in the HAMMER life. Their mailing lists are mostly silent now.

 

[Oko]

Not really, because of DragonFlyBSD's choice of GCC.
I agree with this, but I'd use it any ways if their network drivers and configurations were reliable.

DF guys tend to be very Linux tolerable I would cold heartedly agree with your second argument. I tried to use DragonFly very hard at work but between the lack of developers and "unstable" base it was terrifying experience putting my livelihood on the line. Hopefully once HAMMER(2) is finished they will stabilize the base. Until then it is a cool thing but not worthy a serious risk.

 

[kpa]

DF guys tent to be very Linux tolerable I would cold heartedly agree with your second argument. I tried to use DragonFly very hard at work but between the lack of developers and "unstable" base it was terrifying experience putting my livelihood on the line. Hopefully once HAMMER(2) is finished they will stabilize the base. Until then it is a cool thing but not worth serious risk.

I haven't really taken a look at DF recently. Is it that they are stuck with the old 4.2.1 GCC just like OpenBSD is (and FreeBSD before the CLang import) or are actually using a newer GCC as a system compiler?

 

[Oko]

I haven't really taken a look at DF recently. Is it that they are stuck with the old 4.2.1 GCC just like OpenBSD is (and FreeBSD before the CLang import) or are actually using a newer GCC as a system compiler?

DragonFly is already using GCC 5.1. as a system compiler. Check out the latest snapshots. I use 4.9.2 for developing on OpenBSD but you are right system is compiled with 4.2.1. Interestingly enough I see lots of activity with LLVM on OpenBSD these days. I would not be too surprised to see that thing becoming OpenBSD system compiler on supported platforms. My hope was that PCC will be resurrected but that effort went nowhere.

 

[gofer_touch]

Wow, this has spiralled into something quite interesting.

I tend to follow the the FreeBSD and DragonflyBSD projects. Mainly because I use NFS rather extensively and both offer high bandwidth performance and filesystems which make working with large storage systems and backups easier.

I'll chime in on a few items that I've found to differ from what has been said above.

Dragonfly's web-based documentation is dated, its true but their man pages are very good. What FreeBSD has in its corner is the fact that there are so many more users and those users have blogs, websites and BOOKS(!) which offer far more literature for the beginner up to the advanced user. This is a BIG plus for FreeBSD...its documentation is not only more widespread it is more diverse.

As far as Dragonfly going GCC 5.1, it typically has two compilers in base and the system currently does build using LLVM/Clang. I imagine they will make the switch when they are good and ready. They tend to make design and maintenance choices that are suitable for a small team (which they are). FreeBSD can probably switch to different compilers all day long and deal with the fall out more elegantly because there are far more developers and far more users which report issues (not to mention companies that also report issues and pay developers to fix things).

As far a DF being unstable, LOL!, the DF base is extremely stable. Even the master branch (i.e. current) is stable for day to day use and there are those that even run it in production environments. Any issues that are discovered are reported via the mailing lists by the team and promptly looked into. In particular there have been a lot of changes in the graphics stack, but issues get fixed quickly.

There is also an ARM64 port of Dragonfly in the works. If you have time, hang out over at EFNet #dragonflybsd, a lot of things are discussed among the developers (which all hang out there). They are easily accessible. The mailing lists are in fact usually quiet.

I've done some dabbling in OpenBSD as well. It is a REALLY nice OS, but the lack of a robust filesystem and slow NFS make it not yet ready for my needs. NetBSD has never worked for me on anything that I've tried it on and their installer does weird things.

All in all I do think all of the BSD's benefit by having the diversity that they have as well as having a "flagship BSD" which is likely to mean different things to different people.

 

[Oko]

As far a DF being unstable, LOL!, the DF base is extremely stable. Even the master branch (i.e. current) is stable for day to day use and there are those that even run it in production environments. Any issues that are discovered are reported via the mailing lists by the team and promptly looked into. In particular there have been a lot of changes in the graphics stack, but issues get fixed quickly.

I use word stable as in "stable features". I was not saying that DF code is not stable I was saying that features are so rapidly changing that if I deploy file server now I will not be sure that all features will be there year from now let alone 5-7 years down the line which is typical life of the a server. You can check their mailing list for my post about LDAP. In June of 2014 DragonFly didn't support LDAP authentication and authorization (I am not sure what is the status of Kerberos because I don't use it for authentication). In November feature simply worked and I posted howto. Now guess what? I deployed 4 FreeBSD file server prior to DF having LDAP and primarily because DF didn't have LDAP. Do you think that I will be migrating now to DF just because support was added? Similarly I had trouble monitoring DF with m/monit and more importantly collectd. collectd port fails to compile on DF more than 5 years. Yes I know that they lack the man support and that is probably trivial issue, but guess what? When you run 80 something servers it is no longer trivial issue to me and I don't have time to fix it myself.

 

[gofer_touch]

It is good to see ARM64 being worked on by DFBSD, but I'd still like to see some things in the project change. That is why, if the Apple shills (iXSystems, Jordan Hubbard etc) get their way with FreeBSD and it becomes a little brother to OS X, I'm gonna setup a fork of the last version without all of that versus jumping ship to another camp.

I'd swim the length of the Nile if you gave us FreeBSD with a version of the HAMMER filesystem!

 

[Maelstorm]

I don't think that Hammer will be in FreeBSD anytime soon. As I understand it, DFBSD was forked off FreeBSD due to differences of opinion between developers which eventually resulted in one developer's write access to the repository getting revoked. That developer then forked the FreeBSD tree into DFBSD. They still use GCC as their system compiler.

 

[Beastie7]

That is why, if the Apple shills (iXSystems, Jordan Hubbard etc) get their way with FreeBSD and it becomes a little brother to OS X

Care to elaborate on this? Shills? Any proof of such entity having power over the project? I'd really like to hear your thinking behind this sentiment.

If I did, it would be in lieu of ZFS. I like ZFS, but the CDDL encumbrance and its massive memory consumption among other things is just, too much. HAMMER would be the preferred FS for multidisk systems, but I would probably improve UFS and soft updates further as well as I have a fondness for how fast UFS is, and set it to default for single disks.

Just how is ZFS "too much" exactly? Re-inventing everything that's been well designed, useful, and battle tested for over a decade would be a complete anti-pattern.

As far as licensing is concerned, it hasn't had any detrimental effects on the project; so I'm sure all is fine.

 

[beastDemian]

I don't have proof but there's a few developers on the project and some people who pop up constantly about launchd and other Apple-derived parts that are in the open and integrating them into FreeBSD. Just google launchd FreeBSD and if you go back a bunch there's a ton of info on it. iXSystems is using launchd in FreeBSD derived projects, for what reason, I dunno because rc init still serves well enough. We don't need a full featured systemd suite or anything close to that.

You are probably talking about NextBSD ( http://www.nextbsd.org/ ). Right now it's a hobby project and a semi-fork of FreeBSD. There are so many Apple technologies that they could easily rename it "Free-OSX". Anyway, for now, most of that work is staying in that branch (heck, most of it is not even stable). I hope they keep most of their stuff in that branch and just pull the changes from FreeBSD current since the amount of shoehorning they have to do is

 

[sidetone]

FreeBSD can get influenced by any project. I'd rather that FreeBSD stay true to being BSD, and fork anything that wants to go it's own way.

At least I can say of Apple is that they released LLVM, Clang and will release Swift later this year. This actually helps FreeBSD be it's own. Some companies donate to FreeBSD and ensure their drivers work. I believe a lot of developers worked for Apple or were a former Unix system developer. It's just my opinion.

 

[beastDemian]

They have the most up-to-date port of launchd and they are the ones who (in the past) considered integrating some of their work into the main branch. What is certain is that launchd won't go into FreeBSD right now because its not stable enough (AFAIK its not even working and panics every now and then). I'm fine with launchd staying in their semi-forked FreeBSD distro.

I'm sure there will be a discussion about it in the mailing lists in due time. The problems those utilities solve are still there. Until someone comes up with another solution (better or not) FreeBSD can still "be targetted".

 

[sidetone]

FreeBSD is always influenced. FreeBSD's startup is better than Linuxism types. Why people want that over what FreeBSD has, I don't understand.

Here's what I find on Apple's Launchd https://wiki.freebsd.org/launchd . Despite worries of that being too much influence, once it works could that be better? It has a clear argument for start up times, but if it's not clearer than what FreeBSD has, I rather keep what FreeBSD has.

 

[beastDemian]

Despite worries of that being too much influence, once it works could that be better? It has a clear argument for start up times, but if it's not clearer than what FreeBSD has, I rather keep what FreeBSD has.

Depends on your definition of "better". It introduces centralized dependency managment (basically you only start programs once their dependencies are fully satisfied instead of polling all the time) it saves you procesing power (and battery life) but it brings a lot of complexity to PID1...which is supposed to be simple and easy to understand. IMO it's still better than systemd, but the problems it brings to the table are very similar.