FreeBSD vs. Illumos

The biggest reason I chose FreeBSD over illumos was hardware support and steam gaming. I sometimes like to play some hitman 2 and metal gear phantom pain sometimes metro exodus as well. So I went with a system where I had that option available.

Edit: I came to FreeBSD from Linux. I was in the market for a replacement system. So I may not be the best person to actually comment relevant to what you're needing. But it is what it is.
 
_al said:
Has anyone tried bhyve on OmniOS?
Click to expand...

not on omniOS but on smartOS. We've been running bhyve VMs on it from right after it was available. bhyve has *much* better performance and less overhead than KVM, the VMs (especially windows guests) are much more responsive even with the exact same VM ressources (vCPU/RAM)

We are slowly replacing our smartOS hosts with FreeBSD though, because it better fits my workflow and offers some advantages for our small-ish infrastructure (i.e. more flexible host configuration/usage compared to an immutable smartOS image).

As for noteable differences/advantages: service management and networking on illumos are vastly different in logic and configuration approach. SMF is quite a mighty beast; same goes for the network stack and its virtualization capabilities. Of course this comes with a learning curve, and not everything might be "better" - but the majority depends on personal preferences.

illumos/solaris has been built with the strict mantra of "always production safe and observable", so illumos is fantastic in terms of integration and interoperability of tools and underlying architecture. There are lots of tools to analyze system behaviour and if those are exhausted, you still have dtrace (which is also available on FreeBSD).
Also the manpages are exceptionally good, especially because *everything* has examples. FreeBSD is close, but given that a lot of GNU/Linux stuff is creeping into the base system, some spots are not so well (e.g. wireguard, which has completely wrong and useless manpages on FreeBSD).


But at the end of the day it all depends on your requirements and what you are more comfortable with. E.g. SMF is rather opaque and can be quite a PITA, rc is plain, simple and easily observable (just add 'echo' statements to the scripts...). Both work very well and reliably for what they are designed to do, so it's all down to requirements and preferences - as for the rest of the OS.
 
sko said:
not on omniOS but on smartOS. We've been running bhyve VMs on it from right after it was available. bhyve has *much* better performance and less overhead than KVM, the VMs (especially windows guests) are much more responsive even with the exact same VM ressources (vCPU/RAM)

We are slowly replacing our smartOS hosts with FreeBSD though, because it better fits my workflow and offers some advantages for our small-ish infrastructure (i.e. more flexible host configuration/usage compared to an immutable smartOS image).

As for noteable differences/advantages: service management and networking on illumos are vastly different in logic and configuration approach. SMF is quite a mighty beast; same goes for the network stack and its virtualization capabilities. Of course this comes with a learning curve, and not everything might be "better" - but the majority depends on personal preferences.

illumos/solaris has been built with the strict mantra of "always production safe and observable", so illumos is fantastic in terms of integration and interoperability of tools and underlying architecture. There are lots of tools to analyze system behaviour and if those are exhausted, you still have dtrace (which is also available on FreeBSD).
Also the manpages are exceptionally good, especially because *everything* has examples. FreeBSD is close, but given that a lot of GNU/Linux stuff is creeping into the base system, some spots are not so well (e.g. wireguard, which has completely wrong and useless manpages on FreeBSD).


But at the end of the day it all depends on your requirements and what you are more comfortable with. E.g. SMF is rather opaque and can be quite a PITA, rc is plain, simple and easily observable (just add 'echo' statements to the scripts...). Both work very well and reliably for what they are designed to do, so it's all down to requirements and preferences - as for the rest of the OS.
Click to expand...
Hello,
Why didn't you use Xen?
 
FreeBSD is better integrated and the documentation is also better.

Weird things I found on both OpenIndiana & OmniOS.
  • A lot of binaries are 32-bits installed non-stripped. I wonder if they work at all with 64-bit file offsets.
    Bash: 
    $ file /bin/ls
/bin/ls:        ELF 32-bit LSB executable 80386 Version 1, dynamically linked, not stripped, no debugging information available
  • Solaris make sucks. I have to put /usr/gnu/bin first in the PATH.
  • I still can't find an authoritative source on how to compile kernel & world.
  • They still import closed binaries in the system.
But on the other hand:
  • I have the feeling that the package management is somehow better.
  • Same with services.
 
Zones management is unmatched. It takes only few keystrokes and few moments to create new zone. On the other hand, I played only once with jails, but I found it cumbersome and time consuming to manage. Another nice-to-have for Solaris/illumos is SMF. With built-in converter for init-style services, adding own services to it, is a piece of cake.
 
homeadm said:
Zones management is unmatched. It takes only few keystrokes and few moments to create new zone. On the other hand, I played only once with jails, but I found it cumbersome and time consuming to manage. Another nice-to-have for Solaris/illumos is SMF. With built-in converter for init-style services, adding own services to it, is a piece of cake.
Click to expand...

I completely agree. Other goodies I wish FreeBSD imported (other than ZFS, and Dtrace) would be zoneadmd, zoneschedd, and SMF. Maybe throw in Doors IPC too. They made multi-tenancy ("cloud-ish") based administration a breeze. SMF even allowed something called "soft-reboots"; where you could restart the entire userland without touching the kernel; a great feature for troubleshooting IMO (especially for desktops). I remember watching an SMF demo from a Google TechTalk sometime ago and the backwards compatibility it had with rc-init style services was pretty awesome; and it was all built-in. Solaris 10 was really ahead of it's time.
 
Beastie7 said:
… SMF even allowed something called "soft-reboots"; where you could restart the entire userland without touching the kernel; …
Click to expand...

Does that differ significantly from the effect of reboot -r (alone) in FreeBSD?

From reboot(8), with added emphasis:

The system kills all processes, unmounts all filesystems, mounts the new root filesystem, and begins the usual startup sequence. After changing vfs.root.mountfrom with kenv(1), reboot -r can be used to change the root filesystem while preserving kernel state. This requires the tmpfs(5) kernel module to be loaded because init(8) needs a place to store itself after the old root is unmounted, but before the new root is in place.
Click to expand...
 
homeadm said:
Zones management is unmatched. It takes only few keystrokes and few moments to create new zone. On the other hand, I played only once with jails, but I found it cumbersome and time consuming to manage. Another nice-to-have for Solaris/illumos is SMF. With built-in converter for init-style services, adding own services to it, is a piece of cake.
Click to expand...

I completely agree with you. I used to have OpenSolaris/OpenIndiana servers and their zone management and zfs are quite superior. However, it lacks software support which often times I have to compile from source code. When Oracle decided to end OpenSolaris and many original developers left.

I switched to FreeBSD because of Jail, ZFS and most important of all... packages or ports are easily available to install and use. If Solaris had good software support then I would have stayed with it but Oracle killed it. Current Solaris isn't that great anymore like it used to and it hasn't changed much.
 
homeadm said:
Zones management is unmatched
Click to expand...
I've read this type of comment a lot, so I suppose it must be something.
I know nothing about Solaris, SmartOS, illumOS so I searched and found few information and also this doc from solaris , I took a quick look at it and it sounds like a jail/container thing.
In which way it is better than jail exactly?
It is a real question though, I am not trying to start a war or something.
May be I should give it a try in VM to get an answer but if one of you could give an idea that would be great, thank you.
 
gotnull said:
I've read this type of comment a lot, so I suppose it must be something.
I know nothing about Solaris, SmartOS, illumOS so I searched and found few information and also this doc from solaris , I took a quick look at it and it sounds like a jail/container thing.
In which way it is better than jail exactly?
It is a real question though, I am not trying to start a war or something.
May be I should give it a try in VM to get an answer but if one of you could give an idea that would be great, thank you.
Click to expand...

Solaris Zone management doesn't need additional software to create zones, similar to FreeBSD jails. However, Solaris zones offer better containment and security because they don't share system resources with other zones. In contrast, FreeBSD jails, while improved with VNET, may face challenges when running two web servers on the same port in separate jails.

Although I like Solaris, my main concern is compiling third-party software like PHP, Python, and web servers, and the limited software support. While OpenIndiana has a package repository, it's been a while since I've explored it. I might revisit OpenIndiana to see if there have been any improvements.

On the other hand, FreeBSD has been reliable for me, with no security issues in jails when properly configured. It gets the job done well, and security is manageable with careful configuration. Solaris, known for its robust security like Fort Knox armed to the tooth, is often used by financial institutions.
 
I am a Solaris user, so my opinion may be slightly biased.

To create a zone, you need:
- open zonecfg tool,
- set path to the zone (it's one-liner),
- few one-liners to add and configure net (address, gateway etc.),
- another few one-liners to mount external directory to the zone,
- commit & exit,
- run zoneadm install and wait, let's say, 30 seconds (for sparse-root zones).

If you know zonecfg and zoneadm, you can have sparse-root zone ready in less than a minute.

Regarding FreeBSD, forgive me if I made something incorrect or my knowledge is outdated. But when I was testing jails some time ago, I was forced to:
- install sources of everything,
- create a jail by compiling entire FreeBSD system onto it,
- wait, eat lunch, still wait.

I found jails management tools counterintuitive and difficult to learn. I am sure that after some time, one can be as good with managing jails as with zones. However, jails have key disadvantages compared to zones:

1) To create a jail, you need to compile from sources. I don't understand why compile it, if all files are already on disk?

2) I haven't been able to find anything like a "sparse-root jail". A jail is complete FreeBSD system, which is what makes deployment process so slow. In contrast, sparse-root Solaris zone, inherits most of files from global zone ("mounts" live system directories from the real OS) and only takes up about 150 MB of disk space. This makes deployment of multiple zones quick and pleasant. And no compilation!

Sparse-root zones were removed in Solaris 11, but I can't find similar information for illumos, so I expect they're available there.

ps.: Very interesting topic. I know that future of Solaris user is illumos or FreeBSD. Certainly not Linux!
 
I don't know where you get your information from but you don't need to compile anything from source. you can have a jail up and running in 30 seconds(from scratch). a minimal(thin/sparse-root) jail is ~65 MB. every jail management tool out there can create thick or thin jails or you can do it manually yerself.
 
I'm intrigued by Illumos/OmniOS' kernelbased SMB server that fully integrates NFS4 ACL. I have never used it, but it seems that you can start a SMB server just by setting zfs sharesmb=on. Some claim better performance because it is integrated at kernel level.
 
suntzu00 said:
I don't know where you get your information from but you don't need to compile anything from source. you can have a jail up and running in 30 seconds(from scratch). a minimal(thin/sparse-root) jail is ~65 MB. every jail management tool out there can create thick or thin jails or you can do it manually yerself.
Click to expand...

Therefore, features of both technologies are on par. I definitely did something wrong, or jails have matured since then. I played with them many years ago, probably during FreeBSD 7 era. Time to get familiar with jails again! Thank you for updating my knowledge, and please ignore my earlier post.

blazingice said:
it seems that you can start a SMB server just by setting zfs sharesmb=on
Click to expand...

You are correct, that's the way to start sharing a dataset. It's in Solaris 11, too. Main difference from Samba (or MS Windows) is that shares are per dataset, not traversable between mountpoints.
 
It has been a lot said about advantages of Solaris. Even Solaris ZFS implementation avoids double buffering as in FreeBSD, but the eco-system is limited too much. When migrated from Linux (Jan 2022) I've made a check list to choose between the two. And if we speak about workstation, there is no port of Google Chrome for Solaris and that's a deal breaker. Also it's package manager sucks. Also you can run FreeBSD both on hosting and at home. So FreeBSD is a compromise between Linux that develops chaotically so much, that it doesn't make sense for me as a user. FreeBSD is kinda a patchwork of different techs (3 firewalls?! are you kidding me?!), but with that it manages to be the same i used to use for the last 20 years. And I miss an old closed source gtk2 software like Nero burner and Adobe reader. I like when things just work as before for a long time.
 
crypt47 said:
It has been a lot said about advantages of Solaris. Even Solaris ZFS implementation avoids double buffering as in FreeBSD, but the eco-system is limited too much. When migrated from Linux (Jan 2022) I've made a check list to choose between the two. And if we speak about workstation, there is no port of Google Chrome for Solaris and that's a deal breaker. Also it's package manager sucks. Also you can run FreeBSD both on hosting and at home. So FreeBSD is a compromise between Linux that develops chaotically so much, that it doesn't make sense for me as a user. FreeBSD is kinda a patchwork of different techs (3 firewalls?! are you kidding me?!), but with that it manages to be the same i used to use for the last 20 years. And I miss an old closed source gtk2 software like Nero burner and Adobe reader. I like when things just work as before for a long time.
Click to expand...

That's why I prefer using macOS on my computer and FreeBSD for my server. I've had trouble keeping FreeBSD stable when using it as a desktop, mainly due to issues with package updates and the numerous packages it uses, which sometimes cause things to break. I switched from Microsoft Windows a while back in 2003 and have no interest in going back. However, I'm not happy with the direction Apple is taking macOS, making it more locked down and resembling iOS, which I don't like. Strange enough having a macOS desktop, I use an Android phone – go figure! :D I'm considering OpenBSD as an alternative for a UNIX desktop because they conduct thorough audits to ensure everything works as intended. While Linux is nice, as you mentioned, it develops chaotically, and it's not purely UNIX, so I'm avoiding it too.

The only thing that bothers me about FreeBSD is they dropped the original Solaris ZFS and adopted OpenZFS which are developed mainly by Linux developers. My concern is the stability of OpenZFS's data integrity as if a bug is introduced which could compromise the dataset. Solaris ZFS is extremely stable and there is no reason to adopt OpenZFS. Why fix things when it ain't broken.
 
Remington said:
The only thing that bothers me about FreeBSD is they dropped the original Solaris ZFS and adopted OpenZFS which are developed mainly by Linux developers. My concern is the stability of OpenZFS's data integrity as if a bug is introduced which could compromise the dataset. Solaris ZFS is extremely stable and there is no reason to adopt OpenZFS. Why fix things when it ain't broken.
Click to expand...

The recent corruption bug ws debugged and fixed by a FreeBSD person. OpenZFS has a lot more person-power than Solaris.
 
cracauer@ said:
The recent corruption bug ws debugged and fixed by a FreeBSD person. OpenZFS has a lot more person-power than Solaris.
Click to expand...

Exactly. That tells you the quality of Linux developer's coding. FreeBSD developer was able to catch it but how many more are not caught? It only takes one bug to destroy the entire dataset if Linux developers don't take the time to test and audit the updated codes.

I don't want to criticize Linux developers, but some of them tend to hurry and introduce new features without thorough testing. In Linux, frequent updates can be a bit chaotic and may lead to breakages, but that's generally accepted. On the other hand, FreeBSD prioritizes security and stability, so adopting the fast-paced approach of Linux might not be suitable. Personally, I don't want FreeBSD to become more similar to Linux in that aspect.
 
Thank you for the detailed answers Remington homeadm

I obviously don't know this Solaris world but it seems interesting, considering the fact that I've read countless times how this OS was/is great I needed to ask why at some point.
I've looked into IllumOS Tribblix and more specifically OmniOS website and read a bit about them, BTW the mailing list over there looks slick.
These are pure OS server that can explain why some like it, and indeed zones look fun, there is also LX Branded Zone which is LX Linux container so I would not be surprised if people consider those as an alternative to FreeBSD for a NAS usage only.
I found a YT channel with related content and read some blog posts too, looks like an active community, that's good.
Hopefully I'll give OmniOS/Tribblix a try one day, at least in VM to see how it goes.
 
Remington said:
Exactly. That tells you the quality of Linux developer's coding. FreeBSD developer was able to catch it but how many more are not caught? It only takes one bug to destroy the entire dataset if Linux developers don't take the time to test and audit the updated codes.
Click to expand...

No, the bug has been in ZFS forever, before ZFS even ran on FreeBSD. Nobody in OpenZFS at fault.
 
You must log in or register to reply here.
Back
Top