FreeBSD as storageserver using zfs?!

swemattias

New Member

Reaction score: 2
Messages: 7

I got an idea a few weeks back, I want to "upgrade" my storage-server, från Ubuntu Server 19.04, with Kernal 5.x, using 8 2Tb disks in RAID-6 with XFS.
Hardware is an AMD Ryzen 2200G, 16 GB of non-ECC RAM (please don't comment this, I know), Asus TUF board, with two PCI-e cards, one extra NIC and one RAID/SATA-port. All in all I got 10 SATA 6 GB/s ports.

My goal is:
Dual SSDs in RAID-1 for the system.
2 pools with 4 2 TB disks each using RaidZ2.

The things I wonder about is:
- Does FreeBSD play nice with AMD Ryzen?
- Does FreeBSD come with zfs, particularly v0.8.2?

Do you see anything worth mentioning about what I am about to do?
 

roccobaroccoSC

Aspiring Daemon

Reaction score: 163
Messages: 651

I got an idea a few weeks back, I want to "upgrade" my storage-server, från Ubuntu Server 19.04, with Kernal 5.x, using 8 2Tb disks in RAID-6 with XFS.
Hardware is an AMD Ryzen 2200G, 16 GB of non-ECC RAM (please don't comment this, I know), Asus TUF board, with two PCI-e cards, one extra NIC and one RAID/SATA-port. All in all I got 10 SATA 6 GB/s ports.
Nice hardware!

- Does FreeBSD play nice with AMD Ryzen?
AFAIK - yes. A friend has a Ryzen system and it seems to work fine. I use Threadripper and it works well, except for sound. Sometimes the whole system loses sound and playing audio or video is not possible until restart. I have not been able to track down the cause :(.

- Does FreeBSD come with zfs, particularly v0.8.2?
FreeBSD does come with zfs. I don't know how to check the version though. Why do you need 0.8.2 specifically? In the man page of zpool it's written that the pool version is independent from the ZFS software version.

Do you see anything worth mentioning about what I am about to do?
No, it seems just fine for storage. It depends on your concrete requirements though.
 

bjs

Member

Reaction score: 42
Messages: 98

If you are not planning on the box doing anything else but being a file server, have a look at XigmaNAS or FreeNAS, although the people over at FreeNAS will give you a hard time about the non-EEC RAM... They both are based on FreeBSD... Personally, I use XigmaNAS...
 

roccobaroccoSC

Aspiring Daemon

Reaction score: 163
Messages: 651

If you are not planning on the box doing anything else but being a file server, have a look at XigmaNAS or FreeNAS, although the people over at FreeNAS will give you a hard time about the non-EEC RAM... They both are based on FreeBSD... Personally, I use XigmaNAS...
A friend used FreeNAS but dropped it for FreeBSD. Upgrading FreeNAS was quite a hassle at the time, and their kernel was a bit older than it should have been.
 

bjs

Member

Reaction score: 42
Messages: 98

I haven't used FreeNAS since it was bought by iXSystems so I don't know much about it... However, it doesn't surprise me... XigmaNAS is using current kernels...
 

xtaz

Well-Known Member

Reaction score: 133
Messages: 461

FreeBSD has traditionally used OpenZFS from Illumos and so the version numbering from ZFSonLinux is irrelevant. You only need to care about the pool versions to know if they are compatible or not.

That said, FreeBSD is ditching this and moving to ZFSonLinux itself. This is likely to be the default version in FreeBSD 13.0, but I don't know which version it's currently based on. Support was added back in July. https://github.com/zfsonlinux/zfs/pull/8987
 
OP
swemattias

swemattias

New Member

Reaction score: 2
Messages: 7

If you are not planning on the box doing anything else but being a file server, have a look at XigmaNAS or FreeNAS, although the people over at FreeNAS will give you a hard time about the non-EEC RAM... They both are based on FreeBSD... Personally, I use XigmaNAS...
Tell me about it! When I built the current server I was thinking fo using FreeNAS but the hate I got on the FreeNAS forums told me to never ever thouc h that system with a 10 foot pole ever again.
 

bjs

Member

Reaction score: 42
Messages: 98

Yeah, it's a shame they have to be that way, there is really no call for it... Much better people and forum support for XigmaNAS...
 
OP
swemattias

swemattias

New Member

Reaction score: 2
Messages: 7

I should add some more info to what the system does today.
First off it is the storage for my plex media library, running as a VM inside proxmox, connected via nfs.
Second, it is used as common storage for lots of Docker containers and it is also the storage for my Proxmox setup.
So at times, it is under heavy load but not that much I presume....haven't given me any problems at all.

I am worried about the safety, I have no motioner solution for the storage atm, no nothing really to check the health. That is the main reason for going zfs.
 
OP
swemattias

swemattias

New Member

Reaction score: 2
Messages: 7

So what do I want to do at installtion?
Create and RAID-1 for the system disks, 2 250 GB SSDs.
Create an NIC bond with the two NICs in the machine, set an static IPv4 address to it.
After that the rest can be done after install I hop.
 

Geezer

Active Member

Reaction score: 72
Messages: 138

I don't quite understand this FreeNAS thing. What can it do that FreeBSD can't?

So why not just have FreeBSD as a fileserver?
 

Terpentijn

Member

Reaction score: 9
Messages: 82

FreeBSD has traditionally used OpenZFS from Illumos and so the version numbering from ZFSonLinux is irrelevant. You only need to care about the pool versions to know if they are compatible or not.

That said, FreeBSD is ditching this and moving to ZFSonLinux itself. This is likely to be the default version in FreeBSD 13.0, but I don't know which version it's currently based on. Support was added back in July. https://github.com/zfsonlinux/zfs/pull/8987
Ditching openzfs for ZFS on Linux? Hm... Will ZFS be still in the kernel or is fbsd gonna use modules? Big advantage of kernel support is speed. I'm a little worried reading this.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,283
Messages: 33,826

Will ZFS be still in the kernel or is fbsd gonna use modules?
It's using modules now anyway.

Big advantage of kernel support is speed.
There is no difference between a loadable kernel module or something that's statically linked into the kernel. It's all the same. Interesting detail, on FreeBSD everything is built as a module, even the stuff that's statically linked into the kernel.
 
OP
swemattias

swemattias

New Member

Reaction score: 2
Messages: 7

I don't quite understand this FreeNAS thing. What can it do that FreeBSD can't?

So why not just have FreeBSD as a fileserver?
That is the goal and why I am here to get to know if my hardware is up to the task.
 
OP
swemattias

swemattias

New Member

Reaction score: 2
Messages: 7

For some reason I have a hard time accepting ready-made solutions like FreeNAS, Unraid and all of those, I like to build my own solutions so I know what's installed and used and everything.
 

Geezer

Active Member

Reaction score: 72
Messages: 138

Well then,
FreeBSD, ZFS, NFS, Samba,
and you can still run X and put a desktop on so it looks nice.
 

`Orum

Well-Known Member

Reaction score: 38
Messages: 264

You only need to care about the pool versions to know if they are compatible or not.
Pool versions are meaningless now as everything should be 5000 AFAIK (except for maybe de jure ZFS). What you should care about are feature flags and what's enabled there.
 

xtaz

Well-Known Member

Reaction score: 133
Messages: 461

Ditching openzfs for ZFS on Linux? Hm... Will ZFS be still in the kernel or is fbsd gonna use modules? Big advantage of kernel support is speed. I'm a little worried reading this.
This was the original announcement about it: https://lists.freebsd.org/pipermail/freebsd-fs/2018-December/027085.html and it's moved on quite a lot since then. There are ports for it in sysutils/openzfs and sysutils/openzfs-kmod.

The reasoning behind it is that ZFSonLinux has moved forward significantly from the Illumos code that FreeBSD use and so it now makes more sense to move to ZOL with a single code base. Will get features etc. a lot quicker. Same reason FreeBSD now uses the KMS graphics drivers from Linux as well for X11.

And yes, ZFS has never been in the kernel anyway, it's always been a kernel module because of the GPL licenced code I believe.
 

Terpentijn

Member

Reaction score: 9
Messages: 82

This was the original announcement about it: https://lists.freebsd.org/pipermail/freebsd-fs/2018-December/027085.html and it's moved on quite a lot since then. There are ports for it in sysutils/openzfs and sysutils/openzfs-kmod.

The reasoning behind it is that ZFSonLinux has moved forward significantly from the Illumos code that FreeBSD use and so it now makes more sense to move to ZOL with a single code base. Will get features etc. a lot quicker. Same reason FreeBSD now uses the KMS graphics drivers from Linux as well for X11.

And yes, ZFS has never been in the kernel anyway, it's always been a kernel module because of the GPL licenced code I believe.
My confusion comes from reading that zfs on freebsd was/ is much faster than ZoL, because it was supported by the kernel. I guess this info was incorrect. The way I read it now is that zfs on Linux makes no difference. It's as fast as the way it works on freebsd. Am I right?
 
OP
swemattias

swemattias

New Member

Reaction score: 2
Messages: 7

So after running my tests in 3 different VMs, Debian 10.1 ZoL, Ubuntu Server 19.10 beta and FreeBSD ZFS, I've made my decision.
I am way more at home in the Linux world than the Unix world, I wish it wasn't so, since I began with Unix, that is a reason why I write this on a Mac OS X machine.
So my choise is Ubuntu Server 19.10 with zfs 0.8.2 that is released next week.
Thanks for all the input and knowledge shared!
 

`Orum

Well-Known Member

Reaction score: 38
Messages: 264

I don't know of any recent benchmarks that show a major difference except for the BlogBench results on the Phoronix site. However, these benchmarks have to be taken with a grain of salt, as:
  • He's using a pool consisting of 1 SSD and nothing else. This is atypical for any production environment I've ever seen.
  • ZFS is notoriously hard to benchmark as the ARC fills up. You can benchmark several ways, e.g. for cached data vs uncached, but great care must be taken and what you're testing is often closely tied to the hardware it's running on.
  • The BlogBench results are so wildly different that I suspect their surprising results are not related in any way to ZFS/ZoL, but instead probably due to differences in the disk access (e.g. CAM) or methodology. This can be further emphasized by the fact that the iX systems results (which uses the ZoL ported to FreeBSD) show values in line with FreeBSD and not Ubuntu ZoL.
Since the ZoL FreeBSD port is already showing improvements over the existing ZFS code in all but one benchmark, I certainly wouldn't be concerned. There was a lot of FUD spread in another thread on this forum about ZoL coming to FreeBSD, but I've yet to see any complaints that are well founded. I for one am looking forward to it making pools easy to transfer between OSes, as well as its new features (such as native encryption).
 

vermaden

Son of Beastie

Reaction score: 1,361
Messages: 2,916

I got an idea a few weeks back, I want to "upgrade" my storage-server, från Ubuntu Server 19.04, with Kernal 5.x, using 8 2Tb disks in RAID-6 with XFS.
Hardware is an AMD Ryzen 2200G, 16 GB of non-ECC RAM (please don't comment this, I know), Asus TUF board, with two PCI-e cards, one extra NIC and one RAID/SATA-port. All in all I got 10 SATA 6 GB/s ports.

My goal is:
Dual SSDs in RAID-1 for the system.
2 pools with 4 2 TB disks each using RaidZ2.

The things I wonder about is:
- Does FreeBSD play nice with AMD Ryzen?
- Does FreeBSD come with zfs, particularly v0.8.2?

Do you see anything worth mentioning about what I am about to do?
Maybe this will help you:
 

roccobaroccoSC

Aspiring Daemon

Reaction score: 163
Messages: 651

I don't quite understand this FreeNAS thing. What can it do that FreeBSD can't?

So why not just have FreeBSD as a fileserver?
It has a web based system management interface and some fancy customizing and tuning.
It can do basically the same stuff, it just lets you do it in a pointy-clicky way.
It's a neat idea, the devil is however in the proverbial details.
 

roccobaroccoSC

Aspiring Daemon

Reaction score: 163
Messages: 651

So after running my tests in 3 different VMs, Debian 10.1 ZoL, Ubuntu Server 19.10 beta and FreeBSD ZFS, I've made my decision.
I am way more at home in the Linux world than the Unix world, I wish it wasn't so, since I began with Unix, that is a reason why I write this on a Mac OS X machine.
So my choise is Ubuntu Server 19.10 with zfs 0.8.2 that is released next week.
Thanks for all the input and knowledge shared!
It's most important that your data is safe and accessible. I think that you can do this with any OS and ZFS when used intelligently. Just use whatever you are comfortable with. You could even run ZFS with Windows.
 

Geezer

Active Member

Reaction score: 72
Messages: 138

It has a web based system management interface and some fancy customizing and tuning.
It can do basically the same stuff, it just lets you do it in a pointy-clicky way.
It's a neat idea, the devil is however in the proverbial details.

Exactly. There is nothing that demands another 'distribution', merely some configuration of packages, or at the most, some new programs that runs ontop of existing packages.
 
Top