zfs on Freebsd, Linux and Mac any tips

NapoleonWils0n

Well-Known Member

Reaction score: 115
Messages: 259

My Synology ds112j nas which i used to share files between Freebsd, Linux and Mac osx just died on me

I have a 2 terabyte backup of the nas on an external drive which is formatted as Linux ext4 which is the filesystem used by the synology nas

So i need to copy the backup to an external usb drive formatted as zfs which i can then plug into a Freebsd machine
or Linux machine after installing zfsutils-linux, and hopefully on the mac using openzfsonosx

Does anyone have any tips on the best way to create a zfs formatted drive that will work on Freebsd and linux,
and for bonus nerd points mac osx as well

I have 2 old Macbook Airs from 2011 one running Freebsd and the other running ubuntu,
they are set up exactly the same and the only way to tell them apart at a glance is the wallpaper

But the problem is they only have two usb2 ports and one port supplies more power than the other,
the upshot is i cant plug in 2 usb drives at once to the same machine to copy the files from one drive to another,
because only one drive gets mounted

The plan is to plug in the backup of the nas which is formatted as ext4 into the ubuntu laptop,
and then plug in a usb drive into the Freebsd and format the drive as zfs

To sync the files from Linux to Freebsd i think syncthing would be the best method

Using syncthing on the Linux side i can select directories from the mounted ext4 nas backup drive,
and then on Freebsd select directories on the mounted zfs usb drive

I already have syncthing installed on both machines to sync files anyway
 
OP
NapoleonWils0n

NapoleonWils0n

Well-Known Member

Reaction score: 115
Messages: 259

I created a zfs formatted usb drive on Freebsd and then plugged into an ubuntu machine with zfsutils-linux installed

The short answer is that ubuntu wouldnt import the pool because of the spacemap_v2 feature,
but it did offer a command to import the pool in read only mode

Heres what i found

Plug in the usb drive and use geom disk list to find the name of the drive

Bash:
geom disk list
list disk as da0

use gpart to destroy the drive
and dd to zero out the drive as well

Bash:
# gpart destroy -F da0
# dd if=/dev/zero of=/dev/da0 bs=1m count=128
use gpart to create a gpt partition on the drive
and add a label

Bash:
# gpart create -s gpt da0
# gpart add -t freebsd-zfs -l zusb da0
create a new zpool and give it the same name as the gpt label to make things easy to remember
set the mount point and use chown to change the owner of the mount point,
replacing username:username with your username

Bash:
# zpool create zusb gpt/zusb
# zfs set mountpoint=/mnt/usb zusb
# chown username:username /mnt/usb
export the pool

Bash:
# zpool export zusb
Next i plugged the zfs formatted usb drive into ubuntu

You can find the drive by the gpt label we added to the drive by running the following command

Bash:
# ls -lh /dev/disk/by-partlabel/
We can then try importing the pool using the gpt label and zpool name

Bash:
# zpool import -d /dev/disk/by-partlabel zusb
But thats when i get the error about spacemaps_v2 listed at the top of the post

So i guess its a questions of which features are enabled on the pool,
im wondering if creating the pool on linux is worth a shot and then seeing if that pool with import on freebsd
 

k.jacker

Aspiring Daemon

Reaction score: 449
Messages: 739

If you have syncthing already installed, why not sync the data from your Ubuntu machine to an empty ZFS dataset on your FreeBSD machine?
Then you don't need to worry about filesystems at all.
 
B

badbrain

Guest


It's a portable device why don't you use a portable filesystem like NTFS? Read/Write everywhere NTFS 3G exists and don't have to care about user permissions.
 

Zirias

Aspiring Daemon

Reaction score: 392
Messages: 874

It's a portable device why don't you use a portable filesystem like NTFS? Read/Write everywhere NTFS 3G exists and don't have to care about user permissions.
This is a very bad advice. NTFS is by no means a "portable" fs. It is a complex fs without public specs, so the only "reference" is the existing closed-source implementation. No doubt it is technically "better" than FAT, but at least, FAT is simple and well understood. You can never fully trust a re-implementation of NTFS, so using it for write access is always an additional risk you should be aware off.

That said, it IS something one could try for one-time usage ... but then, why not stick with FAT or even EXT2 (which is supported by FreeBSD as well)?
 
B

badbrain

Guest


This is a very bad advice. NTFS is by no means a "portable" fs. It is a complex fs without public specs, so the only "reference" is the existing closed-source implementation. No doubt it is technically "better" than FAT, but at least, FAT is simple and well understood. You can never fully trust a re-implementation of NTFS, so using it for write access is always an additional risk you should be aware off.

That said, it IS something one could try for one-time usage ... but then, why not stick with FAT or even EXT2 (which is supported by FreeBSD as well)?
This is not bad choice. NTFS 3G has never failed me. One of it biggest plus is the fact it doesn't support unix user permission, I don't have to use chown everytime. The default behavior on Linux is any user could read/write without permission issues. FAT is not good for big files. ExFAT is nothing better than NTFS. Ext2 is a good option, though. There's driver for Windows.
 

ralphbsz

Daemon

Reaction score: 1,486
Messages: 2,437

NTFS 3G has never failed me.
I love your statistical argument, based on a single observation.

Old joke, found on a T-shirt: "There are two kinds of people. 1: Those that extrapolate from insufficient data."

Now serious answer: NTFS is indeed quite a high-quality file system, when using the native implementation (meaning on Windows). It is also a relatively complex file system; if you develop file systems for Windows, you can see that complexity in the sheer volume of the SDK for NTFS-compatible file systems which Windows distributes.

However, the various open-source re-implementations of NTFS are highly problematic (those are the ones that are used on Linux and the various BSDs). They were all done without any knowledge of the NTFS source code, without knowing which algorithms NTFS uses, simply by observing the on-disk data structures and using the for educated guesses. They should not be relied on for data that is important, and where data loss or corruption would be harmful.
 

malavon

Active Member

Reaction score: 144
Messages: 244

NTFS 3G has never failed me
I can attest to the exact opposite. NTFS 3G has failed me, not on Windows but with fuse-ntfs. It was not critical since I had the data somewhere else, but I did lose data back then.
The same has never happened to me in those (quite basic) conditions on Windows or with any of the well-known filesystems on FreeBSD.

If I need a filesystem to share between different OS'es (read: Windows), it's FAT/FAT32. If it's only FreeBSD it's UFS or ZFS. I wouldn't recommend anything else.
 

ralphbsz

Daemon

Reaction score: 1,486
Messages: 2,437

I would go further. Sharing on-disk file systems between different file system implementations is fundamentally dangerous. In particular if the implementations come from different software development organizations, or if some of them had to be reverse engineered, or if the file system is not well documented, or if suspend/resume and other forms of uncontrolled caching are involved. There are cases where it works well; some file systems are explicitly engineered to run on multiple OSes (ZFS is in that category, if you are careful with turning features on and off); others are even designed to allow concurrent access to the same device from multiple computers of different OSes (but those are exotics, not available to standard FOSS users).

But with the easy availability of virtual machine technology and of small inexpensive computers, there is a much better option: Connect the disk device to a dedicated server. So if you want to use NTFS on a disk, the find a suitable machine, set up a VM, boot Windows on it, and use it as a network server (CIFS and Samba for example). Or buy a Raspberry Pi, connect the disk to it, and use the whole thing as a small NAS. Suddenly all these problems of having to do perverse things with file systems go away.
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,275
Messages: 4,099

You'll need to find a list of features that ZFS-on-Linux supports, and compare that to the list of features that FreeBSD supports. Find that features that are common between the two. Then use that list to create the pool with only compatible features enabled (you list them via -o optionX=on -o optionY=off etc, on the zpool command line).

Then be sure to never upgrade the pool or filesystems, and only enable features that are specifically supported on each of the OSes.

This is one area where all of the OpenZFS ports fall down. It would be nice if they either defaulted to "common/compatible features only" at pool creation time. Or, at the very least, included an option for "most compatible" that could be given at pool creation time.

There is work going on to develop a common baseline of features that's discussed during the monthly OpenZFS admin meetings, so they're working in that direction.

For now, you have to do the legwork yourself.
 
Last edited:

stratacast1

Well-Known Member

Reaction score: 33
Messages: 303

I'd also like to echo the differences in ports of OpenZFS. I went from an Ubuntu 16.04 server to using FreeBSD 11.1 a couple years ago and found importing my backup drive to my new FreeBSD system had incompatible features. It did import as read only which was fine for what I needed it for at that time, but in the end I ended up making an all new pool. It really would be neat to see a baseline pool creation option like what phoenix said. OpenZFS is portable, but the features sets aren't as portable at this time. Maybe one day we will see more parity among the OSes and their implementation of OpenZFS
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,932
Messages: 33,331

Maybe one day we will see more parity among the OSes and their implementation of OpenZFS
That's probably going to happen sooner than you expected.
 
OP
NapoleonWils0n

NapoleonWils0n

Well-Known Member

Reaction score: 115
Messages: 259

You'll need to find a list of features that ZFS-on-Linux supports, and compare that too the list of features that FreeBSD supports. Find that features that are common between the two. Then use that list to create the pool with only compatible features enabled (you list them with the zpool command).

Then be sure to never upgrade the pool or filesystems, and only enable features that are specifically supported on each of the OSes.

This is one area where all of the OpenZFS ports fall down. It would be nice if they either defaulted to "common/compare features only" at pool creation time. Or, at the very least, included an option for "most compatible"that could be given at pool creation time.

There is work going on to develop a common baseline of features that's discussed during the monthly OpenZFS admin meetings, so they're working in that direction.

For now, you have to do the legwirk yourself.
I figured that was the issue trying to make sure that the pools shared the same features,
and then as you say dont upgrade the pool to add new features.

The zfs drive i created on Freebsd use the spacemap_v2 feature which isnt supported on the linux side,
but it seems like its possible to mount the drive read only which is better than nothing
 
OP
NapoleonWils0n

NapoleonWils0n

Well-Known Member

Reaction score: 115
Messages: 259

In the end i used rsync over ssh instead of syncthing because i had to move directories around,
and it it takes syncthing a while to scan the directories.

I was averaging 11.2mb transfers speeds over rsync and ssh using ethernet,
just another 750 gig to move across
 

stratacast1

Well-Known Member

Reaction score: 33
Messages: 303

In the end i used rsync over ssh instead of syncthing because i had to move directories around,
and it it takes syncthing a while to scan the directories.

I was averaging 11.2mb transfers speeds over rsync and ssh using ethernet,
just another 750 gig to move across
Yeah I learned syncthing isn't good at a one time shoveling over of data. I really wish they'd see a new client and just shovel the data over and then sync chunks as needed
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,275
Messages: 4,099

Here's info from the recent BSDCan conference covering ZFS on FreeBSD, what it supports, where it's going, and how it's going to get there. There's currently only the slides available, but I believe they're working on getting a recording of the session as well.
About a third of the way through the slides, there's info on the pool creation compatibility work that's going on, and an indication of what the CLI options will look like. Creating a ZFS pool on OS A that's compatible with OS B will soon be very easy. :)

https://www.bsdcan.org/2019/schedule/events/1060.en.html
 
Top