Other Are There Any Distributed Block Storage Filesystems that support FreeBSD

[majortom]

I know that Min.io is available for FreeBSD, so there is at least a distributed object store available, but I have not been able to find something like Ceph or Gluster that currently supports FreeBSD. Does anyone know of any?

With the work being done on OCI containers, and Kubernetes in the Enterprise Working Group, we are close to being able to deploy Cloud Native solutions on FreeBSD but without storage we will not get far.

 

[SirDice]

Does anyone know of any?

HAST

 

[majortom]

HAST

Thanks for the quick reply, but HAST is not really distributed storage, it is a two node Primary/secondary system. Ceph, Rook, Gluster, etc. allow one to increase the amount of storage available by increasing the number of nodes, and by increasing the number of nodes one can increase the overall throughput.

 

[Beastie7]

You can scale up with HAST/CARP/LAGG instead of scaling out with something like Gluster/Ceph. The former is much simpler and requires less leg work. If you're not Google/Amazon, you probably don't need Gluster/Ceph.

 

[covacat]

latest versions of openzfs support live expandable zraids, so in theory you can create a zpool out of iscsi disks and grow it forever and iscsi export zvols from it.
it will most likely suck badly

 

[Beastie7]

I think the iSCSI stuff works good now. It's in kernel now and it's fully integrated into the CAM Target Layer. Haven't tried it though. I'd just use NFS to keep it simple.

 

[covacat]

its not that good over wan (at least in 13.x). sometimes the disk gets unresponsive on the initiator and you have to reboot to reconnect it.
i actually had a zpool locally and the disk was "vps block storage". i had this setup because the vps did not have enough memory to run zfs there. the purpose was offsite backup and zfs send always failed with memory problems. the iscsi / zpool frankenstein worked but with the above problems.

 

[majortom]

You can scale up with HAST/CARP/LAGG instead of scaling out with something like Gluster/Ceph. The former is much simpler and requires less leg work. If you're not Google/Amazon, you probably don't need Gluster/Ceph.

Can you explain how one would use HAST with CARP and LAGG to scale the amount of storage available and the performance of it?

 

[cyrille_v]

I know that Min.io is available for FreeBSD, so there is at least a distributed object store available, but I have not been able to find something like Ceph or Gluster that currently supports FreeBSD. Does anyone know of any? With the work being done on OCI containers, and Kubernetes in the Enterprise Working Group, we are close to being able to deploy Cloud Native solutions on FreeBSD but without storage we will not get far.

Hi! Have a look at glusterfs (net/glusterfs)
Regards

 

[DtxdF]

* Good support for FreeBSD, but CE only supports one master which means SPOF: https://github.com/moosefs/moosefs
* Abandoned: https://github.com/leo-project/leofs
* MinIO (it's not block storage, but worth of mention): https://blog.min.io/filesystem-on-object-store-is-a-bad-idea/
* Ported and seems promising but: https://github.com/seaweedfs/seaweedfs/issues/6645
* Not tested yet: https://man.freebsd.org/cgi/man.cgi?query=pnfs
* GlusterFS: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280148

There are others worth mentioning, but their use cases are completely different from clustering or similar:

* Works in FreeBSD but not ported: https://github.com/tahoe-lafs/tahoe-lafs

 

[majortom]

Hi! Have a look at glusterfs (net/glusterfs)
Regards

Unfortunately:

glusterfs GlusterFS distributed file system
8.4_3 net BROKEN: Fails to build, ld: error: version script assignment of 'global' to symbol 'client_dump' failed: symbol not defined
EXPIRATION DATE: 2025-01-31
IGNORE: is marked as broken: Fails to build, ld: error: version script assignment of 'global' to symbol 'client_dump' failed: symbol not defined