GlusterFS

fullauto2012

fullauto2012

A little bird told me that net/glusterfs is capable of creating a ZFS file system that spans multiple computers/pools. However, I have not been able to find any decent "howto's" or the such on how exactly one would go about implementing it or "best practices" and the such. Would be eternally grateful for any input anyone could offer.
 
Last edited by a moderator:
(Disclaimer: I've never actually used or installed GlusterFS, but am familiar with other cluster file systems.)

A: Other than some details like how to configure firewalls and how to set up / start services (systemd vs. init), the instructions for Fedora or RHEL should work just fine on FreeBSD; the gluster commands should all be the same. I've read the gluster documentation a few months ago, and I remember seeing a howto guide or a simple install guide, and the commands manual was unusually clear.

B: But cluster file systems tend to be complex and powerful beasts. When using high-end networking hardware and storage backends, they have obvious advantages (in particular for high performance), but the price you pay is extra work configuring and maintaining. Unless you are doing this just for fun to learn something, you might want to consider simpler solutions, like one machine with the disks in it (suitably RAIDed) acting as a file server, or an active/standby pair of nodes doing the same.
 
I made a test two-node cluster with net/pacemaker, net/corosync, net-mgmt/crmsh and net/glusterfs in a VM environment on ESXi. It took a while to set it up right but it was a fun learning experience. I used ufs file systems for it and didn't try zfs.

Regardless of the filesystem, I could not mount the cluster locally on the host nodes until after the server was completely up - even mount late would not work, however that's not a requirement for a true clustered host. Client's could mount it without any problems and failover worked great.

Be sure to put fuse_load="YES" in /boot/loader.conf
 
Hello,
I'm not familiar with GlusterFS and I'm just researching this topic now.

I checked this wiki and it sounds pretty official to me https://wiki.freebsd.org/GlusterFS which is:
GlusterFS (last edited 2018-05-22 09:40:16 by LapoLuchini)
Click to expand...
And the wiki says:
As of date, GlusterFS server is tested on Linux, FreeBSD and Opensolaris, and client runs on only Linux machines.
Click to expand...
The key here - client runs only on Linux?
Does it mean that FreeBSD can manage this filesystem as a server, but you cannot mount and browse files it as a client?

Did you manage to do it all on FreeBSD? Client side + server side?

Cheers/Beers
 
Send me a PM and I will send you a link to my howto for setting up pacemaker/corosync and glusterfs. I won't post anything publicly because I don't want to be accused of giving "bad advice".
 
Datapanic said:
Send me a PM and I will send you a link to my howto for setting up pacemaker/corosync and glusterfs. I won't post anything publicly because I don't want to be accused of giving "bad advice".
Click to expand...
Could I also get a copy of your guide? I want to try it out. Thank you!

Vincent
 
Vincent See said:
Could I also get a copy of your guide? I want to try it out. Thank you!

Vincent
Click to expand...
I would also like a copy of your guide. I have been playing with this in my spare time and having something that has worked for someone else would be a great lace to start.
 
Hello,

I initiated a mail communication with the glusterfs, please follow the 2 mails, I think this is a lot of useful information.



Code: 
From: "Kaloyan" <k_georgiev@trud.bg>
To: "craig001" <craig001@lerwick.hopto.org>
Cc: "mat" <mat@FreeBSD.org>, "rene" <rene@FreeBSD.org>, "eugen" <eugen@FreeBSD.org>, "dbaio" <dbaio@FreeBSD.org>, "antoine" <antoine@FreeBSD.org>, "gerald" <gerald@FreeBSD.org>, "sunpoet" <sunpoet@FreeBSD.org>, "olgeni" <olgeni@FreeBSD.org>, "tobik" <tobik@FreeBSD.org>
Sent: Wednesday, February 5, 2020 8:35:21 PM
Subject: FreeBSD Port: net/glusterfs

On 5 Feb 2020 1:22 pm, Kaloyan <k_georgiev@trud.bgwrote:

Dear port maintainer, Dear contributors,
I see that you are still doing changes and contribute to the net/glusterfs port.
However the current version of the port in FreeBSD is 3.11.
There are 2 major issues that make this port quite unusable:
1. Memory leak reported here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235305
2. Gluster tools not showing proper status/statistics reported here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223671

And there's another thing - in the gluster website there's now version 5,6 and the newest is 7. We are still on version 3.x.
I would like to request some information about the future development of this port and if there's a plan to port modern versions of net/glusterfs port on FreeBSD (or should I use Linux instead for gluster stuff).
Thank you in advance.

This is a very good reason why you should not use gluster on FreeBSD. Please keep in mind that the 2nd issue "Gluster tools not showing proper status/statistics reported here:" isn't just about statistics reported. The wrong statistics prevent you from triggering some actions like self heal because the command checks which nodes are connected and exits right away because the statistics shows no online nodes. For me it's to be considered as a major blocker.

The response I got is:

Code: 
From: "craig001" <craig001@lerwick.hopto.org>
To: "Kaloyan" <k_georgiev@trud.bg>
Cc: "mat" <mat@FreeBSD.org>, "rene" <rene@FreeBSD.org>, "eugen" <eugen@FreeBSD.org>, "dbaio" <dbaio@FreeBSD.org>, "antoine" <antoine@FreeBSD.org>, "gerald" <gerald@FreeBSD.org>, "sunpoet" <sunpoet@FreeBSD.org>, "olgeni" <olgeni@FreeBSD.org>, "tobik" <tobik@FreeBSD.org>
Sent: Wednesday, February 5, 2020 5:40:53 PM
Subject: Re: FreeBSD Port: net/glusterfs

The port is bitrotten and never updated, tried to bump up the revisions a while back but hit brick walls at every turn.
There was not a lot of support upstream for FreeBSD at the time, just seemed like we got stuck with that version.
I don't think there is a lot of call for gluster on freebsd as only 2 people have ever reached out to the maintainer in the last couple of years.
FreeBSD has also gone in a few directions that I didn't like and has pulled support for the ARCH I was working on.
Since moved onto other distros, and change of pace in the real world, have not been monitoring FreeBSD at all.
Sorry its not better news.

Regards
Craig B

I'm not sure where is the proper place of posting these emails, so I kept them for myself but since I saw posts regarding glusterfs on the forum I decided to share them here.
I advice you to read them carefully and even more carefully make your decision about using this version of glusferfs .

Hope this helps,
cheers/beers
 
You must log in or register to reply here.
Back
Top