FreeBSD support being added to GlusterFS

Hi all,

The GlusterFS project is looking to add official support for FreeBSD to our next release. Does anyone have time to try out a tarball snapshot (from today) of our code so far? https://download.gluster.org/pub/gluste ... 21.tar.bz2.

In theory(!), it should compile okay using:
Code:
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install

I'm not familiar enough with the ports system in FreeBSD any more to remember how to make it work with that. Hopefully someone else can help with that too. :)

Feedback/thoughts?

[ Note, if I've posted this in the wrong forum section, feel free to move it and/or yell at me appropriately :) ]

Regards and best wishes,

Justin Clift
justin@gluster.org
 
Ouch. :(

The "Not a git repository" is my fault. I removed the .git directory, forgetting it would be needed by autogen. Would you be okay to try this (much bigger) tarball instead: https://download.gluster.org/pub/gluster/experimental/glusterfs-freebsd_20140622.tar.bz2? It's a full Git repository, so it isn't the way a final tarball would be. But, it should get past that "Not a git repo" warning.

This line in that fpaste is interesting too:
Code:
  "automake-1.14: warning: possible forward-incompatibility."
It sounds like you have a newer version of automake installed than what our tarball knows how to deal with. Is there an older version you could try? (maybe the same thing for autoconf too, I'm unsure).
 
I tried out the tarball. I had to install the following packages to get glusterfs to conifgure and compile: autogen, automake and bison. With those packages on the system, I was able to compile glusterfs using the Clang compiler on FreeBSD 10.0. The make install script worked and I was able to run glusterfs --version.

There were a lot of warnings during compile time, almost all of them mentioned GNU-isms being used in the code. But, in the end, the software built properly. If I get time this week I will try to put together a port.
 
JustinClift said:
Hi all,

The GlusterFS project is looking to add official support for FreeBSD to our next release.

This looks very interesting. Could you point me to the documentation describing project goals? Any plans to make things more portable so that Gluster can compile on other BSDs and OSs?
 
NewGuy said:
I had to install the following packages to get glusterfs to conifgure and compile: autogen, automake and bison.

Thanks NewGuy, that's good info. We should add that information to our wiki, so other people can get it going simpler. :)

NewGuy said:
With those packages on the system, I was able to compile glusterfs using the clang compiler on FreeBSD 10.0. The make install script worked and I was able to run glusterfs --version.

Cool, that's good news. I wasn't personally sure if our make requirement was GNU-make only or not. So not. :)

NewGuy said:
There were a lot of warning during compile time, almost all of them mentioned GNU-isms being used in the code. But, in the end, the software built properly. If I get time this week I will try to put together a port.

Matthew D. Fuller whipped up a quick one based upon the first tarball yesterday: http://lists.freebsd.org/pipermail/free ... 19673.html.

The mailing list archive doesn't seem to make attachments available, so I've saved the .shar file he created here: https://download.gluster.org/pub/gluste ... 40621.shar.

Does that help?
 
Last edited by a moderator:
Oko said:
This looks very interesting. Could you point me to the documentation describing project goals?

Hmm, what sort of information do you want? :)

"Project goals" is a bit unclear to me. We're a distributed filesystem, and we keep on adding new features/goals/etc. with every release. Ideally we'd completely replace the need for SANS's and Filer's. Though that's a bit aspirational. ;)

Our website is here if that helps: http://www.gluster.org.

Oko said:
Any plans to make things more portable so that Gluster can compile on other BSDs and OSs.

Definitely. GlusterFS works on NetBSD already due to the awesome effort of Emmanuel Dreyfus (NetBSD guy). He's actively involved, and keeps us on our toes. ;)

The upcoming 3.6 release of GlusterFS will support MacOS X too (but only as a client) as well as FreeBSD (fully supported), if things go well.
 
Justin, I have a few questions regarding glusterfs.

1. What about file AIO support by glusterfs under FreeBSD?
2. Does glusterfs support kqueue or does it use the "fallback" method pool?
3. Are there any recommendations regarding tune ZFS datasets to work with glusterfs?

P.S.: I've successfully compiled glusterfs. Thanks.
 
Heh, that's way beyond my technical knowledge level. I'll ask on the gluster-devel mailing list, and paste answers back here. :)
 
From Harsha on gluster-devel:

ironudjin said:
1. What about file AIO support by glusterfs under FreeBSD?
AIO support exists if they are talking about libaio.

ironudjin said:
2. Does glusterfs support kqueue or it uses "fallback" method pool?
AFAICS kqueue is FreeBSD poll functionality. GlusterFS by default is epoll which is nonexistent on BSD'S. In such cases it has a fallback to poll.

ironudjin said:
3. Are there any recommendations regarding tune ZFS datasets to work with glusterfs?
ZFS worked out of the box on Solaris for GlusterFS 1.3.x so I would say yes.

(This one seems a bit off target to the question. I think Harsha means it might be helpful to look for Solaris ZFS recommendations, though I have no idea where they'd be.)
 
JustinClift said:
AIO support exists if they are talking about libaio.
AFAIK, libaio doesn't exist in FreeBSD. AIO is implemented in FreeBSD through the kqueue and aio modules. It seems glusterfs currently doesn't support file AIO in FreeBSD.
JustinClift said:
AFAICS kqueue is FreeBSD poll functionality. GlusterFS by default is epoll which is non existent on BSD'S. In such cases it has a fallback to poll.
AFAIK, poll exist in each *nix implementation as a generic (i.e. legacy) method which is not as fast as system dependant and effective methods like epoll and kqueue. For Linux glusterfs has support for epoll but in FreeBSD it doesn't support kqueue. It's not fair. :)

Let me clarify a little bit why I'm asking all these questions. When a typical system administrator is going to build a new cluster environment, he do some research about the best way to implement the required functionality. In case glusterfs doesn't support effective methods like aio and kqueue in FreeBSD, he will prefer to build an environment on a Linux platform. Or he will build on FreeBSD but without glusterfs use any other similar solution. Both ways are not so good. I'm not a fanatic of Linux or FreeBSD. My approach is to use the most effective "tool" for each task. Of course all that 's written above is only my opinion as a system administrator with the same upcoming research and task to build an HA cluster in our project.

If I'm wrong somewhere - please feel free to correct me.

Thanks.
 
In a testimony to just how awkward FreeBSD ports can be, I tried out the above port and could not get it to work in a clean environment. GlusterFS builds fine if I install the dependencies manually, but building from the port didn't work. I think the problem is the port requires autotools and the ports system doesn't build/install these until after the configuration step, which needs to come after autotools are used. So, basically, the ports system executes steps in the wrong order. I played around with the ordering a bit, but in the end it is just so much easier to build manually. Maybe the GlusterFS project will offer FreeBSD packages in the future rather than relying on the official ports tree?

For some reason, the latest ports snapshot I downloaded is also missing the tar.mk file, which was included the last time I downloaded the ports tree. This strikes me as strange as it meant no ports with the tar USES flag would build. I was able to import the tar.mk file from another machine, but I'm curious as to why it was missing in the first place.
 
It's not too surprising that a two-day old port might have some teething problems. Documenting them and either entering a PR (shiny new Bugzilla) or corresponding with the port maintainer is important. Otherwise, those problems often persist.
 
ironudjin said:
Let me clarify a little bit why I'm asking all these questions. When typical system administrator going to build new cluster environment, he do some research about the best way to implement required functionality. In case glusterfs doesn't support effective methods like aio and kqueue in FreeBSD, he will prefer to build environment on Linux platform. Or he will build on FreeBSD but without glusterfs use any other similar solution. Both ways are not so good. I'm not a fanatic of Linux or FreeBSD. My approach is to use the most effective "tool" for each task.

Sure, there's nothing wrong with that thinking.

But also, we have to start somewhere. :D What generally happens with new-platform-support for open source software, is something like this:
  1. Someone gets it kind of working, and tells other people how to make it work too.
  2. Someone with more knowledge improves things further, so more people can use it in a much easier fashion.
  3. Someone with more a demanding situation wants to use it, but it's not quite right for them. That person takes the time to "fix it" so that it works in the demanding situation as well (and contributes the fix to the main project).

So, we're kind of at stage a) or b) at the moment. To get the better performance stuff happening, it'll probably take some c)'s :)
 
JustinClift said:
But also, we have to start somewhere. :D
Yes, I agree. I'm glad to see that the glusterfs community pay attention to support FreeBSD. I hope we'll have glusterfs functionality under FreeBSD not worse than in other OSes. But of course not right now. :)
 
Can I ask some questions?
  1. What is the difference between glusterfs-freebsd and glusterfs-3.5.1 versions of GlusterFS?
  2. And is this version (glusterfs-freebsd) stable?
Thank you.
 
freebsd said:
Can I ask some questions?
  1. What is the difference between glusterfs-freebsd and glusterfs-3.5.1 versions of GlusterFS?
  2. And is this version (glusterfs-freebsd) stable?
Thank you.

Apologies for the delay, I didn't see this until now.

The glusterfs-freebsd file is a snapshot of the latest GlusterFS development code, where we are adding FreeBSD support. The early snapshot tarballs had patches (under development) that weren't yet merged into the main GlusterFS code base. The later snapshot tarballs are (from what I know) the main codebase itself, which has had the FreeBSD support merged in. We're still improving things too, fixing bugs that show up, getting everything more solid/reliable, and so on.

This is because we're aiming to have FreeBSD fully supported in the next major GlusterFS release, version 3.6.0. The 3.5.1 version you mention above doesn't officially support FreeBSD, and I kind of doubt it would actually work very well, if at all. ;)

For the "is this version (glusterfs-freebsd) stable?" question, I'd say probably not. Definitely don't put production data on it. :D It could have bugs in it from the FreeBSD work being done, but it's also extremely likely to have bugs in it from the general GlusterFS 3.6 development work going on. Things should be in a much more stable/reliable state closer to our 3.6.0 release. Though from past experience I'd suggest waiting until the .1 or .2 releases after that before putting critical data on it.

Hopefully that helps. :)
 
I've installed GlusterFS on my FreeBSD 10-RELEASE storage.

Needed FreeBSD packages:
  • pkg install automake libtool git bash bison autotools fuse-utils libexecinfo
  • I had to install cmockery2 from the ports: cd /usr/port/sysutils/cmockery2 && make install clean

Latest version of GlusterFS: http://download.gluster.org/pub/gluster/experimental/.

Start building GlusterFS:
  • Extract the tar file: tar -xz glusterfs-3.7dev-xxxx.tar.gz followed by cd glusterfs.
  • Run autogen to check all dependencies: ./autogen.sh.

After you have installed all the missing dependencies you’ll be able to do:
  • without the glusterfs client:
    ./configure
    make
    make install
    make clean
  • with the glusterfs client:
    ./configure -enable-fuse-client
    make
    make install
    make clean

I'm missing only a startup/boot script. If someone could help me with that..;)
 
Last edited by a moderator:
During the ./configure process it gives me an error.
Code:
./configure: 12727: Syntax error: word unexpected (expecting ")")
 
rmahangoe said:
During the ./configure process it gives me a error.

Code:
./configure: 12727: Syntax error: word unexpected (expecting ")")

Interesting. It's obviously not supposed to do that. ;)

I've emailed Harsha about this, and CC'd you on the email, so we'll see what he says.

Hopefully it's something simple, as FreeBSD (in general) is known to compile okay. We have a FreeBSD 10 VM setup in Rackspace that runs a basic smoke test (just ./configure; make so far) for every proposed GlusterFS patch, and every commit in GlusterFS Git: http://build.gluster.org/job/freebsd-smoke/.
 
Back
Top