Has anyone setup MariaDB 5.5 with Galera on FreeBSD 10?

Since there's no port or package, I'm trying to install MariaDB 5.5 with Galera from source, which I downloaded from https://downloads.mariadb.org/mariadb-galera/5.5.41/. After changing my present working directory to the directory containing the tree extracted from the source tarball, I compiled and built from source as follows:

cmake . -DBUILD_CONFIG=mysql_release -DWITH_WSREP=ON -DWITH_INNODB_DISALLOW_WRITES=1
make
make install


The above code installs the software under /usr/local/mysql, which is a mount point for a ZFS file system with the non-default options of setuid=off, atime=off, and compression=lz4.

I installed the wsrep provider software via databases/galera:

cd /usr/ports/databases/galera ; make -DBATCH install clean

Here I install the initial MariaDB tables:

cd /usr/local/mysql
scripts/mysql_install_db


My configuration settings are stored in /usr/local/mysql/my.cnf:
Code:
[mysqld]
user=mysql
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0

# Galera Provider Configuration
wsrep_provider=/usr/local/lib/libgalera_smm.so
#wsrep_provider_options="gcache.size=32G"

# Galera Cluster Configuration
wsrep_cluster_name="staging_cluster"

# Initialize the cluster; switch after --wsrep-new-cluster has been run and other nodes have joined.
wsrep_cluster_address="gcomm://"

# Set cluster node addresses (hostnames work too)
#wsrep_cluster_address="gcomm://192.168.0.1,192.168.0.2,192.168.0.3"

# Galera Synchronization Configuration
wsrep_sst_method=rsync
#wsrep_sst_auth=user:pass

# Galera Node Configuration
wsrep_node_address="192.168.13.246"
wsrep_node_name="aus-mis-db-01.domain.com"
I copied /usr/local/mysql/support-files/mysql.server to /usr/local/etc/rc.d/mysql.server and modified as follows:
Code:
basedir=/usr/local/mysql
datadir=/usr/local/mysql/data
I made sure to set ownership of everything under /usr/local/mysql/data to the mysql user and group. So far so good.

The problem occurs when I try to bootstrap the cluster. When I run service mysql.server bootstrap, MariaDB fails to start:
Code:
root@aus-mis-db-01:/usr/local/mysql # service mysql.server bootstrap
Bootstrapping the cluster.. Starting MySQL.. ERROR!
The aforementioned command is actually running /usr/local/mysql/bin/mysqld_safe --datadir=/usr/local/mysql/data --pid-file=/usr/local/mysql/data/aus-mis-db-01.flatironssolutions.com.pid --wsrep-new-cluster &
When I run that command in the shell, I get a segmentation fault:
Code:
root@aus-mis-db-01:/usr/local/mysql # bin/mysqld_safe --datadir=/usr/local/mysql/data --pid-file=/usr/local/mysql/data/aus-mis-db-01.flatironssolutions.com.pid --wsrep-new-cluster &
[1] 68659
root@aus-mis-db-01:/usr/local/mysql # 150303 13:03:54 mysqld_safe Logging to '/usr/local/mysql/data/aus-mis-db-01.flatironssolutions.com.err'.
150303 13:03:54 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/data
150303 13:03:54 mysqld_safe WSREP: Running position recovery with --log_error='/usr/local/mysql/data/wsrep_recovery.HKUGwW' --pid-file='/usr/local/mysql/data/aus-mis-db-01.flatironssolutions.com-recover.pid'
150303 13:03:56 mysqld_safe WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
Segmentation fault
150303 13:03:56 mysqld_safe mysqld from pid file /usr/local/mysql/data/aus-mis-db-01.flatironssolutions.com.pid ended

[1]  Done  bin/mysqld_safe --datadir=/usr/local/mysql/data --pid-file=/usr/local/mysql/data/aus-mis-db-01.flatironssolutions.com.pid --wsrep-new-cluster
Has anyone setup MariaDB 5.5 with Galera on FreeBSD 10?
 
I was finally able to bootstrap the first node.

First, I rebuilt databases/galera using lang/gcc48 as the compiler instead of the native /usr/bin/clang. To do this, I added the following lines to /etc/make.conf:
Code:
.if !empty(.CURDIR:M/usr/ports/databases/galera) && exists(/usr/local/bin/gcc48)
CC=gcc48
CXX=g++48
CPP=cpp48
.endif
Next, I added these lines to /etc/libmap.conf so that the correct libraries would be used when running the database cluster service:
Code:
libgcc_s.so.1  gcc48/libgcc_s.so.1
libgomp.so.1  gcc48/libgomp.so.1
libobjc.so.3  gcc48/libobjc.so.4
libssp.so.0  gcc48/libssp.so.0
libstdc++.so.6  gcc48/libstdc++.so.6
Finally, I ran service mysql.server bootstrap to perform the bootstrap successfully. Now I just need to have my ports-mgmt/poudriere server build, in the same manner, a package for databases/galera to install on the other two nodes and connect those nodes to the bootstrapped node. Expect future updates to this thread before I declare this cluster setup a success and mark this thread as SOLVED.
 
Awesome work! When you hit that point where you are calling this a success consider documenting this with a quick guide in the HOWTO section. Whoever is next to walk in your footsteps would be sure to appreciate it.
 
Thank you! I'll definitely do that. I wouldn't want anyone to go through the frustration I did. Because of more specific error messages (i.e. not just "Segmentation Fault") that I received when trying to bootstrap the cluster on a FreeBSD 9 host (as opposed to the FreeBSD 10 host), I got the idea to compile databases/galera using lang/gcc48. The idea to setup the mappings in /etc/libmap.conf came from https://www.freebsd.org/doc/en/articles/custom-gcc/article.html.
 
I learned something about compiling.

databases/galera lists gcc as a dependency, and it also seems that ports that require gtk would need gcc too. Is there more to know if a port needs a gcc compiler, or are they hit or miss? I wonder if taking the gcc dependencies out of the galera makefile would allow a clang compile, but its probable that you already did that.

I found FreeBSDwiki - How to test compilation with gcc4, it's in a developer's wiki from 2007, but it says how newly used compilers don't work with all ports until the bugs are worked out.
 
I learned something about compiling.

databases/galera lists lang/gcc as a dependency, and it also seems that ports that require gtk would need lang/gcc too. Is there more to know if a port needs a lang/gcc compiler, or are they hit or miss? I wonder if taking the lang/gcc dependencies out of the databases/galera makefile would allow a /usr/bin/clang compile, but its probable that you already did that.

I found FreeBSDwiki - How to test compilation with gcc4, it's in a developer's wiki from 2007, but it says how newly used compilers don't work with all ports until the bugs are worked out.

Generally, I stick with the port's chosen compiler because I leave it up to the port's maintainer to know how that port should be properly built and installed.

I haven't tried compiling databases/galera with the native /usr/bin/clang on FreeBSD 10; that may also have resolved my issue of not being able to bootstrap the cluster. I do know that lang/gcc48 is required on FreeBSD 9 because the native /usr/bin/gcc is too out-of-date; that being said, the maintainer of databases/galera should add lang/gcc48 as a dependency for FreeBSD 9. I've been contemplating emailing the maintainer.

For FreeBSD 10, my earlier post about having to install and force the use of lang/gcc48 may not be necessary because of the included dependency of lang/gcc which is version 4.8.4:

Code:
root@cacl-mis-poudriere:/usr/ports # cat lang/gcc/distinfo
SHA256 (gcc-4.8.4.tar.bz2) = 4a80aa23798b8e9b5793494b8c976b39b8d9aa2e53cd5ed5534aff662a7f8695
SIZE (gcc-4.8.4.tar.bz2) = 86220648
root@cacl-mis-poudriere:/usr/ports # cat lang/gcc48/distinfo
SHA256 (gcc-4.8-20150212.tar.bz2) = 889a01edf24ac14a2652e3c0615391414d0166af07aa34e4e5245a9b398cfda8
SIZE (gcc-4.8-20150212.tar.bz2) = 82107833

However, the aforementioned modifications to /etc/libmap.conf were still needed to bootstrap the cluster, but I'm currently unable to add additional nodes to the cluster, and I'm wondering if those modifications are conflicting with my source copy of MariaDB 5.5 with Galera having been compiled with /usr/bin/clang. Tomorrow, I plan on either rebuilding databases/galera using /usr/bin/clang or rebuilding both MariaDB 5.5 with Galera and databases/galera using lang/gcc. The latter option will require more effort because I have to force devel/cmake to use a compiler different from what is listed in the source copy's *.cmake files...and I'm a novice with devel/cmake.
 
I remember gcc being a dependency of galera, but not gcc48, when I looked yesterday. Right now it lists gcc48 as a dependency at http://www.freshports.org/databases/galera. I wonder if someone already alerted the maintainer. It's not in the description logs, but the last log only says plain gcc was added. FreeBSD 9 and 10's port makefiles are the same right?

* update - It does seem like gcc48 needs an adjustment. The newer 10.1 system uses gcc48, and it conflicts with the port lang/gcc48 from being compiled. So now I realize, that maybe FreeBSD 9, possibly doesn't use gcc48 as its gcc compiler, even if it is listed in Freshports.
 
I remember gcc being a dependency of galera, but not gcc48, when I looked yesterday. Right now it lists gcc48 as a dependency at http://www.freshports.org/databases/galera . I wonder if someone already alerted the maintainer. It's not in the description logs, but the last log only says plain gcc was added. FreeBSD 9 and 10's port makefiles are the same right?
Yes, the makefiles are the same. No, databases/galera hasn't been updated since last November so I realize now that I didn't need to force the use of lang/gcc48 when installing on either FreeBSD 9 or 10. Still, I might try building databases/galera with /usr/bin/clang on my FreeBSD 10 host, partially out of curiosity. I'm also noting that databases/galera is a young port, having been added last April; that makes me a bit nervous.

* update - It does seem like gcc48 needs an adjustment. The newer 10.1 system uses gcc48, and it conflicts with the port lang/gcc48 from being compiled. So now I realize, that maybe FreeBSD 9, possibly doesn't use gcc48 as its gcc compiler, even if it is listed in Freshports.
I didn't have any trouble compiling lang/gcc48 on my FreeBSD 10 host, which is at version 10.1. What errors are you getting?

FreeBSD 9 uses an older version of GCC as its native compiler:
Code:
root@bou-nix-util:/usr/ports # gcc -v
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
root@bou-nix-util:/usr/ports # uname -r
9.3-RELEASE-p5
So databases/galera will also install lang/gcc48 on a FreeBSD 9 host if the latter hasn't been previously installed.

I never think to look at Freshports. I usually just run cd /usr/ports ; make search name=<query-string> so for databases/galera:
Code:
kkaos@mastersword:/usr/ports % make search name=galera
Port:   galera-25.3.5_2
Path:   /usr/ports/databases/galera
Info:   Synchronous multi-master replication engine
Maint:  horia@racoviceanu.com
B-deps: binutils-2.25 boost-libs-1.55.0_4 cairo-1.12.18_1,2 cups-client-1.7.3_4 cups-image-1.7.3_3 dejavu-2.34_4 doxygen-1.8.9.1,2 encodings-1.0.4_1,1 expat-2.1.0_2 font-bh-ttf-1.0.3_1 font-misc-ethiopic-1.0.3_1 font-misc-meltho-1.0.3_1 font-util-1.3.0_1 fontconfig-2.11.1,1 freetype-1.3.1_5 freetype2-2.5.5 gcc-4.8.4 gcc-ecj-4.5 gettext-runtime-0.19.4 ghostscript9-9.06_10 glib-2.42.1 gmp-5.1.3_2 graphite2-1.2.4 graphviz-2.38.0_6 gsfonts-8.11_6 harfbuzz-0.9.36 icu-53.1 indexinfo-0.2.2 jbig2dec-0.11_4 jbigkit-2.1_1 jpeg-8_6 kbproto-1.0.6 lcms2-2.6_3 libICE-1.0.9,1 libSM-1.2.2_2,1 libX11-1.6.2_2,1 libXau-1.0.8_2 libXaw-1.0.12_2,2 libXdmcp-1.1.1_2 libXext-1.3.3,1 libXft-2.3.2 libXmu-1.1.2_2,1 libXp-1.0.2_2,1 libXpm-3.5.11_2 libXrender-0.9.8_2 libXt-1.1.4_2,1 libcheck-0.9.14 libexecinfo-1.1_3 libffi-3.2.1 libfontenc-1.1.2_2 libgd-2.1.0_5,1 libiconv-1.14_6 libidn-1.29 libltdl-2.4.5 libpaper-1.1.24.3 libpthread-stubs-0.3_6 libxcb-1.11 libxml2-2.9.2_2 m4-1.4.17_1,1 mDNSResponder-561.1.1 mkfontdir-1.0.7 mkfontscale-1.1.1_1 mpc-1.0.2_1 mpfr-3.1.2_2 ncurses-5.9.20150214 openjpeg15-1.5.2_1 pango-1.36.8 pcre-8.35_2 perl5-5.18.4_11 pixman-0.32.6 png-1.6.16 poppler-0.30.0 poppler-data-0.4.7 printproto-1.0.5 psutils-1.17_5 py27-Babel-1.3_2 py27-Jinja2-2.7.3 py27-MarkupSafe-0.23 py27-cloud_sptheme-1.6 py27-docutils-0.12 py27-pygments-2.0.2 py27-pytz-2014.10,1 py27-setuptools27-5.5.1_1 py27-sphinx-1.2.3 python2-2_3 python27-2.7.9 readline-6.3.8 renderproto-0.11.1 scons-2.3.0_1 silgraphite-2.3.1_2 svgalib-1.4.3_7 t1lib-5.1.2_4,1 teckit-2.5.1_1 tex-dvipsk-5.994_1 tex-formats-20140525_1 tex-kpathsea-6.2.0_1 tex-ptexenc-1.3.2 tex-web2c-20140525_1 texlive-base-20140525_6 texlive-texmf-20140525_4 tiff-4.0.3_5 xcb-util-0.4.0,1 xcb-util-renderutil-0.3.9 xextproto-7.3.0 xorg-fonts-truetype-7.7_1 xproto-7.0.26 zziplib-0.13.62_2
R-deps: binutils-2.25 boost-libs-1.55.0_4 gcc-4.8.4 gcc-ecj-4.5 gettext-runtime-0.19.4 gmp-5.1.3_2 icu-53.1 indexinfo-0.2.2 libexecinfo-1.1_3 libiconv-1.14_6 mpc-1.0.2_1 mpfr-3.1.2_2
WWW:    http://galeracluster.com
Perhaps I glue my eyes to the terminal a bit too much. =)
 
I was using poudriere to build gcc48 or gcc, it failed with little verbosity; saying "port lang/gcc failed". Then I removed all of the make.conf and libmap.conf gcc configurations. Then I installed a program requiring gcc using pkg install in my basesystem, it removed lang/gcc48, and put in lang/gcc (which turns out to be gcc 4.8 anyway). I thought it would work this time, but it still failed.

* update - Doing a build outside of the jail, showed
Code:
*** Error code 1
, with the message to uninstall then reinstall print/indexinfo. Doing this worked for getting gcc compiled on my base system.
 
lang/gcc built outside the jail, but inside my jail, it still fails.

I've been tinkering with removing GCC and other GNU options from many ports in Makefile in the basejail. Removing GCC for many instances works. For large programs with plenty of dependencies, removing recursive GNU options tends to break the compile at devel/gettext. I've gotten a small program to work with removing GTK dependencies, it built and worked fine, but I'm unsure if this is because I had its libraries in my base (host) system.
 
You're not going to believe this. I edited a few Makefile's in the ports trees recursively, by removing all gcc, bash, and gmake. After doing this, the compile time dropped from 2 hours to 5 minutes, and the programs I compiled worked. I did this for emulators/wine and its dependencies. Desktop programs worked without gtk. The gcc, and bash dependencies are just bloat, when there is Clang, and sh.

This seems to fail on gnu specific ports, which makes sense; I expect this for linux emulation ports too. Many programs ported from linux can probably make the adaptation though, since they brought everything over from its old build configuration.
 
Last edited:
I have now tried this many times, and I think I am about ready to give it up.

The documentation (here) says that the Galera library cannot be built on FreeBSD "due to certain Linux dependencies".

On the other hand, v. 25.3.5 of this library has a bug in it that prevents MariaDB from starting up (see here). I have tried this at least ten times now.

Galera 25.3.5_2 is the only available version of Galera for FreeBSD 10.2 so, and please oh please do correct me if I'm wrong, it is impossible to have any sort of Galera replication on FreeBSD - I have seen mentions of this same issue from people trying to get galera running with Percona, at least.

I might go all bananas and fetch newer version of this library from a Debian installation or similar, and just place the library as it is on my FreeBSD experiment. I don't think it'll work, but heck, I feel I've tried everything else.
 
Last I checked, MariaDB does not come in FreeBSD with the WSREP patch in any of its variants. Thus, if you want MariaDB to work with Galera (in theory at least), you have no choice but to download he source code from here and manually compile - v5.5 is my choice.

If you still think that you can build a Galera compliant MariaDB from ports on FreeBSD, please point me to wherever you have seen evidence of this being a choice, I'll be much obliged!

/Alan
 
The latest version of MariaDB, 10.1.8 comes with Percona built in. I was able to compile it, though I've yet to use it in conjunction with Galera. You might want to give it a try.
 
I did try to deploy with the Galera library (25.3.10, as far as I remember) present in the brand new Ubuntu and, of course, this did not work: unmet pthread dependencies - I did not try to pursue this further.
 
Back
Top