Porting zimbra

M

Mel_Flynn

I'm going to take a stab at porting zimbra to FreeBSD. No promises yet, as they've not exactly made it easy, by requiring specific versions of software and taking these versions into their build process. So I'm gonna have to take it apart and see where things fail when you're trying to use actual ports, rather then their versions.
What I've seen on the web is one old patchset (no offense Doug), but I may have missed something or maybe I'm doing something somebody is already trying or has tried and failed.
In that case I'd like to know / compare notes.
 
Their build system really inhales fluids. For this to become a real port (respecting $PREFIX), I'm gonna have to patch a lot (think thousands) of hardcoded "/opt/zimbra" references. On the plus side, once this is done, Yahoo might actually see the benefits of not having a hardcoded installer path, but one that respects compile time options.
Matt saw that and decided to just work with the /opt/zimbra path. For a real port that's unacceptable. And I really want a real port, cause Zimbra's main weakness are the hardcoded 3rd party software versions that trail weeks/months behind security patches.
On the plus side, Common/IM and Server compile ok and I've pretty much discected what their installer is trying to accomplish and started working on converting that to sensible BSD makefiles.
 
Porting (that is using /usr/ports for whole ThirdParty) zimbra to FreeBSD is a foolish idea. Even linux doesn't have a true port. Zimbra has a reason for not doing so. Using known versions of third party software and building in a pre-set enviroment greatly simplifies support/development.
I broke my wow not to ever install linux just few days ago. All this 'cos of zimbra! After a nightmare weekend and some google love, I have a happy working FreeBSD zimbra installation (instead of the unlucky linux).
Since I am afraid of sorting out all the licensing issues, I can't provide the packages publicly :( unless someone helps me, however I can provide you with fairly straightforward instructions and the altered zimbra source. There are still some bugs to catch but all of them are purely cosmetical (in the end, we would like one "make install", don't we? and we would also very much like most of the ThirdParty installed from ports. not counting rpm->pkg_add).

Zimbra sources are totally linux specific and almost everything is hardcoded.. so I totally hardcoded it into FreeBSD specific :-). Currently only FreeBSD7/amd64 but i386 is really only a manual work issue.
Since we can't make a real port, we should at least try to provide ports like:
- zimbra5-build-enviroment,
- zimbra5-install-enviroment.
That should simplify it a lot as I ended up with 140 installed packages after building every prerequisite from ports (to build zimbra). Once that is done.. we are free to try that foolish porting ;).

Testers wanted! Send me a message if You are interested.
 
solko-fbsd said:
Porting (that is using /usr/ports for whole ThirdParty) zimbra to FreeBSD is a foolish idea.
Click to expand...

Not really. Breaking up the zimbra port into its constituent parts would be foolish (ie mail/postfix, databases/mysql, www/jetty, and so on, with mail/zimbra being a meta-port). But creating a mega-port (note the g) that took the whole zimbra source tree, patched it to work with FreeBSD, compiled it, and installed it to /usr/local/zimbra, with the appropriate CONFLICTS line, should be doable. You could even install it to /opt, to keep the number of "remove hardcoded path" patches close to 0.

Since I am afraid of sorting out all the licensing issues, I can't provide the packages publicly
Click to expand...

Last time I checked, there was no restriction on packaging the OSS version.

Zimbra sources are totally linux specific and almost everything is hardcoded.. so I totally hardcoded it into FreeBSD specific :-).
Click to expand...

You'd think, with all the work done on creating the Solaris port, and now the Windows port, that there wouldn't be that many Linux-isms left. But that could be wishful thinking. :)

Since we can't make a real port,
Click to expand...

Why not? See above.
 
phoenix_rizzen said:
Not really. Breaking up the zimbra port into its constituent parts would be foolish (ie mail/postfix, databases/mysql, www/jetty, and so on, with mail/zimbra being a meta-port). But creating a mega-port (note the g) that took the whole zimbra source tree, patched it to work with FreeBSD, compiled it, and installed it to /usr/local/zimbra, with the appropriate CONFLICTS line, should be doable. You could even install it to /opt, to keep the number of "remove hardcoded path" patches close to 0.
Click to expand...
Well, that's what I had in mind. Let's not break it up.

phoenix_rizzen said:
Last time I checked, there was no restriction on packaging the OSS version.
Click to expand...
You need to include prepackaged diablo-jdk15/diablo-jdk16 in the zcs package. And I am not quite sure if that's okay.

phoenix_rizzen said:
You'd think, with all the work done on creating the Solaris port, and now the Windows port, that there wouldn't be that many Linux-isms left. But that could be wishful thinking. :)
Click to expand...

Sadly, the opensource version you can download via Perforce is completely linux specific with hacks to allow for a Mac build/install. But since FreeBSD port does not need to care about linux and since we can make a port once every major zcs release.. that's really just an annoyance instead of a real problem.
http://wiki.zimbra.com/index.php?ti...ng_Zimbra_Collaboration_Suite:_.22FRANKLIN.22

phoenix_rizzen said:
Why not? See above.
Click to expand...
Not a true port.. only a mega-port :-). But I guess there is no difference for the end-user.

I Should be able to provide instructions/packages in few days. In the meantime, I will try to make the building process as seamless as possible.
 
So.. zimbra is working as expected:
- all services up and running,
- i can send/receive emails,
- i can add/del users,
- i can log in with users.

I am now in the state of patching all those bash/perl scripts and fixing configuration errors (you would think there can't be any, wouldn't you?).
Startup of java-stuff is insanely slow and some things are kinda irracional (like altering postfix configuration at startup via postconf -e while postfix not running..).
BUT. I started with zimbra 4 days ago so I need to learn how it works first.
 
solko-fbsd said:
You need to include prepackaged diablo-jdk15/diablo-jdk16 in the zcs package. And I am not quite sure if that's okay.
Click to expand...

Hrm, couldn't you just add a DEPENDS line on the diable-jdk, and let the user install that version? Or is this needed internally by Zimbra, and needed during the compilation/installation process?

Sadly, the opensource version you can download via Perforce is completely linux specific with hacks to allow for a Mac build/install.
Click to expand...

That sucks. Maybe we should put in an RFE for FreeBSD support, or to make the code more portable. I know we (as paying Zimbra customers) would really like to be able to put Zimbra onto a box with ZFS support. :)

We're currently looking at using a FreeBSD box for the backend, using ZFS and iSCSI, with Ubuntu Server systems as the frontend, booting off CompactFlash and using iSCSI for storage.

Not having to use Ubuntu anywhere would be even better. :)

I Should be able to provide instructions/packages in few days. In the meantime, I will try to make the building process as seamless as possible.
Click to expand...

Good luck, and keep up the good work!!!
 
Zimbra build/packaging process includes jdk in the core package so that it gets installed first. Some packages (core,mta,apache,spell,proxy.. etc) use java as pre/postinstall package procedure so it needs to be present. However, if we take a different approach than installer and fully hack it to support FreeBSD only, we can ask user to install jdk prior to installing zcs (maybe you can distribute binary jdk but you can't distribute sources).

Easter is screwing my worktime :(, I will post the binary packages tomorrow so that someone can catch the bugs I missed. I am not aware of all the features so I can't test them out. All the ThirdParty should be okay but some of those scripts are really silly and I am thinking of rewriting them.
This is FreeBSD after all and we don't like sloppy stuff :-) (no offense zimbra team!).

Screw the license issues, hopefully noone gonna sue me..
 
Forgot to include ldconfig in the install script.. should be fixed now. If you encounter problems, please try:

ldconfig -m /opt/zimbra/lib
ldconfig -m /opt/zimbra/sleepycat/lib
ldconfig -m /opt/zimbra/openldap/lib
ldconfig -m /opt/zimbra/cyrus-sasl/lib
 
Tried it -- installation was successful, can login (Webclient and Admin Console) but didn't have enough time to test the actual functionality yet. Keep up the good work! (Now we just need to convince Zimbra to officially provide ZCS for FreeBSD... :P)
 
Yay! Glad to hear that. However, that's the only feedback I received so far. I can't fix stuff I don't know about. So I need You (plural) to tell me what to fix! Once that's done, I will send zimbra team my patch towards the actual stable version.

TODO so far:
a) make perl deps from /usr/ports
b) make clamav from /usr/ports 'cos there is a bug every month
c) build FreeBSD package
d) do it for i386
e) remake 600.daily script into /etc/newsyslog.conf, original script rotates log files without notifying daemons so they stop logging..

If i386 package is all that you need, say so. I don't run any recent i386 but I can install some virtual machine.
 
Nice. I'll have to see if I can create a VM somewhere to test this.

We're piloting Zimbra NE on Ubuntu Server right now for staff (need the Outlook Connector and Mobile Sync features). If that works, we're considering using Zimbra OS for students. Maybe we'll put that on FreeBSD. :)
 
I would like the i386 version :)

Great work with the port! Keep it coming!

Hopefully we can get this into the ports tree, even if it does install into /opt/zimbra, atleast its in there for starters.
 
There was no reply/feedback whatsoever in past month so I thought people did not really care and I stopped spending several hours/day by migrating zimbra to FreeBSD-ports.

I am using zimbra on several FreeBSD servers at the moment and I found out that the binary packages I provided don't really work that great out of the box (as I thought).. so I assumed lack of interest as noone ever asked me a question.

I will continue my efforts then.
 
At least for us, it's not lack of interest so much as lack of time for testing. :( We're quite interested in any work toward a port of Zimbra OSS to FreeBSD. Then we can run it on local storage using ZFS, if need be. :)

Keep up the good work, don't be discouraged. :D
 
Keep up the good work!

I'll be testing the AMD64 package soon (hopefully this weekend), I need to free up some more space as its heavily used.

It'll be installed in a jail and I'll write up a summary of what I did to get it going.


I'll try to compile the i386 after that, it was quite a download 2G! :O
 
iceburn said:
It'll be installed in a jail and I'll write up a summary of what I did to get it going.
Click to expand...
I'm very interested in your summary, hopefully with a successful result.

I didn't manage to get a working Zimbra inside a jail so far. Sadly I was only able to spend little time on that topic.

mousaka
 
I already ported Zimbra to FreeBSD and I am happily using it.

BUT.

Integrating Zimbra into ports system is something ENTIRELY different. There are tons of files I need to write from scratch and one bad decision can cost you a day of retesting. Not counting the fact that I had to learn how ports system work (using and creating isn't the same sadly).
Furthermore, ports got some limitations and I won't bug developers with every single thing I encounter. It kinda wasn't designed in a way that part of installation goes to /opt/zimbra and other part to $LOCALPREFIX (/usr/local) (and I decided not to go that way anyway as it causes a lot of trouble in the end). It is wise to use /opt/zimbra as it will help with further upgrades as Zimbra team don't plan to change the prefix.
However, I try to install all might-be-shared-libraries into $LOCALPREFIX so that we can use maximum stuff from ports instead of Zimbra bundled and outdated ones.

If someone needs some help with testing.. just describe the symptoms. Usually, there is problem with java or a problem with postfix-smtpd dlopening the libplain/liblogin libraries (which I still haven't fixed in the binary package).

Testing in jail is perfectly possible if you can solve those stupid mysql/auth problems (jail specific behaviour).

As for ZFS, I am using ZFS on all my new servers but there is this known locking issue (restart only solution) which might not be suitable for big installations.
 
Following your README
Code: 
textbox1# pkg_add pkg/bash-4.0.10_2.tbz
pkg_add: could not find package libiconv-1.11_1 !
pkg_add: could not find package gettext-0.17_1 !
/usr/ports/shells/bash

so I compiled one from ports, its being installed in a jail (ezjail), so had to create the link in the basejail.
Code: 
# pkg_add pkg/rpm-5.0.3_2.tbz
pkg_add: could not find package expat-2.0.1 !
pkg_add: could not find package beecrypt-4.1.2_1 !
pkg_add: could not find package pkg-config-0.23_1 !
pkg_add: could not find package lua-5.1.4 !
pkg_add: could not find package pcre-7.8 !
pkg_add: could not find package sqlite3-3.6.11 !
pkg_add: could not find package db45-4.5.20.0 !
pkg_add: could not find package libxml2-2.7.3 !
pkg_add: could not find package neon28-0.28.3 !
pkg_add: could not find package popt-1.7_5 !
pkg_add: could not find package xar-1.5.2 !
/usr/ports/archivers/rpm5

So installed from ports with WITHOUT_X11=yes and NOPORTDOCS=yes to cut down on the libraries required (esp. QT)
Code: 
# pkg_add pkg/fetchmail-6.3.9.tbz
pkg_add: could not find package ca_root_nss-3.11.9_2 !
/usr/ports/mail/fetchmail


The ./install.sh works

But I'm having issues with my AMD64 server, when it starts importing the MySQL tables which causes it to lockup. So i haven't tested it any further. Seems like there is a problem with my gjournal or something which causes it to lockup.

Hopefully i figure out whats wrong or even remove the gjournal and go back to UFS+SU to test again.
 
You must log in or register to reply here.
Back
Top