Solved Questions about Poudriere and ccache

Sevendogsbsd

Sevendogsbsd

So, user Vermaden suggested I write a HOWTO about my implementation of ccache and tmpfs, based on my success in (possibly) decreasing the build time of packages so other folks who use ports-mgmt/poudriere could potentially benefit as well. Before I do this, I want to be sure the numbers I am basing my assumption on are correct and I am not just saying "yay it's faster" without understanding what is actually happening.

So, with an already "used" ports-mgmt/poudriere install, I built the same static list of packages I always do, but added www/chromium to the list. No ccache and no tmpfs. I ran poudriere ports -u to update the ports tree, did a custom port option configuration of www/chromium and the build of ~200 packages including the never before built www/chromium, took about 11 hours.

A few days later I ran another poudriere ports -u and the same build process without adding or removing any packages for my build list, but adding the configuration for ccache and adding tmpfs to my /etc/fstab. The build time was around 4 hours.

I did the same thing again yesterday but added www/firefox to the ports list and the build took about 4 hours.

Today I ran poudriere ports -u but changed ALLOW_MAKE_JOBS to "yes" in /usr/local/etc/poudriere.conf and reran the same ports list build. The build time was 46 minutes, BUT, when I ran pkg update from my PC which pulls packages from my build box, only 4 or 5 packages were updated, and not any of the "heavy hitters": chromium, firefox, libreoffice. NOTE - I also did this after each time I ran a build on the build machine, unfortunately I didn't note if these large packages were updated on my PC, except for the last build.

ports-mgmt/poudriere still showed these large ports as being built though so now I am confused. What is actually happening here and do my changes by adding ccache, tmpfs and setting ALLOW_MAKE_JOBS to "yes" really make that much difference?

I just want to be sure I am not basing my decreased build time on something that is not really happening. Are these packages in fact being rebuilt?

Thanks in advance.
 
Dear Sevendogsbsd,
what is the size of the ccache? When I have done tests some time ago things did not improve when building "big stuff" because the ccache of 5G has been overwritten multiple times by huge ports. I am not sure about a reasonable size of ccache. I have not done any measurements, but I am sure that I have had no benefit from ccache at that time. Currently I am back to packages but still interested how to improve things with respect to ports.
 
Thanks for your reply - I doubled the size of the ccache to 10G, mainly because I have a lot of disk space on the build box and wanted to experiment.

Since I have made several config adjustments to the system, the build times have gone down, but I was not sure why exactly.

Yes, packages are easier and really, I don't do any really profound configurations, other than eliminating pulseaudio entirely, and adding sndio. I could get away with using packages only but since retiring my previous workstation and building a new one, I wanted to make use of the old workstation as a build server. It was also a learning experiment so I could see how to use ports-mgmt/poudriere and how to speed up the build times using the online help I found.
 
So, implemented some suggestions rigoletto made by increasing PARALLEL_JOBS and PREPARE_PARALLEL_JOBS. I initially set

Code: 
PARALLEL_JOBS=4
PREPARE_PARALLEL_JOBS=12

based on his suggestion, but have doubled PREPARE_PARALLEL_JOBS to see what my ram usage is when building large ports like www/chromium.

Will updates the ports tree in a couple of days and build again to see how fast things go.
 
I usually do the tests after previously removing all built packages, and rebuild everything again late night while I sleep.

poudriere pkgclean -A -j JAILNAME

My machine is far from powerful of yours and so I usually reboot before running each test to start with a more or less standard environment between the tests, but I assume that would not make any difference with the amount of memory you have. ;)

[EDIT]

Btw, the poudriere web interface is quite useful to look for logs, specially if you are using a remote builder.
 
Started a clean build at 0915hrs (CST) - 701 packages, let's see how long this takes :). I doubled
Code: 
PREPARE_PARALLEL_JOBS=12
to 24 so we'll see what the ram usage is.
 
Wow, 2 hours and 6 minutes for 701 packages, starting clean as you referenced above. I'd call that a success! Thanks for the pointers rigoletto. RAM usage stayed around 7-15GB so I can probably try 36 or even 48 for the prepare parallel jobs setting.
 
Try also putting one more builder later to see what happens. 😃

[EDIT]

I suppose at some point increasing PREPARE_PARALLEL_JOBS will not make too much difference anymore, and then that is the point I would start increasing PARALLEL_JOBS.
 
You must log in or register to reply here.
Back
Top