Troubleshooting a hung Poudriere

astyle

astyle

I'm trying to use Poudriere to bulk-build a successful base install on host A. Basically, I have an install with a very bare-bones KDE desktop (no Konsole, or anything like that, but it starts Wayland successfully). So, I'm trying to use Poudriere to bulk-build the packages, populate a repo that I prepared, so that I can replicate the install from host A just by pointing host B's pkg to the repo on A.

So far, so good, but - Poudriere process got hung 3 packages short of completing the build. So, with help from awk and grep, I managed to figure out that those 3 packages are:
  1. www/qt5-webengine
  2. www/qt5-webview
  3. x11-toolkits/plasma5-kdeplasma-addons
I tried rebooting my build rig and restarting the Poudriere build. Fortunately, Poudriere was smart enough to pick up just those 3 ports that still need to be built. But it's taking so long, that I'm beginning to ponder some 'cheat moves'. The rationale behind them is:
Code: 
# poudriere ports -l
default   null   2021-11-23 17:10:39 /usr/ports/poudriere_tree
And thing is, I already have all those ports built in the regular /usr/ports tree. I have my reasons for the redundancy here. So, my thinking goes - since I already have the stuff built in the regular ports tree, maybe I can copy the ports directory (for the missing ports) from regular ports tree into Poudriere's 'default' tree. I don't run make clean in the ports directory, so all compiling work is intact :P

Any pitfalls to watch out for?
I know that www/qt5-webengine takes forever to compile even on a decent CPU. I think on mine (Quad-core Ryzen 5 1400 3.4 GHz) it took at least 3 hours.
 
Compiling gcc,llvm, some qt packages can take a long time. Poudriere only uses one core for each. Compiling llvm on my pc takes 18hours
 
astyle said:
I don't run make clean in the ports directory, so all compiling work is intact
Click to expand...
Poudriere uses a different WRKDIRPREFIX to build ports.
Code: 
---Begin make.conf---
USE_PACKAGE_DEPENDS=yes
BATCH=yes
WRKDIRPREFIX=/wrkdirs
PORTSDIR=/usr/ports
PACKAGES=/packages
DISTDIR=/distfiles
FORCE_PACKAGE=yes
PACKAGE_BUILDING=yes
PACKAGE_BUILDING_FLAVORS=yes
 
Thanks for your replies, everyone! This is really helpful.
--
What I did this time around is just reboot and try to let Poudriere finish on another overnight run. The 'cheat moves' I was considering - that was in case that reboot didn't work. However, it looks like it did work - according to logs, after another nearly 5 hours.

Total for my processor: 28 hours, 48 minutes, and 26 seconds.... Good thing this is done. Update sets (I only plan to update plasma5 and KF5) shouldn't take nearly this long. including the entire QT5 set - that was an upfront investment of processor effort, because until the KDE project switches to QT6.2, they're building against that same QT 5.15.2 set.
 
astyle said:
What I did this time around is just reboot and try to let Poudriere finish on another overnight run. The 'cheat moves' I was considering - that was in case that reboot didn't work. However, it looks like it did work - according to logs, after another nearly 5 hours.
Click to expand...
If you break off a build (for whatever reason) it's going to continue where it left off when you start it again. Thankfully, it doesn't start completely over from scratch. You can force a bulk build from scratch, that's sometimes necessary if you've been mucking around with dependency changes a lot. That's what -c is for. See poudriere-bulk(8):
Code: 
     -C       Clean only the packages specified on the command line or in the
              file given by -f file.  Implies -c for -a.

     -c       Clean all previously built packages and logs.
 
Yeah, that's useful if I change my mind in the middle of a build. poudriere.conf does offer something nice that I plan to make use of: KEEP_OLD_PACKAGES=yes, and the default is up to 5 versions back. ;)

Right now, my next step is figuring out how to manage those versions to make it easier on pkg on other machines. Once I point pkg to my repo, I'd want to specify a version to pass to pkg, like v.0 for baseline, v1. for upgrades, etc.
 
The initial build is always the most time consuming. It has to build everything it needs, all build, run and library dependencies for everything on your package wish list. Bigger, more cores, more memory, faster storage will all help speed up the process. Your next update, after you updated the ports tree, it will only (re)build what is needed. It will use the packages from the previous builds as much as possible.

astyle said:
Once I point pkg to my repo, I'd want to specify a version to pass to pkg, like v.0 for baseline, v1. for upgrades, etc.
Click to expand...
You can create symlinks in the /usr/local/poudriere/data/packages directory:
Code: 
root@molly:/usr/local/poudriere/data/packages # ll
total 112
drwxr-xr-x  7 root  wheel  18 Oct  9 19:17 122-release-server/
drwxr-xr-x  5 root  wheel  16 Nov 22 14:12 13-stable-desktop/
drwxr-xr-x  7 root  wheel  18 Nov 21 23:42 13-stable-server/
drwxr-xr-x  7 root  wheel  18 Nov 22 09:25 130-release-desktop/
drwxr-xr-x  7 root  wheel  18 Nov 21 23:10 130-release-server/
lrwxr-xr-x  1 root  wheel  18 Nov  7  2020 FreeBSD:12:amd64@ -> 122-release-server
lrwxr-xr-x  1 root  wheel  18 Feb 15  2021 FreeBSD:13:amd64@ -> 130-release-server

On the same host I also have nginx running. There's a sample configuration for poudriere in /usr/local/share/examples/poudriere/. I added a /packages location to it pointing at /usr/local/poudriere/data/packages.

I added two symlinks so URLs in my repository URL can use ${ABI}, that was useful when I upgraded my machines from 12.2 to 13.0. The other repositories have predictable names due to the way I used different ports trees and jail names. You can probably figure out some structure that'll work for you.
 
You must log in or register to reply here.
Back
Top