Solved llvm40 package invalid after being built with poudriere

T

tobiam

Hi,

I am experiencing a strange problem. For a while now I have been using poudriere to build my packages. I am running FreeBSD 11 Stable on amd64 and the poudriere jail is on the same revision as the host system.

Poudriere, packaging upgrading, etc. works for the most part. But a problem comes up in relation to the llvm40 package. For some reason building this package repeatedly results in a faulty package.

However I don't understand how and why it is this package. It is the very same machine as they are built on. Of course I checked the build logs and they are fine, no errors or anything xz related when building the package. However, when I do check the packge, right inside poudriere's data directory I get this:

Code: 
xz -t llvm40-4.0.1.r1_4.txz
xz: llvm40-4.0.1.r1_4.txz: Unexpected end of input

This looks like the compression was faulty and normally I'd assume that the data got corrupted in transit, but here I am on the very same system, it was just stored on disk. It happens even after rebuilding the package. So I don't know where the data corruption occurs. I also didn't see any signs of other files having corruptions.

Btw. this happens both inside and outside of poudriere, meaning that installing the package fails when poudriere tries to install it into a jail, as a dependency or when I try to install this package on the host system.

The size is 123MB, compared to the 237MB llvom39 (which works perfectly btw.).

I am really not sure what to loo for next. The whole thing looks like I'd have to check pkg/libarchive, but what confuses me is that it works for other packages. This one isn't particularily big or so. Also I'd expect some kind of warning at least at compression time. It doesn't make any sense to me that only this particular package would be affected and no others. xz should always see some tar, and it's also not particularly big.

Sum up:
  • FreeBSD 11-stable on amd64
  • xz error when decompressing llvm package, both on host and in poudriere
  • No other packages appear to be affected
  • Jail and Host do system match
  • No errors at package creation time
 
I also tried rebuilding pkg, in case pkg create is the issue here, but without success. Seeing that it depends on libarchive, but the base one I wonder whether it could be related to the recent 3.3.1 update.
 
Make sure you're not running out of disk space. That would result in broken packages and may not be noticed. Also try doing a run with -c to force a complete rebuild of everything. I've had a few instances where the caching caused errors on the "updated" packages.
 
I did complete rebuilds and always get stuck at the problem describe above. Shouldn't an out of disk space condition result in some kind of error? Anyway, there is plenty of it.

I don't think I have the caching problem, as I run locally and the update results in an actual update. Also using a local package repository pkg seems to understand that it makes sense copying around package files. But I really tried to get them from the repo, manually deleted the file and rebuilt it.
 
You were right!

It must have been the disk space topic. Thank you!

I don't fully understand though. So I checked the poudriere code and it seems that creating a package and the compression failing does actually gets swallowed/ignored.

For the background. I use TMPFS and the whole thing is actually pretty surprising for two reasons:

Big packages with many dependencies, for example LibreOffice build fine in 16GB of memory, so I am very surprised that llvm suddenly started to have this problem, even though there was nothing concurrently building.

There were out of memory/disk space errors in logs, even though I'd expect something at least in kernel log, but maybe the compression part makes sure and just errors out, which gets ignored though.
 
You must log in or register to reply here.
Back
Top