Error extracting the archive: 'Lzma library error: Corrupted input data'

Hi,

I have: 13.2-RELEASE FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC amd64

I get this error often, it usually happened with larger packages, and failed/succeeded randomly. First suspicion was about older internet connection which is not very fast, but then again, it only happens with FreeBSD 13.2, I can run SmartOS on same server, and download packages via MacOS on same network.
I tried:
Code:
fetch  https://pkg.freebsd.org/FreeBSD:13:amd64/latest/packagesite.pkg
unxz -t -v packagesite.pkg
xzcat packagesite.pkg|tar tf -

No errors.

Code:
# fetch http://pkg.FreeBSD.org/FreeBSD:13:amd64/quarterly/packagesite.txz
packagesite.txz                                       6735 kB  985 kBps    07s
# tar -xvf packagesite.txz
x packagesite.yaml.sig
x packagesite.yaml.pub
x packagesite.yaml

No errors.

I tried changing mirrors, but returned to default, it was the same when I tried installing something with multiple packages and it would fail on larger ones:
Code:
# cat /etc/pkg/FreeBSD.conf
# $FreeBSD$
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#
FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
#  signature_type: "fingerprints",

#  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
Problem is here:

Code:
# pkg update -f
Updating FreeBSD repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01   
Fetching packagesite.pkg: 100%    7 MiB 383.2kB/s    00:18   

pkg: Error extracting the archive: 'Lzma library error: Corrupted input data'
tar: Damaged tar archive
tar: Retrying...

Reinstalled pkg.

But also when doing portsnap fetch, I get "damaged tar" in a row.
 
Now I rebooted server, and it completed sudo pkg update. Fetching was a tiny bit faster, but last time it was still relatively fast. I will need to troubleshoot this because problem will always return.
 
If you have no known issues with your internet connection then try to check the RAM of the computer in case of issues with archives like "CRC error" or "corrupted/damaged archive".
Memory (RAM) failures very ofthen affect on tasks like archiving/unpacking.
Try to run any memory test like Memtest86 or try to compress some large file(s) (~10-20GB) using some archiver with compression enabled and try to test that archive when compressed.
 
I have to respond here again, because its been fixed now after installing 14.1.

I did memtest and one pass came out with no errors. Some people wanted to convince me my machine is failing (cpu, ram, mb), but no - this fujitsu celsius workstation has been aging well, and with previous SmartOS and now FreeBSD, I hope it will be stable for many years forwards.

So I don’t know what was the problem with FreeBSD 13 and random lzma and checksum errors when fetching packages that took some time (still under a half a minute). Sometimes when I retried package it would succeed but any larger fetch and it was corrupted. The same thing happened on FreeBSD 13 as SmartOS VM on same machine. But there was no problem with packages on SmartOS global zone.

I am aspiring porter, and sys programmer and contributer to FreeBSD but at this stage I’m total noob so I dont have sudficient knowledge to look up the sources, however its not worth it now since it will be EoL. I do remain curious though..
 
I couldn’t even upgrade to 14 because of these problems, so I had to do fresh install. But I also bought a new drive so maybe it was faulty SSD?
 
If you have no known issues with your internet connection then try to check the RAM of the computer in case of issues with archives like "CRC error" or "corrupted/damaged archive".
Memory (RAM) failures very ofthen affect on tasks like archiving/unpacking.
Try to run any memory test like Memtest86 or try to compress some large file(s) (~10-20GB) using some archiver with compression enabled and try to test that archive when compressed.
I guess now with 14.1 I will buy another identical drive for ZFS mirror. I have ECC ddr3 6x2 GB, if RAM ever fails I can just exchange it.
 
If it happened on multiple filetypes for larger transfers, there may be network stack issues to review. Testing a download of a file with known checksum and seeing that checksum vary would be a sign to look there. Once the checksum is a match then it should have worked.
If corruption only showed on LZMA with a good checksum and no other content, then I'd start suspecting an issue with LZMA code or the hardware executing it. An updated 13.2 uses xz v5.4.1 while 14.1 uses xz 5.4.5 (with a 5.6 archive compatibility patch).
 
I had file-copying (scp) issues (on OpenBSD) that turned out to be RAM - I assumed it was anything but RAM - surely it would be network or drives or OS bugs - but no, it was bad RAM.

You've changed the OS and the SSD so it's hard to know what really fixed it (and if it is definitely fixed! Hopefully.)
 
I had file-copying (scp) issues (on OpenBSD) that turned out to be RAM - I assumed it was anything but RAM - surely it would be network or drives or OS bugs - but no, it was bad RAM.

You've changed the OS and the SSD so it's hard to know what really fixed it (and if it is definitely fixed! Hopefully.)
I have pulled out that disk, still have it. If I wanted to I could use DTrace or download a file verifying checksum like Mirror176 suggested. Currently just enjoying the new system :beer:
 
Back
Top