Problem with pkg upgrade

F

Friend Of Jolly Devil

From time to time I had problems with pkg download speed, but everything else was ok, and starting it again was solving the issue. At the beginning of this week 'pkg upgrade' didn't work, after downloading pkg complained about package size mismatch error. Starting it again and again didn't help, when left alone it was just downloading one package in a loop, but it was increasing downloaded package count. Question is, what happened? How to repeat that behaviour on my own infrastructure? Thanks in advance.
 
Looks like some hardware problem with network card (is it wifi interface?) - try using another interface (maybe some usb stick?)
or with memory - run memtest and check if your RAM is ok.
 
3301 said:
Looks like some hardware problem with network card (is it wifi interface?) - try using another interface (maybe some usb stick?)
or with memory - run memtest and check if your RAM is ok.
Click to expand...

No, it's Realtek 8168/8111 ethernet card. I don't have any wifi card, I have a 3g gsm modem but last time I checked it didn't work at all. Memtest returns no errors

eternal_noob said:
Maybe try another pkg mirror. See https://forums.freebsd.org/threads/slow-package-fetching.83230/page-2#post-546049
Click to expand...

According to this thread it is not a good idea, but maybe I'll try.

grahamperrin said:
That's a peculiar symptom. Please share outputs from the commands below.

tunefs -p /

freebsd-version -kru ; uname -aKU

pkg -vv | grep -e url -e enabled

pkg prime-origins | sort
Click to expand...

I have zfs, not ufs.
Code: 
fbsdzroot               9.26G   274G       96K  /fbsdzroot
fbsdzroot/ROOT          6.92G   274G       96K  none
fbsdzroot/ROOT/default  6.92G   274G     6.92G  /
fbsdzroot/tmp            280K   274G      280K  /tmp
fbsdzroot/usr           2.33G   274G       96K  /usr
fbsdzroot/usr/home       648M   274G      648M  /usr/home
fbsdzroot/usr/ports     1.01G   274G     1.01G  /usr/ports
fbsdzroot/usr/src        702M   274G      702M  /usr/src
fbsdzroot/var           1.01M   274G       96K  /var
fbsdzroot/var/audit       96K   274G       96K  /var/audit
fbsdzroot/var/crash       96K   274G       96K  /var/crash
fbsdzroot/var/log        516K   274G      516K  /var/log
fbsdzroot/var/mail       120K   274G      120K  /var/mail
fbsdzroot/var/tmp        112K   274G      112K  /var/tmp

pool: fbsdzroot
 state: ONLINE
config:

    NAME           STATE     READ WRITE CKSUM
    fbsdzroot  ONLINE       0     0     0
      ada0p4       ONLINE       0     0     0

errors: No known data errors

Code: 
13.0-RELEASE-p6
13.0-RELEASE-p6
13.0-RELEASE-p6
 
FreeBSD *** 13.0-RELEASE-p6 FreeBSD 13.0-RELEASE-p6 #0: Mon Jan 10 06:28:50 UTC 2022     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 1300139 1300139

Code: 
 url             : "pkg+http://pkg.FreeBSD.org/FreeBSD:13:amd64/quarterly",
    enabled         : yes,

Maybe it should be better to change it to https? Now it's the default.

Code: 
archivers/cabextract
archivers/php80-phar
archivers/php80-zip
archivers/php80-zlib
archivers/unzip
audio/faac
audio/nas
audio/pulseaudio
converters/php80-iconv
databases/mariadb105-client
databases/mariadb105-server
databases/pecl-redis
databases/php80-mysqli
databases/php80-pdo_mysql
databases/php80-pdo_sqlite
databases/php80-sqlite3
databases/phpmyadmin5
devel/automake
devel/git
devel/gmake
devel/libburn
devel/libtool
devel/libublio
devel/p5-Locale-gettext
devel/php80-readline
devel/php80-tokenizer
devel/pkgconf
editors/emacs
editors/libreoffice
editors/vim
emulators/linux_base-c7
ftp/php80-curl
graphics/drm-kmod
graphics/evince
graphics/feh
graphics/phototonic
graphics/php80-gd
graphics/ristretto
japanese/lynx
java/icedtea-web
lang/php80
mail/thunderbird
math/galculator
math/php80-bcmath
misc/freebsd-doc-en
misc/freebsd-doc-pl
misc/help2man
misc/mc-nox11
misc/xfce4-wm-themes
multimedia/mp4v2
multimedia/transcode
multimedia/vlc
net/ifstat
net/php80-soap
net/rsync
polish/libreoffice
ports-mgmt/dialog4ports
ports-mgmt/pkg
print/foomatic-db-engine
print/gutenprint
print/hplip
print/qpdfview
print/texinfo
security/openssl
security/php80-filter
security/php80-openssl
sysutils/automount
sysutils/cdrdao
sysutils/desktop-installer
sysutils/dmidecode
sysutils/dvd+rw-tools
sysutils/exfat-utils
sysutils/fusefs-exfat
sysutils/fusefs-ext2
sysutils/fusefs-hfsfuse
sysutils/fusefs-libs
sysutils/fusefs-ntfs
sysutils/fusefs-simple-mtpfs
sysutils/php80-fileinfo
sysutils/php80-posix
sysutils/xfburn
sysutils/xfce4-battery-plugin
sysutils/xfce4-wavelan-plugin
textproc/php80-ctype
textproc/php80-dom
textproc/php80-simplexml
textproc/php80-xml
textproc/php80-xmlwriter
www/chromium
www/elinks
www/firefox
www/links
www/nginx
www/php80-session
www/w3m-img
x11-drivers/xf86-video-intel
x11-drivers/xf86-video-mga
x11-drivers/xorg-drivers
x11-fm/pcmanfm
x11-fonts/bitstream-vera
x11-fonts/droid-fonts-ttf
x11-fonts/fira
x11-fonts/liberation-fonts-ttf
x11-fonts/webfonts
x11-themes/lumina-themes
x11-themes/lxappearance
x11-themes/qtcurve-gtk2
x11-themes/qtcurve-qt5
x11-toolkits/tk87
x11-wm/i3
x11-wm/lxqt
x11-wm/xfce4
x11/dmenu
x11/dzen2
x11/hs-xmobar
x11/i3blocks
x11/i3lock
x11/i3status
x11/j4-dmenu-desktop
x11/lemonbar
x11/lumina
x11/qterminal
x11/rofi
x11/slim
x11/xdm
x11/xfce4-screensaver
x11/xfce4-screenshooter-plugin
x11/xlockmore
x11/xorg
x11/xsm
x11/zenity
 
Last edited by a moderator:
It could be my ethernet kernel driver, as in this thread, but I don't have watchdog timeouts in /var/log/messages. And isos downloading works fine. I've also found suggestion to force bootstrap of pkg, it may solve some issues. I think I'll make a fresh install on another computer first and see what'll happen.
 
Access to your particular mirror could be problematic. It pays to try another mirror every now and then, just to check. In some cases that might even make things worse though. Downloaded packages are cached in /var/cache/pkg/, especially with checksum errors it can help to remove that cached version. You can also use pkg-clean(8) to clean up the cache.
 
I think I've finally got it. It seems that it was a mixture of bad luck and some pkg imperfection. At the exact moment I was upgrading, there was a minor version change of openjdk8, and in the result pkg tried to download a frankensteined version, it started download with one version, and ended with another. It noticed the mismatch, but it didn't noticed version change, so it repeatedly tried to download the old version, while the contents were from the new one, hence the mismatch. I should pkg update after the first mismatch, it would solved it. Can someone confirm it is plausible?
 
Friend Of Jolly Devil said:
… I should pkg update after the first mismatch, it would solved it. …
Click to expand...

I doubt it.

Update is part of the routine, for example:

Code: 
root@mowa219-gjp4-8570p-freebsd:~ # pkg upgrade
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
Checking for upgrades (2 candidates): 100%
Processing candidates (2 candidates): 100%
Checking integrity... done (0 conflicting)
Your packages are up to date.
root@mowa219-gjp4-8570p-freebsd:~ #
 
Friend Of Jolly Devil said:
From time to time I had problems with pkg download speed, … 'pkg upgrade' … mismatch … Starting it again and again didn't help, when left alone it was just downloading one package in a loop, …
Click to expand...

In this recent case <https://old.reddit.com/comments/sbeekd/-/> the end user diagnosed a side effect with a socks5 proxy / ssh tunnel.

Friend Of Jolly Devil said:
… increasing downloaded package count. …
Click to expand...

This symptom is vaguely familiar. Probably a cosmetic bug, which I never bothered to report, because it's sometimes difficult to reproduce.
 
grahamperrin said:
I doubt it.

Update is part of the routine, for example:

Code: 
# pkg upgrade
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
Click to expand...

But it does this only once at the beginning. When change in repositories happens after this updating, what then? Does pkg auto-detect it?

grahamperrin said:
In this recent case <https://old.reddit.com/comments/sbeekd/-/> the end user diagnosed a side effect with a socks5 proxy / ssh tunnel.
Click to expand...

I don't have any socks5 proxy / ssh tunnel. It seems that he had the exact same problem, but probably it solved itself independently while he was trying socks5 proxy / ssh tunnel setting, hence his statement.
In my case I allowed the second fetch to complete, it was broken too. After SirDice post I checked /var/cache/pkg. I have two openjdk8 files there: 1-st version 8.302.08.1_2, its size matches the first download, and 2-nd v. 8.312.07.1 which size matches the second download, so there was a change in package version after 'pkg update' completed itself. It looks like pkg saw this change only partially, and the second time it claimed downloading the exact same package - at least exactly the same version number in both cases was printed. I can't tell if it's just a cosmetic bug or not, it's hard to find more detailed info about how pkg exactly works.
Is there any way to download both versions manually to compare them with the ones in my cache?

grahamperrin said:
This symptom is vaguely familiar. Probably a cosmetic bug, which I never bothered to report, because it's sometimes difficult to reproduce.
Click to expand...

I suspect it's because of combination of random factors, maybe the above explanation will help somehow to reproduce it.
 
I've installed FreeBSD 13.0 p6 on a different computer with intel network card, I have issues with pkg there too. At the beginning pkg didn't work at all, I had to manually set a working mirror like eternal_noob suggested. It seems that pkg insists on using isc mirror, which is completely down for quite some time now.
 
Try to manually set another mirror by editing /etc/pkg/FreeBSD.conf and changing 'pkg.freebsd.org' to one from the list at the bottom of the https://pkg.freebsd.org/ page. The page tells you which mirror you are hitting (e.g. pkg0.pkg.freebsd.org), so you can choose another one from the list.

If the problem persist with other mirrors and on different machines, there may be something fishy with your connection. E.g. proxies with TLS-termination and/or intrusive DPI come to mind...

Most likely you are just hitting the mirror during a resync. This sometimes can lead to inconsistencies between available packages and the database. Trying again a few hours later (or sometimes even days e.g. after a major release or some other event that causes lots of packages to be upgraded/rebuilt) and maybe cleaning the cache and rebuilding the catalogue ( pkg clean -a && pkg update -f) usually works.
If the problem is only reproducible with one particular mirror (the pkg.freebsd.org page usually tells you which mirror is selected for your location) over a longer period of time, open a PR so someone with insights to that mirror can take a look at it.
 
sko said:
Try to manually set another mirror by editing /etc/pkg/FreeBSD.conf and changing 'pkg.freebsd.org' to one from the list at the bottom of the https://pkg.freebsd.org/ page.
Click to expand...
Please don't edit /etc/pkg/FreeBSD.conf, make a /usr/local/etc/pkg/repos/FreeBSD.conf and put the URL in there. No need to copy anything else:
Code: 
FreeBSD: {
  url: <url of the mirror>
}

The reasoning for this is that /etc/pkg/FreeBSD.conf might get changed with an update and that could remove your changes. Setting it in /usr/local/etc/pkg/repos/FreeBSD.conf will make sure your changes are going to survive this.
 
  • Thanks
Reactions: sko
SirDice said:
The reasoning for this is that /etc/pkg/FreeBSD.conf might get changed with an update and that could remove your changes. Setting it in /usr/local/etc/pkg/repos/FreeBSD.conf will make sure your changes are going to survive this.
Click to expand...
The changes I mentioned were only meant as a temporary measure to verify/locate the problem; I should have pointed that out.
But yes - good practice is to always use /usr/local/etc for such configuration overrides if possible.
 
pkg0.isc.freebsd.org

Friend Of Jolly Devil said:
… It seems that pkg insists on using isc mirror, …
Click to expand...

sko said:
… Most likely you are just hitting the mirror during a resync. …
Click to expand...

Friend Of Jolly Devil said:
…

pkg0.isc.FreeBSD.org doesn't work at all.

it's reproducible, …
Click to expand...

<https://github.com/ehaupt/fastest_pkg/issues/2#issuecomment-1000897777> (2021-12-24), again on 2021-12-27:

pkg0.isc.freebsd.org: 0.0 B/s
Click to expand...

<https://github.com/ehaupt/fastest_pkg/issues/2#issuecomment-1001326332> (2021-12-27):

pkg0.isc.freebsd.org seem to be unreachable ATM.
Click to expand...

<https://github.com/ehaupt/fastest_pkg/issues/2#issuecomment-1002160401> (2021-12-28):

That's correct. If you run it with -v you'll see the cURL error message.
Click to expand...

01:09 today:

Code: 
% date ; fastest_pkg -v
Fri 28 Jan 2022 01:09:47 GMT
pkg0.tuk.freebsd.org: 69%
(28, 'Operation timed out after 5000 milliseconds with 4672768 out of 6699232 bytes received')

pkg0.tuk.freebsd.org: 934.5 kB/s
pkg0.nyi.freebsd.org: 88%
(28, 'Operation timed out after 5000 milliseconds with 5908905 out of 6699232 bytes received')

pkg0.nyi.freebsd.org: 1.2 MB/s
pkg0.isc.freebsd.org: 0%
(28, 'Failed to connect to pkg0.isc.freebsd.org port 80 after 2746 ms: Operation timed out')
pkg0.isc.freebsd.org: 0.0 B/s
pkg0.bme.freebsd.org: 100%
pkg0.bme.freebsd.org: 6.9 MB/s
pkg0.pkt.freebsd.org: 100%
pkg0.pkt.freebsd.org: 6.8 MB/s

Fastest:
pkg0.bme.freebsd.org: 6.9 MB/s


Write configuration:
mkdir -p /usr/local/etc/pkg/repos/
echo 'FreeBSD: { url: "http://pkg0.bme.freebsd.org/${ABI}/latest" }' \
        > /usr/local/etc/pkg/repos/FreeBSD.conf


%

Friend Of Jolly Devil said:
… It seems that pkg insists on using isc mirror, …
Click to expand...

pkg-update(8) has no option -v for verbosity.

Friend Of Jolly Devil said:
… It seems that https://pkg.freebsd.org/ has no idea that isc doesn't work, and it redirects to it sometimes. …
Click to expand...

When http://pkg0.isc.freebsd.org/ occurs, in the web browser, does the page time out?
 
You must log in or register to reply here.
Back
Top