pkg segfault abort, signal 6, runs out of memory, package not installed

pkg consumes too much memory and finally it crashes with signal 6, abort. I have tried too many times without success. I've installed the newest development version with debug symbols from ports tree, it did not change anything.

I have just recently updated to FreeBSD 11.0-RELEASE from 10.3-RELEASE probably because of this same reason.

# pkg -v
1.10.99.4
# uname -a
FreeBSD my_hostname 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: Thu May 18 12:10:33 EEST 2017 my_machine_name_here i386


gdb -c ./pkg.core <path to>/pkg:
Code:
Loaded symbols for /libexec/ld-elf.so.1
#0  0x28745f63 in thr_kill () from /lib/libc.so.7
[New Thread 28c14000 (LWP 101056/<unknown>)]
(gdb) bt
#0  0x28745f63 in thr_kill () from /lib/libc.so.7
#1  0x28745f48 in raise () from /lib/libc.so.7
#2  0x28745e83 in abort () from /lib/libc.so.7
#3  0x280bb61b in xcalloc (n=1, size=2168) at xmalloc.h:16
#4  0x280c11f9 in pkg_addfile_attr (pkg=0x2ad9aa00, path=0x7fbfd680 "/usr/local/include/boost/chrono/chrono_io.hpp",
    sum=0x7ece45a0 "1$5c90efc00d030aca3338af63039aef1c9656b0dfca4050cfc4c175e65d14734e", uname=0x0, gname=0x0, perm=0, fflags=0,
    check_duplicates=false) at pkg.c:715
#5  0x280c0efd in pkg_addfile (pkg=0x2ad9aa00, path=0x29f4dc90 "/usr/local/include/boost/chrono/chrono_io.hpp",
    sum=0x7ece45a0 "1$5c90efc00d030aca3338af63039aef1c9656b0dfca4050cfc4c175e65d14734e", check_duplicates=false) at pkg.c:688
#6  0x28112ee0 in pkg_obj (pkg=0x2ad9aa00, obj=0x60ffe4d0, attr=4294967293) at pkg_manifest.c:539
#7  0x28115339 in parse_manifest (pkg=0x2ad9aa00, keys=0x6151d920, obj=0x616e3ac0) at pkg_manifest.c:763
#8  0x2810f628 in pkg_parse_manifest_ucl (pkg=0x2ad9aa00, obj=0x616e3ac0, keys=0x6151d920) at pkg_manifest.c:799
#9  0x2810ef70 in pkg_parse_manifest (pkg=0x2ad9aa00,
    buf=0x7f60c080 "{\"name\":\"boost-libs\",\"origin\":\"devel/boost-libs\",\"version\":\"1.64.0\",\"comment\":\"Free portable C++ libraries (without
Boost.Python)\",\"maintainer\":\"office@FreeBSD.org\",\"www\":\"http://www.boost.org/\",\"abi\""..., len=1646381, keys=0x6151d920) at pkg_manifest.c:829
#10 0x280c6c84 in pkg_open2 (pkg_p=0x7fbfde90, a=0x7fbfde2c, ae=0x7fbfde28, path=0x7fbfdeac "/var/cache/pkg/boost-libs-1.64.0-16686fd8af.txz",
    keys=0x6151d920, flags=4, fd=-1) at pkg.c:1469
#11 0x280c67f7 in pkg_open (pkg_p=0x7fbfde90, path=0x7fbfdeac "/var/cache/pkg/boost-libs-1.64.0-16686fd8af.txz", keys=0x6151d920, flags=4)
    at pkg.c:1360
#12 0x282f8d25 in pkg_repo_binary_ensure_loaded (repo=0x28c37070, pkg=0x2a2fa000, flags=36) at query.c:439
#13 0x2814818c in pkgdb_ensure_loaded (db=0x28c249a0, pkg=0x2a2fa000, flags=36) at pkgdb_iterator.c:1245
#14 0x280f171e in pkg_jobs_solve (j=0x28c29430) at pkg_jobs.c:1854
#15 0x0806aecb in exec_upgrade (argc=0, argv=0x7fbfec18) at upgrade.c:402
#16 0x0805f168 in main (argc=2, argv=0x7fbfec10) at main.c:886
(gdb)
The line in the file is:

Code:
    12  static inline void *xcalloc(size_t n, size_t size)
    13  {
    14          void *ptr = calloc(n, size);
    15          if (ptr == NULL)
    16                  abort();
    17          return (ptr);
    18  }
And so, indeed memory runs out. I had to install more swap to overcome this issue, it did not help. The actual cause is the calloc, not the function. Am I right, the pointer if it is returned, is copied? I would replace the return line with &(*ptr) to be sure to point to the allocated memory only even after the segment changes (I should check this). This change did not do anything.

With pkg -v debug upgrade or pkg -d install

DBG(1)[87878]> Binary> loading /var/cache/pkg/ca_root_nss-3.30.2-c1809a469f.txz
DBG(1)[87878]> Binary> loading /var/cache/pkg/ca-gimp-help-html-2.8.2-aca12492a6.txz
DBG(1)[87878]> Binary> loading /var/cache/pkg/c64bdf-1.0_3-8f4816e7c7.txz
DBG(1)[87878]> Binary> loading /var/cache/pkg/brasero-3.12.1_1-9c583f4747.txz
DBG(1)[87878]> Binary> loading /var/cache/pkg/bovo-4.14.3_1-60d910acd8.txz
DBG(1)[87878]> Binary> loading /var/cache/pkg/boost-python-libs-1.64.0-2298f21166.txz
DBG(1)[87878]> Binary> loading /var/cache/pkg/boost-libs-1.64.0-16686fd8af.txz
DBG(1)[87898]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[87898]> PkgRepo: verifying update for FreeBSD
DBG(1)[87898]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'

# Abort trap (core dumped)
# signal 6


Many have found a resolution to fix this issue from gettext and gettext-runtime dependency as in the next links. Many have reported the same error, pkg has stayed in a loop for a long time and finally crashed without notifying the reason (ie. a dependency after reading the newsgroups and debugging).

Other posts:
[ https://forums.freebsd.org/threads/49475/ keywords: wget pkg, gettext, gettext-runtime ]
[ https://forums.freebsd.org/threads/55772/ keywords: pkg abort trap ]

pkg:
[ https://github.com/freebsd/pkg ]
[ https://wiki.freebsd.org/pkgng ]

I had found a bug report as well in an URL not found from above links. Some hints in posts were to remove duplicate rows from the database file with an SQL-editor.

My pkg crashes now almost every time when I'm installing a package. Almost every package (recent was lxde-meta, previous was probably texmaker, almost any smaller ones as well). What is the reason in my case? Do I have to update the database somehow?

How do I find the missing dependencies? It would be nice if the pkg would print lines to debug these issues instead of using all of the memory (2GB + 1,5GB swap here) and crashing.

Any other resolution before listing everything, deleting everything and installing everything again? (if this helps.)

esc
 
An example. This one would delete kde4 I'm using at the moment. It would download 3 GB of files to update three libraries.

Code:
# pkg check --dependencies
Checking all packages: 100%
cvsup is missing a required shared library: libXaw7.so.7
opera is missing a required shared library: libxml2.so.5
texmaker has a missing dependency: qt5-webkit

>>> Missing package dependencies were detected.
>>> Found 1 issue(s) in the package database.

Updating database digests format: 100%
The following packages will be installed:

Installed packages to be REMOVED:
   cvsup-16.1h_6
   dri-11.2.2,2
   libEGL-11.2.2
   x264-0.148.2728
   gbm-11.2.2
   doxygen-1.8.13,2
   pdftk-2.02_4
   opera-12.16_6
   intltool-0.51.0_1
   gnome-speech-0.4.25_3
   libGL-11.2.2
   libglesv2-11.2.2
   py35-gobject3-3.18.2
   guile2-2.0.11_3
   opencv2-2.4.13.1_5
   kdenetwork-4.14.3_1
   gnome3-lite-3.18.0_1
   gimp-2.8.18,2
   py35-dbus-1.2.0_1
   py35-cairo-1.10.0_3
   py35-speech-dispatcher-0.8.6
   mate-1.12.0
   memcached-1.4.34_2
   openvpn-2.4.0
   wget-1.18_2
   gnome-games-3.18.0
   aisleriot-3.18.2
   py35-atspi-2.18.0
   samba36-libsmbclient-3.6.25_2
   mono-4.6.2.7_2
   libaacplus-2.0.2_7
   kdeutils-4.14.3_1
   kde-4.14.3_1
   qca-ossl-2.0.0.b3_4
   py34-dbus-1.2.0_1
   py35-libpeas-1.16.0
   libwps03-0.3.1_2
   libodfgen-0.0.4_1
   libetonyek-0.0.4_1
   seed-3.8.1_1
   mate-calc-1.8.0_1
   mate-dialogs-1.8.0_1
   liborcus07-0.7.1_2
   gnupg1-1.4.21_1
   zxid-1.22
   libzeitgeist-0.3.18_2
   libbzrtp-1.0.3
   llvm37-3.7.1_4
   p5-MIME-Base64-3.15

New packages to be INSTALLED:
   qt5-webkit: 5.7.1
   mesa-libs: 17.0.4
   llvm40: 4.0.1.r1_4
   gcc5: 5.4.0_2
   py27-setuptools: 32.1.0_1
   harfbuzz-icu: 1.4.6_1
   gstreamer1-plugins-gl: 1.8.0
   mozjpeg: 3.2_1
   libidn2: 2.0.2
   rhash: 1.3.4
   python36: 3.6.1_1
   libgnome-games-support: 1.2.1
   py36-gobject3: 3.18.2
   py36-cairo: 1.10.0_3
   libsunacl: 1.0
   py27-dnspython: 1.15.0
   py27-iso8601: 0.1.11
   ldb: 1.1.29_1
   openldap-sasl-client: 2.4.44
   samba44: 4.4.13
   opencv-core: 2.4.13.1_1
   grilo2: 0.2.15
   grilo-plugins2: 0.2.17
   kf5-oxygen-icons5: 5.34.0
   py36-xdg: 0.25_1
   py36-setuptools: 32.1.0_1
   py36-dbus: 1.2.0_1
   py36-speech-dispatcher: 0.8.6
   py36-atspi: 2.24.0
   gspell: 1.4.1
   py36-libpeas: 1.16.0
   opencv: 2.4.13.1_6
   x10: 2.2.1_5
   mesa-dri: 17.0.4
   s2tc: 1.0+20151228
   libunwind: 20170113
   compat7x-i386: 7.4.704000.201310.1
   postgresql95-client: 9.5.7
   emojione-color-font-ttf: 1.3
   libstaroffice: 0.0.3
   libzmf: 0.0.1

Installed packages to be UPGRADED:
   libXi: 1.7.8,1 -> 1.7.9,1
   libX11: 1.6.4,1 -> 1.6.5,1
   libxcb: 1.12 -> 1.12_2

... too many lines here to fit in the post ...

   p5-XML-LibXML: 2.0128,1 -> 2.0129,1
   llvm35: 3.5.2_2 -> 3.5.2_4
   llvm38: 3.6.2_2 -> 3.8.1_8
   python33: 3.3.6_6 -> 3.3.6_7
   p5-Locale-gettext: 1.06 -> 1.07
   snappy: 1.1.3 -> 1.1.4
   leveldb: 1.19 -> 1.20_1

Installed packages to be REINSTALLED:
   libXrender-0.9.10
   libXfixes-5.0.3
   libXdmcp-1.1.2
   libXxf86vm-1.1.4_1
   libXext-1.3.3_1,1
   libXp-1.0.3,1
   libXau-1.0.8_3
   xsetroot-1.1.1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libSM-1.2.2_3,1
   libICE-1.0.9_1,1
   xcb-util-wm-0.4.1_3
   xcb-util-0.4.0_2,1
   xbacklight-1.2.1_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXrandr-1.5.1
   mkfontscale-1.1.2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXfont-1.5.2,2
   pixman-0.34.0
   startup-notification-0.12_4
   libXdamage-1.1.4_3
   gtkhtml3-3.32.1_5 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXinerama-1.1.3_3,1
   libXv-1.0.11,1
   libXvMC-1.0.10
   expat-2.2.0_1
   tidy-lib-090315.c_3
   libXmu-1.1.2_3,1
   libXt-1.1.5,1
   nspluginwrapper-1.4.4_7 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXcursor-1.1.14_3
   mousetweaks-3.12.0_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXtst-1.2.3
   fontconfig-2.12.1,1
   libXcomposite-0.4.4_3,1
   libXScrnSaver-1.2.2_3
   qimageblitz-0.0.6_2
   libXpm-3.5.12
   libffi-3.2.1
   libyaml-0.1.6_2
   libxml2-2.9.4
   trousers-0.3.14_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   tpm-emulator-0.7.4_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   pinentry-gnome3-1.0.0 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   teckit-2.5.1_2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   xpdfopen-0.86 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libxkbui-1.0.2_4 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libxkbfile-1.0.9
   xclock-1.0.7_2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXaw-1.0.13,2
   xterm-327 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXft-2.3.2_1
   libXxf86misc-1.0.3_3
   mate-screensaver-1.12.0_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   mate-session-manager-1.12.1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   xdpyinfo-1.3.2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libdmx-1.1.3_3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   gdm-3.16.4_2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   zenity-3.18.0 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   tiff-4.0.7_1
 ... too many lines here to fit in the post ..
   cagibi-0.2.0 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   hupnp-1.0.0_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   psutils-1.17_5 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   spandsp-0.0.6 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   t1lib-5.1.2_4,1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   xcb-2.4_3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   firebird25-client-2.5.6_2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   lua51-5.1.5_9 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   gmime24-2.4.33_3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   gnome-menus-3.13.3_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   meanwhile-1.0.2_6 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libgee06-0.6.8_2 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libotr-4.1.1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   xml2-0.5_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   idnkit-1.0_6 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   xanalyser-1.32_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   alsa-lib-1.1.2
   alsa-plugins-1.1.1_1 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   silgraphite-2.3.1_4 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXTrap-1.0.1_3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   liboldX-1.0.1_3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libXevie-1.0.3_3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   libexttextcat-3.4.4 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   lp_solve-5.5.2.3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   CoinMP-1.8.3 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')
   xlsfonts-1.0.5 (ABI changed: 'freebsd:10:x86:32' -> 'freebsd:11:x86:32')

Number of packages to be removed: 49
Number of packages to be installed: 41
Number of packages to be upgraded: 532
Number of packages to be reinstalled: 537

The process will require 2 GiB more space.
3 GiB to be downloaded.

>>> Try to fix the missing dependencies? [y/N]:

esc
 
After the upgrade from 10.x to 11.x you need to reinstall all ports/packages. Start with this:
Code:
pkg-static install -f pkg # this will 'fix' pkg 
pkg upgrade -f # this will reinstall [i]everything[/i]
 
The development version was newer so I left it.

Installed packages to be DOWNGRADED:
pkg: 1.10.99.4 -> 1.10.1


This script was left on for more than 9 hours:

while true
do
pkg -d upgrade -y -f -U 2>> /var/log/upgrd.log 1>> /var/log/upgrd.log &&
break ||
rm pkg.core
done


These lines occur frequently in every run:

...
DBG(1)[38933]> trying to delete local package samba36-libsmbclient-3.6.25_2 on install/upgrade, reiterate on SAT
pkg: sqlite error while executing UPDATE packages SET name=?1 WHERE name=?2; in file pkg_jobs.c:1714: UNIQUE constraint failed: packages.name
pkg: sqlite error while executing UPDATE packages SET name=?1 WHERE name=?2; in file pkg_jobs.c:1714: UNIQUE constraint failed: packages.name
...


Otherwise the problem remains unsolved. Should the pkg to be downgraded to 10.3 or 10.2 version.

And there were "Abort" -messages in the console one after another, the core dump size is huge.

The pkg starts downloading the packages even if there were the same packages already in the directory causing the update time to increase.

What to do? Should something to be deleted first?

esc
 
Should the pkg to be downgraded to 10.3 or 10.2 version.
All versions of FreeBSD use the same ports tree and therefor have the same versions of applications, including ports-mgmt/pkg

he pkg starts downloading the packages even if there were the same packages already in the directory causing the update time to increase.
That's because it needs to replace all 10.x packages for their 11.x versions. The version of the package stays the same but the old packages have been built for 10.x and you need packages for 11.x now.
 
The final solution was to make a list of installed packages pkg info | awk '{ print $1 }' | awk '{ FS="-[0-9]"; print $1 }', delete all of them, rename file /var/db/pkg/local.sqlite and install only the needed back.

Old configurations were mostly found after installing. Easiest could have been to remove the UNIQUE constraint sql -error from the sqlite database with an sqlclient. This was not tested.
 
Back
Top