Solved Power loss while updating ports

freethread · Jun 2, 2026

The power went out while I was updating the ports on a VM:

Code:

# uname -a
FreeBSD iantomarus 15.0-RELEASE-p9 FreeBSD 15.0-RELEASE-p9 GENERIC amd64
# zfs list
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
zroot                                         57.4G   101G    96K  /zroot
zroot/ROOT                                    41.0G   101G    96K  none
zroot/ROOT/15.0-RELEASE-p1_2026-02-01_183205     8K   101G  18.7G  /
zroot/ROOT/15.0-RELEASE-p2_2026-03-01_105108     8K   101G  18.4G  /
zroot/ROOT/15.0-RELEASE-p4_2026-04-01_080659     8K   101G  21.3G  /
zroot/ROOT/15.0-RELEASE-p5_2026-05-01_142206     8K   101G  19.7G  /
zroot/ROOT/15.0-RELEASE-p7_2026-05-01_193749     8K   101G  19.9G  /
zroot/ROOT/15.0-RELEASE-p8_2026-06-01_190344     8K   101G  21.1G  /
zroot/ROOT/default                            41.0G   101G  21.2G  /
zroot/home                                    1.94G   101G    96K  /home
zroot/home/paolo                              1.94G   101G  1.94G  /home/paolo
zroot/tmp                                      128K   101G   128K  /tmp
zroot/usr                                     14.3G   101G    96K  /usr
zroot/usr/ports                               13.4G   101G  13.4G  /usr/ports
zroot/usr/src                                  977M   101G   977M  /usr/src
zroot/var                                     70.8M   101G    96K  /var
zroot/var/audit                                 96K   101G    96K  /var/audit
zroot/var/crash                               68.5M   101G  68.5M  /var/crash
zroot/var/log                                 1.79M   101G  1.79M  /var/log
zroot/var/mail                                 160K   101G   160K  /var/mail
zroot/var/tmp                                  104K   101G   104K  /var/tmp

Now running pkg info:

Code:

# pkg info
7-zip-26.01                    Console version of the 7-Zip file archiver
Andika-6.200                   Sans-serif font designed for literacy use
AppStream-1.1.2                Machine-readable software metadata for desktop environments
GraphicsMagick-1.3.47,1        Fast image processing tools based on ImageMagick
ImageMagick7-7.1.2.24          Image processing tools
Imath-3.2.2_4                  C++/Python lib of 2D/3D vector, matrix, math ops for computer graphics
aalib-1.4.r5_14                ASCII art library
abseil-20250127.1_1            Abseil Common Libraries (C++)
accountsservice-26.13.3_1      D-Bus interface for user account query and manipulation
adwaita-icon-theme-47.0        GNOME Symbolic Icons
adwaita-icon-theme-legacy-46.2 Full color icon theme for legacy applications
alef-1.0_4                     Free multi-lingual font designed for screens
alegreya-sans-2.008_1,1        Humanist sans serif font with a calligraphic feeling
alsa-lib-1.2.15.3              ALSA compatibility library
anonymous-pro-1.002_3          Fixed width sans designed especially for coders
aom-3.14.0                     AV1 reference encoder/decoder
appres-1.0.7                   Program to list application's resources
appstream-glib-0.8.3_1         Library to help with AppStream metadata
asciidoc-10.2.1                Text document format for writing short documents and man pages
at-spi2-core-2.60.4            Assistive Technology Service Provider Interface
atkmm-2.28.4                   C++ wrapper for ATK API library
autoconf-2.73                  Generate configure scripts and related files
autoconf-switch-20220527       Wrapper script to switch between autoconf versions
automake-1.18.1                GNU Standards-compliant Makefile generator
avahi-app-0.8_6                Service discovery on a local network
b612-1.008                     Highly legible font family for aircraft cockpit screens
babl-0.1.124                   Dynamic pixel format conversion library
bash-5.3.9                     GNU Project's Bourne Again SHell
Bus error (core dumped)

I think I can get the old pkg backup database in /var/backups but I do not know how to use it. The files in /var/backups are:

Code:

  > ls /var/backups/
total 25 MB
-rw-r--r--  1 root wheel uarch  1.5K 2026.01.06 11:07 aliases.bak
-rw-r--r--  1 root wheel uarch  512K 2026.05.02 03:01 boot.ada0p2.bak
-rw-r--r--  1 root wheel uarch  512K 2026.01.07 03:01 boot.ada0p2.bak2
-rw-r--r--  1 root wheel uarch  189B 2026.01.07 03:01 gpart.ada0.bak
-rw-r--r--  1 root wheel uarch  706B 2025.12.12 17:59 group.bak
-rw-r--r--  1 root wheel uarch  571B 2026.05.02 03:01 kern.geom.conftxt.bak
-rw-r--r--  1 root wheel uarch  571B 2026.01.07 03:01 kern.geom.conftxt.bak2
-rw-------  1 root wheel uarch  2.8K 2025.12.13 15:19 master.passwd.bak
-rw-r--r--  1 root wheel uarch   11M 2026.01.07 04:01 pkg.sql.xz
-rw-r--r--  1 root wheel uarch   13M 2026.05.02 03:13 pkg.sql.zst

How can I restore the pkg sql database? And. The error is a pkg sql database error? Many thanks

freethread · Jun 2, 2026

Well, I guess it is gone. It no more starts. No way to restart it with old kernel and with saved BE, in single user mode, no way.

It freeze with this screen, no access with SSH

Emrion · Jun 2, 2026

I don't see the relation with the power outage, but you should read this: PR 295463

bakul · Jun 2, 2026

The above PR may apply (see comment #10 in it). In the screenshot above I notice the size of intel-ucode.bin is over 16MB.

freethread · Jun 2, 2026

Wait, I turned off the machine and turned off my brain too (swearing enough) to go to dinner. I rebooted the machine (Windows) started the VM (VirtualBox) and FreeBSD boot normally, maybe the problem was in the host too. It was a very short brownout (a storm lightning strike), the server, TV and other electronic devices didn't even shut down, only my desktop, probably because it was under heavy load, I not turned off the machine after power loss.
In any case, pkg gives the same error. When I update the ports, I disable LightDM and the desktop environment (plus dbus, avahi, samba, and a few other services) to keep the system lighter. Then, when everything was fine, I re-enabled all the services, LightDM started normally, I logged into XFCE (X11), and everything is fine except for the XFCE whisker menu. It was most likely the update that was running when the power went out.Now the only problem I have is pkg (I hope). I searched the forums for threads about problems with the pkg database, but I couldn't find a solution. I don't know how to use the files in /var/backups to restore the database. Any pkg command gives me the error

Code:

Bus error (core dumped)

bakul · Jun 2, 2026

Since it is a VM, first I would run "zpool scrub" to ensure the zpool/zfs structures are consistent and not damaged due to powerfail. Assuming the FS is not damaged (or fixed up), you can try something like the following as root:

Code:

mv /var/db/pkg/local.sqlite /var/db/pkg/local.sqlite.damaged
zstdcat /var/backup/pkg.sql.zstd | pkg shell
pkg check

You will likely find some issues that you can fix by reinstalling affected packages but the db should be consistent. But Note: follow the above at your own risk.

freethread · Jun 2, 2026

"zpool scrub", I have to read manpage, I'm totally ignorant about ZFS. Well I will try all tomorrow in late afternoon. Many thanks bakul.

Charlie_ · Jun 3, 2026

pkg.sql.* is a compressed file created by pkg shell .dump.
See /usr/local/etc/periodic/daily/411.pkg-backup
You should be able to restore it with .read.

The /var/log/message file records the installation and uninstallation of packages.
Please check if any changes have been made since the backup.

freethread · Jun 3, 2026

I have installed nothing new. Anyways I have to add /var/db/pkg/local.sqlite in my backup. I make backup before update system and ports.

bakul · Jun 3, 2026

freethread said:
I have installed nothing new. Anyways I have to add /var/db/pkg/local.sqlite in my backup. I make backup before update system and ports.

This is already backed up daily as /var/backup/pkg.sql.zst and prev 7 days saved with suffix .1 .. .7. The code I showed above will recreate local.sqlite. No idea why the backup is saved as a script to "pkg shell".

freethread · Jun 3, 2026

Yes, but it should more easy to replace local.sqlite with the backup, I think, otherwise I have to remember the commands to recreate it. However I have to write a note about this and place it on my local web to easily reach the solution (if it happens again). Now I ran zpool scrub zroot, the result is the following (I skipped a lot of www/firefox files in work directory:

Code:

  pool: zroot
 state: ONLINE
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 84K in 00:04:44 with 159 errors on Wed Jun  3 22:08:21 2026
config:

    NAME        STATE     READ WRITE CKSUM
    zroot       ONLINE       0     0     0
      ada0p4    ONLINE       0     0   344

errors: Permanent errors have been detected in the following files:

        /usr/ports/www/firefox/work/.build/dom/media/webrtc/transport/third_party/nICEr/nicer_nicer/ice_peer_ctx.o
        /usr/ports/www/firefox/work/.build/dom/media/webrtc/transport/third_party/nICEr/nicer_nicer/.deps/ice_peer_ctx.o.pp
        /usr/ports/www/firefox/work/.build/dom/media/webrtc/transport/build/.deps/nr_socket_proxy_config.o.pp

        ... [skipped]

        /usr/ports/www/firefox/work/.build/dom/media/webrtc/transport/third_party/nICEr/nicer_nicer/ice_util.o
        /usr/ports/www/firefox/work/.build/x86_64-unknown-freebsd/release/deps/libstyle-47a86e47ca55751e.rlib
        /usr/ports/www/firefox/work/.build/dom/media/webrtc/transport/third_party/nrappkit/nrappkit_nrappkit/.deps/Unified_cpp_nrappkit0.o.pp
        //var/db/pkg/local.sqlite
        //usr/local/lib/firefox/libxul.so

Now recreate the sqlite database of pkg

bakul · Jun 3, 2026

Note that normally zfs won't *create* permanent errors so the issue may be the underlying storage damage due to the power failure. Is zroot a file on the Windows host or a disk partition? If it is a file, you may wish to run fsck equivalent on the host (you'll have to ask windows expert). Make sure the Windows filesystem is repaired and then run zpool scrub again. After that you will need to research on how to deal with zfs permanent errors....

You should do all this *before* further work like restoring pkg database etc.

freethread · Jun 3, 2026

Done. The command pkg check get me a lot of missing file and checksum mismatch, this is the end of the result:

Code:

... [skipped]

xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xextproto.pc
xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xf86bigfontproto.pc
xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xf86dgaproto.pc
xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xf86driproto.pc
xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xf86vidmodeproto.pc
xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xineramaproto.pc
xorgproto-2024.1: checksum mismatch for /usr/local/share/pkgconfig/xproto.pc
Checking all packages:  98%
xrandr-1.5.3: checksum mismatch for /usr/local/bin/xrandr
xrandr-1.5.3: missing file /usr/local/share/licenses/xrandr-1.5.3/LICENSE
xrandr-1.5.3: missing file /usr/local/share/licenses/xrandr-1.5.3/MIT
xrandr-1.5.3: missing file /usr/local/share/licenses/xrandr-1.5.3/catalog.mk
xrandr-1.5.3: checksum mismatch for /usr/local/share/man/man1/xrandr.1.gz
Checking all packages:  98%
xterm-409: checksum mismatch for /usr/local/bin/resize
xterm-409: checksum mismatch for /usr/local/bin/xterm
xterm-409: missing file /usr/local/share/licenses/xterm-409/LICENSE
xterm-409: missing file /usr/local/share/licenses/xterm-409/MIT
xterm-409: missing file /usr/local/share/licenses/xterm-409/catalog.mk
xterm-409: checksum mismatch for /usr/local/share/man/man1/koi8rxterm.1.gz
xterm-409: checksum mismatch for /usr/local/share/man/man1/resize.1.gz
xterm-409: checksum mismatch for /usr/local/share/man/man1/uxterm.1.gz
xterm-409: checksum mismatch for /usr/local/share/man/man1/xterm.1.gz
Checking all packages:  99%
xxd-9.2.0387: checksum mismatch for /usr/local/bin/xxd
xxd-9.2.0387: missing file /usr/local/share/licenses/xxd-9.2.0387/LICENSE
xxd-9.2.0387: missing file /usr/local/share/licenses/xxd-9.2.0387/VIM
xxd-9.2.0387: missing file /usr/local/share/licenses/xxd-9.2.0387/catalog.mk
Checking all packages:  99%
zstd-1.5.7_1: checksum mismatch for /usr/local/bin/zstd
zstd-1.5.7_1: missing file /usr/local/share/licenses/zstd-1.5.7_1/BSD3CLAUSE
zstd-1.5.7_1: missing file /usr/local/share/licenses/zstd-1.5.7_1/GPLv2
zstd-1.5.7_1: missing file /usr/local/share/licenses/zstd-1.5.7_1/LICENSE
zstd-1.5.7_1: missing file /usr/local/share/licenses/zstd-1.5.7_1/catalog.mk
Checking all packages: 100%

bakul said:
Note that normally zfs won't *create* permanent errors so the issue may be the underlying storage damage due to the power failure. Is zroot a file on the Windows host or a disk partition? If it is a file, you may wish to run fsck equivalent on the host (you'll have to ask windows expert). Make sure the Windows filesystem is repaired and then run zpool scrub again. After that you will need to research on how to deal with zfs permanent errors....

You should do all this *before* further work like restoring pkg database etc.

Ehm, too late. Anyways I will check the windows disk (scandisk or how is called today). Hovever VM disk is a .vdi file on an SSD partition.

Now let's see if everything goes back to normal. Thank you all so much, time will tell. In the while I mark thread as 'solved'.

freethread · Jun 5, 2026

UPDATE:
Yesterday I reinstalled all ports to update the database of pkg (2129 ports) all went ok. I restored XFCE Whisker Menu, it was gone for I do not know the reason. Now seems all works as before.

I checked the Windows disks (where the VM disk reside) as suggested by bakul with chkdsk /F, no errors, and zpool scrub zroot again, no errors (the only one error is the old pkg database file /var/db/pkg/local.sqlite.damaged, that I will delete).

I'm just adding a fix to bakul's commands for restoring the pkg database, in case anyone runs into this issue (nothing serious: /var/backup/pkg.sql.zstd -> /var/backup/pkg.sql.zst):

Code:

mv /var/db/pkg/local.sqlite /var/db/pkg/local.sqlite.damaged
zstdcat /var/backup/pkg.sql.zst | pkg shell
pkg check

Thanks for halp me restore the system. It's not the first time a power outage has occurred, but this time it caused damage. With UFS, it happened every time it was down; with UFS+journaling, it happened much less often. With ZFS, it's the first time, and I can even understand why, because it happened during a system and mass storage load.

freethread · Jun 7, 2026

Ah, emrion and bakul were right in posts #3 and #4. The power loss caused errors in the pkg database, but the sysutils/cpu-microcode port block the system one time on three (screenshot in post #2). When I re-enabled the desktop environment services, the system is permanently blocked. I am trying to know if cpu.microcode port is blocking the system, I enable it in both loader.conf and in rc.conf, maybe enabling it in rc.conf only, it might work or at least not blocking the system. However, I don't know if that port is necessary in VirtualBox VM, not so much for the type of CPU but for the fact that it is virtualized.