Missing required shared library after "pkg upgrade"

V

vieron

Hi guys,

for over a month I have been unable to upgrade my packages with pkg upgrade because after the upgrade the newly installed databases/mysql80-server package is missing the "libprotobuf.so.3.19.4" shared library. I followed, and posted on, the PR 270289 bugzilla report and at a certain point Toshimichi Masubuchi attached a patch which was then committed to the main branch and the report was given status "Closed FIXED". I am currenty on the quarterly branch and if I try to upgrade my packages the library is still missing so my guess is that the fix hasn't made it to the quarterly branch.

Here I read that
Quarterly branches aim to receive (on a best effort basis) only the following merge (cherry-pick -x) commits, including, but not limited to:
Security fixes (that may be version updates, or backports of commits)
Build, run, packaging, or other bug fixes
Ports compliance/framework changes
Click to expand...

So I have several questions:
  1. How can I resolve my issue? What is the correct way to go about it?
  2. Why did this problem make it to the FreeBSD quarterly repository? Don't automated tests catch this sort of issue beforehand?
  3. If my assumption that the fix hasn't been included in FreeBSD quarterly branch is true, I would like to understand why.

Here is the version of my current system:
Code: 
root@homebsd:~ # freebsd-version -r
13.2-RELEASE
root@homebsd:~ # freebsd-version -k
13.2-RELEASE
root@homebsd:~ # freebsd-version -u
13.1-RELEASE-p7
root@homebsd:~ #

Currently installed mysql server:
Code: 
root@homebsd:~ # pkg info | grep mysql
mysql57-client-5.7.40          Multithreaded SQL database (client)
mysql57-server-5.7.40          Multithreaded SQL database (server)
qt5-sqldrivers-mysql-5.15.7p177 Qt MySQL database plugin (KDE patched)
root@homebsd:~ #

Post upgrade mysql server version and missing shared library:
Code: 
root@homebsd:~ # pkg -r /tmp/be_mount.ATPU/ info | grep mysql
mysql80-client-8.0.32          Multithreaded SQL database (client)
mysql80-server-8.0.32          Multithreaded SQL database (server)
qt5-sqldrivers-mysql-5.15.8p157 Qt MySQL database plugin (KDE patched)
root@homebsd:~ # pkg -r /tmp/be_mount.ATPU/ check -nd
Checking all packages: 100%
mysql80-server is missing a required shared library: libprotobuf.so.3.19.4
root@homebsd:~ #

List of packages which depend on mysql80-server:
Code: 
root@homebsd:~ # pkg -r /tmp/be_mount.ATPU/ info -r mysql80-server
mysql80-server-8.0.32:
        akonadi-22.12.3_2
root@homebsd:~ # pkg -r /tmp/be_mount.ATPU/ info -r akonadi
akonadi-22.12.3_2:
        akregator-22.12.3
        pim-data-exporter-22.12.3
        kgpg-22.12.3
        kalarm-22.12.3
        akonadiconsole-22.12.3
        korganizer-22.12.3
        grantlee-editor-22.12.3
        kontact-22.12.3
        kmail-22.12.3
        kdepim-addons-22.12.3
        mbox-importer-22.12.3
        kmail-account-wizard-22.12.3
        kaddressbook-22.12.3
        akonadi-import-wizard-22.12.3
        incidenceeditor-22.12.3
        eventviews-22.12.3
        kdepim-runtime-22.12.3_1
        mailcommon-22.12.3
        libksieve-22.12.3
        mailimporter-22.12.3
        messagelib-22.12.3
        calendarsupport-22.12.3
        akonadi-notes-22.12.3
        akonadi-calendar-22.12.3
        kmailtransport-22.12.3
        pimcommon-22.12.3
        libkdepim-22.12.3
        kaccounts-integration-22.12.3
        akonadi-search-22.12.3
        akonadi-contacts-22.12.3
        akonadi-mime-22.12.3
        kalarmcal-21.12.3
root@homebsd:~#

Any help, advise or personal opinion would be very much appreciated.

Thanks in advance!


 
The quarter has been running for more than a month. So if it was fixed in head 4 weeks ago it is not in quarterly.

The correct-est way to fix your system is to apply the patch to a git checkout of the ports tree and install from there.

In this particular case you can probably get away with installing protobufs manually.
 
cracauer@ said:
So if it was fixed in head 4 weeks ago it is not in quarterly.
Click to expand...
Isn't that a kind of problem that *must* be fixed in quarterly?
Code: 
Quarterly branches aim to receive (on a best effort basis) only the following merge (cherry-pick -x) commits, including, but not limited to:

[*]Security fixes (that may be version updates, or backports of commits)
[*]Build, run, packaging, or other bug fixes
[*]Ports compliance/framework changes
 
Thank you very much for your reply cracauer@ .

cracauer@ said:
The correct-est way to fix your system is to apply the patch to a git checkout of the ports tree and install from there.
Click to expand...
So far on my home computer I have only installed packages from the FreeBSD Quarterly branch without installing from ports.
Wouldn't it be a problem to build and install a package from ports and thus mix ports and packages? On many threads I read warnings against doing so.
I'm guessing that that single databases/mysql80-server port would also require me to build and install many other packages from ports thus worsening the "mix" of ports and packages. I initially chose to use Quarterly in the hope of a bit more stability and easiness in upgrading my packages.

cracauer@ said:
In this particular case you can probably get away with installing protobufs manually.
Click to expand...
Are your referring to the devel/protobuf port? How could I do that since the library version is not the most recent one available in the Quarterly repository?

The most recent version of devel/protobuf port and library seems to already be installed:
Code: 
root@homebsd:~ # pkg -r /tmp/be_mount.RjSC info -b -g "*" | grep libprotobuf -B5
mplayer-skins-1.1.5:
mypaint-brushes-1.3.1:
mysql80-client-8.0.32:
        libmysqlclient.so.21
mysql80-server-8.0.32:
        libprotobuf-lite.so.3.19.4
--
proj-9.2.0,1:
        libproj.so.25
proj-data-1.12:
protobuf-3.21.12,1:   <-----------
        libprotoc.so.32
        libprotobuf.so.32
        libprotobuf-lite.so.32
root@homebsd:~ #

But databases/mysql80-server requires a specific version of the library:
Code: 
root@homebsd:~ # pkg -r /tmp/be_mount.RjSC info -B -g "*" | grep libprotobuf -B5
        libgmodule-2.0.so.0
        libglib-2.0.so.0
        libgirepository-1.0.so.1
        libgio-2.0.so.0
libphonenumber-8.13.7:
        libprotobuf.so.32
--
        libfido2.so.1
        libedit.so.0
mysql80-server-8.0.32:
        libzstd.so.1
        libunwind.so.8
        libprotobuf.so.3.19.4 <-----------
root@homebsd:~ #
 
Yes.

In practice I think breakage that breaks building is more likely to be fixed in quarterly than run errors.
 
vieron said:
Here is the version of my current system:
Code: 
root@homebsd:~ # freebsd-version -r
13.2-RELEASE
root@homebsd:~ # freebsd-version -k
13.2-RELEASE
root@homebsd:~ # freebsd-version -u
13.1-RELEASE-p7
root@homebsd:~ #
Click to expand...
Am I the only one to see this?

Maybe not related to your problem, but your system is half upgraded. Userland is still in 13.1 version.
You should use freebsd-update and complete the upgrade.
 
vieron said:
Wouldn't it be a problem to build and install a package from ports and thus mix ports and packages? On many threads I read warnings against doing so.
Click to expand...
It wouldn't be a problem.

For clarification: There are two ports trees, a "quarterly" and a "latest"( also refered as "main").

"quarterly" packages, installed from the official package repositories, are build from a "quarterly" ports tree.

That "quarterly" ports tree should be fetched to apply the patch. See FreeBSD handbook 4.5.1. Installing the Ports Collection, Procedure: Git Method, 3. Or, check out a copy of a quarterly branch.

Problems with mixing ports and packages can arise when "quarterly" packages (which is configured as default on -RELEASE versions) are mixed with a "latest" ports tree.

vieron said:
I'm guessing that that single databases/mysql80-server port would also require me to build and install many other packages from ports
Click to expand...
Build dependencies can be installed as packages, no need to build from source, see ports(7), EXAMPLES:
Code: 
     Example 2: Installing Dependencies with pkg(8)

       The following example shows how to build and install a port without
       having to build its dependencies.  Instead, the dependencies are
       downloaded via pkg(8).

         # make install-missing-packages

To remove those build dependencies after see pkg-autoremove(8).
 
Emrion said:
Am I the only one to see this?

Maybe not related to your problem, but your system is half upgraded. Userland is still in 13.1 version.
You should use freebsd-update and complete the upgrade.
Click to expand...
I actually thought that was normal. I had initially noticed it post upgrade and had thought it was strange but then I read Thread kernel-and-userland-versions-different.82127 and Thread uname-shows-different-patch-level-than-freebsd-version.81378/ in which, if I'm not mistaken, they say that userland and kernel patch level versions can diverge if there are changes, for example, which only affect the kernel and not the userland.
 
vieron said:
but then I read Thread kernel-and-userland-versions-different.82127 and Thread uname-shows-different-patch-level-than-freebsd-version.81378/ in which, if I'm not mistaken, they say that userland and kernel patch level versions can diverge if there are changes, for example, which only affect the kernel and not the userland.
Click to expand...
Patch level, yes (-p1, -p2, etc). Not entire minor or major versions. Finish the upgrade, you only upgraded the kernel and forgot to update the userland.
 
Adding to that, with patch levels, it's a normal/recurring thing that the kernel version is older than the userland (if some patches just didn't affect the kernel), but never the other way around!
 
Awesome information T-Daemon , thank you very much! I guess I'll have to "fold up my sleeves", since the patching procedure is quite new for me, and try apply the patch to the port from Quarterly. Really cool to know that I can use make install-missing-packages to install the dependencies from the repository.

Alain De Vos said:
You can change pkg config to pull this one port from main if it is fixed there.
Click to expand...
By what T-Daemon said, if I didn't misunderstand, I should work on the Quarterly port and just try apply the patch that was attached in the PR 270289 otherwise I guess I would possibly be mixing "latest" and "quarterly" and not just selectively applying the patch to the current Quarterly databases/mysql80-server port.
 
If you have time to compile, a deterministic way is to define the specific mysql or mariadb version in make.conf. Then everything is compiled against that.
[Currently i run poudriere compile tasks while i'm asleep]
 
I would like to build my PC packages with poudriere but unfortunately the build time is an issue for me mainly because on the home computer I have huge programs (KDE, Firefox, Chromium etc.) and when, for example, there is a security issue affecting a small package on which many others depend on it forces me to rebuild all the others. Other than that I recently connected my PC to a UPS and the UPS fans spin quite often making quite a bit of noise. This last one is just an added reason for which I just use the official FreeBSD repositories and only build from ports in jails or on server hosts.
 
Unfortunately I am having issues trying to upgrade the userland. I honestly don't understand what I'm doing wrong.

Code: 
root@homebsd:~ # freebsd-update install
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
root@homebsd:~ # freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 13.2-RELEASE from update1.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.
No updates needed to update system to 13.2-RELEASE-p0.
root@homebsd:~ # freebsd-update -r 13.2-RELEASE upgrade
freebsd-update: Cannot upgrade from 13.2-RELEASE to itself
root@homebsd:~ #

I have a feeling this might have to do with a specific step I have been following from this guide: https://klarasystems.com/articles/managing-boot-environments/
At a certain point they suggest issuing rm -Rf /var/db/freebsd-update on the currently running boot environment. Is it possible that I somehow "broke" something by eliminating the directory before completing the upgrade with freebsd-update install? If so could you please advise me on how I might fix this issue or what the best way to go about it may be?

Just for clarity, this is the script I launch to update inside a boot environment:
Code: 
aaron@homebsd ~ % cat bectl-patch-in-be.sh
bname=`date +%Y%m%d`_patchfrom_`freebsd-version`
bectl create $bname
bectl mount $bname
bectl_mount=`bectl list | grep $bname | awk '{print $3}'`
rm -Rf /var/db/freebsd-update
mkdir $bectl_mount/var/db/freebsd-update
freebsd-update -b $bectl_mount -d $bectl_mount/var/db/freebsd-update fetch
freebsd-update -b $bectl_mount -d $bectl_mount/var/db/freebsd-update install
bectl activate -t $bname
echo
echo
echo
echo
bectl list
aaron@homebsd ~ %
 
At the moment only the kernel has been upgraded. It's probably easier just to roll this back, that's as simple as renaming /boot/kernel.old to /boot/kernel. Then reboot to boot the 'old' kernel and start the upgrade procedure from scratch.
 
Not very useful, freebsd-update(8) already creates boot environments nowadays.

Code: 
root@fbsd-test:~ # bectl list
BE                                Active Mountpoint Space Created
13.1-RELEASE-p1_2023-01-04_175450 -      -          1.20G 2023-01-04 17:54
13.1-RELEASE-p5_2023-02-16_141243 -      -          875M  2023-02-16 14:12
13.1-RELEASE-p6_2023-04-12_021149 -      -          1.79M 2023-04-12 02:11
13.1-RELEASE_2022-08-13_152420    -      -          667M  2022-08-13 15:24
13.2-RELEASE_2023-04-12_021412    -      -          2.00M 2023-04-12 02:14
default                           NR     /          7.72G 2022-07-13 21:54
 
SirDice said:
At the moment only the kernel has been upgraded. It's probably easier just to roll this back, that's as simple as renaming /boot/kernel.old to /boot/kernel. Then reboot to boot the 'old' kernel and start the upgrade procedure from scratch.
Click to expand...
Awesome. I'm quite amazed how that worked out so simply. Thank you very much SirDice

Code: 
root@homebsd:~ # freebsd-version -u
13.2-RELEASE
root@homebsd:~ # freebsd-version -k
13.2-RELEASE
root@homebsd:~ # freebsd-version -r
13.2-RELEASE
root@homebsd:~ #

Next step for me will be patching databases/mysql80-server port from the Quarterly branch as T-Daemon suggested.

Alain De Vos said:
Is this script really needed and usefull? Or thus it shoot yourself in the foot?
Click to expand...
I just put the commands I was issuing on the command line in a script to speed things up for me. The reason I like the approach indicated by Klara Systems is that it allows me to upgrade my system or it's packages without having to touch my currently running system which for me is very comfortable since I'm often using my computer for work for example and I like rebooting only once I'm pretty "confident" things will work out.

Other than that digging a bit deeper I finally understood why I wasn't seeing any boot environments when issuing bectl list even though freebsd-update install's output said that it was going to be created: freebsd-update.conf(5) says that a boot environment isn't created by freebsd-update if an alternative root is specified with "-b".

Going back to Klara Systems guide I mentioned earlier, I wonder if it is really necessary to delete the currently active boot environment's /var/db/freebsd-update directory prior to issuing freebsd-update with an alternative root. In any case my issues where very probably caused by me not issuing the commands in the correct order and maybe deleting /var/db/freebsd-update erroneously before issuing freebsd-update install.
 
Hi guys,

I finally cloned the 2023Q2 branch to /usr/ports/ and then "copied" the differences from the following commit with which PR 270289 was closed: https://cgit.freebsd.org/ports/commit/?id=567557abbfc0a4deec492983ffc01da78c62bae4
I then used make install-missing-packages as suggested by T-Daemon to install the dependencies directly from the Quarterly repository without having to build them from source and built only databases/mysql80-server and databases/mysql80-client from /usr/ports/ with make install clean. Once that was done I issued pkg upgrade and everything went well, finally with no missing dependencies post upgrade:

Code: 
root@homebsd:~ # pkg info | grep mysql
mysql80-client-8.0.32_2        Multithreaded SQL database (client)
mysql80-server-8.0.32_3        Multithreaded SQL database (server)
qt5-sqldrivers-mysql-5.15.8p157 Qt MySQL database plugin (KDE patched)
root@homebsd:~ # pkg check -dn
Checking all packages: 100%
root@homebsd:~ #

The only thing that I honestly still don't understand is why the fix from main wasn't "backported" to Quarterly. It's something I would certainly like to understand better. A part from that FreeBSD always seems more and more an awesome system to me especially for the amazing flexibility it allows.

Thanks to everyone once again for all the advice, information, and patience.
 
vieron said:
The only thing that I honestly still don't understand is why the fix from main wasn't "backported" to Quarterly. It's something I would certainly like to understand better.
Click to expand...
If a commit fixes something that is clearly "broken" before, and the version in quarterly is also affected, it should be merged to quarterly. I guess the explanation here is simple: There's no team responsible for this (and it wouldn't make sense trying to change this, as we just don't have the required resources). Therefore, it's every individual committer's responsibility to do a "MFH" (merge from head) when needed. Sometimes, this is just forgotten.

So, what you could do: Just ask for MFH in the PR ?
 
You must log in or register to reply here.
Back
Top