How to get sources installed and have these updated by freebsd-update(8) after a base-install

E

Erichans

At base install time using bsdinstall(8) one can choose to install its sources and these get updated automatically with every patch/minor/major release update/upgrade.

If one has choosen not to, is there an after the fact possibility to get the source tree included in such a way that it gets updated automatically?


P.S. Just to be clear: I'm not referring to installing git(1) and using it to update like:
git clone --branch releng/14.1 https://git.FreeBSD.org/src.git /usr/src
whereby it comes under git control and has to be kept up-to-date manually. When I installed FreeBSD and its source tree, the source tree is controlled by freebsd-update(8) and not under git control, where /usr/ports is (by my explicit action to do so):
Code: 
# cd /usr/src
# git remote --verbose
fatal: not a git repository (or any of the parent directories): .git
# cd /usr/ports
# git remote --verbose
origin  https://git.FreeBSD.org/ports.git (fetch)
origin  https://git.FreeBSD.org/ports.git (push)
 
I didn't test when /usr/src get populated by git if it isn't updated with freebsd-update. Are you sure?
I always install sources during bsdinstall.

An other way is (case of 14.1-RELEASE):
Code: 
fetch https://download.freebsd.org/ftp/releases/amd64/amd64/14.1-RELEASE/base.txz
tar -xf base.txz -C /usr/src/ -
rm base.txz
But, like git method, I think you're out of sync for security patches. I guess the problem resolves itself at the next update/upgrade. Or, maybe you can try a freebsd-update fetch.
 
So I tested it in a VM. Starting from 14.0-RELEASE with no security patches. I didn't select to install src at bsdinstall. Then, I used the git way to get sources.

After that, I got security patches, I saw that freebsd-update found the sources and updated them.

If you don't have sources when you got security patches, a new freebsd-update fetch install updates the newly downloaded sources.

I upgraded to 14.1-RELEASE and once again it updated the sources. So, to get them by git isn't a problem for binary updates / upgrades.

However, I guess it may cause trouble with subsequent uses of git.
 
This what I found so far, based on my 14.1-RELEASEp4. I don't have a VM setup, so my experimentation is limited. I've used:
Code: 
[0] /usr/src/bin # uname -aK
FreeBSD q210 14.1-RELEASE-p4 FreeBSD 14.1-RELEASE-p4 GENERIC amd64 1401000
[1] /usr # fetch --output /home/eric/usr/tmp/src-14.0.txz https://download.freebsd.org/ftp/releases/amd64/amd64/14.0-RELEASE/src.txz
/home/eric/usr/tmp/src-14.0.txz                       194 MB   20 MBps    09s
[2] /usr # fetch --output /home/eric/usr/tmp/src-14.1.txz https://download.freebsd.org/ftp/releases/amd64/amd64/14.1-RELEASE/src.txz
/home/eric/usr/tmp/src-14.1.txz                       205 MB   20 MBps    09s
These are the source files for the release version; but they lack the specific tags for being of use to a real version control system such as git(1); they are mostly used for "human" (i.e. user) reference.

FreeBSD sources (base-install) for reference​

At installation time one can select the option to include the source code for reference. By selecting this option /usr/src comes under control of freebsd-update(8). Upgrading by means of freebsd-update upgrade ... will update the sources when src is specified for the Components option in freebsd-update.conf(5).

For updates (patches) as in freebsd-update fetch (and subsequent install), the description in freebsd-update(8) states:
Rich (BB code): 
COMMANDS
   [...]
       fetch     Based on the currently installed world and the configuration
                 options set, fetch all available binary updates.
IMO, this implies also that the sources at /usr/src when selected as such in freebsd-update.conf are not updated. However, using src.txz as the initial download basis, seems to trigger the update of some source files:
Code: 
[1](0) /usr # rm -rf /usr/src/
[2](0) /usr # tar -xf /home/eric/usr/tmp/src-14.0.txz -C /
[3](0) /usr # freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching metadata signature for 14.1-RELEASE from update2.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.
The following files will be updated as part of updating to
14.1-RELEASE-p4:
/usr/src/contrib/llvm-project/libcxx/include/string
/usr/src/crypto/openssh/log.c
/usr/src/crypto/openssh/sshd.c
/usr/src/crypto/openssh/version.h
/usr/src/crypto/openssl/crypto/x509/v3_utl.c
/usr/src/sbin/ifconfig/af_inet.c
/usr/src/sys/cam/ctl/ctl.c
/usr/src/sys/cam/ctl/ctl_private.h
/usr/src/sys/conf/newvers.sh
/usr/src/sys/contrib/libnv/bsd_nvpair.c
/usr/src/sys/contrib/libnv/nvlist.c
/usr/src/sys/contrib/openzfs/module/zfs/dbuf.c
/usr/src/sys/fs/nfsclient/nfs_clrpcops.c
/usr/src/sys/kern/kern_ktrace.c
/usr/src/sys/kern/kern_umtx.c
/usr/src/sys/netpfil/pf/pf.c
/usr/src/sys/netpfil/pf/pf_lb.c
/usr/src/usr.bin/calendar/calendar.c
/usr/src/usr.sbin/bhyve/pci_xhci.c
/usr/src/usr.sbin/bhyve/tpm_ppi_qemu.c
[4](0) /usr # freebsd-update install
Installing updates...
Restarting sshd after upgrade
Performing sanity check on sshd configuration.
Stopping sshd.
Waiting for PIDS: 12064.
Performing sanity check on sshd configuration.
Starting sshd.
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
 done.
[5](0) /usr #
My guess is that patches containing security updates (and perhaps more) are being updated or newly supplied. However, it seems that all other files are not being considered, as can be seen by deleting a complete subdir that will not be "restored" by a fetch. A simple example of changing one source Makefile:
Code: 
[1](0) /usr/src/bin # mv Makefile Makefile.org
[2](0) /usr/src/bin # sed 's/SUBDIR=cat/SUBDIR=catXX/' <Makefile.org >Makefile
[3](0) /usr/src/bin # freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching metadata signature for 14.1-RELEASE from update2.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 14.1-RELEASE-p4.
[4](0) /usr/src/bin #
I think that freebsd-update fetch(1) & install wil not perform a full (three-way) merge. I haven't tried an upgrade, such as 14.0-RELEASE -> 14.1-RELEASE; but I expect that then all source files will be fully merged and all source files will get updated, or installed when not present.

For the use of sources for the base system, I see the following options when not having these installed at the time of installation.

A - Sources under freebsd-update(8) control
Having not selected the sources at installation, freebsd-update.conf(5) can bring it under control of freebsd-update(8) afterwards:
  1. Specify src for the Components option in /etc/freebsd-update.conf as in: Components src world kernel
  2. Download the sources (e.g. use fetch(1) or curl(1)), like the following commands for 14.1 sources:
    fetch https://download.freebsd.org/ftp/releases/amd64/amd64/14.1-RELEASE/src.txz
    tar -xf src.txz -C /
    rm src.txz

    Edit: I'm not sure if and when these are updated with patch updates, but a freebsd-update fetch and subsequent install won't hurt.


B - Sources under git(1) control
- I don't recommend this option, unless you want to build and maintain the base yourself. -

Instead of the above two steps one could use git:
git clone -b releng/14.1 --depth 1 https://git.freebsd.org/src.git /usr/src
This would have to be kept up-to-date manually for every new patch like 14.1-RELEASE-p4; notably to stay in sync with any upgrade, for example from FreeBSD 14.1-RELEASE to 14.2-RELEASE, you'd have to change to releng/14.2, something easy to forget when an upgrade is months away.

Note that git(1) is a specific tool for version control, where the source under its control is intended to be used as input for a compilation/build; see for example Tracking a Development Branch. The sources selected at install time are for reference not specifically meant to be used as (a part) to be used for a build. Even though it seems that for building kernel modules in ports like /graphics/drm-61-kmod is satisfied by having access to an a file like /usr/src/sys/conf/kern.opts.mk and others in order to build this kernel module (like Emrion noted here).


C - Using git(1) to download the sources; then set to be controlled by freebsd-update(8)
- I don't recommend this option, option A seems much more appropriate. -

Using git-clone(1) can be combined with freebsd-update.conf(5) for use by freebsd-update(8). However, having two methods that both control /usr/src is not a good idea. To rescind git control of /usr/src you can delete .git. After configuring /etc/freebsd-update.conf (step 1 of A), use as step 2:
Rich (BB code): 
[1] /usr # git clone --branch releng/14.1 --depth 1 https://git.freebsd.org/src.git /usr/src
Cloning into '/usr/src'...
   [...]
[2] /usr # cd /usr/src
[3] /usr/src # git remote -v
origin  https://git.freebsd.org/src.git (fetch)
origin  https://git.freebsd.org/src.git (push)
[4] /usr/src # rm -rf \.git
[5] /usr/src # git remote -v
fatal: not a git repository (or any of the parent directories): .git  <-- no longer under git control
[6] /usr/src #

Even after rescinding git control over /usr/src, you'll probably get warnings (like here) during an upgrade. Apart from the .git file, git-clone(1) deposits more files than freebsd-update(8) expects.
 
Last edited:
You must log in or register to reply here.
Back
Top