Can this be avoided ? This takes a week on my PC.
It is not. But no matter whether packages are fetched or were built locally, strict rules apply: If a package has different options, a different version, different linked library versions or even different versions of dependencies, it is deleted and rebuilt. This is done recursively, so any transitively dependent packages are built as well.
fetch -qo- http://beefy16.nyi.freebsd.org/data/131amd64-default/latest-done/.data.json \
| head -c80 | sed 's/.*buildname":"\([^"]*\)",.*/\1/'[00:01:04] Package fetch: Will fetch 3 packages from remote or local pkg cache
Updating database digests format: 100%
The following packages will be fetched:
New packages to be FETCHED:
llvm11: 11.0.1_7 (107 MiB: 28.11% of the 381 MiB to download)
llvm12: 12.0.1_10 (114 MiB: 30.00% of the 381 MiB to download)
llvm13: 13.0.1_3 (159 MiB: 41.89% of the 381 MiB to download)
Number of packages to be fetched: 3
The process will require 381 MiB more space.
381 MiB to be downloaded.
[freebsd_12-4x64-HEAD-12] Fetching llvm12-12.0.1_10.pkg: 100% 114 MiB 2.0MB/s 00:59
[freebsd_12-4x64-HEAD-12] Fetching llvm11-11.0.1_7.pkg: 100% 107 MiB 1.9MB/s 00:58
[freebsd_12-4x64-HEAD-12] Fetching llvm13-13.0.1_3.pkg: 100% 159 MiB 2.4MB/s 01:11
[00:04:17] Package fetch: Using cached copy of llvm11-11.0.1_7
[00:04:17] Package fetch: Using cached copy of llvm12-12.0.1_10
[00:04:17] Package fetch: Using cached copy of llvm13-13.0.1_3
[00:04:17] Sanity checking the repository
[00:04:17] Checking packages for incremental rebuild needs
[00:04:17] Deleting llvm11-11.0.1_7.pkg: missing dependency: libedit-3.1.20221030,1
[00:04:17] Deleting llvm12-12.0.1_10.pkg: missing dependency: libedit-3.1.20221030,1
[00:04:17] Deleting llvm13-13.0.1_3.pkg: missing dependency: libedit-3.1.20221030,1
[00:04:17] Deleting stale symlinks... done
[00:04:17] Deleting empty directories... done
[00:04:17] Package fetch: Generating logs for fetched packages
[00:04:17] Unqueueing existing packages
[00:04:17] Unqueueing orphaned build dependencies
[00:04:19] Sanity checking build queueYou could add an additional
git checkout <hash> to these scripts with a hash obtained from one of the official builders...
DEFAULT_VERSIONS+= ssl=opensslI have a VMWare vm building all the time with 4 jails - FreeBSD_12-4-RELEASE amd64 64GB . It always uses the latest ports branch. It runs forever in a cycle of Upgrade Jails -> Upgrade Ports -> PortShaker -> Bulk Builds -> Clean Logs -> Clean Distfiles -> Clean Old Pkgs. Flags are checked and any of these steps can be skipped or added. It sleeps at the end for X amount of time and can be set to sleep less if there is a failed build. It also uses the poudriere hooks script bulk.sh to handle what to do on different statuses. It's a pretty cool script that is configured under rc.conf.
git checkout $(fetch -qo- \
http://beefy6.nyi.freebsd.org/data/123amd64-default/latest-done/.data.json \
| head -c80 | sed 's/.*buildname":"\([^"]*\)",.*/\1/')git checkout main before updating your ports tree again.
DEFAULT_VERSIONS+= ssl=opensslDuh ?
Datapanic, well, try adding something like this to your scripts before starting the next bulk build:
(The official builders still build packages for 12 on 12.3, you'll have to change this URL once 12.3 is EOL)Bash:git checkout $(fetch -qo- \ http://beefy6.nyi.freebsd.org/data/123amd64-default/latest-done/.data.json \ | head -c80 | sed 's/.*buildname":"\([^"]*\)",.*/\1/')
Then, you'll probably also needgit checkout mainbefore updating your ports tree again.
This assumes you're already getting your ports tree directly from git....
edit: as for hijacking Alain's thread, I think package fetching might help him solve his problem as well, so it's IMHO not too far off-topic, but of course up to the mods to decide
beefy6 doesn't have an IPv4 address... if it did, any IPv4-only machine would be able to ping it. Yeah, there's lots of details I'm omitting here, but long story short, beefy6 is IPv6-only.
Well, the problem certainly isn't IPv6 here, maybe a broken IPv6 connectivity (in which case you could try for example tunnelbroker.net for a free IPv6-tunnel (on v4)). I've never had any issues with the official builders so far
$ host beefy6.nyi.freebsd.org
beefy6.nyi.freebsd.org has IPv6 address 2610:1c1:1:6080::16:15
beefy6.nyi.freebsd.org mail is handled by 0 .
$ ping beefy6.nyi.freebsd.org
PING6(56=40+8+8 bytes) [MY_HOST] --> 2610:1c1:1:6080::16:15
16 bytes from 2610:1c1:1:6080::16:15, icmp_seq=0 hlim=47 time=116.228 ms
16 bytes from 2610:1c1:1:6080::16:15, icmp_seq=1 hlim=47 time=97.309 ms
^C#!/bin/sh
RELEASE=${1:-13.1}
ARCH=${2:-amd64}
BRANCH=${3:-latest}
if [ "$BRANCH" = "quarterly" ]; then PORTS=quarterly; else PORTS=default; fi
if [ "$RELEASE" = "CURRENT" ]; then
JAIL=main-$ARCH
HASHPAT='p[0-9a-f]*_s[0-9a-f]*'
HASHCAP='p\([^_]*\)_s.*'
else
JAIL="$(echo $RELEASE | sed -e 's:\.::')$ARCH"
HASHPAT='[0-9a-f]*'
HASHCAP='\([^"]*\)'
fi
fetch -qo- \
"https://pkg-status.freebsd.org/api/1/builds?type=package&latest=True" \
| sed -e 's@{"_id":"@\n@g' \
| grep ":${PORTS}:" \
| grep '"jailname":"'$JAIL'"' \
| grep '"remaining":0' \
| grep '"buildname":"'$HASHPAT'"' \
| sed -e 's@.*"buildname":"'$HASHCAP'".*@\1@'$ ./getporthash.sh
908640ce3450
$ ./getporthash.sh 12.3 i386
bdb5353bacce