Solved Questions re building packages for a plurality of computers

M

mefizto

Greetings all,

upon building my server, I used ports-mgmt/synth to build packages. Later I added a laptop, and since I have never had problems with ports-mgmt/synth, I installed it on the laptop too. Then I added a workstation, and it seems like a waste to have three different instances of ports-mgmt/synth, so I had an idea to have the server, which is idle most of the time to do the building.

The question is, whether it is possible to have ports-mgmt/synth or ports-mgmt/poudriere to set-up in such a manner, considering, that (i) some packages are common to all three computers and (ii) other are specific to a particular computer. By means of an example, the server does not need any X-related packages, the laptop does not need, e.g., KiCAD, Openscad, etc.

I started doing searches, but most results deal with basic set-up and not management of the repositories. So, if someone can andise if this is possible and point me to the right direction, I would appreciate it.

Kindest regards,

M
 
mefizto said:
By means of an example, the server does not need any X-related packages, the laptop does not need, e.g., KiCAD, Openscad, etc.
Click to expand...
I have two repositories per version. One is desktop oriented, the other server. The server setup doesn't have any X dependencies (or at least as little as possible).
Code: 
root@molly:/usr/local/etc/poudriere.d # poudriere jail -l
JAILNAME    VERSION         ARCH  METHOD                                    TIMESTAMP           PATH
122-release 12.2-RELEASE-p4 amd64 ftp                                       2021-03-06 17:51:50 /usr/local/poudriere/jails/122-release
130-release 13.0-RC2        amd64 ftp                                       2021-03-13 15:27:27 /usr/local/poudriere/jails/130-release
13-stable   13.0-STABLE     amd64 url=file:///storage/release/13-stable/ftp 2021-03-13 17:04:18 /usr/local/poudriere/jails/13-stable
Code: 
root@molly:/usr/local/etc/poudriere.d # poudriere ports -l
PORTSTREE METHOD    TIMESTAMP           PATH
desktop   svn+https 2021-03-14 17:48:31 /usr/local/poudriere/ports/desktop
server    svn+https 2021-03-14 17:54:16 /usr/local/poudriere/ports/server
There are probably better ways to do this but this was the easiest to set up and maintain. The end result is 4 repositories (I don't need the desktop for all versions):
Code: 
root@molly:/usr/local/etc/poudriere.d # ll /usr/local/poudriere/data/packages/
total 90
drwxr-xr-x  7 root  wheel  15 Mar 14 21:46 122-release-server/
drwxr-xr-x  7 root  wheel  15 Mar 14 18:24 13-stable-server/
drwxr-xr-x  7 root  wheel  15 Mar 15 08:06 130-release-desktop/
drwxr-xr-x  7 root  wheel  15 Mar 17 09:28 130-release-server/
lrwxr-xr-x  1 root  wheel  18 Nov  7 22:41 FreeBSD:12:amd64@ -> 122-release-server
lrwxr-xr-x  1 root  wheel  18 Feb 15 21:24 FreeBSD:13:amd64@ -> 130-release-server

Each build has it's own list of packages as I don't need everything everywhere either.
Code: 
root@molly:/usr/local/etc/poudriere.d # ll *.lst
-rw-r--r--  1 root  wheel  1399 Jan 16 21:30 12-stable-server-package.lst
-rw-r--r--  1 root  wheel  1085 Feb 14 12:46 122-release-desktop-package.lst
-rw-r--r--  1 root  wheel  1220 Mar 13 23:46 122-release-server-package.lst
-rw-r--r--  1 root  wheel   765 Feb 20 21:07 13-stable-server-package.lst
-rw-r--r--  1 root  wheel  1011 Mar  9 18:54 130-release-desktop-package.lst
-rw-r--r--  1 root  wheel  1162 Mar 17 09:23 130-release-server-package.lst
Building is done with a small script:
Code: 
#!/bin/sh

POUDRIERE=/usr/local/bin/poudriere

BASEDIR=/usr/local/etc/poudriere.d

${POUDRIERE} ports -u -p desktop
${POUDRIERE} ports -u -p server

for j in 13-stable 130-release 122-release ; do
  ${POUDRIERE} bulk -j ${j} -p server -f ${BASEDIR}/${j}-server-package.lst
done

for j in 130-release; do
  ${POUDRIERE} bulk -j ${j} -p desktop -f ${BASEDIR}/${j}-desktop-package.lst
done
 
Hi SirDice,

thank you for the reply.

This is very close to what I had in mind. The only missing part is to be able to install different packages from the /usr/local/poudriere/ports/desktop. I was wondering, whether pkg install could be given a list - different for the workstation and the laptop, but the pkg(8) does not specify, whether this is possible. If not, perhaps a script invoking the pkg install for each member of a list may be used.

Kindest regards,

M
 
They're repositories, nothing more. Of course you can just run pkg install <whatever>. It doesn't matter whatever else is in the repository. When you run pkg-install(8) on the official repository it doesn't install all 43000+ packages either, it only installs the package (and its dependencies) you told it to. The primary reason I've split up things up in a 'desktop' and 'server' repository is because the server repositories have a make.conf that includes OPTIONS_UNSET+= X11, which is a setting that's certainly going to conflict with building packages for a desktop environment.

The server repositories contain things like Kibana, Jenkins, Gitlab, Phabricator and a whole bunch of server applications I use or test on various different machines. I don't install all those applications on the same server, I have multiple servers, each with a specific task or application. The repository contains everything I need for all those machines.
 
My approach is similar to Sirdice's, but I use a single ports directory and Poudriere package sets:
Code: 
doas poudriere bulk -c -v -j 12amd64 -p portbuild -z workstation -f ~/jails/pkglist-workstation
doas poudriere bulk -c -v -j 12amd64 -p portbuild -z server -f ~/jails/pkglist-server
 
Hi SirDice, Jose,

it appears that we are miscommunication. I understand the concept of repositories and the fact that the pkg install must be given a specific package, the latter being clearly disclosed in pkg(8).

The issue is, that, as explained in my OP, once I build the packages for the desktop computers - the workstation and the laptop, I want to install different sets on the workstation and on the laptop. Hence the idea of a package lists for the different sets.

Kindest regards,

M
 
mefizto said:
it appears that we are miscommunication. I understand the concept of repositories and the fact that the pkg install must be given a specific package, the latter being clearly disclosed in pkg(8).

The issue is, that, as explained in my OP, once I build the packages for the desktop computers - the workstation and the laptop, I want to install different sets on the workstation and on the laptop. Hence the idea of a package lists for the different sets.
Click to expand...
The pkg-install(8) command can take more than a single package to install:
Multiple package names can be specified on the command line, either explicitly or by matching against package names (or origins) in the repository catalogues using shell globbing or regular expressions.
Click to expand...

You could keep the package names in a file and pass them to pkg-install using xargs(1):
Code: 
$ cat laptop.txt
pkg1
pkg2
pkg3
$ cat laptop.txt | xargs echo pkg install                                            
pkg install pkg1 pkg2 pkg3
You would remove the "echo" to actually run the pkg-install command. I like to inspect what's actually going to be invoked by using echo in this way when doing something potentially risky.
 
Hi Jose,

thank you once again, this solves my issue.

So, I am reading the pkg(8), and like an idiot miss the pkg-install(8), which is clearly specified in the SEE ALSO section. There is not hope for me.

Kindest regards,

M
 
Make sure your list only contains so-called "leaf" packages. That way, when you remove something, pkg-autoremove(8) will be able to automatically clean up unused dependencies.
 
Hi SirDice,

thank you for the warning. However, the "leaf" package is a new term for me. Search does not return a definition, but as best as I was able to gather from the pkg_autoremove(8), the term refers to a package that does not depend on any other package.

If that is correct, how do I determine whether a package is a "leaf" package?

Kindest regards,

M
 
You must log in or register to reply here.
Back
Top