poudriere storage question

J

jbo@

Developer
I run poudriere in a bhyve VM for ease of maintenance. Up until now, the VM had one virtual disk which was located on the fast NVMe zpool of the host. However, since I am now building several different repositories I'm starting to run low on NVMe space. I've decided to add a second virtual disk to the VM which instead is located on the slow HDD based storage pool.

I'm running into a bit of confusion regarding the poudriere configuration. What I would like to happen is that poudriere is building on the NVMe backed disk and then moves the finished packages & logs to the HDD backed disk.
The configuration provides these options which I deem relevant:
  • ZPOOL - The pool where poudriere will create all the filesystems it needs.
  • ZROOTFS - Root of the poudriere zfs filesystem
  • BASEFS - The directory where poudriere will store jails and ports
  • POUDRIERE_DATA - The directory where the jail will store the packages and logs
I'm somewhat confused here and I couldn't really get more out of the documentation. Can somebody tell me whether what I want to achieve is even possible and if so, how to tweak those settings assuming that I have a zroot and a storage pool?

Or should I maybe just not care and keep all poudriere related aspects (jails, ports, packages & logs) on the HDD backed storage pool given that the host has 128 GB of RAM and an L2ARC? It's not like I care whether building ports takes 10 hours or 20.
 
ports-mgmt/poudriere or ports-mgmt/poudriere-devel?

The TMPFS size revealed by SIGINFO can help to decide how much use to make of this type of filesystem.

1646867265017.png

To help get my head around things, I sometimes like to look at the array of mounts that are present before and during (or during and after) a run of poudriere-bulk(8).

For example (during and after):

Code: 
% zfs get mountpoint | grep poudriere
august/poudriere                                         mountpoint  /usr/local/poudriere                    local
august/poudriere/data                                    mountpoint  /usr/local/poudriere/data               inherited from august/poudriere
august/poudriere/data/.m                                 mountpoint  /usr/local/poudriere/data/.m            inherited from august/poudriere
august/poudriere/data/cache                              mountpoint  /usr/local/poudriere/data/cache         inherited from august/poudriere
august/poudriere/data/logs                               mountpoint  /usr/local/poudriere/data/logs          inherited from august/poudriere
august/poudriere/data/packages                           mountpoint  /usr/local/poudriere/data/packages      inherited from august/poudriere
august/poudriere/data/wrkdirs                            mountpoint  /usr/local/poudriere/data/wrkdirs       inherited from august/poudriere
august/poudriere/jails                                   mountpoint  /usr/local/poudriere/jails              inherited from august/poudriere
august/poudriere/jails/13r                               mountpoint  /usr/local/poudriere/jails/13r          inherited from august/poudriere
august/poudriere/jails/13r@clean                         mountpoint  -                                       -
august/poudriere/jails/main                              mountpoint  /usr/local/poudriere/jails/main         inherited from august/poudriere
august/poudriere/jails/main@clean                        mountpoint  -                                       -
august/poudriere/ports                                   mountpoint  /usr/local/poudriere/ports              inherited from august/poudriere
august/poudriere/ports/default                           mountpoint  /usr/local/poudriere/ports/default      inherited from august/poudriere
august/poudriere/ports/portoverlay                       mountpoint  /usr/local/poudriere/ports/portoverlay  inherited from august/poudriere

Code: 
% mount | grep poudriere | sort
/usr/local/poudriere/data/.m/main-default/ref/rescue on /usr/local/poudriere/data/.m/main-default/01/rescue (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/rescue on /usr/local/poudriere/data/.m/main-default/02/rescue (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/lib32 on /usr/local/poudriere/data/.m/main-default/01/usr/lib32 (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/lib32 on /usr/local/poudriere/data/.m/main-default/02/usr/lib32 (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/share on /usr/local/poudriere/data/.m/main-default/01/usr/share (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/share on /usr/local/poudriere/data/.m/main-default/02/usr/share (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/src on /usr/local/poudriere/data/.m/main-default/01/usr/src (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/src on /usr/local/poudriere/data/.m/main-default/02/usr/src (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/tests on /usr/local/poudriere/data/.m/main-default/01/usr/tests (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/usr/tests on /usr/local/poudriere/data/.m/main-default/02/usr/tests (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/.m/main-default/ref/var/db/ports on /usr/local/poudriere/data/.m/main-default/01/var/db/ports (nullfs, local, read-only)
/usr/local/poudriere/data/.m/main-default/ref/var/db/ports on /usr/local/poudriere/data/.m/main-default/02/var/db/ports (nullfs, local, read-only)
/usr/local/poudriere/data/packages/main-default/.building on /usr/local/poudriere/data/.m/main-default/01/packages (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/packages/main-default/.building on /usr/local/poudriere/data/.m/main-default/02/packages (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/data/packages/main-default/.building on /usr/local/poudriere/data/.m/main-default/ref/packages (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/jails/main/rescue on /usr/local/poudriere/data/.m/main-default/ref/rescue (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/jails/main/usr/lib32 on /usr/local/poudriere/data/.m/main-default/ref/usr/lib32 (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/jails/main/usr/share on /usr/local/poudriere/data/.m/main-default/ref/usr/share (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/jails/main/usr/src on /usr/local/poudriere/data/.m/main-default/ref/usr/src (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/jails/main/usr/tests on /usr/local/poudriere/data/.m/main-default/ref/usr/tests (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/ports/default on /usr/local/poudriere/data/.m/main-default/01/usr/ports (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/ports/default on /usr/local/poudriere/data/.m/main-default/02/usr/ports (nullfs, local, noatime, read-only, nfsv4acls)
/usr/local/poudriere/ports/default on /usr/local/poudriere/data/.m/main-default/ref/usr/ports (nullfs, local, noatime, read-only, nfsv4acls)
/usr/ports/distfiles on /usr/local/poudriere/data/.m/main-default/01/distfiles (nullfs, local, noatime, nosuid, nfsv4acls)
/usr/ports/distfiles on /usr/local/poudriere/data/.m/main-default/02/distfiles (nullfs, local, noatime, nosuid, nfsv4acls)
/usr/ports/distfiles on /usr/local/poudriere/data/.m/main-default/ref/distfiles (nullfs, local, noatime, nosuid, nfsv4acls)
/var/cache/ccache on /usr/local/poudriere/data/.m/main-default/01/root/.ccache (nullfs, local, noatime, nfsv4acls)
/var/cache/ccache on /usr/local/poudriere/data/.m/main-default/02/root/.ccache (nullfs, local, noatime, nfsv4acls)
/var/cache/ccache on /usr/local/poudriere/data/.m/main-default/ref/root/.ccache (nullfs, local, noatime, nfsv4acls)
august/poudriere on /usr/local/poudriere (zfs, local, noatime, nfsv4acls)
august/poudriere/data on /usr/local/poudriere/data (zfs, local, noatime, nfsv4acls)
august/poudriere/data/.m on /usr/local/poudriere/data/.m (zfs, local, noatime, nfsv4acls)
august/poudriere/data/cache on /usr/local/poudriere/data/cache (zfs, local, noatime, nfsv4acls)
august/poudriere/data/logs on /usr/local/poudriere/data/logs (zfs, local, noatime, nfsv4acls)
august/poudriere/data/packages on /usr/local/poudriere/data/packages (zfs, local, noatime, nfsv4acls)
august/poudriere/data/wrkdirs on /usr/local/poudriere/data/wrkdirs (zfs, local, noatime, nfsv4acls)
august/poudriere/jails on /usr/local/poudriere/jails (zfs, local, noatime, nfsv4acls)
august/poudriere/jails/13r on /usr/local/poudriere/jails/13r (zfs, local, noatime, nfsv4acls)
august/poudriere/jails/main on /usr/local/poudriere/jails/main (zfs, local, noatime, nfsv4acls)
august/poudriere/ports on /usr/local/poudriere/ports (zfs, local, noatime, nfsv4acls)
august/poudriere/ports/default on /usr/local/poudriere/ports/default (zfs, local, noatime, nfsv4acls)
august/poudriere/ports/portoverlay on /usr/local/poudriere/ports/portoverlay (zfs, local, noatime, nfsv4acls)
devfs on /usr/local/poudriere/data/.m/main-default/01/dev (devfs)
devfs on /usr/local/poudriere/data/.m/main-default/02/dev (devfs)
devfs on /usr/local/poudriere/data/.m/main-default/ref/dev (devfs)
fdescfs on /usr/local/poudriere/data/.m/main-default/01/dev/fd (fdescfs)
fdescfs on /usr/local/poudriere/data/.m/main-default/02/dev/fd (fdescfs)
fdescfs on /usr/local/poudriere/data/.m/main-default/ref/dev/fd (fdescfs)
linprocfs on /usr/local/poudriere/data/.m/main-default/01/compat/linux/proc (linprocfs, local)
linprocfs on /usr/local/poudriere/data/.m/main-default/02/compat/linux/proc (linprocfs, local)
linprocfs on /usr/local/poudriere/data/.m/main-default/ref/compat/linux/proc (linprocfs, local)
procfs on /usr/local/poudriere/data/.m/main-default/01/proc (procfs, local)
procfs on /usr/local/poudriere/data/.m/main-default/02/proc (procfs, local)
procfs on /usr/local/poudriere/data/.m/main-default/ref/proc (procfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/01 (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/01/.p (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/01/usr/local (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/02 (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/02/.p (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/02/usr/local (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/ref (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/ref/.p (tmpfs, local)
tmpfs on /usr/local/poudriere/data/.m/main-default/ref/var/db/ports (tmpfs, local)

Code: 
% mount | grep poudriere | sort
august/poudriere on /usr/local/poudriere (zfs, local, noatime, nfsv4acls)
august/poudriere/data on /usr/local/poudriere/data (zfs, local, noatime, nfsv4acls)
august/poudriere/data/.m on /usr/local/poudriere/data/.m (zfs, local, noatime, nfsv4acls)
august/poudriere/data/cache on /usr/local/poudriere/data/cache (zfs, local, noatime, nfsv4acls)
august/poudriere/data/logs on /usr/local/poudriere/data/logs (zfs, local, noatime, nfsv4acls)
august/poudriere/data/packages on /usr/local/poudriere/data/packages (zfs, local, noatime, nfsv4acls)
august/poudriere/data/wrkdirs on /usr/local/poudriere/data/wrkdirs (zfs, local, noatime, nfsv4acls)
august/poudriere/jails on /usr/local/poudriere/jails (zfs, local, noatime, nfsv4acls)
august/poudriere/jails/13r on /usr/local/poudriere/jails/13r (zfs, local, noatime, nfsv4acls)
august/poudriere/jails/main on /usr/local/poudriere/jails/main (zfs, local, noatime, nfsv4acls)
august/poudriere/ports on /usr/local/poudriere/ports (zfs, local, noatime, nfsv4acls)
august/poudriere/ports/default on /usr/local/poudriere/ports/default (zfs, local, noatime, nfsv4acls)
august/poudriere/ports/portoverlay on /usr/local/poudriere/ports/portoverlay (zfs, local, noatime, nfsv4acls)
%
 
All are doable. In the last few days, we have shifted poudriere load to our new desktop build. Despite being a client, it provides nfs & smb shares. The poudriere, before now, was running on our public server.

Today, poudriere runs at the background moments after turning on the client and the server connects to its nfs share to rsync poudriere_data to the jail previously running it. All other servers and jails continue to use the long existing repo and builds on the client can now be monitored live across the globe.

You may just use the hdd for poudriere_data.
 
Note that poudriere has hooks you can use to kick off scripts when a build is done for example. You could use that to automatically copy the built repository somewhere else.

Code: 
   Hooks
     Hook scripts may be loaded in any of the following paths:

           /usr/local/etc/poudriere.d/hooks/<hook>.sh
           /usr/local/etc/poudriere.d/hooks/plugins/<plugin>/<hook>.sh

     For specific hook documentation see:
     https://github.com/freebsd/poudriere/wiki/hooks
See poudriere(8) and https://github.com/freebsd/poudriere/wiki/hooks
 
You must log in or register to reply here.
Back
Top