Setting up poudriere to use existing infrastructure

[dhw]

I have a local "build machine" that tracks & rebuilds FreeBSD (stable/10 & head) daily , from a private SVN mirror. While I also update its installed ports daily, there is very little activity on that front, as the build machine has few ports actually installed. (The way I used the build machine, I would NFS-mount its /usr/src & /usr/obj read-only from a couple of "target" machines every Sunday, and thus update the target machines -- which worked a treat for base FreeBSD. But I have been updating the installed ports on all machines on themselves (using portmaster), based on an NFS-mounted /usr/ports (that is a working copy updated from the local private SVN mirror).

One of the reasons I am doing things from my local private SVN mirror is to enforce consistency: I only update my mirror overnight; until I update it again, it presents a consistent image of the state of the repositories in question.

Historically, the build machine was both rather slow and very space-constrained (total 80GB disk). A couple of weeks ago, I finally retired that machine in favor of a shiny new machine that is quite a bit faster, and has plenty of disk space.

I would now like to have the new build machine also build the ports for the target machines; I thought that this seemed to be a task ideally suited to poudriere.

But I'm finding the poudriere.conf a bit confusing:

• I wish to use my mirror of the SVN repositories, which are accessible from the build machine:
  

  freebeast(10.2-P)[2] ls -F /repo/svn/freebsd/
  doc/    ports/  src/

• Perhaps more to the point, I wish to use my existing SVN working copies (such as /usr/src and /usr/ports).
• And I'd prefer to use my just-built userland, as well (for the jails) -- after all, I will have just built it before I try to build any ports, and I know that userland is consistent with my working copy and with the running system, because that 's the way I do things. And I have no confidence that something fetched from somewhere else will match what I just built or with my sources or the current state of my mirrors.

I seem to be unclear about how poudriere goes about populating jails -- from my perspective, the simplest, most straightforward approach (in my situation) would be to perform a make installworld DESTDIR=... But I don't see a way to do that.

The problem I'm trying to solve (by using poudriere) is that, while my updates to base FreeBSD are simple, short, and straightforward, the follow-on ports updates (for the "target" machines, at least) can take hours (during which I have services such as email delivery disabled, as that part of my infrastructure depends (in part) on ports (which may be in the process of being updated). The build machine, however, is not directly involved in the "production" activities -- so if it is unable to do anything but build ports for a few hours, that's not a big deal.

Is there a way to persuade poudriere to do this? Is there some other approach I might consider?

Thanks!

david

 

[kpa]

I'm using poudriere jail -m src=/usr/src -c -j ... method to create and then the usual poudriere jail -u -j ... to update my poudriere jails after updating the host, it should be what you're after.

 

[SirDice]

Another way is to use /usr/src/release and create a release(7). Then use that to update your jails. Added bonus, you get installation files, CD, FTP, etc.

dice@molly:~ % poudriere jail -l
JAILNAME   VERSION             ARCH      METHOD                                    TIMESTAMP           PATH
10-desktop 10.1-STABLE         amd64     url=file:///storage/release/10-stable/ftp 2015-04-28 14:35:49 /usr/local/poudriere/jails/10-desktop
10-server  10.1-STABLE         amd64     url=file:///storage/release/10-stable/ftp 2015-04-28 14:36:55 /usr/local/poudriere/jails/10-server
9-server   9.3-STABLE          amd64     url=file:///storage/release/9-stable/ftp  2015-04-29 23:37:51 /usr/local/poudriere/jails/9-server

 

[dhw]

kpa -- thanks; that seems to have set up the sources (initially, anyway). SirDice: Understood, but I don't expect to have the resources to be building releases and updating the ports every day: Building a release is a lot of overhead compared to what I've been doing. (The build machine is actually tracking stable/10 and head for both amd64 and i386 daily as I type this, and it took (for example) 1:08:26 actual elapsed time this morning, checking the timestamps on the typescript files for all 4 builds.) My intent is to switch my target machines from i386 to amd64, but I'd prefer to do that after I have poudriere building my ports, so that after the initial switch (which I've tested), I can just install my packages, rather than rebuild all of the ports in-place.

And once that switch is done (and had a week or so to work out any kinks), I expect to drop the i386 environment entirely.

Back to the original thread topic: From the above, it's apparent that I already have a lot of ports built; accordingly, I have a lot of options set. Is there a way I can collect all of those options and just make them all available to poudriere (at least initially)? (As a point of reference, as of the moment, I have 953 ports installed on my laptop: checking/setting options for each of these to ensure consistency with my current environment seems like something much better-suited to automation, vs. direct human interaction.)

Thanks!

 

[kpa]

To use your existing set options for ports you can sync your /var/db/ports to /usr/local/etc/poudriere.d/options. After that you would use poudriere options to set options and those saved options would have life of their own independent of /var/db/ports.

 

[dhw]

OK; thanks. After doing that, I tried my first poudriere bulk ... run (generally following the steps in 5.6 of the Handbook), but it seems that I have some other issues. (While some ports built OK; others failed with a claim that the port directory didn't exist... while a subsequent check shows it being there. But at that point, I believe we're off-topic for this thread.)