Setting up a mirrored update/ports server?

FreeBIE

FreeBIE

Hi All,

How do I go about setting up a mirrored update/ports server? Preferably a private one but I might even go so far as to set up a public one if that's what it takes to get one.

Thanks
 
I can only tell you how I've setup mine..

/usr/ports is on it's own 4GB filesystem. I've got it mounted using nullfs in a jail. To store everything I've used nullfs again to mount /storage/FreeBSD/{distfiles,packages} on /usr/ports/{distfiles,packages} inside the jail. In this jail I build and package all my ports (I always start with a clean jail).

The /storage/FreeBSD/{distfiles,packages} directories are also mounted (again using nullfs) read-only in a separate jail running apache.
 
OK, that looks nice. How do you fetch these files? I imagine that you use postsnap fetch/update for the local repository of ports as one would normally. Am I right?

How are these files then fetched by your other servers? I guess you configure portsnap.conf like SERVERNAME=myupdateserver.mydomain.com. Is your portsnap server just an anonymous ftp server? How do you generate the public/private keys?
 
FreeBIE said:
OK, that looks nice. How do you fetch these files? I imagine that you use postsnap fetch/update for the local repository of ports as one would normally. Am I right?
Click to expand...
More or less, I just do a csup on the host to update the ports tree. Fetching the distfiles happens automatically when I start building things inside the jail. I currently use portmaster to do all the building. Add the -g switch to it so it'll create packages of the built ports.

How are these files then fetched by your other servers? I guess you configure portsnap.conf like SERVERNAME=myupdateserver.mydomain.com. Is your portsnap server just an anonymous ftp server? How do you generate the public/private keys?
Click to expand...
I simply use NFS to mount the read-only exported /storage/FreeBSD/packages directory. My update strategy mainly consists of just nuking all installed packages with pkg_delete -a after which I just pkg_add everything I need. Doing this on my workstation (which has the most installed packages) usually takes about 20-30 min.
 
I think I'll have a look at cvsup-mirror before trying to build something on my own. Thanks for your replies.
 
FreeBIE said:
Hi All,

How do I go about setting up a mirrored update/ports server? Preferably a private one but I might even go so far as to set up a public one if that's what it takes to get one.

Thanks
Click to expand...

If you're still interested, in net/cvsup-mirror, I'd recommend to use packages with this port. There is a dependency that's pretty useless (lang/ezm3) after the build process (basically unused by other ports).
 
There doesn't seem to be a package of cvsup-mirror and compiling the source fails on lang/ezm3. What's up with that?

OK. A workaround was pre-installing:

pkg_add -r cvsup-without-gui
 
to mirror only the ports tree over my machines i use the following.

/etc/make.conf
Code: 
WRKDIRPREFIX?=  /data/work
DISTDIR?=       /data/distfiles
PACKAGES?=      /data/packages

/usr/local/etc/rsyncd.conf
...
Code: 
[portsnap]
    path        = /var/db/portsnap
    comment     = FreeBSD portsnap files
    max connections = 2
    use chroot  = yes
    read only   = true
    refuse options  = c delete

I have a script which generates script's like the following and scp this to remote servers, then the script is executed over ssh

Code: 
#!/bin/sh
# file: update_ports_client.sh
#
BASE_DIR=/root
RSYNCD_MASTER=MasterServer/portsnap/
PORTSNAP_DIR=/var/db/portsnap/
UPDATE_LOG=${BASE_DIR}/.portsnap_update_$(date +%F).log
MAX_AGE="+7d"

[ -d ${PORTSNAP_DIR} ] || ( /bin/mkdir -p ${PORTSNAP_DIR} || echo "`hostname`: faild to create ${PORTSNAP_DIR}" && exit 2)

DMZS=0
hostname | grep -v dmz 2>&1 > /dev/null
DMZS=$?
if [ ${DMZS} -eq 0 ]; then
        rsync -4 -a -q --delete --no-motd rsync://${RSYNCD_MASTER} ${PORTSNAP_DIR}
fi
echo -n "$(hostname) ==> portsnap update .... "
portsnap update >> ${UPDATE_LOG}
RETVAL=$?
if [ ${RETVAL} -eq 0 ];then
        echo " OK done"
else
        echo " failed"
fi
# cleanup old logs
/usr/bin/find ${BASE_DIR} -maxdepth 1 -name .portsnap_update_20\*.log -mtime ${MAX_AGE} -delete 2>/dev/null 1>/dev/null

/usr/sbin/pkg_version -vIL=

servers in the dmz's are rsyncd from the MasterServer since they do not have access to the lan, the script execute then only portsnap update

this works verry quick for more than 20 machines
 
You must log in or register to reply here.
Back
Top