Solved Setting up a private pkg and upgrade mirror

R

Randall Le-Fevre

Hi folks.

Here's the situation I find myself in...

I'm migrating from Debian where I've got a local private apt mirror (just the packages in 'main') totalling 130GB for 2 releases and uses about 2GB of my monthly bandwidth for updates. If you're not familiar, apt is (very) roughly equivalent to pkg plus freebsd-upgrade. There are currently 6 servers, a dozen or so containers (jails) and 16 or so notebooks in my home network (my house is where the still-useful things go when where I work sends things to recycling). I'd like to have a similar setup for FreeBSD--not only to reduce the amount of bandwidth I use (especially while I migrate all of my network), but also to reduce the amount of strain I put on FreeBSD's resources.

What I'd like more specifically, is a local private mirror of the 'latest' pkg repository (the .txz files, like at http://pkg.freebsd.org/freebsd:10:x86:64/latest/All/) and updates, like what "freebsd-update fetch install" retrieves. Even better would be to mirror only the packages and updates that I need (for the base system and packages I've installed).

I've read "Requirements for FreeBSD mirrors" and "How to Mirror FreeBSD", but 1.4TB seems a bit excessive for my needs. I also don't have the monthly bandwidth to provide a public mirror and my ISP has a strong dislike for public-facing servers on their network.

I've got a server with "FreeBSD 10.1-RELEASE-p9" and can set up any services required. I'm also getting a handle on (t)csh and scripting in sh.

So, where do I go from here?

Thank you in advance for any guidance you can provide.

Cheers!
Randall
 
I would suggest setting up your own repository with ports-mgmt/poudriere instead. That will give you the benefit of building from ports while keeping the ease of management of packages.
 
Thank you to both SirDice and getopt for your input.

I had looked at poudriere, but wasn't sure if there was an easier way to a private repository set up (something like rsync'ing the packages at http://pkg.freebsd.org/freebsd:10:x86:64/latest/All/ and updates with a couple changes to config files). That both of you suggest it tells me I should look into it further. Thank you for that.

My motivation is that I've only got 125GB monthly bandwidth before my ISP starts charging more. With the 36 machines (physical plus virtual) I have on my network at 2GB per month per machine (not sure what the FreeBSD load will be yet) would use a significant portion of that (around 72GB) getting the updates for each machine. Whereas having a local repository means around 2GB per month of bandwidth (currently) and the rest is local.

In addition, I often set up disposable containers that get used for a of couple days while I learn something or experiment, then get destroyed. Updating and fetching packages several times a month adds up.

My wife uses an average of 55GB per month (Netflix and other streaming media), plus the 72GB in updates would put me over the limit and that doesn't include anything _I_ do (averages 25GB per month, putting us at around 152GB of 125GB). My ISP caps out the increased charges at $15 per month, but that comes out of _my_ monthly allowance, since it's _my_ use that puts us over the limit.

We don't have to have so many machines in the house. It is nice, however to have machines in almost every room--centrally authenticated with nfs home (and media) directories where you log in and your stuff is there. Not having to remember where you put _your_ computer (with all your files) or wait until someone else is finished with it is also nice.

It's also handy for guests. Need to look something up? Here you go... username and password are 'guest'. Log out when you're done please.

How's that for motivation? :) I hope none of this is taken as rude or stand-offish.

I'll keep the thread open for a few days if that's okay, just in case there are other ideas while I'm looking into poudriere.

Thanks for your help folks.

Cheers!
Randall
 
ports-mgmt/poudriere will rock your world. It's not difficult to set up and you can have one server compiling all your binary packages; your only bandwidth cost would be fetching the ports tree. You can setup multiple jails if you want several configs.

I'd recommend creating a baseline build (a complete deployment-ready desktop/server), then from this pick out the list of ports you want Poudriere to make.

The handbook has advice for setting it up, also the Poudriere project page is an excellent how-to resource.
 
sulman said:
It's not difficult to set up and you can have one server compiling all your binary packages; your only bandwidth cost would be fetching the ports tree.
Click to expand...
When building ports it's going to fetch a bunch of distfiles too, applications don't appear out of thin air ;) But these will all be cached so you only have to download them once. When ports are updated it will automatically fetch and cache the new files.
 
hukadan: That's something I'll keep my eye on, and more what I had envisioned--thanks!

While that matures a bit...

I've had some other tasks to attend to in my migration, but I've spent some time getting to know poudriere now.

Once I got it to work in a jail (boy was that a struggle using mfsBSD) it immediately started returning value. The major issue I had was:
Code: 
poudriere bulk -f /usr/local/etc/buildlist -j testjail
[00:00:00] ====>> Creating the reference jail... done
[00:00:11] ====>> Mounting system devices for testjail-default
[00:00:12] ====>> Mounting ports/packages/distfiles
[00:00:12] ====>> Using packages from previously failed build
[00:00:12] ====>> Mounting packages from: /usr/jails//data/packages/testjail-default
/etc/resolv.conf -> /usr/jails//data/.m/testjail-default/ref/etc/resolv.conf
[00:00:12] ====>> Starting jail testjail-default
jail: jail_set: Operation not permitted
[00:00:12] ====>> Cleaning up
[00:00:12] ====>> Umounting file systems

After some digging through the script (/usr/local/share/poudriere/common.sh), I found that it was trying to set an IPv6 address :):1) in the jail. That's all good and well, but the host jail doesn't have one either (at least I think that's the issue). A quick comment-out of that and everything works as it should--and does on a test VM.

Thanks to all who contributed. I feel very comfortable marking this as SOLVED now.

Cheers!
Randall
 
So I looked if there was something behind the scenes that needed to be fixed and /usr/local/share/poudriere/include/common.sh.freebsd does indeed check the output of sysctl -n kern.features.inet and sysctl -n kern.features.inet6 to determine if it should try to assign an IPv4 and IPv6 address. Do you have any specific configuration that prevents assigning any IPv6 by default even though the kernel supports it?
 
After Googling for how to disable IPv6, I can say that I have definitely not disabled it--at least not on purpose.

There are no IPv6 addresses assigned on any interfaces other than loopback on the host (host0, bare-metal). There are also no IPv6 addresses on the first-level jail (on host0).

Output of ifconfig from host0:
Code: 
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:25:90:38:e6:9c
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:25:90:38:e6:9c
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:25:90:38:e6:9c
        inet 172.17.50.216 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.50 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.51 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.53 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.54 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.55 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.56 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.57 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.58 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.59 netmask 0xffffff00 broadcast 172.17.50.255
        inet 172.17.50.52 netmask 0xffffff00 broadcast 172.17.50.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

Output of ifconfig from pkg0 (jail on host0):
Code: 
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:25:90:38:e6:9c
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:25:90:38:e6:9c
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:25:90:38:e6:9c
        inet 172.17.50.52 netmask 0xffffff00 broadcast 172.17.50.255
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

Relevant entry from jail.conf on host0:
Code: 
pkg0 {
        host.hostname = "pkg0";
        path = "/usr/jails/pkg0";
        ip4.addr = lagg0|172.17.50.52/24;
        ip4.addr += 127.0.0.1;
        children.max=10;
        enforce_statfs=1;
        allow.mount;
        allow.mount.tmpfs;
        allow.mount.devfs;
        allow.mount.procfs;
        allow.mount.zfs;
        allow.mount.nullfs;
        allow.raw_sockets;
        allow.socket_af;
        allow.sysvipc;
        allow.chflags;
        exec.clean;
        exec.system_user = "root";
        exec.jail_user = "root";
        exec.start += "/bin/sh /etc/rc";
        exec.stop = "";
        exec.consolelog = "/var/log/jail_pkg0_console.log";
        mount.fstab = "/etc/fstab.pkg0";
        mount.devfs;
        mount.fdescfs;
        mount +=  "procfs /usr/jails/pkg0/proc procfs rw 0 0";
        allow.set_hostname = 0;
        children.max=10;
}

There are some things in jail.conf that I haven't completely figured out yet. I just went the dangerous route of adding stuff from what I found on Google until it worked. I'll whittle it down when I get a chance.

I'm totally okay with just modifying /usr/local/share/poudriere/common.sh to the following:
Code: 
#       localipargs="ip4.addr=${LOIP4} ip6.addr=${LOIP6}"
        localipargs="ip4.addr=${LOIP4}"

The learning curve from Debian to FreeBSD has been quite steep, but it's been fun so far.

Cheers!
Randall
 
Randall Le-Fevre said:
...
Once I got it to work in a jail (boy was that a struggle using mfsBSD) it immediately started returning value. The major issue I had was:
...
Click to expand...

I understand now. I missed this comment earlier so what you are seeing may just be an oddity from running Poudriere in a jail with your particular config.
 
You must log in or register to reply here.
Back
Top