Local repo for freebsd-update (not for pkg)

[bedrich]

Hi everyone,

I spent a lot of time searching about local repos for updating via freebsd-update but I'm finding only articles and guides how to install/update software vial pkg. That's not what I'm trying to do.

I'm long time Linux user, new to FreeBSD. I have local mirror of Debian repo (with private IP on my LAN). The reason is that I have some servers with no internet access and also I have unreliable internet connection. The Debian repo is rsynced over night. The server with rsync cron job has configured NGINX and all my Debian clients have something like this in /etc/apt/sources.list:

deb http://swmirror1/debian.org/update_mirror/debian/ buster main contrib

I rsynced amd64 of FreeBSD repo and now same as Debian it's locally accessible over HTTP (I can install FTP server if FTP protocol is required). For example I can now browse the repo in Firefox and when I go to http://swmirror1/freebsd.org/update_mirror/freebsd/releases/amd64/amd64/12.0-RELEASE/ I see these files:

MANIFEST              07-Dec-2018 05:04      1045
base-dbg.txz          07-Dec-2018 05:04 143874908
base.txz              07-Dec-2018 05:04 154325028
doc.txz.empty_archive 07-Dec-2018 05:04       180
kernel-dbg.txz        07-Dec-2018 05:04  80406936
kernel.txz            07-Dec-2018 05:04  41358500
lib32-dbg.txz         07-Dec-2018 05:04  15642480
lib32.txz             07-Dec-2018 05:04  61684984
ports.txz             07-Dec-2018 05:04  40711528
src.txz               07-Dec-2018 05:04 159564768
tests.txz             07-Dec-2018 05:04   7839360

Could you please help me to point my first FreeBSD server to this local mirror?

The only thing I found (just by poking around in /etc) is ServerName update.FreeBSD.org in /etc/freebsd-update.conf. I can replace it by ServerName swmirror1 and add private IP of swmirror1 into /etc/hosts but that's pretty much all I found so far.

Thank you very much for any tips and thank you for the great work. So far my FreeBSD journey is fantastic I have a lot of fun with it ;-).

Kind regards,
Bedrich Perina

 

[getopt]

First read https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/updating-upgrading-freebsdupdate.html

Also read freebsd-update(8) and freebsd-update.conf(5)

The server directory you found is not being updated and stays on patch version "p0". See the date of the files there. So this is not the way to go.

What you could try is to freebsd-update fetch on one box and deploy/copy the fetched to the other boxes. Then go on with freebsd-update install on them.

Disclaimer: I have not tested this.

 

[SirDice]

You can easily proxy and cache the freebsd-update(8) data. If you enable cache, only the first host will trigger the downloads, consecutive hosts will get their data from the cache. This has worked well for me.

Apache virtual host:

<VirtualHost *:80>
        ServerAdmin info@example.com
        ServerName fbsd-update.example.com

        ProxyRequests Off
        ProxyPreserveHost Off

        <Proxy *>
          Require all granted
        </Proxy>

        ProxyPass / http://update.freebsd.org/

        <Location />
                ProxyPassReverse /
                Require all granted
        </Location>

        <IfModule cache_module>
                <IfModule cache_disk_module>
                        CacheEnable disk /
                        CacheRoot /var/cache/freebsd-update/
                </IfModule>
        </IfModule>

</VirtualHost>

 

[bedrich]

First read https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/updating-upgrading-freebsdupdate.html

Also read freebsd-update(8) and freebsd-update.conf(5)

No, I've been RTFMed ;-).

Seriously I did read both man pages and the specific page in Handbook and I didn't get answer. The only interesting field I found is ServerName as I mentioned above but that's not enough to set up the local mirror.

The server directory you found is not being updated and stays on patch version "p0". See the date of the files there. So this is not the way to go.

You're right, that was not a good example. I mentioned that I rsynced amd64 so I also have this:

http://swmirror1/freebsd.org/update_mirror/freebsd/snapshots/amd64/12.0-STABLE/

../
BUILDDATE       01-Aug-2019 06:35          9
MANIFEST        01-Aug-2019 02:37       1046
REVISION        01-Aug-2019 06:35          8
SRCBRANCH       01-Aug-2019 06:35         12
base-dbg.txz    01-Aug-2019 02:37  157317460
base.txz        01-Aug-2019 02:37  160506568
doc.txz         01-Aug-2019 02:37        180
kernel-dbg.txz  01-Aug-2019 02:37   78267356
kernel.txz      01-Aug-2019 02:37   42185904
lib32-dbg.txz   01-Aug-2019 02:37   15600616
lib32.txz       01-Aug-2019 02:37   60307644
ports.txz       01-Aug-2019 02:37   39524440
src.txz         01-Aug-2019 02:37  165782228
tests.txz       01-Aug-2019 02:37   10571948

Is that something I can point FreeBSD to?

What you could try is to freebsd-update fetch on one box and deploy/copy the fetched to the other boxes. Then go on with freebsd-update install on them.

That's sounds functional but I'd love to find a way to automate it. The rsync solution is the way how to automate it.

You can easily proxy and cache the freebsd-update(8) data. If you enable cache, only the first host will trigger the downloads, consecutive hosts will get their data from the cache. This has worked well for me.

Thank you. Both seems like a valid approach. The only thing I don't understand is how it is possible that there is not official way to update from local mirror? I worked with Windows Servers and Linux and in both cases it is possible to setup a server that will host all updates and other servers and clients can reach out to this server and get updates even when they can't reach out to the internet.

With Linux it's very easy - just rsync data, change URL to the repo on LAN (or even file://) and you can update over LAN. I'm not trying to be offensive comparing FreeBSD and Linux. I'm just suprised that there is no official way how to do something that is a security requirement (servers can't go to internet) in many companies I worked for.

 

[SirDice]

Is that something I can point FreeBSD to?

You cannot update a -STABLE with freebsd-update(8). What you copied are the full installation files, not an update.

The only thing I don't understand is how it is possible that there is not official way to update from local mirror? I worked with Windows Servers and Linux and in both cases it is possible to setup a server that will host all updates and other servers and clients can reach out to this server and get updates even when they can't reach out to the internet.

If you use my example you can simply point your clients to this server by setting ServerName in /etc/freebsd-update.conf. No other changes would be needed.

 

[bedrich]

Ouh, now I get it.

freebsd-update fetch doesn't download anything from ftp://ftp.freebsd.org/pub/FreeBSD/. So what I rsynced (amd64 from ftp://ftp.freebsd.org/pub/FreeBSD/) is useless for updating.

I'm also assuming that it is not possible to rsync data in from update.FreeBSD.org that freebsd-update fetch downloads from. So proxy is the only way to go...

 

[SirDice]

Besides the Apache config I showed you can also use Squid or some other proxy. If you already have one of those set up it might be worthwhile to use them instead.