Automatic Updating?

[4S4K4N]

So iI want to have one of my servers automatically pull the latest stable version via subversion, compile it, and install it... Maybe once a month?

Doing this via a cron script would be pretty easy, but can anyone think of a way to compare the existing /usr/src revision to the newest one and only continue if the revision is newer? I don't plan on using this for any major upgrades, for example 10 -> 11. I feel like those are better off being done manually.

I also want to do this for currently installed ports, but since that belongs in a different forum iI wont go there here.

 

[Oko]

So i want to have one of my servers automatically pull the latest stable version via subversion, compile it, and install it... Maybe once a month?

Doing this via a cron script would be pretty easy, but can anyone think of a way to compare the existing /usr/src revision to the newest one and only continue if the revision is newer? I don't plan on using this for any major upgrades, for example 10 -> 11. I feel like those are better off being done manually.

I also want to do this for currently installed ports, but since that belongs in a different forum i wont go there here.

I strongly feel that you will benefit from reading the official documentation.

https://www.freebsd.org/doc/handbook/updating-upgrading-freebsdupdate.html

I would also suggest that you familiarize yourself with sysutils/beadm.

I personally would not trust you to update my machines based on what you wrote.

 

[4S4K4N]

I strongly feel that you will benefit from reading the official documentation.

https://www.freebsd.org/doc/handbook/updating-upgrading-freebsdupdate.html

I would also suggest that you familiarize yourself with sysutils/beadm.

I personally would not trust you to update my machines based on what you wrote.

I have read that, and because iI use sparc64 servers iI cant use freebsd-update(8). There are no binary updates for tier 2 architectures that iI can find. The only option iI have is subversion and compiling. Well except for CTM, but iI prefer subversion. instead iI plan on doing a ZFS snapshot before the update, and if the update succeeds then the snapshot can be deleted. If it fails iI can just roll back.

That’s why iI only want to do minor version upgrades automatically, not major as iI will break nearly every application due to the ABI change. I probably should have mentioned the sparc64 thing in the first post.

Oh and yes, thanks for the link to beadm. That could actually be quite useful.

 

[Oko]

I have read that, and because i use sparc64 servers i cant use freebsd-update.

OpenBSD has support for Sparc 64 in par with Solaris 10. I would not run FreeBSD, NetBSD or Linux for that mater on Sparc 64 hardware.

Until 2 years ago most OpenBSD development has been done on sparc64.

 

[ShelLuser]

You don't want to automate updates like this, not on the scale of a whole OS update. That can only lead you to serious issues.

 

[protocelt]

I agree with ShelLuser, auto-updating your system unattended is a bad idea. There are manual steps that need to be taken sometimes while updating both the system and your packages, such as instructions in /usr/src/UPDATING and /usr/ports/UPDATING. Not watching these files can possibly lead to a corrupted and unbootable system regardless of snapshots.

 

[4S4K4N]

OpenBSD has support for Sparc 64 in par with Solaris 10. I would not run FreeBSD, NetBSD or Linux for that mater on Sparc 64 hardware.

Until 2 years ago most OpenBSD development has been done on sparc64.

The only things holding me back from OpenBSD are jails and ZFS. FreeBSD actually runs quite well on sparc64 if the machine has driver support. I have yet to experience any issues other than having to do a few things manually. I used to run OpenSolaris and later a custom build of Illumos. As time went on maintaining my own OS build and package repository became a pretty daunting task.

You don't want to automate updates like this, not on the scale of a whole OS update. That can only lead you to serious issues.

Would you care to elaborate a bit more on why this would be an issue? I have manually updated this machine since 9.1 and the only time iI ever ran into any issues was going from 9.3->10.0 and this was mostly because it was a major revision update and a lot of the core system changed. I'm not planning on doing any major revision updates automatically, iI can see that being a disaster waiting to happen. But even if iI did do a major update automatically, iI don't plan on having anything overwrite existing system files.

All iI really intend on doing is minor updates as in 10.1->10.2. Ideally iI would design the script like this.

1. ZFS Snapshot /usr/src (just in case, probably not needed)
2. Create zvol rpool/ROOT/10.2
3. SVN download new source
4. Build source
5. install to rpool/ROOT/10.2
6. Activate system diagnostics light / email administrator

At this point someone would come in during the next maintenance period and set the system to boot from rpool/ROOT/10.2 and make sure it all comes up correctly. After copying over any needed configuration of course. Worst case scenario, it doesn't boot correctly and we roll back to 10.1 while we figure out what happened to 10.2.

The other option would be to just create a snapshot of zroot/ROOT/default for every update, but there could be a lot of issues doing this. Especially if something doesn't work coming out of single user mode, so i would try to avoid this.

 

[ShelLuser]

Would you care to elaborate a bit more on why this would be an issue? I have manually updated this machine since 9.1 and the only time iI ever ran into any issues was going from 9.3->10.0 and this was mostly because it was a major revision update and a lot of the core system changed.

Simple: the very moment you run into a bug (which wouldn't be the first time) you risk the whole system to come crashing down.

And that's just stating the obvious; /usr/src/UPDATING was put in place for a good reason; same applies to /usr/ports/UPDATING.

This is the classic issue of: usually things will probably go right (or did they?) but when things do go wrong then you got one heck of a job to repair the damages.

Basically this is no different to me than turning on automated updates on a Windows server. In general all goes right, until that one time when....

If you can afford downtime then by all means, but in general a server is just that. Which means that automatic updates are the worst thing you can apply to it.

 

[4S4K4N]

Simple: the very moment you run into a bug (which wouldn't be the first time) you risk the whole system to come crashing down.

And that's just stating the obvious; /usr/src/UPDATING was put in place for a good reason; same applies to /usr/ports/UPDATING.

This is the classic issue of: usually things will probably go right (or did they?) but when things do go wrong then you got one heck of a job to repair the damages.

Basically this is no different to me than turning on automated updates on a Windows server. In general all goes right, until that one time when....

If you can afford downtime then by all means, but in general a server is just that. Which means that automatic updates are the worst thing you can apply to it.

The big question is how does one keep up with security updates if you don't have freebsd-update(8) and a relatively small maintenance window of around 24 hours per month that is pre-scheduled? I agree fully with running into bugs and it should be avoided at all costs. But the difference here is that I'm not trying to install the updates automatically, but rather only compile them automatically.

Lets say this window is on the second Sunday of every month. At 12AM on this day the script will pull the latest SVN stable branch, compile both the kernel and world, and install them on a empty zvol.

For example, you normally boot from the zvol zroot/ROOT/default after the CD installation. This script will install the newly compiled system to zroot/ROOT/(SVN Revision). At this point when an admin comes in around 9AM to do maintenance, they can check current bugs, the UPDATING files, build logs, etc... to make sure everything went as planned. If everything checks out the admin can copy over configuration and execute zpool set bootfs=zroot/ROOT/(SVN Revision) zroot and reboot the system. After it's verified that everything came up correctly the job is done. If the system doesn't boot up correctly or experiences some unexpected behavior they can just set the zpool bootfs entry back to the old root which will undo any updates as the build scripts never touched the old zvol.

So in reality i suppose it's not automatically updating, but rather automatically installing a second copy of FreeBSD into a separate volume and having an admin choose whether its safe to boot that new volume or not. If not, they can just zfs destroy it. The Windows update analogy can't really apply to this process, because nothing is being overwritten or replaced. This would be more along the lines of having Windows automatically download the updates and letting an admin choose whether to install them or not.

The point of this is to avoid the amount of time make buildworld and make buildkernel takes when doing this manually, which on the machine in question (a low powered embedded server acting as an internet gateway) takes around 8 hours.

I'm not trying to be argumentative here, but I'm rather trying to think of any ways this could fail and cause the system to crash to a point to where we would have to restore from a backup. The only thing that comes to mind would be a bug in which the newly booted system corrupts the entire zpool. But with how I'm handling this, the admin should catch that before booting into the new system. Manually doing this update could have the same result if the admin where to overlook the bug. I would also surely hope that a bug like that could never make it's way into a stable branch.

 

[ANOKNUSA]

The point of this is to avoid the amount of time make buildworld and buildkernel takes when doing this manually, which on the machine in question (a low powered embedded server acting as an internet gateway) takes around 8 hours.

Do you have some reason to believe it would take less time if you weren't around to observe it? Typing

$ cd /usr/src
$ svn update
$ make buildworld && make buildkernel

and then walking away doesn't make the process any slower than having a cron job do the same. The concern people have isn't with the build but with the installation of the new files--sooner or later you'll need to make choices in merging or replacing config files, replacing init scripts, or some similar low-level system management decision. You can't know in advance what that will entail, so given enough time and a great enough degree of change (like upgrading to a new version of FreeBSD), automating the process will inevitably be disastrous. Since you're already half-way to that conclusion, just take the extra step: don't automatically install the newly built system. Run the build at the end of the day, and come back the next morning to finish the install (after you're properly caffeinated, of course). Maybe one time out of ten, or maybe a hundred, the build will fail, making automation moot. The rest of the time, you plan on not being around for the build anyway. That's all fine. The problem is trying to get a machine to automatically make rational decisions to manage its own affairs, and if that were possible then you wouldn't be here asking this and half the people here would be out of work.