Solved Looking for Advices - distribution of configurations across multiple systems

I am just in the process to setup a new server at home, while the former server still runs. Beside these I have also multiple RaspBerry PI's running. This as a background to my question: is there a simple software to allow the setup of FreeBSD as a "script" ?

I currently use git for versioning, sh-script for installation of dotfiles and follow my own notes e.g. to install the essential packages ( e.g. sudo, zsh, git, tmux ). I also follow my notes for hardening ( e.g. sshd.conf ). the RPi's are using RaspOS/Linux so, the setup is a bit diffrent. But zsh or my own user are on all systems. Now, I am doing this at home for a few hw pieces. I just wonder how you do that if you host a server farm ? Images ? VM-Images or Terraform ? Terraform looks a bit like a group of elephants, while I only need a mule ...

Any suggestions ?
 
This as a background to my question: is there a simple software to allow the setup of FreeBSD as a "script" ?
I just wonder how you do that if you host a server farm ?
Puppet or Ansible. Neither are "simple" but Ansible is probably a bit easier to start with, Puppet has a fairly steep learning curve.

Terraform looks a bit like a group of elephants, while I only need a mule ...
Red Hat is heavily pushing AAP; Ansible Automation Platform.
 
Ansible rings a bell.
Yeah, you see it a lot in bigger environments. It's quite popular these days. Should work just fine on FreeBSD; sysutils/ansible. While it's just the basic functionality (nothing like Ansible Tower or AAP) you can create playbooks and have those configure your machines automagically. Put your Ansible playbooks, collections and whatnot in git, attach a Jenkins or something similar and you can even create a rudimentary CI/CD for your configurations.

I haven't done much with Ansible yet. Had invested a lot of time in learning Puppet and I still very much like it but it requires a bit more "infrastructure" to properly set up (Puppetserver and puppet agents). Ansible can simply be run from a central system and doesn't require much on your hosts.
 
For two or three machines, and with configurations that change, I think that a little bit of discipline is better. Here's what I do: Anytime I perform sys admin, I keep an editor window open and write down a log. If I run complex commands, or edit something interestingly weird into a config file, I cut and paste those into my log. I add a sentence or two justifying why I'm doing it, and I use it to keep track of tasks that need to be done.

Then, when I have to duplicate a configuration, or re-install a machine, I just read those logs (there are hundreds and hundreds of lines), which usually takes an hour or two. I often use the opportunity to clean up, deal with to-do items that are now easy, and doing things a better way.
 
I agree. Ansible, Puppet, and similar, play into the corporate deployment scene, where new machines are required on a regular basis. In this context, keeping configuration absolutely consistent for each operational environment (test, production, hot fix, etc.), and automated quick deployment onto virtual machines is the name of the game.

For my home systems, I have a spiral bound notebook, to keep note of the basics (what and why). But for complex changes (how), I still write and keep action plans with code ready to cut and paste.
 
+1 for Ansible. Minimal infrastructure requirements and easy to deploy. For a home setup you might only need 5% of its capabilities, especially if you just re-use service configurations from your already existing git repositories. This isn't as flexible/scalable as configuration templates, but you can easily use them for manual setups as well. If you need slightly different configs across multiple hosts (e.g. multiple nameservers), just use git branches. I use this approach for 8 slave NS in our infrastructure and i find it more accessible and re-usable than ansible templates...
 
For two or three machines, and with configurations that change, I think that a little bit of discipline is better. Here's what I do: Anytime I perform sys admin, I keep an editor window open and write down a log. If I run complex commands, or edit something interestingly weird into a config file, I cut and paste those into my log. I add a sentence or two justifying why I'm doing it, and I use it to keep track of tasks that need to be done.

Then, when I have to duplicate a configuration, or re-install a machine, I just read those logs (there are hundreds and hundreds of lines), which usually takes an hour or two. I often use the opportunity to clean up, deal with to-do items that are now easy, and doing things a better way.
Actually , that is also what I do. I just wanted to a) get rid of MS Notes ( as a dependency) and hoped to get some automation done to reduce the manual tasks.
 
I just found etcupdate diff >report.txt is excellent to grab all the *conf changes I have done. It will not allow to build the pkg's , but it is an excellent help for the configurations.
I might have a look at ansible - specially because I have multiple RaspBerry PI's as music boxes, where sometimes the SD Card crashed. Ansible looks like a OS independent tool , so I could use it either on Linux distributions or FreeBSD
+1 for Ansible. Minimal infrastructure requirements and easy to deploy. For a home setup you might only need 5% of its capabilities, especially if you just re-use service configurations from your already existing git repositories. This isn't as flexible/scalable as configuration templates, but you can easily use them for manual setups as well. If you need slightly different configs across multiple hosts (e.g. multiple nameservers), just use git branches. I use this approach for 8 slave NS in our infrastructure and i find it more accessible and re-usable than ansible templates...
Yes, I also use a git repository for "dotfiles" to setup my personal environment on a system. Especially things like shell (zsh) and add-ons (znt, tmux,ssh) I have written an installation shell script to work on different environments and that works fine. But it needs as a pre-requisition that the underlying pkg's (like sudo, zsh,...) are allready installed.

I also want to automate the first jobs on a fresh installed system ( like portsnap or apt-get update ...) which you only call once after the fresh setup.

Overall a bit big for some home systems, but I want to get prepared as the systems become older now and I am afraid that I need to replace the one or other hardware. So - if time is available - I will have a look on ansible to understand how I could use it ...

Thank you all for your advices !
 
Ansible looks like a OS independent tool , so I could use it either on Linux distributions or FreeBSD
Combined with something like chocolatey you could even manage windows clients. at least in theory - in reality you still have to deal with the usual f*ckups and failures of random clients, even if all have been set up identical...

I have written an installation shell script to work on different environments and that works fine. But it needs as a pre-requisition that the underlying pkg's (like sudo, zsh,...) are allready installed.
Initially ansible will only need ssh access to the system and needs to know what OS and package manager it is dealing with. Playbooks *can* be written completely OS-agnostic, so no matter if you point ansible at a freebsd, linux or illumos host, it will set it up e.g. with LAMP or whatever you specified in your playbook.
 
I just started working on a little setup I call system0, which is kind of like beefed up dot files. I realized that I’m dealing with dependencies, and that make and packages are a good fit for it. I don’t have packages in there yet, but they’re easy to make with `make package` from ports or pkg-create(8). No need even for anything like poudriere yet.

It’s for my dev machine for now, but I am giving some thought to how I would configure my NAS, router, and other servers.

One cool thing for me at the moment is that it’s hooked up to CI, so I know it all works.

We’ll see how it goes… I’m also thinking of automated bootstrapping based on Tailscale tags. At which point it may be getting in to Ansible et al territory anyway :)

I did feel for now though that those tools are overkill for my purposes. They’re intended to be used for all things in all environments, so they’re quite complex. Declaring the stuff I care about with Makefile makes sense to my brain right now.

Maybe it’s an interesting approach to you, maybe not. In any case, I encourage you to try automated tests w/ CI so you can be confident that your config works when run from scratch. Cirrus CI supports FreeBSD as you can see in .cirrus.yml.
 
Combined with something like chocolatey you could even manage windows clients. at least in theory - in reality you still have to deal with the usual f*ckups and failures of random clients, even if all have been set up identical...
Thanks God , I am very happy that I don't have to deal with windows clients. I am an Apple lover and using FreeBSD where for all Server activities ( or RaspBerryOS for that type of HW) . I currently only use Windows on a VM for some special tasks, but not for multiple clients

One cool thing for me at the moment is that it’s hooked up to CI, so I know it all works.
CI is a a big rocket for a small bird like my home setup. Cirrus CI looks great , but also cost $10 for my private use-case. I may use for some tests some VM I could run in Parallels on my Mac, but currently I just will do the minimum and may learn this winter Ansible. It may allow to make even the installation of my dotfiles a bit easier. We will see ....
 
+1 for Ansible. A few things to add to the discussion above:
  • You can bootstrap the use of Ansible on a target machine without Python installed. See Ansible and BSD on the Ansible documentation site.
  • Note that you can run an Ansible playbook against localhost. I've done this on my desktop as a means of keeping track of configuration changes I've made, so I can reproduce it on other workstations later.
  • I've found that most Ansible core modules support FreeBSD just fine, with exceptions being in edge cases. For example, I found just last night that you can use the Ansible "user" module to create a user with a disabled password (by simply not using the password attribute), but I can't find a way to use the module to disable the password on a user that already exists. Might be that I've missed something, though.
  • The only FreeBSD-specific Ansible package I know of is "pkgng" for managing software installation with FreeBSD's pkg().
  • Benedict Reuschling has a good overview to Ansible generally, with examples configuring FreeBSD systems. See his talk on YouTube: BSD Systems Management.
Good luck!
 
Back
Top