Scripting Desktop Installs

decuser

decuser

After installing and reinstalling my personal desktop some zillion times over the past couple of decades, I have recently gotten tired of following and updating my notes, and taking days to get stuff set up. So, as part of the process of cleaning up my notes, I started thinking like the programmer I am and decided that "all this" could be scripted... not in the traditional scripted to the n'th degree in painfully difficult to read and maintain but awesomely efficient shell scripts way, but rather in the, I'd be able to run a script and get my stuff installed and configured automagically as much as possible and with minimal intervention. So, I've launched the plan and it's working out pretty well.

Here's what I done... First I created a directory structure for my work in ~/0wa/_CD_DVD/freebsd14 (where I put install stuff):


packages/
scripts/
scripts/post-install
logs

Then I created a 00-install-stuff.sh at the top level. This script sets up the install process and runs any scripts it finds in scripts, then it runs any scripts in scripts/post-install. It logs everything (tee and sh +x stuff to logs. So far, it's pretty extensible, I've only made super minor tweaks to the top level script and when I figure out a new thing to install and how it works, I just add a script in scripts. I'm not trying real hard to manage dependencies (but I do try to write scripts that don't depend on each other) but the poor man's dependency control is to name the scripts 01-whatever, 01-whatever-else, 02-something-else (depends on 01-whatever). Any files that need to be installed are put in packages.

A sample of files in packages:

mint-backgrounds.tar.gz
mysql-connector-j-8.0.33.jar
org.apache.commons.commons-codec_1.16.0.jar
org.jkiss.dbeaver.ext.generic_2.3.209.202402111410.jar
org.jkiss.dbeaver.slf4j_2.0.93.202402111410.jar
org.osgi.service.event_1.4.1.202109301733.jar
otf.zip
slf4j.api_2.0.9.jar
spain1.ovpn

A sample of files in scripts:
01-firefox-fonts.sh
01-fonts.sh
01-javadocs.sh
01-jdftweak.sh
01-jdiskreport.sh
01-latex.sh
01-mint-backgrounds.sh
01-mysql-connector.sh
01-nordvpn.sh
01-perl.sh
01-ruby.sh

Post-install stuff is like adding users and stuff.

Here's the 01-ruby.sh script to give you a feel for the scripts:
Code: 
#!/usr/local/bin/bash -x

#********** ruby
pushd ~
rm -fr ~/.rbenv
git clone https://github.com/rbenv/rbenv.git ~/.rbenv

# skip if ark restore
cat << "EOF" >> ~/.bashrc
export PATH=$HOME/.rbenv/bin:$PATH
eval "$(rbenv init -)"
EOF

export PATH=~/.rbenv/bin:$PATH
eval "$(rbenv init -)"

mkdir -p "$(rbenv root)"/plugins
git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build
curl -fsSL https://github.com/rbenv/rbenv-installer/raw/HEAD/bin/rbenv-doctor | bash

MAKE=make rbenv install 3.3.0
rbenv global 3.3.0
gem update --system
gem install rubocop

echo "alias sl='/usr/local/bin/sublime'" >> ~/.bashrc
popd

nevermind that sublime isn't ruby... it's cruft that I should clean up, but hey, it's a work in progress.

I also mirror the directory in test and that's where I try out new scripts.

Anyhow, I thought I'd be transparent and share to see if y'all are doing anything similar, but maybe better/easier. My objective isn't to perfect a script system, just to make it so I can simplify my build process. With the current setup, I can be back up and running from bare metal in about two hours, most of the time is waiting on the scripts. It used to be a couple of days of tweaking this setting, running that custom installer, copy this, copying that, etc.

Note: the scripts have some interactions cuz I'm not gonna spend the time to figure out how to do it without answering prompts unless the package provides --no-interaction type arguments that are easy to follow (an example is texlive - it's much faster to download the iso, save it into packages, and let the script mount it, install it, add environment variables, etc. than it is to do it via download, thankfully it has --no-interaction and reasonable arguments, otherwise, you'd have to run the tui/gui and they are decidedly not script friendly).

Cheers.
 
rbranco said:
Use Ansible. It's better
Click to expand...
Hmm.. I'm a little skeptical...

Not that it isn't better than my homebrewed solution... I'm sure it is, just that it doesn't look simple even though it says simple everywhere all over the docs. It looks great for it pros managing a handful to 65000 machines with only minor variations here and there and a slow change cycle of the catalog, but it doesn't look great for desktop use. Maybe I'm wrong, thinking this way. I see a bunch of how to do ansible videos and tutorials and they're gobbledegook - lots of here's how you id machines, connect to them, check if stuff's installed on them (checksumming and whatnot), add what's missing, etc. But super light on actual small, real life, examples. Actually, it kind of reminds me of ANT for some reason... Looks like a lot of setup to me, even for the individual tasks (none of the videos/tutorials I saw actually showed anything more complex than unpacking a package and adding users, stuff pkg already does).

Do you have some great stuff you can point to?
 
grahamperrin said:
I guess, some of what's done by outpaddling sysutils/desktop-installer might avoid reinventing a wheel or two.

github.com

GitHub - outpaddling/desktop-installer: Quickly configure a FreeBSD or NetBSD desktop system

Quickly configure a FreeBSD or NetBSD desktop system - outpaddling/desktop-installer
github.com github.com
Click to expand...

That's an interesting project. I wonder what it would take to add the ruby example to the workflow... Looking at their quickstart, it looks geared toward automating the "desktop" install, meaning getting the desktop running. I'm able to do that manually in about the same amount of time the tool does (really not too much more than sudo pkg install the right packages, sysrc the right services and loader settings, add the user to the right groups). My idea of getting the "desktop" up is having all of my packages in place and configured, where all I need to do mimic a restore from backup, but with no leftover cruft, is to restore my user data (Pictures, Music, Zotero files, fossil sandbox, git sandbox, Thuderbird emails, that sort of thing).
 
decuser said:
Hmm.. I'm a little skeptical...
Click to expand...
Understandable. ansible has a learning curve, as any reasonable configuration tool.

That being said, if you invest time into learning ansible, you can end up with this:
1) install (or reinstall) a machine, including adding user accounts and getting it onto the network
2) run one ansible-playbook command to install and configure ansible on that machine
3) run another ansible-playbook command to have the whole recorded configuration for the machine installed on it.
Steps 2 and 3 can be combined into one.

The downside is that you will have to maintain that ansible configuration for the machines you have in ansible. Everytime you make a change on the machine (that you want to keep) you will need to update the ansible configuration for it.

I believe this is common for most / all configuration tools in use.
However for me it is easier to force myself to update the ansible configuration in question than remember to keep a backup of all the important files everytime I make a change on a machine.

Full disclosure: I currently use ansible only on my servers / virtual machines, I haven't statred using in on my workstations / desktop machines yet.
 
Having had plenty of use with Ansible in the past, I tried using it to configure my Linux desktop a few years ago. The thing that made me give up and roll something pretty close to what decuser has was conditionals.
While Ansible is really nice for installing packages and making sure that a few lines are in a specific file, etc, when there isn't a module that does what you want you're stuck either writing a script to do it, or using several steps to register something then conditionally acting depending on the check. I found I was doing this enough that I could roll my own solution using shell scripts simply enough.

Plus, between using Ansible daily and for my desktop, then stopping using it daily at work, then stopping using it for my deskop, I found I wasn't changing/running it much on my desktop and so modules changed/became deprecated/etc. It was just another layer of management that I didn't need for one machine.

I still use Ansible to configure my simple home DHCP/DNS server, mostly because I use templating heavily to take a yaml file of all my home machines and create the correct configuration for them - wouldn't fancy doing that in a shell script!

tingo said:
The downside is that you will have to maintain that ansible configuration for the machines you have in ansible. Everytime you make a change on the machine (that you want to keep) you will need to update the ansible configuration for it.
...
However for me it is easier to force myself to update the ansible configuration in question than remember to keep a backup of all the important files everytime I make a change on a machine.
Click to expand...
In all the servers I configured Ansible with I had a simple policy: you manually do anything on the server and I will come to remove your fingers - or at least that was the threat 🤣
Even with simple shell framework for my desktop, I never install packages/edit system files/etc manually, instead it gets added to the relevant file(s) and I run the scripts again.
 
tingo said:
Full disclosure: I currently use ansible only on my servers / virtual machines, I haven't statred using in on my workstations / desktop machines yet.
Click to expand...
I also use it for my workstation and I'm comfortable reinstalling the OS with my separate /home partition knowing that Ansible will leave it configured as before.
 
decuser said:
Hmm.. I'm a little skeptical...

Not that it isn't better than my homebrewed solution... I'm sure it is, just that it doesn't look simple even though it says simple everywhere all over the docs. It looks great for it pros managing a handful to 65000 machines with only minor variations here and there and a slow change cycle of the catalog, but it doesn't look great for desktop use. Maybe I'm wrong, thinking this way. I see a bunch of how to do ansible videos and tutorials and they're gobbledegook - lots of here's how you id machines, connect to them, check if stuff's installed on them (checksumming and whatnot), add what's missing, etc. But super light on actual small, real life, examples. Actually, it kind of reminds me of ANT for some reason... Looks like a lot of setup to me, even for the individual tasks (none of the videos/tutorials I saw actually showed anything more complex than unpacking a package and adding users, stuff pkg already does).

Do you have some great stuff you can point to?
Click to expand...
With Ansible it's better to learn by doing, not by reading tutorials or videos. Start small. With 20% of Ansible knowledge you can solve 80% of your problems. It's very Pareto. No need to know it all. When I don't know something for a specific task I just look it up and find the answer.

I use this for my VM:
github.com

GitHub - ricardobranco777/ansible-freebsd: Ansible configuration for FreeBSD 14

Ansible configuration for FreeBSD 14. Contribute to ricardobranco777/ansible-freebsd development by creating an account on GitHub.
github.com github.com
 
Once you have a system installed/configured with the applications you want, save the output of this:
pkg prime-list

That easily gives you a list of the packages to install, so you can do something like this:
pkg install < pkgs.that.i.must.have

Configuration changes should be relatively easy to track, the bulk wind up in:
/etc/rc.conf
/etc/periodic.conf
 
Before Ansible I was tracking /etc and /usr/local/etc in git like this:

```
cd /etc
sudo git init
sudo git add .
sudo git commit -m init
sudo chmod 700 .git
```

Then I generated a diff that I could apply with patch. I now only use it to track any changes in the configuration with updates.
 
You must log in or register to reply here.
Back
Top