How to create a scripted installer?

[madwebness]

I've been browsing the forum and the documentation, and it seems like the right kind of information must be there somewhere, but basically what I would like to create is a following:

A custom bootable USB-stick from which FreeBSD installer would launch. I say "custom", because I would like
the installation process to install all the things I would normally install - but without any input or minimum of it.

Example: I would like to send my parents an image they can burn onto a USB and install the system from it - granted, I know the architecture of each target system. I would like the script to automatically install GPU drivers, WiFi drivers (and prompt for WiFi password), Wireguard, Desktop Env (say, xfce) and slim. Apart from that, I'd like the installer to NOT ask my parents questions about which filesystem and on which drives to use - it should pick zfs with encryption and prompt for password.

Ideally, I would also like to packages pre-fetched and installed from the same flash-drive as opposed to downloading them from the internet. Right now, it seem like only the ports tree contains the software locally and my god does it take time to compile...

The particulars of the installation process - whether through a fancy GUI installer, or the regular BSD installer or even through a command prompt asking yes or no questions - does not matter. What matters is lack of friction. Now I know I had some friction installing FreeBSD, but that was within limits and I was motivated. My parents would do the regular installation if I ask them to, but I'd like to save them - and myself - the time. Because the thing is, I'd like to have an installer that has a ready-to-use system for me, yet I don't want an image of a working system, as these are quite different things.

So far, I failed to see the difference between a mini-memstick installation and a full DVD image installation. It seems to be behaving in exactly the same way during installation - specifically downloading stuff from the internet and NOT downloading packages I would have to install manually. Why the size, then?

Would you kindly let me know what would be my best options at building such an installer?

 

[grahamperrin]

See for example christhegeek's experimental <https://forums.freebsd.org/threads/81709/> although when I last checked, it uses UFS (not matching your requirement for ZFS). Other aspects of what's there might be food for thought.

More food: <https://github.com/yangzhong-freebsd/lua-httpd#readme> (related: Technology Roadmap).

Re: <https://github.com/nomadbsd/NomadBSD#readme> and the Handbook, I wonder whether NomadBSD (or some adaptation) can come closest to what you'd like.

… Example: I would like to send my parents an image they can burn onto a USB and install the system from it …

Imagine: burning to a (fast) external drive on USB and then simply running the system – without the need to subsequently install. An adaptation might write NomadBSD to a (faster) internal drive.

 

[T-Daemon]

I would like to send my parents an image they can burn onto a USB and install the system from it - granted, I know the architecture of each target system. I would like the script to automatically install GPU drivers, WiFi drivers (and prompt for WiFi password), Wireguard, Desktop Env (say, xfce) and slim. Apart from that, I'd like the installer to NOT ask my parents questions about which filesystem and on which drives to use - it should pick zfs with encryption and prompt for password.

Ideally, I would also like to packages pre-fetched and installed from the same flash-drive as opposed to downloading them from the internet. Right now, it seem like only the ports tree contains the software locally and my god does it take time to compile...

All of those can be done with an official FreeBSD installer USB image, not sure about WiFi password. There I need to look into.

This is a crude how to prepare a official USB installer image for a non-interactive (except for the encryption password and root password) encrypted Root-on-ZFS base system and packages:

Assuming a running FreeBSD system:

• Download a official USB installer image
• Resize the image with truncate(1) to harbor all the packages to be installed (~2G)
• mdconfig(8) the image
• gpart(8) resize the slice and root partition
• growfs(8) the root partition
• mount(8) root partition

mkdir -p /mnt/packages/All
mkdir /mnt/packages/Latest
pkg fetch  -o /mnt/packages/Latest pkg
mv /mnt/packages/Latest/pkg-* /mnt/packages/Latest/pkg.pkg
pkg fetch -d -o /mnt/packages/All <package_name> , repeat for each primary package you want installed on the target PC
pkg repo /packages

[UPDATE]

• Edit /mnt/etc/pkg/FreeBSD.conf, replace url: with "file:///packages", remove mirror_type, signature_type, fingerprints
• create an /mnt/etc/installerconfig file (see bsdinstall(8) "ENVIRONMENT VARIABLES" and "SCRIPTING" preamble for details)
• copy installer shell script (see post #6) under /root
• set in /mnt/etc/installerconfig below the preamble a setup script, i.e.:

bsdinstall script /root/installer

• umount /mnt
• mdconfig -du 0
• image ready to send

Be advised, if the target PC is a UEFI system this needs extra configuration as a workaround in /etc/installerconfig.

For the time being 13.0-RELEASE unattended scripted Root-on-ZFS installation on UEFI is buggy. I don't know about -STABLE and -CURRENT, haven't tested them, but they are supposed to be patched.

Check the target PC's firmware and report back so I can provide the workaround, and also check if there is more than one disk.

Please ask if there is something unclear or not working as it supposed to work.

 

[christhegeek]

You can try my last image it automatically detect the gpu if its amd it will load amdgpu driver if its an old radeon it will detect the id and load radeonkms , if its nvidia it will load the latest nvidia driver if their nvidia gpu can work with the latest nvidia driver then it will be ok , also if they had intel intergrated it will load the respective driver for intel.
The only problem is that on my latest image i don't include LibreOffice .The installer is pretty simple and fast and they will have a nicely customized (anime themed) Kde Plasma environment.
I will also create a Plasma based Studio Edition with plenty of software .
You can find everything on my blog cultbsd.blogspot.com

PS1: when you have installed it you might have to go to bios and select the correct disk (efi) . Also after you have installed cultbsd on your system after a while you might have more software and data etc you might want to Clone it to an external HDD you can do this by simply running the cultbsd installer from the installed system and just enter the same username you have currently and it will clone it to your external drive (have in mind that performance will be much slower cause external disks usually are very slow)

PS2: FreeBSD has its Desktop installer you can find the package and install it
PS3: You can use this script to install your favorite desktop (it might not work 100% for all desktops it shows) https://pastebin.com/raw/Jm2ZsaKu

I've been browsing the forum and the documentation, and it seems like the right kind of information must be there somewhere, but basically what I would like to create is a following:

A custom bootable USB-stick from which FreeBSD installer would launch. I say "custom", because I would like
the installation process to install all the things I would normally install - but without any input or minimum of it.

Example: I would like to send my parents an image they can burn onto a USB and install the system from it - granted, I know the architecture of each target system. I would like the script to automatically install GPU drivers, WiFi drivers (and prompt for WiFi password), Wireguard, Desktop Env (say, xfce) and slim. Apart from that, I'd like the installer to NOT ask my parents questions about which filesystem and on which drives to use - it should pick zfs with encryption and prompt for password.

Ideally, I would also like to packages pre-fetched and installed from the same flash-drive as opposed to downloading them from the internet. Right now, it seem like only the ports tree contains the software locally and my god does it take time to compile...

The particulars of the installation process - whether through a fancy GUI installer, or the regular BSD installer or even through a command prompt asking yes or no questions - does not matter. What matters is lack of friction. Now I know I had some friction installing FreeBSD, but that was within limits and I was motivated. My parents would do the regular installation if I ask them to, but I'd like to save them - and myself - the time. Because the thing is, I'd like to have an installer that has a ready-to-use system for me, yet I don't want an image of a working system, as these are quite different things.

So far, I failed to see the difference between a mini-memstick installation and a full DVD image installation. It seems to be behaving in exactly the same way during installation - specifically downloading stuff from the internet and NOT downloading packages I would have to install manually. Why the size, then?

Would you kindly let me know what would be my best options at building such an installer?

 

[christhegeek]

Be more specific about the installer , i have created mine using Qt you can use the QtCreator for easier development .
Also you can just create a script and use zenity gui dialogs on script.

"Would you kindly let me know what would be my best options at building such an installer?"

 

[T-Daemon]

Just for completeness to the above post #3. See also UPDATE there.

I couldn't figure out how to install packages from the installation image to the new system from the /etc/installerconfig setup script part, and it seems not possible to do so. The setup script runs under chroot(8).

Instead the below script does the job of an non-interactive (except for the root password) system installation, not relying on scripting in /etc/installerconfig.

This script is not very sophisticated due to my limited knowledge about shell scripting, but it should work when it's tailored to the exact specification of the target PC:

#!/bin/sh

# If the system is BIOS, those sections dealing with ESP
# can be skipped.
#
# The partitoning, zpool and ZFS creation were taken from
# an existing Root-on-ZFS installation log (/var/log/bsdinstall_log)
#
# Assuming the target PC's disk has a partion scheme.
# If there isn't, remove the first gpart line.

gpart destroy -F ada0
gpart create -s gpt ada0
gpart add -t efi -s 260m -a 4k -l efiboot0 ada0
gpart add -t freebsd-boot -a 4k -s 512k -l gptboot0 ada0
gpart add -t freebsd-swap -a 1m -s 200m -l swap0 ada0
gpart add -t freebsd-zfs -a 1m -l zfs0 ada0

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0

# Create and populate ESP, skip on BIOS system

newfs_msdos /dev/ada0p1
clear
mount_msdosfs /dev/ada0p1 /mnt
mkdir -p /mnt/efi/boot/
mkdir /mnt/efi/freebsd
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd
umount /mnt

zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot ada0p4
zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ zroot/ROOT/default
zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp
zfs create -o mountpoint=/usr -o canmount=off zroot/usr
zfs create zroot/usr/home
zfs create -o setuid=off zroot/usr/ports
zfs create zroot/usr/src
zfs create -o mountpoint=/var -o canmount=off zroot/var
zfs create -o exec=off -o setuid=off zroot/var/audit
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o atime=on zroot/var/mail
zfs create -o setuid=off zroot/var/tmp
zfs set mountpoint=/zroot zroot

clear
echo
echo "Installing system, please wait ....."

tar xf /usr/freebsd-dist/base.txz -C /mnt
tar xf /usr/freebsd-dist/kernel.txz -C /mnt

zpool set bootfs=zroot/ROOT/default zroot
zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot
zfs set canmount=noauto zroot/ROOT/default

mkdir /mnt/packages
mount_nullfs /packages /mnt/packages

# Not sure if this is necessary, but pkg(8) complains about /dev/null not found.
mount_nullfs /dev /mnt/dev

mkdir -p /mnt/usr/local/etc/pkg/repos

cat << EOF > /mnt/usr/local/etc/pkg/repos/local.conf
FreeBSD: { enabled: no }

local: {
    url: "file:///packages"
    enabled: yes
}
EOF

chroot /mnt pkg bootstrap -fy
chroot /mnt pkg install -yg '*'

rm /mnt/usr/local/etc/pkg/repos/local.conf

clear
echo "Please select a password for the system management account (root):"
chroot /mnt passwd root

clear
chroot /mnt pw adduser -n parents -c "My parents" -s /bin/csh -m -w yes
echo
echo "User 'parents' has been created, password 'parents.'"
sleep 5

cat << EOF > /mnt/boot/loader.conf
kern.geom.label.disk_ident.enable=0
kern.geom.label.gptid.enable=0
cryptodev_load="YES"
zfs_load="YES"
EOF

cat << EOF > /mnt/etc/rc.conf
hostname="home-pc.home"
zfs_enable="YES"
ifconfig_DEFAULT="DHCP"
sshd_enable="YES"
kld_list="amdgpu nvidia"
slim_enable="YES"
EOF

# Remove efi line on a BIOS system
cat << EOF > /mnt/etc/fstab
/dev/ada0p1    /boot/efi  msdosfs  rw  1   1
/dev/ada0p3    none       swap     sw  0   0
EOF

umount /mnt/packages
umount /mnt/dev

# If the target PC is BIOS, remove next line

efibootmgr -c -a -L FreeBSD -l ada0p1:/efi/boot/bootx64.efi

clear
echo
echo "Installation complete, rebooting ..."
echo
reboot

 

[madwebness]

Thank you all, you've been incredibly helpful, this community is the best. I might ask a few questions later on as I proceed or improve the script, but your replies are nothing but great and to the point (I read the rules to avoid making "Thank you posts", but this is my first interaction on this forum and I've been reading a lot of other's threads. Mostly great.

 

[T-Daemon]

As for the wireless, I can think of a non-interactive configuration from the installer script or a post-install configuration with the help of third party applications.

Configuration from the installer script could be as follows, assuming the wireless device is supported, the driver, the SSID and passphrase known. Taking here run(4) as example:

Replace the scripts rc.conf configuration section with

cat << EOF > /mnt/etc/rc.conf
hostname="home-pc.home"
zfs_enable="YES"
kld_list="amdgpu nvidia"
slim_enable="YES"

# Wireless configuration
wlans_ifconfig_run0="wlan0"
ifconfig_wlan0="ssid <ssid_name> WPA DHCP"
EOF

# This configures the parameters for the WPA key
# negotiation with the router/access point.

cat << EOF > /mnt/etc/wpa_supplicant.conf
network={
              ssid="<ssid_name>"
              psk="<passphrase>"
}
EOF

The passphrase can be in clear text or a 256-bit pre-shared WPA key, see wpa_passphrase(8).

Alternatively, after installation, configure wireless with net-mgmt/networkmgr or net-mgmt/wifimgr (an editor for the wpa_supplicant.conf(5).

 

[Sergei_Shablovsky]

Amazing community! Thank You all for perfect detailed answers!

Just not to create extra+ one tread, lets add my sub-question:
Please share bash script to automate installing bunch of packages

I mean
pkg_to_install.txt - text file that contain name of packages that I would to install. Each name in separate row. Full name of package (“zsh-5.2.4”), or short version (“zsh”)
—
zsh
zsh-highlighting
py38-speedtest-cli
librespeed
fping
—

pkg-autoinstaller.sh - bash script

pkg_autoinstaller.log - log file, that also contain copy of screen output of each pkg installation process.
With timestamps.

Script parameters:
-fpkg #force update pkg system
-f #force reinstall package even installed already
-e #not open log-file in default system editor at the end of installation
-rst #restart system after installation complete if no errors (mean no “Error”, “Failed” in every package from list)
Logic:
- clean pkg system cache
- update pkg system before installation
- check/reserve 300kb space for log file
- read each package name from pkg_to_install.txt, install it, write screen output to pkg_autoinstaller.log log file
- [OPT] open log file in editor on tty0 (or where from script started)
- [OPT] standard restart server

Thanks you all another one time!

P.S. I more that sure that this is trivial script, that may be exist in Your scripts library...