Solved Scripting the bsdinstall utility - running into problems when trying the given example from the manpage

[jimkoen]

Hello,

I am currently trying to create a custom script for bsdinstall for my server. I want to apply the partition layout for that specific machine, as well as install certain packages and create some users at install time. My end goal is to have a non-root user account created on install, as well as an SSH server up and running, after the machine restarts. The SSH daemon should be configured to have password authentication disabled at install time and instead accepts my private key, which I supplied via my script during install. Once I log in via SSH, I'm prompted to set my password (makes sense to not hardcode a passwort for either root or the non-root user).

I'm already struggling to get the basic stuff working however.
My current setup involves following the directions in the bsdinstall(8) manpage under 'SCRIPTING', I inject the following script under
/etc/installerconfig:

PARTITIONS=DEFAULT
DISTRIBUTIONS="kernel.txz base.txz"
#!/bin/sh
sysrc ifconfig_DEFAULT=DHCP
sysrc sshd_enable=YES
pkg install puppet

I'm testing the script on a VMware virtual machine.
This is the most basic example that is also given in the manpage.
Yet, this fails with a weird error display:

I can't investigate further, as I cannot login on the system (login fails with "/etc/login.conf is not owned by root", which makes sense, given that the installer runs in a chroot environment).

Anyone knows what I'm doing wrong?

 

[jimkoen]

Okay, so it seems that the example from the manpage has trouble with the default values of some environment variables. For example, this bug report in the FreeBSD issue tracker (PR 264911) mentions a similar error when trying to use bsdinstall to install a FreeBSD guest on KVM/Proxmox whike having set PARTITIONS=DEFAULT in the script. The bug report affects FreeBSD 13.1, the author mentions that they ran into similar behavior on 13.0, but could avoid the bug by setting PARTITIONS to the name of the default virtual block device, vtbd0 in their case.

I have tried the same, my default block device name is ada0 as I am on virtualbox. I have also removed the line to install puppet, to eliminate other sources for errors.

PARTITIONS=ada0
DISTRIBUTIONS="kernel.txz base.txz"
#!/bin/sh
sysrc ifconfig_DEFAULT=DHCP
sysrc sshd_enable=YES

This seems to have fixed my issue, the installation is successful and the machine installs successfully. I have tried once more with setting PARTITIONS to DEFAULT and this also works. So the line to install puppet seems to be the source of error. It prompts me for my confirmation to bootstrap pkg, which I then confirm. As soon as I press enter, a similar error message as above appears.

 

[Sergei_Shablovsky]

avoid the bug by setting PARTITIONS to the name of the default virtual block device, vtbd0 in their case.

I have tried the same, my default block device name is ada0 as I am on virtualbox.

Is that possible to realize “if-this-than-that” technology to making universal installation script in case I have several identical servers but with DIFFERENT disks (HDD, SSD)?

I mean on one machine this would be da but on other nvd/nda

Is possible to using there the method from this thread “

Identify type of disk”​

P.S.
The rest of software I planning to install by Ansible AFTER the FreeBSD installation was successful and first cold reboot.

 

[W.hâ/t]

I am not sure what's going on here. Can the script query input?

pkg bootstrap -fy
pkg install -y prog

 

[jimkoen]

I am not sure what's going on here. Can the script query input?

pkg bootstrap -fy
pkg install -y prog

According to the official manpage, it should be able to:
DISTRIBUTIONS="kernel.txz base.txz" export ZFSBOOT_VDEV_TYPE=stripe export ZFSBOOT_DISKS="ada0 ada1" export nonInteractive="YES" #!/bin/sh echo "ifconfig_DEFAULT=DHCP" >> /etc/rc.conf echo "sshd_enable=YES" >> /etc/rc.conf pkg install puppet

Look under the heading "SCRIPTING"

 

[W.hâ/t]

I've read the thread again, I missed a few things out... Where do you setup the network?
According to /usr/libexec/bsdinstall/script installerconfig is run in ${BSDINSTALL_CHROOT}

 

[T-Daemon]

To be able to install packages from the chroot(8) environment when running the /etc/installerconfig script, two conditions are required:

1. A configured network interface on the installer media system:
   
   Assuming the FreeBSD installer is a USB stick, mount the disk, edit /etc/rc.conf:
   

   sendmail_enable="NONE"
   hostid_enable="NO"
   ifconfig_DEFAULT="DHCP"

   
   If the installer is a ISO image, extract, modify and rebuild the image as described in bsdinstall(8), BUILDING AUTOMATIC INSTALL MEDIA at the end of the manual, then burn on disk.
   .
2. A temporary DNS resolver address on the newly installed system (it will be overwritten when the new system is up, running and had a lease from a DHCP server):
   
   /etc/installerconfig
   

      ...
      #!/bin/sh
      ...
      echo "nameserver 8.8.8.8" > /etc/resolv.conf
   
      pkg bootstrap -y
      pkg install -y puppet

 

[T-Daemon]

For installing packages have also a look at sysutils/firstboot-pkgs.

For system update sysutils/firstboot-freebsd-update.

 

[jimkoen]

I've read the thread again, I missed a few things out... Where do you setup the network?
According to /usr/libexec/bsdinstall/script installerconfig is run in ${BSDINSTALL_CHROOT}

The example I am referring to is this, from the official manpage of bsdinstall, for which it notes:

A typical bsdinstall script, using the default filesystem layout and the
UFS filesystem, looks like this:

PARTITIONS=DEFAULT
DISTRIBUTIONS="kernel.txz base.txz"
#!/bin/sh
sysrc ifconfig_DEFAULT=DHCP
sysrc sshd_enable=YES
pkg install puppet

I have tried this as a noninteractive approach as well. I want to keep referring to the official example though, because the official example is currently not working (I'm using it without any modifications) and I'd like to know why, so I can debug the root cause.

To be able to install packages from the chroot(8) environment when running the /etc/installerconfig script, two conditions are required:

1. A configured network interface on the installer media system:
   
   Assuming the FreeBSD installer is a USB stick, mount the disk, edit /etc/rc.conf:
   

   sendmail_enable="NONE"
   hostid_enable="NO"
   ifconfig_DEFAULT="DHCP"

   
   If the installer is a ISO image, extract, modify and rebuild the image as described in bsdinstall(8), BUILDING AUTOMATIC INSTALL MEDIA at the end of the manual, then burn on disk.
   .
2. A temporary DNS resolver address on the newly installed system (it will be overwritten when the new system is up, running and had a lease from a DHCP server):
   
   /etc/installerconfig
   

      ...
      #!/bin/sh
      ...
    echo "nameserver 8.8.8.8" > /etc/resolv.conf
   
      pkg bootstrap -y
      pkg install -y puppet

I've read your previous posts on the topic and have followed these steps (minus the DNS resolver, because the installer is able to use my networks resolver by getting it from DHCP). An installation sort of works, but I still cannot install any packages, because the install script is not able to write to a file system. I am moderately certain that this is because it needs a temporary FS on my bootable USB to work (pkg seems to not want to use the newly created volume as a target for installs, or rather, it does, but it seems to still need to request write access to the live usb stick in order to function properly.)
My script exits by rebooting the machine, which avoids the above error message. I am still curious as to why it appears however, because I can sometimes avoid it by not installing packages at all. The installer will instead show an "install successful" screen, which seems to be the intended behavior.

For installing packages have also a look at sysutils/firstboot-pkgs.

For system update sysutils/firstboot-freebsd-update.

How would using packages from Ports be of any use, when I can't install software reliably during my scripted installation whatsoever?

 

[W.hâ/t]

How would using packages from Ports be of any use, when I can't install software reliably during my scripted installation whatsoever?

Click the link and read the description maybe?

The example I am referring to is this, from the official manpage of bsdinstall, for which it notes:

Yes, I've 'read' that, it doesn't tell the whole story.

 

[jimkoen]

Click the link and read the description maybe?

I did, I assume I'm supposed to include this in a live USB disk image somehow?

Yes, I've 'read' that, it doesn't tell the whole story.

Heres the script that I've now resorted to. It works, but vim is never installed, as pkg can't install it

installerconfig:

DISTRIBUTIONS="kernel.txz base.txz src.txz ports.txz"
PARTITIONS=nvd0
export nonInteractive="YES"

#!/bin/sh

echo jimkoen::::01-01-1970::John Username Doe::tcsh:none | adduser -w none -G wheel -f -
pw usermod jimkoen -p -

mkdir /home/jimkoen/.ssh
touch /home/jimkoen/.ssh/authorized_keys
chown jimkoen:jimkoen /home/jimkoen/.ssh/authorized_keys
chown jimkoen:jimkoen /home/jimkoen/.ssh
echo "ssh-rsa <my-ssh-key>" > /home/jimkoen/.ssh/authorized_keys


echo "ifconfig_ix0=DHCP" >> /etc/rc.conf
echo "sshd_enable=YES" >> /etc/rc.conf

dhclient ix0

env ASSUME_ALWAYS_YES=YES pkg bootstrap -f | cat
pkg install -y sudo wget vim
poweroff

rc.conf:

sendmail_enable="NONE"
hostid_enable="NO"

 

[W.hâ/t]

but vim is never installed, as pkg can't install it

This doesn't tell us much. Can you provide an error message? Can you install it once booted?
Also which version/architecture are you installing?

 

[T-Daemon]

An installation sort of works, but I still cannot install any packages, ...

Why not just try it exactly as I described, in a VM, with a minimal installer script?

Configure the network interface of the installer media and add a DNS name server IP to the newly installed system.

/etc/rc.conf

...
ifconfig_DEFAULT="DHCP"

/etc/installerconfig

DISTRIBUTIONS="kernel.txz base.txz src.txz ports.txz"
PARTITIONS=ada0
export nonInteractive="YES"

#!/bin/sh

echo "nameserver 8.8.8.8" > /etc/resolv.conf

pkg bootstrap -y
pkg install -y sudo wget vim

I wouldn't install ports.txz here, that package is outdated. Better install after booting the new system or from the installer script with portsnap(8) or devel/git.

I've checked the script in a VirtualBox VM. Packages are installed as expected with the above configuration.

... , because the install script is not able to write to a file system.

That's a wrong conclusion. The install script (/etc/installerconfig) did install the system (by invoking bsdinstall(8)), chrooted in the newly installed system, executed echo(1), pw(8), mkdir(1), touch(1), chown(8), did it not? That is only possible when the file system of the newly installed system is writable.

I am moderately certain that this is because it needs a temporary FS on my bootable USB to work (pkg seems to not want to use the newly created volume as a target for installs, or rather, it does, but it seems to still need to request write access to the live usb stick in order to function properly.)

Nothing of the sort is necessary.

How would using packages from Ports be of any use, ...

sysutils/firstboot-pkgs is an alternative of installing packages on first boot on the new system. Granted, from a installer script it is not much of an improvement, it's easier to pkg-install(8).

sysutils/firstboot-freebsd-update is definitively of use in a non-interactive scenario to update the system (and /usr/src). It can be installed and configured by the installer script.

 

[T-Daemon]

This doesn't tell us much. Can you provide an error message?

That is difficult. Error messages are covered by the error dialog(1) window.

 

[W.hâ/t]

That is difficult. Error messages are covered by the error dialog(1) window.

I did not test it but it should be pretty straightforward.

export debug=1
export debugFile="+/tmp/bsdinstall_debug.log"
...
#!/bin/sh
whatever cmd | tee -a /tmp/post-install.log

{
     ...
} | tee -a /tmp/post-install.log

## comment poweroff
#poweroff

When done, inspect /tmp/bsdinstall_debug.log and /mnt/tmp/post-install.log

 

[W.hâ/t]

So not that simple...

/usr/libexec/bsdinstall/script resets $debugFile if it differs from $BSDINSTALL_LOG

# Re-initialize a new log if preamble changed BSDINSTALL_LOG                                                                                       
if [ "$BSDINSTALL_LOG" != "${debugFile#+}" ]; then                       
        export debugFile="$BSDINSTALL_LOG"                                                                                                         
        f_quietly f_debug_init                                                                                                                     
        # NB: Being scripted, let debug go to terminal for invalid debugFile                                                                       
        f_dprintf "Began Installation at %s" "$( date )"                                                                                           
fi

I am not sure if it is the intended behavior, anyway this force the following.

export BSDINSTALL_LOG="/tmp/bsdinstall_debug.log"
export debugFile="+${BSDINSTALL_LOG}"  # if output is also wanted on stdout, otherwise it can be discarded.

I also noticed that /usr/libexec/bsdinstall/config copies $debugFile to /var/log before 'chrooting'
So now we can simply append the commands to $debugFile

...
#!/bin/sh
export debug=1
export debugFile="+/var/log/bsdinstall_debug.log"
# The following MUST be set or it will destroy $debugFile ( : >${debugFile#+} )
export DEBUG_SELF_INITIALIZE=0

. /usr/share/bsdconfig/common.subr

...
f_eval_catch "post-install-script" "pkg-install" "pkg install -y %s" "vim tmux"
...
f_dprintf "post-install-script finished at $(date)"

 

[W.hâ/t]

I got curious and gave it try... Well... Defenetly not that simple. I got lost in the code and I don't even know who I am anymore, haha.

For those interrested, I did find a bug PR 204506

I recommend using the default /tmp/bsdinstall_log as log file.

So to print the output to stdout and the log:
- A plus `+' sign must be prenpeded to $debugFile in /etc/rc.local.
- $debugFile and $BSDINSTALL_LOG must be set to the same value without the `+' sign in /etc/installerconfig.

Here is the little bugger. tee will erase the log each time bsdinstall is called.

        +*) exec "/usr/libexec/bsdinstall/$VERB" "$@" \
                2>&1 >&$TERMINAL_STDOUT_PASSTHRU | tee "${debugFile#+}" ;;

Other than that, I also didn't have any trouble installing pkgs. Maybe it's a disk space issue?

 

[jimkoen]

Why not just try it exactly as I described, in a VM, with a minimal installer script?

Configure the network interface of the installer media and add a DNS name server IP to the newly installed system.

[...]

I've checked the script in a VirtualBox VM. Packages are installed as expected with the above configuration.

Other than that, I also didn't have any trouble installing pkgs. Maybe it's a disk space issue?

I finally now had time to look into this again and having a misconfigured network interface was indeed the issue. I only found out after I video recorded the installer log from my servers KVM, where afterwards I could see that the interface I was trying to get a DHCP lease on was inactive. Meaning dhclient would time out.

But now everything is working as expected. I settled on the this configuration, which also set's up an initial user and an SSH key. The user has an initial empty password and is prompted to change it on first login via SSH:

DISTRIBUTIONS="kernel.txz base.txz src.txz"
PARTITIONS=nvd0 # Replace with your drive name
export nonInteractive="YES"

#!/bin/sh
echo "nameserver 1.1.1.1" > /etc/resolv.conf

echo johndoe::::01-01-1970::John Doe::tcsh:none | adduser -w none -G wheel -f -
pw usermod johndoe -p -

mkdir /home/johndoe/.ssh
touch /home/johndoe/.ssh/authorized_keys
chown johndoe:johndoe /home/johndoe/.ssh/authorized_keys
chown johndoe:johndoe /home/johndoe/.ssh
echo "<ssh-key>" > /home/johndoe/.ssh/authorized_keys


echo "ifconfig_ix0=DHCP" >> /etc/rc.conf
echo "sshd_enable=YES" >> /etc/rc.conf

ifconfig ix0 up
dhclient ix0

env ASSUME_ALWAYS_YES=YES pkg bootstrap -f | cat
pkg install -y sudo wget vim
poweroff

Thanks for all the help! My only remaining issue is modifying the installation media so that I can overwrite default configurations such as the sshd_config with my own custom ones, but that may be a topic for a different post.