Solved How is bsdinstall started

Question, can you use mfbsd to create an USB pendrive image ?
I tried dd'ing the ISO image to a pendrive but that didn't work, however reading through the docs (BUILD.md):-

## Creating images

You may create three types of output: disc image for use by dd(1),.
ISO image or a simple .tar.gz file

##Examples

1. disc image

```bash
make BASE=/cdrom/usr/freebsd-dist

so it looks like you can just by changing the target of the Makefile. I'll give it a try shortly.
 
Question, can you use mfbsd to create an USB pendrive image ?
Having just tried, indeed you can dd the .img file to a USB pendrive and it boots up straight to a BSDINSTALL panel. i'm still trying to figure out how to incorporate an installerconfig script and there is no sign of the installation files, but I'm working on it...
 
The magic happens here,
Code:
${_v}${MKDIR} ${WRKDIR}/mnt ${WRKDIR}/trees/base/boot
    ${_v}${INSTALL} -m 0444 ${WRKDIR}/disk/boot/boot ${WRKDIR}/trees/base/boot/
    ${_v}${DOFS} ${BSDLABEL} "" ${WRKDIR}/disk.img ${WRKDIR} ${WRKDIR}/mnt 0 ${WRKDIR}/disk 80000 auto > /dev/null 2> /dev/null
    ${_v}${RM} -rf ${WRKDIR}/mnt ${WRKDIR}/trees
    ${_v}${MV} ${WRKDIR}/disk.img ${.TARGET}
And if you want to hear the voice of the original author,
 
I'm sorry but you've lost me there... if the sample code you provided is part of doFS.sh then I can't find it.
 
One thing I can't figure out is how to get the mfsbsd build to install packages... According to BUILD.md

## Additional packages and files
If you want any packages installed, copy the .tbz files that should be.
automatically installed into the packages/ directory

I want to install sysutils/lfm but how do I get a .tbz version of it? Tried pkg-fetch(), but that didn't work.
 
1. uninstall lfm , install lfm, and then you find the .txz in your /var/cache/pkg
or,
2.
Code:
wget http://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/lfm-3.1.txz
 
After test install from local media, you can makefs from the memstick image, and put it on tftp server and then boot from this image. Which like mfsbsd works, but load whole rootfs of install media to memory.
Code:
mfs_load="YES"
mfs_type="mfs_root"
mfs_name="/mfsroot"
Not sure where this code is supposed to go or how it is supposed to work. Are you talking about remote booting?

I can put the ISO image on my tftp server and boot from it using PXELINUX.
 
That was part of the makefile which calls dofs.sh
I guess you mean this section... I couldn't see it too easily in the Makefile, but I've copied the section here to provide better context:-
Bash:
image: install prune config genkeys customfiles customscripts boot compress-usr mfsroot fbsddist ${IMAGE}                                                                                    
${IMAGE}:                                                                                                                                                                                    
        @echo -n "Creating image file ..."                                                                                                                                                    
.if defined(BSDPART)                                                                                                                                                                          
        ${_v}${MKDIR} ${WRKDIR}/mnt ${WRKDIR}/trees/base/boot                                                                                                                                
        ${_v}${INSTALL} -m 0444 ${WRKDIR}/disk/boot/boot ${WRKDIR}/trees/base/boot/                                                                                                          
        ${_v}${DOFS} ${BSDLABEL} "" ${WRKDIR}/disk.img ${WRKDIR} ${WRKDIR}/mnt 0 ${WRKDIR}/disk 80000 auto > /dev/null 2> /dev/null                                                          
        ${_v}${RM} -rf ${WRKDIR}/mnt ${WRKDIR}/trees                                                                                                                                          
        ${_v}${MV} ${WRKDIR}/disk.img ${.TARGET}                                                                                                                                              
.else                                                                                                                                                                                        
        ${_v}${TOOLSDIR}/do_gpt.sh ${.TARGET} ${WRKDIR}/disk 0 ${WRKDIR}/boot ${VERB}                                                                                                        
.endif                                                                                                                                                                                        
        @echo " done"                                                                                                                                                                        
        ${_v}${LS} -l ${.TARGET}

Not really sure what this is doing, but I don't see that it copying [base|kernel].txz to anyware in the img file...
 
in the makefile,
Code:
@echo -n "Extracting base and kernel ..."
${_v}${CAT} ${BASEFILE} | ${TAR} --unlink -xpzf - -C ${_DESTDIR}
and
Code:
${_v}${CAT} ${KERNELFILE} | ${TAR} --unlink -xpzf - -C ${_BOOTDIR}
${_v}${MV} ${_BOOTDIR}/${KERNCONF}/* ${_BOOTDIR}/kernel
${_v}${RMDIR} ${_BOOTDIR}/${KERNCONF}

Destination is a mounted a file-backed device, which later becomes the .img
 
I think this is the section I need:-
Bash:
if defined(SE)                                                                                                                                                                               
. if !defined(CUSTOM) && exists(${BASE}/base.txz) && exists(${BASE}/kernel.txz)                                                                                                               
        @echo -n "Copying base.txz and kernel.txz ..."                                                                                                                                         
. else                                                                                                                                                                                         
        @echo -n "Creating base.txz and kernel.txz ..."                                                                                                                                       
. endif                                                                                                                                                                                       
        ${_v}${MKDIR} ${_DISTDIR}                                                                                                                                                             
. if !defined(NO_ROOTHACK)                                                                                                                                                                     
        ${_v}${CP} -rp ${_BOOTDIR}/${KERNDIR} ${_DESTDIR}/boot                                                                                                                                 
.  if "${KERNDIR}" != "kernel"                                                                                                                                                                 
        ${_v}${MV} -f ${_DESTDIR}/boot/${KERNDIR} ${_DESTDIR}/boot/kernel                                                                                                                     
.  endif                                                                                                                                                                                       
. endif                                                                                                                                                                                       
. if !defined(CUSTOM) && exists(${BASE}/base.txz) && exists(${BASE}/kernel.txz)                                                                                                               
        ${_v}${CP} ${BASE}/base.txz ${_DISTDIR}/base.txz                                                                                                                                       
        ${_v}${CP} ${BASE}/kernel.txz ${_DISTDIR}/kernel.txz                                                                                                                                   
. else                                                                                                                                                                                         
        ${_v}${TAR} -c -C ${_DESTDIR} -J ${EXCLUDE} --exclude "boot/${KERNDIR}/*" -f ${_DISTDIR}/base.txz .                                                                                   
        ${_v}${TAR} -c -C ${_DESTDIR} -J ${EXCLUDE} -f ${_DISTDIR}/kernel.txz boot/kernel                                                                                                     
. endif                                                                                                                                                                                       
        @echo " done"                                                                                                                                                                         
. if !defined(NO_ROOTHACK)                                                                                                                                                                     
        ${_v}${RM} -rf ${_DESTDIR}/boot/${KERNDIR}                                                                                                                                             
. endif                                                                                                                                                                                       
.endif

So maybe I need to define {SE} somewhere... but where..
 
I think this is the section I need:-
...
So maybe I need to define {SE} somewhere... but where..
You are looking at all the wrong places, complicating things unnecessarily. You don't need to modify the Makefile or any other script of 'mfsbsd'.

In BUILD.md is's clearly stated where to put additional files to include them in the created image:
Add any additional files into the customfiles/ directory. These will be copied recursively into the root of the boot image.
customfiles represents the root ( / ) directory of the image. If a file 'B' needs to be included to directory 'A' on the boot image then the directory structure needs to be customfiles/A/B. If directory 'A' is absent it will be created.

In post #21 is a instruction how to include installerconfig, MANIFEST, kernel.txz, base.txz] into the image. The created image has the kernel and base sets in /usr/freebsd-dist, where the installer expects them to be, and the installerconfig in /etc.
 
When I create an .img and write to usb, the usb has a corrupt gpt partition table which does not boot.
But I have old bios with gpt boot problems.
I'll do some test with virtualbox
 
The created .iso boots correctly in virtualbox.
When you login you see a mounted memory device as root.
Something strange is, is it gives mounted read-only when in fact it is read-write, probably normal for a memory device.
Networking in virtualbox did not seem to work but vm_bhyve works fine with mfsroot.
 
Bingo!

Managed to get it working with the following code:-

Bash:
mkdir -p ~/mfsBSD-workdir/DIST
cd  ~/mfsBSD-workdir

pkg install -y git
git clone https://github.com/mmatuska/mfsbsd.git

fetch -o - https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/12.2/FreeBSD-12.2-RELEASE-amd64-memstick.img.xz | xz -d > DIST/FreeBSD.img

mkdir /mnt/md
mount -r /dev/`mdconfig -o readonly DIST/FreeBSD.img`s2a /mnt/md


#- If there is a work directory and an ISO from a previous build remove them
#- Edit Makefile  'MFSROOT_MAXSIZE?=   400m'   # the ISO has  ~300MB with kernel and base.
#                                              # If there are more sets included increase the size
#cp rc.local conf/                             # a rc.local from a installer image
#mkdir -p customfiles/usr/freebsd-dist/        # copy here MANIFEST, kernel.txz, base.txz, ...
#mkdir customfiles/etc/                        # put here installerconfig
#make iso BASE=/dir/to/distfiles_kernel.txz_and_base.txz


cp /mnt/md/etc/rc.local mfsbsd/conf/
cp mfsbsd/conf/hosts.sample mfsbsd/conf/hosts
sed 's/#ifconfig_rl0/ifconfig_em0/' mfsbsd/conf/rc.conf.sample >mfsbsd/conf/rc.conf

sed  -i '' '/MFSROOT_MAXSIZE?/s/120/400/' mfsbsd/Makefile

echo 'lfm' > mfsbsd/tools/packages.sample

mkdir -p mfsbsd/customfiles/usr/freebsd-dist/        # copy here MANIFEST, kernel.txz, base.txz, ...
cp /mnt/md/usr/freebsd-dist/base.txz mfsbsd/customfiles/usr/freebsd-dist
cp /mnt/md/usr/freebsd-dist/kernel.txz mfsbsd/customfiles/usr/freebsd-dist
mkdir mfsbsd/customfiles/etc/                        # put here installerconfig

echo 'creating installerconfig script'

cat > mfsbsd/customfiles/etc/installerconfig <<__INSTALLERCONFIG_
# Installerconfig

PARTITIONS="ada0 gpt { 20G freebsd-ufs /, 4G freebsd-swap }"

DISTRIBUTIONS="kernel.txz base.txz"

#!/bin/sh
gpart bootcode -b /boot/pmbr    -p /boot/gptboot -i 1 ada0
sysrc ifconfig_em0=DHCP
sysrc sshd_enable=YES

__INSTALLERCONFIG_


cd mfsbsd

make iso BASE=/mnt/md/usr/freebsd-dist

I named this script build-mfsBSD and ran it with the following command which automatically creates a log of the process:


time script build-mfsbsd_$(date +"%y%m%d%H%M").log sh ./build-mfsBSD


BEWARE THAT THIS SCRIPT AS IT STANDS WILL DESTROY WHAT YOU CURRENTLY HAVE ON YOUR LOCAL HARD DISK (ada0) SO TAKE CARE
 
Bingo!

Managed to get it working with the following code:-

Bash:
mkdir -p ~/mfsBSD-workdir/DIST
cd  ~/mfsBSD-workdir

pkg install -y git
git clone https://github.com/mmatuska/mfsbsd.git

fetch -o - https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/12.2/FreeBSD-12.2-RELEASE-amd64-memstick.img.xz | xz -d > DIST/FreeBSD.img

mkdir /mnt/md
mount -r /dev/`mdconfig -o readonly DIST/FreeBSD.img`s2a /mnt/md


#- If there is a work directory and an ISO from a previous build remove them
#- Edit Makefile  'MFSROOT_MAXSIZE?=   400m'   # the ISO has  ~300MB with kernel and base.
#                                              # If there are more sets included increase the size
#cp rc.local conf/                             # a rc.local from a installer image
#mkdir -p customfiles/usr/freebsd-dist/        # copy here MANIFEST, kernel.txz, base.txz, ...
#mkdir customfiles/etc/                        # put here installerconfig
#make iso BASE=/dir/to/distfiles_kernel.txz_and_base.txz


cp /mnt/md/etc/rc.local mfsbsd/conf/
cp mfsbsd/conf/hosts.sample mfsbsd/conf/hosts
sed 's/#ifconfig_rl0/ifconfig_em0/' mfsbsd/conf/rc.conf.sample >mfsbsd/conf/rc.conf

sed  -i '' '/MFSROOT_MAXSIZE?/s/120/400/' mfsbsd/Makefile

echo 'lfm' > mfsbsd/tools/packages.sample

mkdir -p mfsbsd/customfiles/usr/freebsd-dist/        # copy here MANIFEST, kernel.txz, base.txz, ...
cp /mnt/md/usr/freebsd-dist/base.txz mfsbsd/customfiles/usr/freebsd-dist
cp /mnt/md/usr/freebsd-dist/kernel.txz mfsbsd/customfiles/usr/freebsd-dist
mkdir mfsbsd/customfiles/etc/                        # put here installerconfig

echo 'creating installerconfig script'

cat > mfsbsd/customfiles/etc/installerconfig <<__INSTALLERCONFIG_
# Installerconfig

PARTITIONS="ada0 gpt { 20G freebsd-ufs /, 4G freebsd-swap }"

DISTRIBUTIONS="kernel.txz base.txz"

#!/bin/sh
gpart bootcode -b /boot/pmbr    -p /boot/gptboot -i 1 ada0
sysrc ifconfig_em0=DHCP
sysrc sshd_enable=YES

__INSTALLERCONFIG_


cd mfsbsd

make iso BASE=/mnt/md/usr/freebsd-dist

I named this script build-mfsBSD and ran it with the following command which automatically creates a log of the process:


time script build-mfsbsd_$(date +"%y%m%d%H%M").log sh ./build-mfsBSD


BEWARE THAT THIS SCRIPT AS IT STANDS WILL DESTROY WHAT YOU CURRENTLY HAVE ON YOUR LOCAL HARD DISK (ada0) SO TAKE CARE
Have you found a way to get mfsbsd to auto-start bsdinstall? I haven't tried the script above so apologies if it does that and I'm missing it, I've gotten mfsbsd to start via an iso over PXE but I still have a few commands I have to execute after it does so - this looks like it would take care of the kernel.txz and base.txz downloads but my ideal would be to have it go straight from selecting the iPXE menu item to the bsdinstall utility without typing additional commands (or even the root/mfsroot login info.)
 
Do you want interactive installation or scripted, unattended installation?
Interactive, I just want to get to the bsdinstall utility (no parameters - didn't know it could take them if it can) without having to do more than select the iPXE menu option for the iso - from there it would be a manual setup.
 
My goal is essentially to have PXE-bootable install discs for FreeBSD and some other things, though most others OS's have a slim iso that works.
 
Grab from an official installer image /etc/rc.local and add it to the build. That shell script includes the bsdinstall interactive dialog. When the mfsbsd iso image is PXE booted, the installation dialog will appear automatically, no additional commands, no log in. Also copy (or download) MANIFEST, base.txz and kernel.txz, eventually the other sets, if you want them included to the image.

From the root mfsbsd source directory proceed with the steps down below:

Origin post #21
Code:
- If there is a work directory and an ISO from a previous build remove them
- Edit Makefile  'MFSROOT_MAXSIZE?=   400m'   # the ISO has  ~300MB with kernel and base.
                                              # If there are more sets included increase the size
cp rc.local conf/                             # a rc.local from a installer image
mkdir -p customfiles/usr/freebsd-dist/        # copy here MANIFEST, kernel.txz, base.txz, ...
make iso BASE=/dir/to/distfiles_kernel.txz_and_base.txz

If this is a mfsbsd installer image and there is no need for additional packages to be included, you can rename file mfsbsd/tools/packages.sample, so they are not fetched and installed.
 
It looks like it's close, but apparently not picking up MANIFEST, kernel.txz, base.txz - any ideas on that one?
 
Have you edited the Makefile and increased MFSROOT_MAXSIZE?= 400m? Cleaned the mfsbsd source directory?
Code:
mfsbsd # rm -r work
mfsbsd # rm *.iso

Builds fine on my system:
Code:
mfsbsd # ls customfiles/usr/freebsd-dist/
MANIFEST    base.txz    kernel.txz

mfsbsd # make iso BASE=/tmp/mfsbsd/freebsd-dist
Extracting base and kernel ... done
Removing selected files from distribution ... done
Installing configuration scripts and files ... done
Generating SSH host keys ... done
Copying user files ...
customfiles/ -> /tmp/mfsbsd/mfsbsd/work/mfs/rw
customfiles/usr -> /tmp/mfsbsd/mfsbsd/work/mfs/rw/usr
customfiles/usr/freebsd-dist -> /tmp/mfsbsd/mfsbsd/work/mfs/rw/usr/freebsd-dist
customfiles/usr/freebsd-dist/base.txz -> /tmp/mfsbsd/mfsbsd/work/mfs/rw/usr/freebsd-dist/base.txz
customfiles/usr/freebsd-dist/kernel.txz -> /tmp/mfsbsd/mfsbsd/work/mfs/rw/usr/freebsd-dist/kernel.txz
customfiles/usr/freebsd-dist/MANIFEST -> /tmp/mfsbsd/mfsbsd/work/mfs/rw/usr/freebsd-dist/MANIFEST
 done
Configuring boot environment ...x ./
x ./linker.hints
x ./kernel
 done
Installing pkgng ... done
Installing user packages ...
  100 %       253.0 MiB / 414.5 MiB = 0.610   2.9 MiB/s       2:21
  done
---cut ---
Creating and compressing mfsroot ... done
Creating ISO image ... done
-rw-r--r--  1 root  wheel  281952256  8 Jun 16:00 mfsbsd-12.2-RELEASE-p7-amd64.iso
Note the image size.
 
Back
Top