Combining tmpfs and unionfs on root filesystem

S

sub_mesa

Hello forum!

I'm building a web-interface to ZFS based on FreeBSD, called Mesa. Ideally i would love to release several versions:

1) script-only version (.php files; requires manual setup)
2) normal binary version (pre-installed binary image based on UFS or ZFS; root bootable filesystem)
3) embedded binary version that runs great on USB flash / CF media due to no writes happening during operation

The reason for opening this thread, is solving the puzzle on how to implement option three, as i already implemented the first two. To describe exactly what i want:

root filesystem: tmpfs (writable filesystem that runs in RAM; contents gone after reboot)
mounted in combination with unionfs that acts as a bridge between the root tmpfs and a binary image that is not writable.

The result of this is a writable root filesystem where no physical writes happen to the system disk, which usually is a small USB pendrive or CompactFlash card. Such devices do not last long with the large number of small writes/modifications happening on system disks.

My question is: how does this work? Not much information turns up in my search queries, except the usual /tmp tmpfs examples and non-root filesystem unionfs mounts. My experience tells me that /boot/loader.conf should hold values to implement such a thing; but i couldn't find much tunables in /boot/defaults/loader.conf other than loading the relevant kernel modules themselves. I briefly looked at FreeNAS, but their /boot/loader.conf appears empty; and the /etc/fstab is pretty much empty too. How is this stuff supposed to be configured?

Anyone who could put me on the right track? Thanks alot!
 
Looks interesting! I'll play with it and see if it suits my needs.

The cool thing of this is that i can do binary updates on a running system. That means: while the systems runs you download a new version (say 9.0 when it comes out), and boot that binary image next time instead.

I also hope to implement something that would save all changes made (for example to configuration files); NanoBSD appears to support that with a writable MSDOS /etc partition. However, i believe it uses mdmfs and not tmpfs. That means the size will not be dynamic like tmpfs (uses all RAM or none depending on howmuch data is stored). Ideally, i would love to use tmpfs as i feel its perfectly suited to my intended use.

Anyway, will update here as i play around with NanoBSD. Thanks!
 
sub_mesa said:
I also hope to implement something that would save all changes made (for example to configuration files); NanoBSD appears to support that with a writable MSDOS /etc partition.
Click to expand...
It's a UFS partition, not an MSDOS partition.

sub_mesa said:
However, i believe it uses mdmfs and not tmpfs. That means the size will not be dynamic like tmpfs (uses all RAM or none depending on howmuch data is stored).
Click to expand...
mdmfs uses swap based md(4) devices by default which behave as you would like. However, FreeBSD's /etc/rc.initdiskless script calls mdmfs with "-M", telling it to use malloc based md devices. Easy work around might be to modify line 201 of /etc/rc.initdiskless. :)
 
Well, i took a different direction and experimented a bit; was a very fruitful experience i'd say!

Right now i ended up with tmpfs running as root filesystem without anything else mounted. That means: i have a tiny preloaded md image that has a script which copies an archive from cdrom (uzip UFS image) to the new tmpfs and then chroots.

The cool thing of this approach is that there is no storage required at all; you could take the CDROM out and it'll run in memory from now on. The performance is pretty cool too, simple dd benches say about 1GB/s write and 3GB/s read, that is within a Virtualbox VM; not bad at all! The best thing here, is that unlike other approaches, tmpfs would not cache data twice in memory.

Normally if you read stuff from UFS contained in md image, you would see the "Inact" value in top rise due to more filecache. So it caches the memory disk; which does not make any sense. I guess under memory constraints the Inact would lower quickly as filecaches can be safely discarded; but this tmpfs approach is much better; Inact does not rise at all when reading all 300MB contents of the system disk image contained in tmpfs.

And best of all, creating a new .iso from all three parts (cdrom, preloaded md image, large system disk image) now takes only 10 seconds before i can re-try in Virtualbox with a brand new .iso. This shortens my development time considerably!

One limitation i came across and the original reason for opening this thread, was that unionfs and tmpfs don't play along all that well; with mdmfs it works great but as soon as i replace with tmpfs the unionfs fails with "Operation not supported". This remains unresolved, but i'll start by experimenting with a real tmpfs setup without unionfs. With the cdrom and preloaded environments more or less done, i will move on tuning the system disk image; which in essence is a full blown FreeBSD install.

My first usable liveCD might not be that far away after all. :)
 
For anyone trying to do anything similar and finding this with Google, this may be of help:

It's the script i use to build everything from scratch. Very cool, and you can see what takes the most time.

Code: 
[MAKEALL] Making all from scratch
[1/5] [SYSTEM] Starting script
* Destroying /mesadev/systemdisk
* Creating /mesadev/systemdisk
* installworld
* (took: 75 seconds)
* installkernel
* (took: 33 seconds)
* distribution
* (took: 0 seconds)
* remove symbols
* compressing kernel
[1/5] [SYSTEM] total time: 109 seconds
[2/5] [POPULATE] Starting script
* adding packages to /mesadev/systemdisk
* bash
* lighttpd
* fcgi-devkit
* php5
* php5-session
* samba34
* sudo
* populating filesystem with tunables
* configuring lighttpd
* copying mesa web interface
[2/5] [POPULATE] total time: 131 seconds
[3/5] [PRELOADED] Starting script
* creating .ufs file from filesystem
* compressing .ufs file into .ufs.gz
* moving .ufs.gz file to /mesadev/cdrom/boot/
[3/5] [PRELOADED] Total time: 1 sec
[4/5] [UZIP] Starting script
* creating .ufs file
* (took: 53 seconds)
* creating .ufs.uzip file from .ufs file
* (took: 59 seconds)
[4/5] [UZIP] total time: 112 seconds
[5/5] [MAKEISO] Starting script
* creating .iso archive
[5/5] [MAKEISO] total time: 20 seconds
[MAKEALL] cumulatative time: 373 seconds

I had much help from this guide: http://wiki.freebsd.org/AvgLiveCD
 
You must log in or register to reply here.
Back
Top