Solved Getting image file from ZFS disk

A reinstall is usually a lot quicker than restoring a backup. Separate data from the OS, backup that, because that's important. Document what you installed and how you configured it. An OS or a bunch of applications can always be easily reinstalled, your data however cannot be easily recreated.

Images are useful in situations where you need to spin up VMs for example, then you can quickly clone them. They're not that useful for installing on "iron".
 
Alain De Vos said:
To backup you only need "cp" in my humble opinion.
Some use "rsync", "clone" is also not bad.
Click to expand...
That easy? LoL yea but sounds true. I can use rsync for do it faster. So can I copy /dev things too? and proc? and why "dd" command exist if its just that?
 
If you have "something" to get the backup (in this example an hard disk, da1)

Code: 
pkg install -y pv pigz
zpool create mybak da1
zfs snapshot -r zroot@mybackup
zfs send -R zroot@mybackup |pigz|pv >/mybak/c.zfs.gz
... or send to "something" (with ssh), USB key etc.

On the new one (for example da0 disk, but whatever), with attached da1
(the disk containing the backup, of course can be whatever, just a snippet)
Boot from ISO, USB Key, whatever (I use https://mfsbsd.vx.sk/)
Code: 
gpart destroy -F da0
gpart create -s gpt da0
gpart add -a 4k -t freebsd-boot -s 512k -l boot da0
gpart add -a 4k -t freebsd-swap -s 16g -l swap0 da0
gpart add -a 4k -t freebsd-zfs -l zfs0 da0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
gnop create -S 4096 /dev/gpt/zfs0
zpool create -f -o altroot=/mnt -O canmount=off -m none zroot /dev/gpt/zfs0.nop

Now restore the image
Code: 
zpool import -o altroot=/mnt -f mybak
gzcat /mnt/mybak/c.zfs.gz | zfs receive -vF zroot
shutdown -r now
Shutdown not really necessay, but I get LOT LESS problems with a reboot

Code: 
zpool import -o altroot=/mnt -f zroot
zpool set bootfs=zroot/ROOT/default zroot
shutdown -r now
 
fbsd_ said:
So can I copy /dev things too? and proc?
Click to expand...
Why do you think you need to copy those? They're dynamic, virtual, filesystems (see devfs(5) and procfs(5)). Devices are created based on the hardware the machine has, and proc only contains information about things that are currently running on that machine.
 
fbsd_ said:
That easy? LoL yea but sounds true. I can use rsync for do it faster. So can I copy /dev things too? and proc? and why "dd" command exist if its just that?
Click to expand...
dd is archaic, with zfs.
It is rarely used, often with virtual machines.
 
The question how do you copy /dev is however interesting. It is said you "make special files".
The freebsd installer does this for you ....
 
SirDice said:
Why do you think you need to copy those? They're dynamic, virtual, filesystems (see devfs(5) and procfs(5)). Devices are created based on the hardware the machine has, and proc only contains information about things that are currently running on that machine.
Click to expand...
Yea I mean this by saing cp not a very nice option. cp / thing copies that dynamic dirs too. So cp seems to not a very clean solution.
 
msplsh said:
He wants a "ghost" image to blast on other computers, although this method will copy SSH keys, not change the hostname, etc.
Click to expand...
SSH keys and I guess it will copy GUID and things like that too. Not very nice but maybe I can randomize them later by a script huh?
 
The "dumb" way to do this is to boot from some other media, don't mount the system and dd the disk to a file on some third media. This does not account for disk sizes, resizing the disk, and isn't space efficient.
 
fbsd_ said:
I want to full back up my FreeBSD 12.2 system. I want a image file that be able to setup other computers. How?
Click to expand...
The main and ZFS-native way is to use send streams. You could save a file of the output of "zfs send" and keep that around for future receive operations.
 
Sorry to revive this solved thread.
chungy answer piqued my interest.

What would happen if said "file output" from a "zfs send" operation becomes corrupted or developed bit rot or flipped bit on a USB stick or other storage media.

Does "zfs receive" have the ability to cope or recover? Detect and scrub?
 
Phishfry said:
Does "zfs receive" have the ability to cope or recover? Detect and scrub?
Click to expand...
Not if you dump the zfs recv into a single file; only if it is received into a proper pool with redundancy (e.g. mirrored vdevs).
You could hack up something like redundant vdevs via multiple image files, but if they reside on the same disk that's completely worthless...
IIRC the zfs recv can only detect checksum errors and error out, but the stream doesn't contain any redundancy, because that would be horribly inefficient. Usually you would just re-send the stream if it got corrupted in transit.

My backup strategies nowadays are usually based on ZFS snapshots and periodic incremental send|recv over multiple hosts. I.e. one host gathers all the backups/snapshots from multiple systems into a dedicated backup pool; snapshots from this pool then are send|recv to the next system (NAS) for long term backups.
Additionally, the *really* important stuff is also backed up to another FS/media and/or tarsnap.

ZFS snapshots and incremental send|recv are cheap and fast (at least after the large, initial sync), so I use them extensively and in varying intervals, down to */10 minutes for snapshots and hourly send|recv jobs, depending on the importance or rate of changes and human involvement on the datasets. E.g. datasets from our windows-fileserver are usually snapshotted in 10 minute intervals, with 6 hours hold time - if I had gotten a buck for every time I had to restore some files that "just disappeared" because the user "did nothing" I could easily retire by now...
 
OK so you must store this zfs 'stream in a file' on a ZFS filesystem then you will be protected from bit rot/ flipped bit.
Is that right?
 
sko said:
I.e. one host gathers all the backups/snapshots from multiple systems
Click to expand...
All that sounds wonderful sort of a cascade system but what if you do detect an anomaly with a file.
How do you do 'version control' of a backed up file? Which one to restore from?
 
Phishfry said:
How do you do 'version control' of a backed up file? Which one to restore from?
Click to expand...
If version control is needed, the file would be in version control (i.e. our local gitlab server)
Otherwise we just restore from a "known good" point in time from the snapshots. 99% of the time it's only "the file is gone" (or very seldomly a truthful "I borked/deleted the file"); I haven't had a single case of "the file looks weird/is broken" yet - at least on those fileshares that reside on ZFS. I don't care what happens on NTFS on some client - all users know that they should store anything remotely important on the fileserver. Some needed to learn that the hard way, but they all got there eventually.
Most of our users consider a fileshare something close to black magic and if something isn't working "our internet is broken". So no, we don't need version control for them...
 
Back to the OP's question you could literally name this zfs stream file freebsd.img if you wanted.
You just can't use dd to restore it. You must use zfs-receive.

So a ZFS send stream at rest should be treated with great care? More so than regular files?
Only use the highest quality media for physically transferring?
 
Phishfry said:
So a ZFS send stream at rest should be treated with great care? More so than regular files?
Only use the highest quality media for physically transferring?
Click to expand...
I think it is the other way around. After import a scrub should inform about inconsistent data. From my understanding it is close to transfer regular files plus their checksum. If something is corrupted one can copy the files again. It is of course necessary that the original files are still there to be copied, the same applies for a snapshot to be send again. And that does not mean to use poor media for the transfer by intention :-).
 
chrbr said:
I think it is the other way around. After import a scrub should inform about inconsistent data.
Click to expand...
from my experience any hiccup in a send|recv stream leads to the process failing and completely reverting the whole recv job. The recv is atomic: either it succeeds, or everything is rolled back to have a consistent state.
If this was caused by e.g. broken metadata on the sending pool, you can re-send all 'good' snapshots and avoid sending the broken one, but that's not possible with a "send-stream at rest" that has been corrupted - it *must* be fully receivable or it is useless.
True - you could just split up your send into multiple small ones (e.g. one for every dataset instead of the whole pool at once) and save them to distinct files, but I don't see any reason why one should consider this tedious procedure a viable solution for whatever problem one tries to solve with those send-stream files...
 
You must log in or register to reply here.
Back
Top