Other Sharing my backup script.

Alain De Vos

Alain De Vos

As we all have lost essential data, or have had failing disks, there is one cure , backups.
I have no knowledge of syncoid or sanoid ....

Here i'm sharing my backup scripts.
It only has only one backup , the latest.
I have had sequential failing of backups using incremental backups ...
My root zfs pool is called SSD

1. Destroying all snapshots
Code: 
zfs list -H -o name -t snapshot | grep "@.*NOW" | xargs -n 1 zfs destroy

2. Creating a snapshot for root zfs pool.
Code: 
zfs list -H -o name -r SSD | grep -v '^SSD/usr/local/poudriere' | xargs -I {} echo {}@NOW | xargs zfs snapshot

3. Creating a snapshot for all non root zfs pools.
Code: 
zpool list -H -o name | grep -v '^SSD$' | xargs -I {} zfs snapshot -r {}@NOW

4. Copying essentials files, like home directory to external usb drive
Code: 
#!/bin/sh
TODAY=$(date "+%Y_%m_%d")
echo "The date is: $TODAY"
rm -fR         /mnt/MYWINZFS/data/BACKUP/2026*
export MYPATH="/mnt/MYWINZFS/data/BACKUP/$TODAY"
echo "Start Cloning"
mkdir $MYPATH
clone -s /etc           $MYPATH/etc
clone -s /usr/local/etc $MYPATH/usr_local_etc
clone -s /root          $MYPATH/root
clone -s /home/x        $MYPATH/home_x
clone -s /var/db        $MYPATH/var_db
clone -s /var/backups   $MYPATH/var_backups
echo "Done Cloning"
cp /etc/src.conf                                     $MYPATH
cp /usr/src/sys/amd64/conf/MYKERNCONF                $MYPATH
cp /usr/doit*                                        $MYPATH
cp /var/db/postgres/data18/postgresql.conf           $MYPATH
cp /var/db/postgres/data18/pg_hba.conf               $MYPATH
cp /MYFREEBSD/jails/a/usr/local/etc/nginx/nginx.conf $MYPATH
pkg prime-list                     >                 $MYPATH/primelist.txt

Feel free what you think of it? Elaborate ? Maybe i forgot something ? Incremental ? Sanoid/Syncoid config ?
 
To me backup starts with organisation, which above all means the question:
Where to keep which files. Or: Which files needed to be backupped, and which ones just to be stored safely.
I stressed my point of view about the topic BU at length in several posts (see bottom.)
Short: It's to keep the backup's footprint as small as possible for the backup routine does not last longer as it needs.
In my case it's 1..20 minutes (depends on the amount of changes done) for my ~14 GB /home/ to my NAS (my LAN is slow.) My desktop workstation has a two NVMe mirror for /, a two SSD mirror for /home/, and a two SSD mirror for /storage/ where I keep all stuff I want to have on my workstation, but neither in / nor /home/ like Downloads, or VM stuff.
I have "outsourced" a lot of storage capacity consuming stuff from my /home/ to other places (mirrors, zfs pools, NAS) and created links ln(1) to them, like VirtualBox VMs -> /path/to/my/vb/, video -> /path/to/my/NAS/videos/ or Downloads -> /path/to/Download. I don't see why to keep TBs of stuff in my /home/ which does not change (videos, music, pictures, PDFs, ebooks, audio books, CD ISO images, ancient stuff from old Windows machines...) or could be downloaded again, and stress my regular BU routine with it. I keep those in other places (ZFS pools) pretty safe from hardware failure (raidz3 pool), plus doing additional redundant BUs to extra drives quarterly. And for the valuable things (a few GB) on my NAS a scipt automatically encrypts (gpg), packs that stuff, and stores additional copies of it via WAN in other locations.

My BU routine is run daily when I shutdown my desktop workstation. For me daily backups are safe enough. It all depends on the certain individual situation, the kind of files, their amount and value, storage capacities, bandwidth, time and frequency.
sh: 
#!/bin/sh

sh /root/shscr/bu_sysconfigs.sh
sh /root/shscr/bu_rootshome.sh
sh /home/[...]/bu_home.sh

exit 0
For
/boot/loader.conf
/etc/
/usr/local/etc/
/var/cron/tabs/
/home/
I'm doing rsync -aq --delete --log-file=/root/bu_logs/bu_home.log --exclude-from=${SrcDir}rsync_excludes.txt ${SrcDir} ${TrgtDir}
via LAN (NFS) to keep 1:1 copies of those on a two HDDs 1 TB mirror in my NAS. The NAS does daily rolling .tar.gz BUs of this mirror's content internally to the raidz3 pool, keeping the last 10 days.
rsync_excludes.txt contains a list of all stuff I don't want/need to backup, like my production directories. For those snapshots are done, and they are under version control, which repository is also on the NAS and backupped daily .
bu_sysconfigs.sh also contains
Code: 
## Do lists of all installed packages
    pkg prime-list > /root/BU-stuff/installed_packages.txt
    pkg leaf > /root/BU-stuff/installed_packages_versions.txt
# when /root/ is backupped, those will be backupped, too
bu_home.sh contains additional lines to cleanup the place before the backup, like
sh: 
rm ${HomeDir}*.core 2>/dev/null
(Which could be done more elegant when first check if any core dumps even exist at all, of course - quick'n'dirty is my middle name. :cool:)

Rolling snapshots can be done like this:
sh: 
#!/bin/sh
# do and keep a number of snapshots of a pool

Z=10 # Amount of snapshots to keep
POOLNAME=home

d=$Z
# delete oldest one
zfs destroy -r ${POOLNAME}@${d}days
# increment all names by 1
while [ $d != 0 ]
do
    b=$(($d - 1 ))
    zfs rename -r ${POOLNAME}@${b}days ${d}days
    d=$(($d - 1 ))
done
# do the recent one
zfs snapshot -r ${POOLNAME}@0days

exit 0
zfs snapshot/rename do not overwrite existing snapshots.
If you wanna use this you first have to create the number of snapshots named correctly manually. Otherwise the loop breaks with the error the snapshot(s) were not found.
(I was capable to do this it without AI.😁)

Backup Plan
About Backups
About Backups
About Backups
About Backups
 
not backups, but I do
sh: 
$ doas crontab -l | fgrep snapshot
*/15    *    *    *    *    S=zroot/ROOT/default@`LC_ALL=C date +\%a`-cron && zfs destroy -r $S && zfs snapshot -r $S
*/15    *    *    *    *    S=zroot/usr/home@`LC_ALL=C date +\%a`-cron     && zfs destroy -r $S && zfs snapshot -r $S
 
Thanks, i will add something like that to fcron.
Something like quarterly & hourly & dayly & weekly & monthly. So i always have 5 and only 5 snapshots.

PS: Indeed snapshots will not help , if like me your SSD suddenly and without warning completely gives up.
 
You must log in or register to reply here.
Back
Top