Solved zfs rollback

fred974

fred974

Hi,

I made a mistake in a jail and I tried to restore from previous snapshot as we have incremental snapshot setup...

I did a rolback
sudo zfs rollback -r
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-14
instead of
sudo zfs rollback -r
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2017-01-14

now when i try sudo zfs rollback -r
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2017-01-14
i get
Code: 
cannot open
'zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2017-01-14':
dataset does not exist

We send all our snapshot to a backup server every nigth.. So I know that I can still save the day but I have no idea how to get the snapshot from our backup server back to the production server..
Could anyone please help.. Also do I need to send all the jail (iocage) data in a recurvise manner?

Thank you in advance for your help
 
getopt said:
Post the command that sent the snapshot to your backup server.
Click to expand...
Code: 
#!/bin/sh
export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:
pool="zroot"
destination="zroot/trinity"
host="62.30.xxx.xxx"

if [ -f /tmp/backupscript.lock ]; then
        logger -p local5.notice "Backup did not complete yesterday FAILED"
        echo  "Backup did not complete yesterday FAILED" | /usr/bin/mail -s "Backup Report" root
        exit 1
else
        touch /tmp/backupscript.lock
fi

today=`date +"$type-%Y-%m-%d"`
yesterday=`date -v -1d +"$type-%Y-%m-%d"`
day=`date -v -30d +"$type-%Y-%m-%d"`

# create today snapshot
snapshot_today="$pool@$today"

# look for a snapshot with this name
if zfs list -H -o name -t snapshot | sort | grep "$snapshot_today$" > /dev/null; then
        logger -p local5.notice "snapshot, $snapshot_today, already exists skipping"
else
        logger -p local5.notice "Taking todays snapshot, $snapshot_today"
        zfs snapshot -r $snapshot_today
fi

# look for yesterday snapshot
snapshot_yesterday="$pool@$yesterday"
if zfs list -H -o name -t snapshot | sort | grep "$snapshot_yesterday$" > /dev/null; then

        if zfs send -R -i $snapshot_yesterday $snapshot_today | mbuffer -q -v 0 -s 128k -m 1G | ssh root@$host "mbuffer -s 128k -m 1G | zfs receive -Fdu $destination" > 0; then
                logger -p local5.notice "Backup OK"
                echo  "Backup OK" | /usr/bin/mail -s "Backup Report" root
        else
                logger -p local5.error "Backup FAILED"
                echo  "Backup FAILED" | /usr/bin/mail -s "Backup Report" root
                exit 1
        fi
        rm /tmp/backupscript.lock
        zfs destroy -r $pool@$day
        exit 0
else
        logger -p local5.error "missing yesterday snapshot Backup FAILED"
        exit 1
fi
I tried this but I could find the snapshot i needed
zfs send -R zroot/trinity/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2017-01-13 | mbuffer -q -v 0 -s 128k -m 1G | ssh root@91.203.xxx.xxx "mbuffer -s 128k -m 1G | zfs receive -Fduv zroot"
So i tried
Code: 
zfs send -R zroot/trinity/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-07 | mbuffer -q -v 0 -s 128k -m 1G | ssh root@91.203.xxx.xxx "mbuffer -s 128k -m 1G | zfs receive -Fduv zroot"
in @  722 KiB/s, out @  0.0 KiB/s,  0.0 KiB total, buffer   0% fullfound clone origin zroot/iocage/jails/d4132390-f7f8-11e5-912b-f04da20055fe/root@cc3c85f7-1853-11e6-b209-f04da20055fe
cannot receive new filesystem stream: destination has snapshots (eg. zroot/trinity/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-12-16)
must destroy them to overwrite it
mbuffer: error: outputThread: error writing to <stdout> at offset 0x10000: Broken pipe

summary:  0.0 KiByte in  5.5sec - average of  0.0 KiB/s
Ay advise please?
 
zfs list
Code: 
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe                                                   1.02G  15.0G   288K  /iocage/jails/cc3c85f7
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-07                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-12                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-13                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-14                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-15                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-16                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-11-17                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@ioc-update_2016-12-07_03:26:20                        0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-08                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-14                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-15                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-16                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-17                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-18                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-19                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-20                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-21                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-22                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-23                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-24                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-25                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-26                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-27                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-28                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-29                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-30                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2016-12-31                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-01                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-02                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-03                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-04                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-05                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-06                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-07                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-08                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-09                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-10                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-11                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-12                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-13                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-14                                           0      -   288K  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root                                              1.02G  15.0G  5.95G  /iocage/jails/cc3c85f7/root
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-07                                  73.9M      -  5.98G  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-12                                  13.2M      -  5.95G  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-13                                  6.91M      -  5.95G  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-14                                  3.25M      -  5.95G  -
I need the rest
Code: 
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-07                                  73.9M      -  5.98G  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-12                                  13.2M      -  5.95G  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-13                                  6.91M      -  5.95G  -
zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2016-11-14                                  3.25M      -  5.95G  -
[missing days]
 
See zfs(8) Example 12. Read the text before the commands. Which means that you should destroy first the dataset that you are going to replace by sending from backup.
 
getopt
Code: 
zfs destroy -fr zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe
zfs create zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe
zfs send -R zroot/trinity/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-13 | mbuffer -q -v 0 -s 128k -m 1G | ssh root@91.203.xx.xxx "mbuffer -s 128k -m 1G | zfs receive -Fduv zroot"
will that be the correct prodecure?
 
Are you sure that you want 2017-01-13? Above you talked about 2017-01-14.
There is "root" missing: cc3c85f7-1853-11e6-b209-f04da20055fe@-2017-01-13

Yes that should make it.
 
SOLUTION

After many hours of try and error, the following command sent all my data back to the server:)
zfs send -R zroot/trinity/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe/root@-2017-01-13 | mbuffer -q -v 0 -s 128k -m 1G | ssh root@91.203.xx.xxx "mbuffer -s 128k -m 1G | zfs receive -Feuv zroot/iocage/jails/cc3c85f7-1853-11e6-b209-f04da20055fe"
Hope this help someone else

getopt this would have not been possible without all you help.. Thank you
 
Hi guys,

Even tough I have managed to get the snapshot back on the server when I started iocage, the jail didn't come back...
Could anyone please provide any advise?
 
A little O/T, but strongly consider not using -F in your automated backup script (on the destination recv command.) You shouldn't need -F for backups; if you find you are running into 'destination changed' errors and that's why you want -F, try setting your backups to be readonly. (You can still zfs recv into a readonly filesystem.)

The reason (IMHO) to avoid -F is that it removes locally deleted (at the source) snapshots from the destination. If you are trying to keep two copies of the data AND snapshots in perfect sync, or migrating a system, it can make sense. For a backup setup, you can easily shoot yourself in the foot. You will need to manage snapshot expiration separately on the backup, but that can be a benefit. (Different retention schedules on the backup vs. live systems.)

Also, you could have likely gotten by with an incremental send from the backup rather than the full send, but if this isn't a lot of data, that's not a big deal. Just something to consider if it happens again.

Sorry. Off soapbox, back on topic.

As for why it's not working, are (user) there any zfs properties that somehow got lost in the process? They look like they should have been backed up and restored by your notes, but just a thought. I can't recall if iocage uses those for metadata.
 
You must log in or register to reply here.
Back
Top