ZFS Files corrupted with ZFS and SAMBA

R

Raynigamy

Hello there,

First of all, I am new to FreeBSD, I have so far only some rather limited experience with Linux though (mostly Debian).

I have a FreeBSD (version 8.1-RELEASE-p11) server running as a Samba and LDAP server.
Just for your information, I am the one who is responsible for managing it (among several other FreeBSD servers) since I replaced the sysadmin who just left the company. He didn't really have the time to explain me how the servers were configured.

Today in the morning, a user complained that he couldn't save a file on the Samba share because he got an error message saying it was full. So I connected to the server using SSH and rapidly figured out that the ZFS quota was reached for this specific share. I was able to increase its quota size which solved the aforementioned issue.

I left my terminal open in the background because I used the find command to search for some files. When I wanted to get the terminal back, I noticed that the session had time out. So I retried to connect, but to no avail. Pings to the IP address of the server also failed. Then I headed to the server room, connected a monitor to the relevant server (physical - no vm or jail) and I saw an error mentioning a kernel panic (something along those lines) and that the system was supposed to automatically reboot in like 10 or 15 sec. After a few minutes, no progression, so I decided to hard reset the server.

Then it rebooted, from what I saw there was nothing wrong or suspicious during the boot process. I logged in locally as root just to make sure nothing obvious was broken, and finally exited the root session.

After some time, the same user told me that he wasn't able to open an .ods (Open Office Calc) file from the samba share after getting a message asking to choose various settings in order to import the file! Rapidly, it was clear that the file was corrupted, opening it in a text editor just displayed loads of repeated "NULL" strings, and that was it. Files on the share were not backed up, otherwise I wouldn't be here :).

Soon after, the person told me that there were other files like this (maybe 10 or 12, I don't know exactly) that were basically left unreadable. Their respective size was not 0 bits, and I think it was actually what it should have been before they become corrupted. Apparently, those were only - and all - the files that have been edited this morning, before all the quota was used up and before I had to hard-reset the server.

Anyway, after attempting different things without success (unzip or extract the .ods files, looking for files starting with the " ~ " character in the local shared directory and so on), I was quite desperate.

Then I thought of the following:
- ZFS is a robust file system, with journalling capabilities and some form of redundancy (that is basically all I know about ZFS :p ), so if there is a sudden power loss before the pending modification could be committed to the HDD's, the differences between the data in RAM and the data written on the zpool must be logged somewhere. I assumed (maybe wrongly though) that the ZFS should finish the writing jobs just after the system rebooted, but apparently that wasn't the case.

Is there a command to check data integrity (compare the journal and the actual writes performed so far), force pending writes to make it to the disks or perhaps to rebuild the zpool?

- Samba should use a cache to manage the sharing/access of/to files, right? But I don't know where it should be located, since I didn't notice anything regarding this aspect in the smb.conf file. So, maybe it uses the system's cache folder, like /tmp? I browsed it and I don't think I have seen anything related to the corrupted files. Do you know by default where the cache would be?

Is there any way to (hopefully) recover those important files?

Please don't tell me files should be regularly backed up, I know it, and trust me, if it depended only on me, they would be. The thing is, the business has a tight budget, and IT is (was) not a priority. Hopefully that will change soon, that those files be eventually recovered or not.

Thank you for reading and I really hope you will be able to help me.
 
FreeBSD 8.1 is not supported anymore. Also, ZFS has changed a lot during the past 6 years. I doubt that you can find a resolution to your problem. If I were you I would backup everything and reinstall a newer version.
 
What was the setup? For example, was this a redundant pool (i.e. RAIDZ)?

ZFS doesn't have fsck but did you try to run a zfs scrub to see if it could detect and repair the data in the pool (assuming you had some redundancy).

gkontos is right though, FreeBSD 8.1 is old, whatever happens in this scenario, it would be wise to update to the latest release of FreeBSD. Tons of stability fixes have happened since then.
 
If you don't have snapshots of the filesystems, or backups of the filesystems, then the files are gone for good.

If you have snapshots, then it's fairly easy to copy the file out of the snapshot into the live filesystem. You'd lose any data between when the snapshot was created and the file was corrupted, but you'd at least have the file back.

You may be able to import the pool and roll-back a few transaction groups, but that might lead to missing data in other files. If more than an hour or so has gone by since you imported the pool, though, this is no longer possible (too many new transaction have been written, it only tracks the last 128-ish).

Root cause of your issue was that Samba and "disk full" situation leads to possibly corrupted files, as the data cached on the client can't be written out to the server. What caused the irrepairable corruption, though, was you running the find process that ran the server out of memory and causing the kernel panic ... which corrupted all the open files and lost all data in open files.

ZFS doesn't "journal" writes. It logs sync writes to the ZIL and then asyncronously writes that data out to the pool, along with all other writes. If (and only if) the system crashes between the time the data is written to the ZIL and the transaction group is flushed to the disk, then the ZIL is read at boot to write out missing data. However, "normal" data sitting in a transaction group waiting to be written to disk is gone.

Moral of the story: at the very least, you need a cron job that creates a snapshot of all filesystems every night. If you aren't doing that ... well ... prepare for more days like this one. :) Even better, get an external USB harddrive, and configure it to zfs send/recv snapshots from the main pool, and use that as your backups. Best setup, though, is to get a proper backups server installed and running, and use that.
 
Thank you all for you replies.

Here is the result of the zpool status command:

Code: 
/root# zpool status                                                                   
  pool: tank
state: ONLINE
scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            da1     ONLINE       0     0     0
            da2     ONLINE       0     0     0

errors: No known data errors

So I guess it's RAID 1 version of ZFS, correct?

Then I ran the zpool scrub command as advised and then re-run the zpool status -v tank to get more detailled informations, here is the result:

Code: 
/root# zpool status -v tank                                                             [root@atlas]
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: scrub in progress for 0h9m, 2.59% done, 5h44m to go
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            da1     ONLINE       0     0     1  4K repaired
            da2     ONLINE       0     0     0

errors: No known data errors
/root#

How does that look to you, good or rather bad ? What will the next steps be?
Will the zpool scrub command automatically try to recover the possible errors it will detect? I have seen that there is the zpool clear but I'm not sure whether it will be required or not, and what exactly it does? Its aim is simply to clear a kind of errors log once the zpool scrub has been performed or it'll do more/something completely different?
Do I need tu run other commands once the zpool scrub is completed?

Thank you for your help.
 
zpool scrub should take care of that problem as it already repaired by comparing data from your 2nd drive.

zpool clear is not required if there is no further error after scrubbing for the 2nd time.
 
So far it looks good. You should let it finish though. Worst case scenario it will find some files with permanent errors. After this is done, get a full backup of your pool (snapshot) to an external drive and use a newer version of FreeBSD.
 
It looks ok now but I would still back up everything and recreate the pool on a FreeBSD 9.3 or 10.1 system. ZFS pools can get into a state where the damage can not be repaired and the pool needs to be recreated from backup, this is from my personal experience.
 
Thank you guys for your replies. I feel somewhat better, but you're right the next step will be backing up all the data. The harder will be to convince the CEO though, since we'll need to acquire new servers for that purpose.

Anyway I ran the command again to get the scrubbing status, below is the result. As you can see it's almost complete:

Code: 
/root# zpool status -v tank                                                         
  pool: tank
state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
scrub: scrub in progress for 5h49m, 93.95% done, 0h22m to go
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            da1     ONLINE       0     0     1  4K repaired
            da2     ONLINE       0     0     0

errors: No known data errors


So I'll need to run the zpool scrub tank command again?
Just after the current process is done, or is it best to wait a bit more time?
What does the "4K repaired" in the checksum mean? I know what a checksum is and what purpose it serves, however how should I interpret this information? Can I know how much data was corrupted and successfully repaired (e.g. in bits)?
Is it possible to get a list of the files that were repaired?

I'll probably post a new thread to get advice on a solid backup plan, it's quite easy to backup a VM, but I find taking care of a whole running physical server is another story.

Your help is much appreciated :)
 
Okay try to run zpool clear after scrub is completed to clear the error and see if that '4K repaired' goes away. Your ZFS is already fixed so you didn't have data loss. If you had real data corruption or error which cannot be fixed then it'll tell you which file is corrupted or unrepairable.
 
It is now finished. Afterwards I ran the zpool clear tank as you've suggested but the message regarding the "4K repaired" didn't go away though:

Code: 
/root# zpool status -v tank                                                             
  pool: tank
 state: ONLINE
 scrub: scrub completed after 6h22m with 0 errors on Wed May 20 18:10:14 2015
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            da1     ONLINE       0     0     0  4K repaired
            da2     ONLINE       0     0     0

errors: No known data errors

Should I now attempt to run the scrubbing a second time?
 
Raynigamy I will be very honest with you here.

The CEO relies on YOU to advise on best practises and data integrity. If you can't convince yourself that you have a problem you will never convince a supervisor.
You are running a very old system that is EOL & EOS and you have no backups. Like you said in your first post, you have limited experience with *nix environments.
If you want to maintain your position, then start reading, asking questions like you do here but most of all, try to take the situation under control. Focus on what needs to be done and document it.
 
OK, I just started the scrubbing again. But unfortunately, the corrupted files still appear to be corrupted :( .
I opened a few (I don't remember all the document names of the complete corrupted file list, and there were many that were not corrupted in the same folders) and I get exactly the same symptoms: only a bunch of repeated "NULL" strings after opening an .ods file in a text editor, and trying to unzip the file just causes an error saying it's not a zipped archive.
Is there any other option I can try besides the scrubbing (when the 2nd one is over and if files are still not repaired)?
 
Definitely very bad if server or hdd fails without a backup. Better make a backup first and talk to your boss about replacing the aging server. Most production servers are usually decommissioned or retired after 3 years and they function as a backup server afterward. 3 years is usually the recommend time to retire the server but some people push it to no more than 5 years which I think it is in your case.

It's easy to make a backup to external storage drive or storage provider. ZFS is NOT a backup even if you have mirror or raidz enabled. Two hdds can fail at the same time, bad drivers, virus, hackers, thief and that's total loss without a backup. Don't depend on ZFS as the sole backup.
 
Raynigamy said:
OK, I just started the scrubbing again. But unfortunately, the corrupted files still appear to be corrupted :( .
Click to expand...

Looks like it was already corrupted before and I'm afraid there's not much you can do without a backup since both drives are identical in mirror mode.

I would suggest you to make a snapshot to external storage drive and try to upgrade to FreeBSD 10.x since ZFS has improved a lot over the years. Maybe that'll fix it but its no guarantee. That way you'll have a complete snapshot backup of your zpool before you go further to try to repair or upgrade to FreeBSD 10.x with USB Live. You will have to export the drive first to take them offline before you import the zpool in FreeBSD 10.1 USB Live or upgrade to FreeBSD 10.1. But first, do the backup because many things can go wrong if you try to import the corrupted zpool or upgrade FreeBSD.
 
I was already convinced that there were problems before this one actually happened. Many (if not all) of their servers are not even covered by their warranty any longer, they have some switches that are now discontinued and virtually impossible to buy new and the list could go on and on.
As I said, if it only depended on me, I'd virtually re-buy everything that already exists and double every single hardware IT equipment. I know that disasters involving the loss of critical data actually happen more that a normal person could imagine, and a non negligible percentage of companies will go bankrupt within 2 years if they do not have any DRP ((if you have exact and up to date numbers about that, please give me a link).
I know that FreeBSD 8.1 is no longer receiving updates or bug fixes, and I should upgrade, or probably better, re-install all the services that were running on this server on a brand new one with the latest release of FreeBSD(10.1). You're preaching to the converted :)

The thing is, it's easier said than done. I'm alone to manage all the IT infrastructure, and as I said before, I'm new to FreeBSD and I don't even have all the different IP addresses of servers, jails or network equipment. Nor do I know how most services (emails, DB, SVN repo, bugtracker, etc. ...) are configured. So am I willing to do it? A part of me says of course I should, but another is thinking: "what will happen should the migration go wrong? Will everything work as before" (well, when it was working at least :) "and no data whatsoever be lost)" So maybe it's better I wait a bit to get to know FreeBSD and the infrastructure better.
 
Raynigamy said:
The thing is, it's easier said than done. I'm alone to manage all the IT infrastructure, and as I said before, I'm new to FreeBSD and I don't even have all the different IP addresses of servers, jails or network equipment. Nor do I know how most services (emails, DB, SVN repo, bugtracker, etc. ...) are configured. So am I willing to do it? A part of me says of course I should, but another is thinking: "what will happen should the migration go wrong? Will everything work as before" (well, when it was working at least :) "and no data whatsoever be lost)" So maybe it's better I wait a bit to get to know FreeBSD and the infrastructure better.
Click to expand...

I know being an owner and administrator of VPS business and I've learned that being a good administrator is to document everything which someone before you didn't do a good job. They should not have waited 5 years to upgrade to the latest FreeBSD since ZFS was new in FreeBSD 8.x.

I understand but your priority is the problematic server that's about to fail so you need to make a backup by making a daily snapshot to an external storage drive until your boss and you can come up with a plan to replace the aging server. Total data loss is unacceptable for any businesses without a data backup. Do you have other retired server or desktop computer that you can use to see if FreeBSD 10.x can try to repair the error? It's worth a shot until your boss and you come up with a plan. It's not your fault that this happened and it was the former administrator's fault and bad plannings too.
 
Raynigamy said:
The thing is, it's easier said than done. I'm alone to manage all the IT infrastructure, and as I said before, I'm new to FreeBSD and I don't even have all the different IP addresses of servers, jails or network equipment. Nor do I know how most services (emails, DB, SVN repo, bugtracker, etc. ...) are configured. So am I willing to do it? A part of me says of course I should, but another is thinking: "what will happen should the migration go wrong? Will everything work as before" (well, when it was working at least :) "and no data whatsoever be lost)" So maybe it's better I wait a bit to get to know FreeBSD and the infrastructure better.
Click to expand...

No, don't just wait till all hell breaks. They will blame you after. Like I told you before, document everything. Explore your network and discover the weaknesses. People here are willing to help you but you have to be more proactive. Of course, you will need to learn a bit more about FreeBSD if you have many servers there. But set your priorities. First and foremost, make sure that you have the ability to backup important data. Start from there.
 
Thanks everybody for your support, honestly I didn't think I'd get so many replies this fast, that's a good surprise :) .
I already tried to document as much as I can, for instance when I was able to fix an error (or at least find a workaround for it, even if that's not perfect). I used nmap to scan the network, the former admin started to migrate servers from 10.1.8.x/21 to 10.1.11.x/21 but not all servers are on the new network. I keep an updated file to which I add servers that I didn't know existed (for some of them) with their IP address as they show up on the network scan and I also mention services I discovered for each one after connecting with a private RSA key (I check /etc/rc.conf but I guess not all services running at boot are shown in this file).
Another problem I faced, is with scripts. I know that a good sys admin must use scripts whenever possible for repetitive tasks or to automate them. He used quite a lot of them, but in python, and I know know where most of them are located and what they are used for.
After using the find command the server had a kernel panic, not sure whether it's this command that caused it or not, but I imagine it stress the CPU and disks (I/O) quite a lot, especially if I have to search form / . I never really wrote a bash script before (just edited a few quite simple scripts), let alone python.
The default shell is quite different from the bash of debian (I find auto-completion more limited, but maybe that's just how FreeBSD works), and I must say I prefer the latter. But changing the default shell scares me a bit, it might easily have a side effect beyond my skills I think.
The good thing is that the official FreeBSD documentation is really good I must admit, so I attempt to use it when I have a doubt.

Regarding backups, ZFS snpashots seem like a good option to me, but I don't know how they work, is it like taking a snapshot of a running VM, with veeam for instance? A snapshot consists necessarily of an entire zpool? And in case you need to roll back for some reason, do you have to restore the whole snapshot or you can select files/folders inside it manually?
What are the advantages / drawbacks compared to, say, rsync or creating .tar.gz archives?

There is an old, well really old server (PowerEdge 860 from March 2008) with a RAID controller and an expansion enclosure with a SAS expander and 10 disks (500GB if I remember correctly). I was thinking of installing FreeNAS (would be easier for me to manage, at least as long as I'm a noobie) on the server and using the enclosure for the storage , but when I checked its specs with the S/N, I reconsidered this option. I think most people recommend having 1 GB of ECC RAM per TB of data on a ZFS pool. Well, this server has 1 GB of RAM and a weakish XEON 3040 (from my understanding, ZFS also needs a powerful enough CPU to perform data integrity checks, manage parity operations and the likes since there is no hardware RAID controller to offload this kind of load. So I don't think it's even worth bothering with this, right?
That was the only 'spare' server I have at hand right now.
If I'm able to find a USB disk (I know that not the best way to do backups) how should I perform the backup?
Mount the external HDD and use the cp or dd comand?
 
Raynigamy said:
Regarding backups, ZFS snpashots seem like a good option to me, but I don't know how they work, is it like taking a snapshot of a running VM, with veeam for instance? A snapshot consists necessarily of an entire zpool? And in case you need to roll back for some reason, do you have to restore the whole snapshot or you can select files/folders inside it manually?
What are the advantages / drawbacks compared to, say, rsync or creating .tar.gz archives?
Click to expand...

Snapshot takes an image of zpool. First snapshot will be a large file and next snapshots will be smaller files containing only the changes to zpool since the last snapshot. It helps a lot for transferring snapshots to backup server using zfs send and zfs receive with bzip2 compression to lessen the bandwidth. rsync will only copy files but not zfs datasets. Snapshot is more reliable and its read-only. You can browse inside the snapshot file to copy file, etc. but you won't be able to write anything to it. Creating .tar.gz is should be avoided as it consumes server resources and large file. That's one of the best features about ZFS is snapshot and its very well documented on website.

There is an old, well really old server (PowerEdge 860 from March 2008) with a RAID controller and an expansion enclosure with a SAS expander and 10 disks (500GB if I remember correctly). I was thinking of installing FreeNAS (would be easier for me to manage, at least as long as I'm a noobie) on the server and using the enclosure for the storage , but when I checked its specs with the S/N, I reconsidered this option. I think most people recommend having 1 GB of ECC RAM per TB of data on a ZFS pool. Well, this server has 1 GB of RAM and a weakish XEON 3040 (from my understanding, ZFS also needs a powerful enough CPU to perform data integrity checks, manage parity operations and the likes since there is no hardware RAID controller to offload this kind of load. So I don't think it's even worth bothering with this, right?
Click to expand...

I know some people run FreeNAS with less memory and CPU. Of course it'll be slow but it won't break. At least you can use the old server to restore the snapshot to analyze the problem or use it to learn FreeBSD.

If I'm able to find a USB disk (I know that not the best way to do backups) how should I perform the backup? Mount the external HDD and use the cp or dd comand?
Click to expand...

Don't use cp or dd. Snapshot is better as you can use zfs send file to USB external drive and use zfs receive to restore snapshot from USB external device on a different computer with ZFS.

Check this doc. http://www.googlux.com/zfs-snapshot.html
 
Firstly snapshots are not a backup. They allow you to access previous versions of a dataset, but if your pool goes wrong, your live data and snapshots will all go with it. For backup, snapshots are usually combined with zfs send, which allows you to send a complete copy of the data referenced by a snapshot to a separate ZFS pool.

Snapshots don't really take an "image" of the pool, and the first snapshot does not create a "large file". When you take a snapshot, no space is used at all. All that happens is a tiny marker is set inside ZFS which points to the current version of the filesystem. From that point on as you change data, if that data existed when the snapshot was taken, both the new & old copy are kept. The live file system points to the new version, and the snapshot points to the old version. This means that as you change or remove data on the file system, the size of the snapshot grows. (If you overwrite a 1MB file that existed when the snapshot was taken, ZFS has to keep a copy of the old and new data. Your live filesystem is the same size, but the snapshot is 1MB bigger because it's had to hold onto the old copy of that 1MB file. Overall the pool now uses 1MB of extra space). In normal use, you can usually keep around days/weeks or even months worth of snapshots without actually using that much additional space.

If you want to rollback, you can easily revert the entire filesystem to as it was when the snapshot was taken. You can also browse the snapshot and retrieve single files if needed.
Code: 
# zfs snapshot pool/dataset@snapname
-- Revert entire dataset back to snapshot --
# zfs rollback pool/dataset@snapname
-- Copy a single file back from the snapshot --
# cd /pool/dataset/.zfs/snapshot/snapname
# cp somefile.txt /pool/dataset/

If you keep a couple of weeks worth of daily snapshots it means that if a file becomes corrupted by some application, you can just go into yesterdays snapshot, or last weeks snapshot and hopefully find a good copy.
Remember though it's not a backup. You still need to copy your files somewhere else, either with normal tools (cp/rsync), or by sending snapshots to a second pool.

Regarding the corruption. If you open files and they are corrupt, zpool scrub isn't going to fix it. If it could, ZFS would of fixed the files on the fly while you were looking at them. A scrub is just pro-active - it goes looking for problems across the entire pool that might not be noticed otherwise unless you actively go and open every file. If a scrub comes up with no data errors, then ZFS considers everything on disk to be perfectly correct and your data was corrupt before being written, either by the application itself or possibly by incredibly bad RAM.

For backup, sending snapshots to a second pool is the best method. There's nothing really wrong with using something like rsync, but it will be slower as rsync has to check the timestamp on every file to find changes, whereas zfs send will just send a stream of the raw data that's been updated between yesterdays snapshot and todays. 1GB of RAM is pretty tight but has been done. I'd prefer to put base FreeBSD on it instead of FreeNAS just to completely minimise the services running, and you'll probably have to limit the size of ZFS ARC to ~500MB or less. Some people suggest leaving the system to tune itself is best (and that may be the case on bigger systems) but I've always found, and still do, that on low end systems ZFS will happily push memory use hard enough to strangle the rest of the system.
 
Snapshot itself not a complete backup without using zfs send as it combines both dataset and file data.

This is proper way to backup to external drive:

zfs send tank/zpool@now > /backup/tank-zpool.zsnd

To restore backup from external drive:

zfs receive tank/zpool < /backup/tank-zpool.zsnd
 
Bear in mind that ZFS is 100% stringent about data integrity. If you snapshot a 500GB dataset and send it to a file, you'll end up with a 500GB file. When you try to recover from that file, if any of the file isn't 100% intact ZFS will complain and abort the restore.

Ideally you want to backup to a second pool, either locally
Code: 
zfs send pool/dataset@snapshot | zfs recv backup-pool/dataset
or on a second backup host
Code: 
zfs send pool/dataset@snapshot | ssh backup.host zfs recv backup-pool/dataset
That way, the integrity of the backup is managed by ZFS. You can see via zpool status if the backup is ok, and run regular zpool scrub's to make sure the backup isn't failing. If ZFS does report errors on the backup at any point, you can do something about it - either by fixing it or in the worst case scrapping the backup and re-creating it while you still have the live system working. If you only find out your backup isn't intact when you actually need it, it's too late.

Additionally, once you've sent the first snapshot and got a complete copy of the dataset on the second pool, you'll want to use incremental send, so it only sends the differences from then on
Code: 
zfs send -i yesterdays-snapshot pool/dataset@today | zfs recv backup-pool/dataset
 
Remington said:
Snapshot itself not a complete backup without using zfs send as it combines both dataset and file data.

This is proper way to backup to external drive:

zfs send tank/zpool@now > /backup/tank-zpool.zsnd

To restore backup from external drive:

zfs receive tank/zpool < /backup/tank-zpool.zsnd
Click to expand...

This is not the safest way to use the zfs send streams for backup. If you do that there is no redundancy or error checking of any kind, one flipped bit in the stream can render the whole stream unusable and you won't notice it until you try to restore the stream with zfs receive. If you must use plain files for backup use at least gzip(1) compression so you can test the resulting files for corruption with gzip -t:

zfs send tank/zpool@now | gzip -2 >/backup/tank-zpool.zsnd.gz

This will of course guard only against on-disk corruption on the backup media, it won't help if the data is already corrupted before it gets to gzip(1).
 
You must log in or register to reply here.
Back
Top