Script - Extract Data & Watch for Errors

I recently had a problem when a drive that was part of a ZFS raidz1 failed and another had previously undetected bad sectors on it; ZFS isn't smart enough to continue on and kept restarting the resilvering process. Using the following script, I was able to extract everything off the bad RAID onto a large USB hard drive. Then I replaced the drives and rebuilt the data in the pool – believe it or not, I ended up losing nothing. :)

Basically, this script will copy files from a given directory to another, pausing slightly between each, and watch the message log for read errors. Figured I'd share in the hope it's useful to someone else.

One word of warning: This script does have some trouble with filenames that have special characters in them, but I wasn't able to determine a pattern or solution; my main goal was extracting data. I don't put special characters in the name when I save something; the directories in question were other people's data backed up onto my box. You can see the 'fix' I tried near the end, but it ended up making things worse for everything but apostrophes; ended up just tar-ing the directories containing this type of file and copying them – thankfully, there were no bad sectors in them.

Another word of warning: Your /var/log/messages file (or whatever you set that variable to) will be destroyed by this script. The usage help also contains this information.

Code:
#!/usr/bin/perl

###############################################################################
#                                                                             #
# Ruler's Common-Sense License:                                               #
#                                                                             #
#   You may use this script however you want to, but I don't warrant it to    #
#   be good for anything in particular, though it happens to work well for    #
#   me.  (I hate putting BS like this in, but I hate more being sued.)  If    #
#   you use this script, you must keep this license and credit to me in it    #
#   in the form of this block, even if you modify it for your own use.  If    #
#   you want to send me money for it, fantastic!  Send me a private message   #
#   on the freebsd.org forums and I'll give you my PayPal address. :-)  Even  #
#   just a simple 'thank you' would be nice.  If not, that's fine too.  All   #
#   hate mail/spam is sent directly to /dev/null                              #
#                                                       - Jim, AKA Ruler2112  #
#                                                                             #
###############################################################################
#                                                                             #
# History:                                                                    #
#                                                                             #
#   2014-02-28 by Ruler2112       Wrote initial version.                      #
#   2014-03-28 by Ruler2112       Released on freebsd.org forums.             #
#                                                                             #
###############################################################################

my $SourceDir = shift;
my $DestDir = shift;
my $ConsoleLog = shift;
my $LogFile = shift;
my ($FromDir, $ToDir, $ExtraRetryCmd);

my $LOC_CAT = "/bin/cat";
my $LOC_CP = "/bin/cp";
my $LOC_ECHO = "/bin/echo";
my $LOC_FSYNC = "/usr/bin/fsync";
my $LOC_LS = "/bin/ls";
my $LOC_MKDIR = "/bin/mkdir";
my $LOC_PING = "/sbin/ping";
my $LOC_RM = "/bin/rm";

# Command executed before retrying
$ExtraRetryCmd = "/sbin/zpool scrub -s tank";

###############################################################################
#   ------------- This is the start of the actual script -----------------    #
#     Do not change below this line unless you know what you're doing!        #
###############################################################################

if(CheckParams() > 0)
  {
  exit;
  }
$FromDir = EscapeName($SourceDir);
$ToDir = EscapeName($DestDir);
`$LOC_ECHO -n "" > \"$ConsoleLog\"`;
open(OUTLOG, ">$LogFile");
CopyDir($FromDir, $ToDir);
close(OUTLOG);
exit;

sub CheckParams()
  {
  my $retval = 0;
  if($SourceDir eq "")
    {
    print "The source Directory must be specified.\n";
    $retval = PrintUsage();
    }
  elsif($DestDir eq "")
    {
    print "The destination directory must be specified.\n";
    $retval = PrintUsage();
    }
  elsif($ConsoleLog eq "")
    {
    print "The system console log file must be specified.\n";
    $retval = PrintUsage();
    }
  elsif($LogFile eq "")
    {
    print "The destination log file must be specified.\n";
    $retval = PrintUsage();
    }
  elsif(! -e $SourceDir)
    {
    print "The source directory must exist!\n";
    $retval = PrintUsage();
    }
  elsif(-e $LogFile)
    {
    print "For safety reasons, the destination log file cannot exist.\n";
    $retval = PrintUsage();
    }
  return($retval);
  }

sub PrintUsage()
  {
  print "This script copies files recursively from a source directory to a destination directory.\n";
  print "It also monitors the system log for read errors.\n";
  print "A log file is created for each file copied with any errors found.\n";
  print "!WARNING!  The console log file will be deleted by this script! !WARNING!\n";
  print "Hint - Disabling cron jobs when running this script is not a bad thing.\n\n";
  print "Usage:\n";
  print "  cpfiles.pl \"Source Directory\" \"Destination Directory\" \"Console Log File\" \"Destination Log File\"\n";
  return(1);
  }

sub CopyDir()
  {
  my $sdir = shift;
  my $tdir = shift;
  my ($dataline, $perlline, $cmdout);
  open($cmdout, "$LOC_LS '$sdir' |");
  while($perlline = <$cmdout>)
    {
    chomp($perlline);
    $dataline = EscapeName($perlline);
    if(-d "$sdir/$perlline")
      {
      `$LOC_MKDIR '$tdir/$perlline'`;
      CopyDir("$sdir/$perlline", "$tdir/$perlline");
      }
    else
      {
      ClearLog();
      `$LOC_CP -p '$sdir/$dataline' '$tdir'`;
      `$LOC_FSYNC '$tdir/$dataline'`;
      `$LOC_PING -c 2 -i 0.25 127.0.0.1`;
      print OUTLOG "File- $sdir/$dataline";
      if(CheckSystemLog("$dataline") > 0)
        {
        print OUTLOG "Deleting file with bad source sectors- $tdir/$dataline\n";
        `$LOC_RM -f '$tdir/$dataline'`;
        if($ExtraRetryCmd ne "")
          {
          `$ExtraRetryCmd`;
          }
        sleep(5);
        print "Retrying $sdir/$dataline...\n";
        ClearLog();
        `$LOC_CP -p '$sdir/$dataline' '$tdir'`;
        `$LOC_FSYNC '$tdir/$dataline'`;
        `$LOC_PING -c 2 -i 0.25 127.0.0.1`;
        print OUTLOG "File- $sdir/$dataline";
        if(CheckSystemLog("$dataline") > 0)
          {
          print OUTLOG "Deleting file with bad source sectors- $tdir/$dataline\n";
          `$LOC_RM -f '$tdir/$dataline'`;
          print OUTLOG "Fatal error for $sdir/$dataline\n";
          }
        }
      }
    }
  close($cmdout);
  return;
  }

sub CheckSystemLog()
  {
  my $filename = shift;
  my($cmdout, $loginfo, $retval);
  $retval = 0;
  open($cmdout, "$LOC_CAT \"$ConsoleLog\" |");
  while($loginfo = <$cmdout>)
    {
    chomp($loginfo);
    if(index($loginfo, "READ_DMA48") > -1)
      {
      print "$filename - $loginfo\n";
      print OUTLOG " - Detected Drive Error! (READ_DMA48)";
      ClearLog();
      $retval = 1;
      }
    elsif(index($loginfo, "FAILURE") > -1)
      {
      print "$filename - $loginfo\n";
      print OUTLOG " - Detected Drive Error! (FAILURE)";
      ClearLog();
      $retval = 1;
      }
    }
  close($cmdout);
  print OUTLOG "\n";
  return($retval);
  }

sub ClearLog()
  {
  `$LOC_ECHO -n "" > \"$ConsoleLog\"`;
  return;
  }

sub EscapeName()
  {
  my $name = shift;
  $name =~ s/\'/\'\\\'\'/g;
# For whatever reason, these break more than they fix...
#  $name =~ s/\!/\'\\\!\'/g;
#  $name =~ s/\@/\'\\\@\'/g;
#  $name =~ s/\#/\'\\\#\'/g;
#  $name =~ s/\$/\'\\\$\'/g;
#  $name =~ s/\%/\'\\\%\'/g;
#  $name =~ s/\^/\'\\\^\'/g;
#  $name =~ s/\&/\'\\\&\'/g;
#  $name =~ s/\*/\'\\\*\'/g;
#  $name =~ s/\(/\'\\\(\'/g;
#  $name =~ s/\)/\'\\\)\'/g;
#  $name =~ s/\</\'\\\<\'/g;
#  $name =~ s/\>/\'\\\>\'/g;
#  $name =~ s/\:/\'\\\:\'/g;
#  $name =~ s/\;/\'\\\;\'/g;
#  $name =~ s/\`/\'\\\`\'/g;
  return($name);
  }
 
Back
Top