rdiff-backup to samba share

S

snerpton

Hi all,

I am trying to back up a FreeBSD 8.2-RELEASE box (target-server.domain.local) to a second FreeBSD 8.2-RELEASE box (backup-server.domain.local) using sysutils/rdiff-backup, where the backup will be stored on a Samba mount. sysutils/rdiff-backup is installed on both machines.

The target to be backed up is accessed as root via ssh, where ssh key authentication has been configured and limited with the following in /root/.ssh/authorized_keys:

Code: 
command="rdiff-backup --server --restrict-read-only /",from="backup-server.domain.local",no-port-forwarding,no-X11-forwarding,no-agent-
forwarding,no-pty ssh-rsa BIG_LONG_KEY rdiff-backup@backup-server.domain.local

The server initiating the backup has the following in its .ssh/config file:

Code: 
Host target-server
  Port my_port_number
  User root

Before applying the ssh restrict command enty I was able to perform passwordless SSH into the machine to be backed up. I am able to test the ssh connection with the command:

Code: 
$ rdiff-backup --test-server target-server::/home/chris
Testing server started by:  ssh -C target-server rdiff-backup --server
Server OK

I am able to perform backups when the destination directory is local to the backup-server, for example:

Code: 
$ rdiff-backup target-server::/home/chris /home/rdiff-backup/chris-test/ ; echo $?
0

However, when I try to backup to a directory which is a Samba share it fails:

Code: 
$ rdiff-backup target-server::/home/chris /mnt/rdiff-backup_storeage/chris-test/; echo $?
Exception '' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()

Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()
AssertionError
1
$ Fatal Error: Lost connection to the remote system

The Samba share is mounted with the following line in fstab:

Code: 
//BACKUP_USER@target-server/RDIFF-BACKUP  /mnt/rdiff-backup_storeage    smbfs  rw      0       0

The Samba mount is mounted with the following permissions:

Code: 
$ ls -ld /mnt/rdiff-backup_storeage
drwxr-xr-x  1 rdiff-backup  wheel  16384 Jan  1  1970 /mnt/rdiff-backup_storeage

$ ls -ld /mnt/rdiff-backup_storeage/chris-test
drwxr-xr-x  1 rdiff-backup  wheel  16384 Jun  1 11:39 /mnt/rdiff-backup_storeage/chris-test

So in summary, I am able to backup when the destination directory is local, but I am unable to backup when the destination directory is a Samba mounted share.

Any help or pointers would be very much appreciated.

Cheers,
Chris


PS I am able to write to /mnt/rdiff-backup_storeage. I've tested this by touching some files etc.
 
snerpton said:
Code: 
File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()
Click to expand...

I'm just guessing, but it looks like rdiff-backup is unhappy with the fact, that your SAMBA share is case-insensitive filesystem. Try adding --override-chars-to-quote option to rdiff-backup.
 
Many thanks for the reply.

I tried your suggestion of adding --override-chars-to-quote, and despite what the man page suggests, it seems it requires a parameter. As per the suggestion found here, I tried using --override-chars-to-quote '-"*/:<>?\\|;' and --override-chars-to-quote 'A-Z-"*/:<>?\\|;' without success. I've shown the output below:

Code: 
$ rdiff-backup --override-chars-to-quote '-"*/:<>?\\|;' target-server::/home/chris /mnt/rdiff-backup_storeage/chris-test/; echo $?
Found interrupted initial backup. Removing...
Exception '' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()

Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()
AssertionError
1
$ Fatal Error: Lost connection to the remote system

Code: 
$ rdiff-backup --override-chars-to-quote 'A-Z-"*/:<>?\\|;' target-server::/home/chris /mnt/rdiff-backup_storeage/chris-test/; echo $?
Found interrupted initial backup. Removing...
Exception '' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()

Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()
AssertionError
1
$ Fatal Error: Lost connection to the remote system

Many thanks,
Chris
 
It's probably because SMB/CIFS file shares don't understand unix ownership and permissions. The script probably tries to set those and fails.
 
Further information... I have ran the command with extra verbosity in case that is instructive.

Code: 
rdiff-backup -v7 --override-chars-to-quote 'A-Z-"*/:<>?\\|;' target-server::/home/chris /mnt/rdiff-backup_storeage/chris-test/
Found interrupted initial backup. Removing...
Unable to import module xattr.
Extended attributes not supported on filesystem at /home/chris
Unable to import module posix1e from pylibacl package.
POSIX ACLs not supported on filesystem at /home/chris
Unable to import win32security module. Windows ACLs
not supported by filesystem at /home/chris
escape_dos_devices not required by filesystem at /home/chris
Exception '' raised of class '<type 'exceptions.AssertionError'>':
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()

Using rdiff-backup version 1.2.8
Executing ssh -C target-server rdiff-backup --server
Registering connection 1
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Windows access control lists                 Off
  Case sensitivity                             On
  Escape DOS devices                           Off
  Escape trailing spaces                       Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Making directory /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24
Touching /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/5-_ a.snapshot.gz
Deleting /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/5-_ a.snapshot.gz
Touching /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/uni\xe1\x84\x89
Deleting /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/uni\xe1\x84\x89
Touching /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/:\"
Touching /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/A
Deleting /mnt/rdiff-backup_storeage/chris-test/rdiff-backup-data/rdiff-backup.tmp.24/a
Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 334, in Backup
    rpout.conn.fs_abilities.backup_set_globals(rpin, force)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 922, in backup_set_globals
    dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 156, in init_readwrite
    self.set_case_sensitive_readwrite(subdir)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/fs_abilities.py", line 304, in set_case_sensitive_readwrite
    assert not upper_a.lstat()
AssertionError
Exception 'Truncated header string (problem probably originated remotely)' raised of class '<class 'rdiff_backup.connection.ConnectionReadError'>':
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/Main.py", line 278, in take_action
    connection.PipeConnection(sys.stdin, sys.stdout).Server()
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/connection.py", line 355, in Server
    self.get_response(-1)
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/connection.py", line 315, in get_response
    try: req_num, object = self._get()
  File "/usr/local/lib/python2.7/site-packages/rdiff_backup/connection.py", line 233, in _get
    raise ConnectionReadError("Truncated header string (problem "

Fatal Error: Lost connection to the remote system

Cheers,
Chris
 
As I expected, it trips on the differences in filesystem capabilities. It simply can't cope with the fact the source and destination filesystems are different.
 
Hi SirDice,

Many thanks for the reply.

One of the reasons I am experimenting with sysutils/rdiff-backup is that I believe it is possible to backup to Samba/CIFS shares -- see here. I presumed by this that file permissions/ownership were taken care of. I understood that the file deltas along with permission/ownership etc. were stored in separate files? Is this not the case?

Incidentally, the primary reason for backing up to a Samba share is that our FreeBSD/Linux backups may then be incorporated into our existing offsite backup plan. If there is a tool you believe is more appropriate for this task I'm open to suggestions. I've tried Amanda, but it appears it relies on a symbolic link in the storage location to point to the most recent virtual tape, and it seems NTFS doesn't support symbolic links (I won't forget that in a hurrry).

Cheers,
Chris
 
So if I were to mount the target server as a samba share and back it up to a second samba share it should work?

Cheers,
Chris
 
snerpton said:
One of the reasons I am experimenting with sysutils/rdiff-backup is that I believe it is possible to backup to Samba/CIFS shares -- see here.
Click to expand...
If you read it carefully you'll see they're suggesting to use SSH. The way you currently have it set up is actually more or less a local copy, not a remote one. The filesystem might be on a remote server but to the system (and application) it's local.
 
Many thanks for the replies.

So if I were to mount the target server as a samba share and back it up to a second samba share it should work?
Click to expand...

I answered this one for myself, and it seems it does not work.

If you read it carefully you'll see they're suggesting to use SSH. The way you currently have it set up is actually more or less a local copy, not a remote one. The filesystem might be on a remote server but to the system (and application) it's local.
Click to expand...

Yes I can see that now, I think I got blinded by it saying it was possible to backup to a Samba mount. It does seem as though people have had a degree of success with this, but it also seems some people have done so via having a disk image (formatted appropriately) on the samba mount that they back up to.

So this looks like this method of backing up is consigned to the scrap heap. I wonder if anyone can recommend an alternative, but I'll start a new thread for that.

Once again, many thanks for your replies SirDice and pbd.

Cheers,
Chris
 
You must log in or register to reply here.
Back
Top