Other Backing up a device which lots of free space

B

balanga

Does anyone have any suggestions about how to backup/archive a disk which has lots of free space?

The only way I can think of is to use dd & gzip.

Is there is a better alternative?
 
dd can only backup your drive 1:1, so also the empty space is backupped, which, as you already see yourself, is pretty bogus, especially when the drive is pretty empty.
gzip like all other packers work on files, not drives or partitions. Depending on what's on that drive, you can use tar or rsync to create a copy of the filesystem, and then use such a packer to store them as a packed file. With BSD and I suppose also Linux partitions this will work. But if it's a Windows system partition/drive, this will not provide a bootable/running system, when you restore from such an archive.
As I said in another thread of yours, you could use gparted to reduce the partitions sizes - shrink them down to have only very few empty space left, and then copy the whole partitions with it to collect them on another drive(s), or afterwards use dd to create image files of the partition, and save that file where you wanna keep it.
In my Windows days I used it a lot, and I have very good experiences with gparted. Was my #1 tool to resize, copy, move and backup my Windows partitions, since there is no other way to really backup a Windows' system drive/partition than to make a full clone of it.
gparted cannot handle ZFS or other BSD filesystems (look at the list, what it can provide on its homepage.)
And it's Linux only (comes by default with Ubuntu.) But it also comes as a live system. You simply download the image, write (just dd) to and then boot and run from a small flash drive.
There are also others for that job, like clonezilla.
 
Maturin said:
dd can only backup your drive 1:1, so also the empty space is backupped, which, as you already see yourself, is pretty bogus, especially when the drive is pretty empty.
gzip like all other packers work on files, not drives or partitions. Depending on what's on that drive, you can use tar or rsync to create a copy of the filesystem, and then use such a packer to store them as a packed file. With BSD and I suppose also Linux partitions this will work. But if it's a Windows system partition/drive, this will not provide a bootable/running system, when you restore from such an archive.
As I said in another thread of yours, you could use gparted to reduce the partitions sizes - shrink them down to have only very few empty space left, and then copy the whole partitions with it to collect them on another drive(s), or afterwards use dd to create image files of the partition, and save that file where you wanna keep it.
In my Windows days I used it a lot, and I have very good experiences with gparted. Was my #1 tool to resize, copy, move and backup my Windows partitions, since there is no other way to really backup a Windows' system drive/partition than to make a full clone of it.
gparted cannot handle ZFS or other BSD filesystems (look at the list, what it can provide on its homepage.)
And it's Linux only (comes by default with Ubuntu.) But it also comes as a live system. You simply download the image, write (just dd) to and then boot and run from a small flash drive.
There are also others for that job, like clonezilla.
Click to expand...
I have used clonezilla in the past but never managed to script a backup so it's like starting from scratch every time.

One of the biggest problems I have is mounting an NFS share.

Is there a FreeBSD equivalent of Clonezilla or has anyone tried porting it?
 
May be it would be better if you could be a bit more specific on what you are actually wanna to do.

First you wanna archive or backup a whole drive with lots of empty space without saving the empty space. That would make sense on Windows-, maybe other, old, currently not used non-FreeBSD drives/partitions.
Now it's to be automated with a script and done to NFS? 🤔
What are you actually going to do?
What I only can guess by your other threads is, you have a bunch of old drives containing lots of stuff from old computers, which you wanna clean up.

Cleaning up old stuff consist of one time tasks - you do it once, but not regulary:
  1. review what's on a drive: system or files
  2. decide if it shall be kept or being deleted
  3. if it shall be kept, decide where and the form, which depends on how often you need to have access to it:
    • files best being copied into some directory, or tared & packed
    • system partitions/drives are either be kept as they are, shrinked & cloned, or saved as images
Depending on the amount you need to clean up, this can be a bit effort. But if you deal with different drives, containing stuff from different computer systems from different eras, you need to review all yourself what's on it, decide yourself what to keep, how and where. You also need to exchange the drives physically anyway, figure out what partition scheme, partitions and filesystems are on those... I don't see how this really can be automated.

What's done regulary, so being automated, so scripts are useful, are backups: Keep at least one copy of the stuff you currently work with on another physical storage.
But it makes no sense to make copies or clones of full FreeBSD system drives/partitions, especially not as routine backups. The system itself doesn't need to be backupped at all. It's throw away, can be reinstalled anytime. All you keep copies of your system are the config files you edited, only. This would be /etc/rc.config, maybe /etc/fstab, and maybe other files under /etc/, maybe /boot/loader.conf if you did any changes in that one, and maybe /usr/local/share/X11/xorg.conf.d/, if you placed some additional config files for some special things for your X server there. If you have cronjobs configured you may also keep a copy of /var/cron/tabs/ ... - anything you did some editing yourself is only worth to be backupped. And those are a couple of files with about some kB - not even remotely worth to copy the whole drive/partition/pool.
You may also do a pkg prime-list > /path/to/myinstalledpackages.txt to have a list of what you have installed, which can be used to install it all again automatically when you want/need to set up a new, equal system. But anything else by the system ain't worth to be backupped.
What periodically needs to be backupped, so being automated, is your /home/ and any other directory keeping your files in. But those are files, and backupped by some way of copying them.
Neither of those are done by cloning or copying whole drives/partitions: wastes too much time and space and is pretty pointless.
 
gparted is not for filesystems but for PARTITIONS ...
It can change any partition with any filesystem !!!
Offcourse the filesystem takes space within the partition , so be carefull.
 
I mostly use partitionmanager on linux to do stuff with filesystems within a partition.
Then afterwards i use gparted on freebsd to change limits of the partition. [partitionmanager can not handle this]
 
Alain De Vos said:
gparted is not for filesystems but for PARTITIONS ...
Click to expand...
Where did I say otherwise?
Point is the term 'filesystem' ain't not 100% clear, since it can mean two things:
  • a directory tree, like /usr/
  • kind of format a partition is formatted: UFS, NTFS, FAT32, ext4, ZFS...
Both are named filesystem. This may the cause when I was not 100% crystal.
 
balanga said:
I have used clonezilla in the past but never managed to script a backup so it's like starting from scratch every time.
Click to expand...
Partzilla (edit: ops Clonezilla) uses Partclone. It can recognize the structure of some/many file-systems, and it avoids to copy not used sectors of the disk (edited: i.e. it is a file-system aware "dd"). But partclone is not ported to FreeBSD.

Maybe you can use Partzilla/Partclone under Linux, for one-time archive, and then set a regular backup at the file system level using something like
archivers/py-borgbackup.
 
Maturin I have no problem with anything related to files/disks using UFS. I'm simply talking about Windows installations.

I think I will try to use Clonzilla which I can start up easily via PXE and then backup via NFS to my server.

From my understanding such a backup only needs as much space as the data on the disk.
 
Worth a try: store the partition table of the entire disk with fdisk -f and treat the content as a usual directory tree Compress it to 1 file with a tar/gzip pipe. To restore the disk from nothing, the partition table must be written back and the file tree archive decompressed.
But this needs testing. At least, I wouldn't trust on anything at the first attempt. Maybe problems with mountpoints, file flags and other special properties of a system file tree.
 
linux partitionmanager of windows disk, you can move filesystem around in partition.
Later you can change "command line" beginning sector of end sector of partition,
test.png
 
balanga said:
From my understanding such a backup only needs as much space as the data on the disk.
Click to expand...
Well, yeah. But you need to see the difference between a mounted filesystem, and some tool handling partitions/drives. The last one needs to detect and distinguish written blocks from empty, or unused blocks.
Depending on how the filesystem works - what is written where when how - (ntfs is more complicated than FAT, and ZFS for sure one of the most complicated ones.) the growing and particulary the shrinking of a partition ain't no trivial job, if even possible. The partition manager needs to rearrange the needed and used blocks, and only remove the not needed ones from the partition.
While copying a partition 1:1 block by block ain't no magic at all. All you need for that is simply pure good old
dd. And most partition managers do not use much more than that for doing that job.
And of course the filesystem on the partition/drive in progress must not be mounted. Otherwise, in an active filesystem, there are write processes - blocks are changed, which leads a corrupted copy.

Since this is all about Windows partitions, my advice would be:
Pick and use any partition manager you feel comfortable with.
Then shrink the according partition(s) to its smallest possible size. If you still work with those, so there is to be some more data to be expected to get there, let it have a bit more additional space.
This way you have to deal with a lot smaller partition, and way less empty space.
Those can be backupped then easily as full partition clones done as I said e.g. with dd, which you can use within scripts. But there are probably other tools producing 1:1 partition/drive clones, but since I stopped using Windows I don't need to do that anymore, so I don't know any. If you use dd you need to tell dd where to start and stop. You get those values e.g. with FreeBSD's gpart, or the partition manager you use. Otherwise dd will copy the full drive from 0 to max. ( dd if=/dev/ada0p1 ... should work for producing partition's clones. :-/)

And again: Besides to do full 1:1 clones of partitions/drives I don't know a working solution to backup whole Windows system drives/partitions.
Also be aware, when you will reboot Windows with the resized partition, it will grump about disk things changed garbage anything, and needs to run its harddisk checker-repair tool (I forget the names, and I'm too lazy to google that. It's Windows. So I don't care anymore.) Anyway: Let it run!
 
balanga, you didn't specify the type of data on that system and whether you need the entire filesystem byte for byte. Assuming it's just regular "user data" files, I'd simply do a: tar -cvzf /mnt/path/to/backup.tar.gz /path/to/my/data
 
Beastie said:
balanga, you didn't specify the type of data on that system and whether you need the entire filesystem byte for byte. Assuming it's just regular "user data" files, I'd simply do a: tar -cvzf /mnt/path/to/backup.tar.gz /path/to/my/data
Click to expand...
Its a disk with an installation of Windows XP which I'd like to keep a backup copy of in case I need again at some time so I don't have to go through the pain of installing it again.
 
Can't you just make a compressed tar of the 5GB data you want to back up? If you use sshfs to mount a drive on a remote machine... you can write the tar file to the remote drive.
Suppose you mounted your remote drive on ~/remote, as follows
$ sshfs remoteip:backups ~/remote/
Then as beastie suggested
$ tar -cvzf ~/remote/backup.tar.gz /path/to/my/data
... and this will write the tar file to the directory 'backups' in your home directory on the remote machine that you are backing up to.
 
Ha... I should have read your last post about windows XP. Egg on face! Still... if you install cygwin on XP, I wonder if you can do something like that. You should at least be able to make a tar, and maybe use Putty to send it, if there's no ssh client on cygwin, I can't remember now whether it has an ssh client or not; if it has, you should be able to do what I suggested from the cygwin shell.

Of course on windows... you can use rar, and unrar on unix, assuming you're backing up to a freebsd box. It's 'unrar e' to unpack.
 
You must log in or register to reply here.
Back
Top