dump freezes every time I try to run it.

LSDave · #1 August 19th, 2011, 10:52

Hello,

Running FreeBSD 9.0 BETA1 (AMD64) on a Dell PowerEdge SC430.

I am mostly a Windows guy turned FreeBSD newb, but have had success updating the source, building world, customizing, compiling and installing the kernel (to remove debugger stuff and devices that don't apply to my system), installing world, importing ZFS raidz pool from previous FreeBSD system, installing Samba, Xorg, KDE4, and TightVNC. In fact, I just got everything working the way I wanted, was about to do a portupgrade -a, but thought to take a backup while everything was working so great...

So I started to do a dump of the system slices--backing them up one at a time onto my large ZFS raidz. I only got as far as the root fs, using the following command:

Code:

#dump -0aLuf /mnt/ZFSstore/more.paths/full-root /

This first attempt was actually working--I saw corresponding hard drive activity on my raidz hard drives enclosure (i.e., the destination drives). But then I stupidly tried to interrupt the process..hitting Ctrl+C repeatedly to try to get it to stop, and when the process did not terminate and the system seemingly froze, I gave up and hard reset the machine.

On reboot, fsck detected errors in the filesystems, and appeared to correct them. I've also since booted to single user mode and ran fsck on all the system partitions. fsck reported that it fixed the errors.

The problem is that each time since that first forced restart, running the dump command has failed. Absolutely no machine activity follows from the command, and the computer becomes largely unresponsive, though strictly speaking it's not "frozen". For example, my Ctrl+C entries appear on the screen, but have no effect--the process does not terminate and I am not returned to the command line and cursor. If I press Alt+F2 I can indeed get to another console, but upon typing "root" at the login prompt the username very sluggishly appears on character at a time, and then that console becomes largely unresponsive as well--the password prompt never appears.

This unresponsiveness does not end prettily--as each time I am obliged to hard reset the machine again, after which fsck reports and repairs more filesystem errors on boot up. This has gone on 4 or 5 times like this with the same result, and now I come here for your assistance.

Any guidance would be appreciated, and please let me know if you require further details. Thanks in advance.

**SirDice** · #2 August 19th, 2011, 11:25

It sounds like the drive may be up the creek. I've had a similar issue once before, turned out the drive had bad sectors which caused things to hang indefinitely.

**wblock@** · #3 August 19th, 2011, 13:06

-L when dumping a live filesystem makes a snapshot first; see mksnap_ffs(8). That can take a couple of minutes where it seems to be doing nothing, although there will be drive activity and mksnap_ffs will show on top(1). Running anything else will be slow due to head contention. Once the snapshot completes, it'll be back to normal and the dump will start.

LSDave · #4 August 30th, 2011, 02:15

Quote:

Originally Posted by SirDice

It sounds like the drive may be up the creek. I've had a similar issue once before, turned out the drive had bad sectors which caused things to hang indefinitely.

I have progressed without backing up, and no issues or other symptoms suggesting a faulty drive. No complaints for fsck on boot up for example. I really don't think this is the problem, but any suggestions on program I can run to rule it out completely?

Quote:

Originally Posted by wblock

-L when dumping a live filesystem makes a snapshot first; see mksnap_ffs(8). That can take a couple of minutes where it seems to be doing nothing, although there will be drive activity and mksnap_ffs will show on top(1). Running anything else will be slow due to head contention. Once the snapshot completes, it'll be back to normal and the dump will start.

Yes, there was certainly a couple of minutes delay the first time when it worked and I foolishly interrupted it. Since that time however, I have left it for 10, 15, and even 20 minutes and there is ZERO activity. So I don't think it's patience matter, but thanks for this info.

Q1: How can I fix this: Rebuild world? Rebuild kernel? Both?

Q2: After many many attempts installing and frustration, I've really just got FreeBSD right where I want it: customized Kernel, Samba + ZFS NAS server playing nice with my Windows network, KDE working and autoloaded, VNC autoloaded, and VirtualBox running. I would consider a rebuild discussed above at Q1, but I really want a back up first.

I have read that while dump/restore is best, you can also back up your system with tar. Any thoughts or advice on me running the following command to effect a full backup:

Code:

tar -cvpzf  /mnt/path.to.ZFSstore/fullbackup.tar --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/dev /

And restored if (and hopefully not!) necessary with:

Code:

tar -xvpzf /mnt/path.to.ZFSstore/fullbackup.tar -C /

Will these commands do the trick? Have I missed some typical exclusions? Any other concerns?

Thanks for your time and any comments.

**wblock@** · #5 August 30th, 2011, 03:40

Wait--if the disk being backed up is ZFS, don't use dump, which is for UFS. net/rsync is a better candidate than tar(1) if you want to go that way, but there should be something native for ZFS.

LSDave · #6 August 30th, 2011, 04:57

Quote:

Originally Posted by wblock

Wait--if the disk being backed up is ZFS, don't use dump, which is for UFS. net/rsync is a better candidate than tar(1) if you want to go that way, but there should be something native for ZFS.

I was not clear. The ~/, var, tmp, and usr filesystems are not ZFS, but rather the standard FreeBSD filesystem that is the only option readily available during the Boot-DVD's installation process. The destination folder for the backup (and incidentally the storage shared by the Samba Server) is a ZFS raidz pool, which is mounted at /mnt.

Against the background of those clarifications, is using tar as I described above acceptable?

**wblock@** · #7 August 30th, 2011, 05:50

Quote:

Originally Posted by LSDave

Against the background of those clarifications, is using tar as I described above acceptable?

Yes, but on restore it may Do Things with hard links and other filesystem weirdness. Might be options to handle that like with rsync(1) (-H).

Trying dump on a small filesystem like / or /var without -L, and maybe with -C8 or -C16, could help to figure out the problem. It really ought to work, and is kind of concerning.

dalescott · #8 January 19th, 2012, 00:23

I'm also getting a non-responsive system after issuing dump on a relatively fresh install of FreeBSD-9.0-RELEASE. The system is a VirtualBox 4.1.2 vm, with a 20GB GPT system drive and a 20GB MBR backup drive. Since it was created, the ports tree has been updated and apache22, mysql55-server and python/django installed (and a number of minor utilities). Everything seems to work ok, except for dump.

Code:

# mount
/dev/ada0p2 on / (ufs, local, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
/dev/ad1as1d on /backup (ufs, local, soft-updates)
#
# cd /backup
# dump -0aLf 20120118.dump /

there is no output after hitting <enter> and the system becomes generally unresponsive. A command typed into either the VirtualBox server console or an ssh terminal is echoed (e.g., whoami), but that's all. I had started "top" in a seperate ssh terminal before issuing the dump command and it shows mksnap_ffs with 98%-100% WCPU for about 55 minutes, at which point the "top" session stops updating. After 70 minutes I give up and yank the virtual power cord.

I the created a new FBSD-9.0 VM (basic install only from dvd), but also create/fdisk/label/mount a new virtual backup drive. On this one, dump works as expected! I've moved the virtual backup drive between systems, and also repartitioned/relabeled the backup drive on the problem system, with no effect. FWIW, on both VMs (the problem vm and the one where dump works), fdisk reports that the system drive chunks do not start on track boundaries (no idea why, I used auto for the install).

Does any of this make any sense to anyone? Is it possible to correct it somehow? I could spend the time to re-configure the second vm like first, but not knowing what went wrong concerns me (and if it's going to happen again). The overall intent is to get version 2 of my live server working as a vm, then restore the dump from the vm onto the live server.

Thanks in advance for any and all assistance,
Dale

**wblock@** · #9 January 19th, 2012, 01:13

Are the two VMs identical in hardware settings, particularly in controller? dump(8) forks several processes. If the VM was really limited on RAM, it might start swapping and really slow down.

kpa · #10 January 19th, 2012, 01:16

Your problem is not likely related to VirtualBox but to a problem with snapshots on an UFS filesystem when using SU+J setup.

There's a thread on freebsd-current mailing list about the very issue:

http://lists.freebsd.org/pipermail/f...ry/030937.html

dalescott · #11 January 19th, 2012, 18:25

Thanks kpa for your link. The problem was caused by UFS SU+J (with journalling). After reverting the file system to SU only (no +J), dump executed correctly.

For the benefit of others, the thread linked to by kpa is very informative and worth reading, but the short version for me was to boot the livecd from a FreeBSD-9.0-RELEASE (dvd1, but disk1 would work too), then:

Code:

# fsck -f (as recommended by Mr. McKusick)
# tunefs -j disable /dev/ada0p2

and then reboot normally from the system drive and delete /.sujournal (to reclaim space).

Hopefully smarter minds than mine will solve the issue with journalling.

P.S. Thanks also wblock@, the two vm's were in fact identical although having the default 128M memory. Before disabling journalling, I tried increasing memory to 256M, and then to 512M, but without effect.

kpa · #12 February 7th, 2012, 13:57

Take note:

http://lists.freebsd.org/pipermail/f...ry/013646.html

**wblock@** · #13 February 7th, 2012, 16:43

Direct link to the PR: kern/161674

Tomse · #14 October 1st, 2012, 19:07

do anyone but me also have this problem ?

running 9.0 p3 amd64

according to the PR the bug should be fixed by now.

swa · #15 October 1st, 2012, 21:32

Last week I had freezes using Amanda dump. Only on one partition (643G) with 24G in use. I couldn't locate the problem, in stead I thought it would be the combination of big partition+active/encrypted jails causing problems. I managed to reproduce it and system just hangs, finally locking myself out of ssh and couldn't do anything but hard reboot. Dump on the smaller / /usr and /var are fine. Using 8.3-RELEASE-p3 with soft-updates but without Journaling.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
"Shared object has no run-time symbol table" bug ?	zero	FreeBSD Development	0	December 12th, 2010 20:17
xorg 7.5, usb keyboard freezes gui for a short time	hedgehog	X.Org	4	August 11th, 2010 21:05
Approx dump restore time	jaymax	General	3	February 28th, 2010 05:06
Xterm run only one time :S	rokpa92	KDE	0	February 6th, 2010 17:09
FreeBSD Run-Time Link-Editor Local r00t Zeroday (with exploit)	kegf	General	4	December 1st, 2009 09:03

The Following User Says Thank You to wblock@ For This Useful Post:
LSDave (January 8th, 2013)

The Following User Says Thank You to wblock@ For This Useful Post:
dalescott (January 19th, 2012)

The Following 2 Users Say Thank You to kpa For This Useful Post:
BostonDriver (January 2nd, 2013), dalescott (January 19th, 2012)

The Following User Says Thank You to dalescott For This Useful Post:
BostonDriver (January 2nd, 2013)

The Following User Says Thank You to kpa For This Useful Post:
wblock@ (February 7th, 2012)