Solved HDD crash - trying to recover

W

walterbyrd

FreeBSD 15.0 - ZFS file system.
I left my system for about ten minutes, came back and it was not working. I did a hard restart. Could not log in.
I booted to single user mode. My entire home directory seems to be empty.
I tried fsck - no luck.
I keep getting "Inappropriate file type or format"
I also got messages about could not find superblock, or alternate superblock.
My system directories seem to be there. The files seem to be there.
a zfs list shows my home direct with the subdirectories in it. But an ls in the home directory show the directory as being empty.
I found a /mnt/home directory, which had my subdirectories in it, but the subdirectories are empty.
I have some backups, but not too recent.
The hard is a 1TB Seagate. Somewhere between 5 and 10 years old.
Is there anything else I can try? Should I just get a new hdd and start over?
 
zpool list shows me the zroot is online

I just discovered that zfs mount -a, gives me access to the home directory, and all my files seem to be there. But if type exit, and the system restarts, I still cannot log in.
 
richardtoohey2 said:
You just type exit and the machine restarts by itself? As in reboots?

Login to what?

I think you need zfs_enable="YES" in /etc/rc.conf to make sure the dataset will be available after a reboot.
Click to expand...

I am using MATE DE. When I enter exit, the system restarts. I am presented with a box to enter my name and password. But the name and password are not accepted.

The exact error I get is "failed to start session"

zfs_enable is set to "YES" in /etc/rc.conf. I was able to login just fine since last December, until yesterday.
 
There are only 2 lines in /etc/fstab file. One for swap, the other for proc.
I think there were more lines than that. I don't know how the file changed.

/dev/ada9p2 none swap sw 0 0
/proc proofs rw 0 0

That's it. Not sure what happened, or what to do about it.
 
I was able to log in by hitting ctrl-alt-f2 when I get the login box. Then I login from command line.

Then I did a: cat ~/.xsession-errors

I noticed this error coming up many times:

fstab: /etc/fstab:3: inappropriate file type or format
Click to expand...

I never touched my fstab file.
 
walterbyrd said:
/dev/ada9p2 none swap sw 0 0
/proc proofs rw 0 0

That's it.
Click to expand...
The ZFS pools do not appear in fstab. So don't panic! The file did maybe not changed.
walterbyrd said:
Not sure what happened, or what to do about it.
Click to expand...
walterbyrd said:
gives me access to the home directory, and all my files seem to be there.
Click to expand...
If you do not already have backups (which was of gross negligence, since hardware simply can die any moment) do at least now one of all of those to not get additionally even more damage by data loss.
 
walterbyrd said:
There are only 2 lines in /etc/fstab file. One for swap, the other for proc.
I think there were more lines than that. I don't know how the file changed.
/dev/ada9p2 none swap sw 0 0
/proc proofs rw 0 0
That's it. Not sure what happened, or what to do about it.
Click to expand...

According to google, my ftabs files should look like this:

# Device Mountpoint FStype Options Dump Pass#
/dev/ada0p2 none swap sw 0 0
proc /proc procfs rw 0 0
Click to expand...

Notice how line 3, the proc line is different? So I made those changes, but still having the same trouble.

I am able to get xterm running. And I can use the dillo browser. MATE will not run.

I had a backup, but it was about a week old. I did a new backup.
 
Assuming your goal is to recover the data from the sick system, stop using the desktop for now.

walterbyrd said:
I tried fsck - no luck.
I keep getting "Inappropriate file type or format"
I also got messages about could not find superblock, or alternate superblock.
Click to expand...
There is no fsck for ZFS.
When did you get "Inappropriate file type or format"?
We can not help debug "got messages". We need to know what messages, what they said, and when they happened.

Suggestion: Bring the system up in single user mode from the console. Determine exactly what file systems are where on the disk, and try to mount them read-only one at a time. See exactly what happens.

It's really hard to simultaneously debug file system and storage hardware problems, and the "dillo browser" and "MATE" (I don't even know what those are, nor do I care). I would start the recovery process from the fundamentals at the bottom.
 
I got "Inappropriate file type or format" when I tried to use fsck, as google suggested.

Dillo is a browser that does not require X11. I was using that because I did not know how to get to my standard desktop environment, which is MATE. MATE is a DE (Desktop Environment) that was forked from the Gnome DE. I have since got my X11 to work, and I have figured out how to start the chromium browser, which I am using now. I start the chromium browser by: $ chrome &. I am still not sure how to start firefox, but chromium is fine.

Here is what is happening now: When my system tries to boot to multi-user, I get a login box. I enter my name and password, but cannot login. The exact error I get is "failed to start session."

To get by that, I press: Ctrl-Alt-F2. This takes me to the command line, and I can login there. I enter zfs mount -a, to mount the file systems. Then I enter startx. This takes me to xterm. xterm provides a text interface screen inside the graphical environment. Once X11, and xterm, have started, I can launch applications from the command line, including my browser.

When the problems first started, I did a power off/on reboot. Then I went to single user. When I check my home directory, it was completely empty. I figured it was a complete HDD crash. But it turned out, the files were still there, the zroot file system was just not mounting. It is still is not mounting at startup, I have to mount it manually. Even after being mounted manually, I still cannot start my MATE desktop environment - although I can start X11 and xterm.

I hope that is more clear.
 
walterbyrd said:
Here is what is happening now: When my system tries to boot to multi-user, I get a login box.
Click to expand...
If you get a graphical login, than you are already in multi-user mode. The system started successfully.
walterbyrd said:
I enter my name and password, but cannot login. The exact error I get is "failed to start session."
Click to expand...
That is to be investigated: post the content of ~/.xsession-errors and the log file of your login manager (shuold be in /var/log).
walterbyrd said:
To get by that, I press: Ctrl-Alt-F2. This takes me to the command line, and I can login there. I enter zfs mount -a, to mount the file systems.
Click to expand...
You login from the console, ok. You don't need however to mount anything, that is needed only when you are in single-user mode; how I said you are in multi-user mode if you got to the login manager.
walterbyrd said:
Then I enter startx. This takes me to xterm. xterm provides a text interface screen inside the graphical environment. Once X11, and xterm, have started, I can launch applications from the command line, including my browser.
Click to expand...
What is the content of your ~/.xinitrc?
walterbyrd said:
But it turned out, the files were still there, the zroot file system was just not mounting. It is still is not mounting at startup, I have to mount it manually.
Click to expand...
That's weird, if you could successfully login from the console, without errors, then your shell config files have been read, and thus your home directory is regularly mounted. Please post the output of zfs list -o name,canmount,mountpoint.
 
$ cat ~/.xsession-errors
(mate-screensaver:3049): GLib-CRITICAL **: 15:49:04.360: Source ID 1005 was not found when attempting to remove it

(mate-screensaver:3049): GLib-GObject-CRITICAL **: 15:49:14.367: g_object_ref: assertion 'G_IS_OBJECT (object)' failed

(mate-screensaver:3049): GLib-GObject-CRITICAL **: 15:49:14.389: g_object_ref: assertion 'G_IS_OBJECT (object)' failed
fstab: /etc/fstab:3: Inappropriate file type or format
fstab: /etc/fstab:3: Inappropriate file type or format
fstab: /etc/fstab:3: Inappropriate file type or format
Click to expand...

When logged in as root:
# cat ~/.xinitrc
exec mate-session
Click to expand...

When logged in as myself (user):
$ cat ~/.xinitrc
cat: /home/walter/.xinitrc: No such file or directory
Click to expand...

$ zfs list -o name,canmount,mountpoint
NAME CANMOUNT MOUNTPOINT
zroot on /zroot
zroot/ROOT on none
zroot/ROOT/default noauto /
zroot/home on /home
zroot/home/joslyn on /home/joslyn
zroot/home/walter on /home/walter
zroot/tmp on /tmp
zroot/usr off /usr
zroot/usr/ports on /usr/ports
zroot/usr/src on /usr/src
zroot/var off /var
zroot/var/audit on /var/audit
zroot/var/crash on /var/crash
zroot/var/log on /var/log
zroot/var/mail on /var/mail
zroot/var/tmp on /var/tmp
Click to expand...
 
walterbyrd said:
When logged in as root:
Click to expand...
You must not start MATE (or everything else meant to be used an an unprivileged user) as root, remove that .xinitrc.
Instead, create an .xinitrc file in your $HOME with this:
Code: 
exec dbus-launch --exit-with-x11 ck-launch-session mate-session
This way you can at least start MATE from the console with startx.
walterbyrd said:
$ zfs list -o name,canmount,mountpoint
Click to expand...
Everything seems fine here.

Now, what login manager do you use? Its log file should shed a light about why it doesn't start the MATE session.

P.S.
Please, leave Gemini, ChatGPT, $whatever_glorified_chatbot out of the equation: they just create more confusion.
 
Made the changes you suggested, sadly it did not work. Here is what I got:

$ startx
xauth: file /home/walter/.serverauth.5052 does not exist

X.Org X Server 1.21.1.24
X Protocol Version 11, Revision 0
Current Operating System: FreeBSD fir 15.0-RELEASE FreeBSD 15.0-RELEASE releng/15.0-n280995-7aedc8de6446 GENERIC amd64

Current version of pixman: 0.46.4
Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.1.log", Time: Thu Aug 20 14:49:47 2026
(==) Using system config directory "/usr/local/share/X11/xorg.conf.d"
scfb trace: probe start
scfb trace: probe done
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Warning: Multiple symbols for level 1/group 1 on key <FK23>
> Using F23, ignoring XF86TouchpadOff
> Warning: Symbol map for key <FK23> redefined
> Using last definition for conflicting fields
> Warning: Symbol map for key <FK24> redefined
> Using last definition for conflicting fields
> Warning: Could not resolve keysym XF86ElectronicPrivacyScreenOn
> Warning: Could not resolve keysym XF86ElectronicPrivacyScreenOff
> Warning: Could not resolve keysym XF86ActionOnSelection
> Warning: Could not resolve keysym XF86ContextualInsert
> Warning: Could not resolve keysym XF86ContextualQuery
Errors from xkbcomp are not fatal to the X server
xinit: connection to X server lost

waiting for X server to shut down (II) Server terminated successfully (0). Closing log file.
Click to expand...

FWIW:
$ ls -l .serverauth*
-rw------- 1 walter walter 96 Jan 2 2026 .serverauth.18221
-rw------- 1 walter walter 144 Jan 2 2026 .serverauth.3094
-rw------- 1 walter wheel 96 Aug 20 14:51 .serverauth.5092
Click to expand...

After it failed to start an x11 session. I renamed .xinitrc to .xinitrc-bu, and did another startx, which made xterm start. That may be where the .serverauth.5092 came from.
 
“Seagate” is the first hint of you being in deep kimshe.

The trade rags love Seagate the best but in my SOHO business they are the chronic failures. Over 26,000 hours of run time you are on borrowed time.

I’m surprised your ZFS system cannot tolerate a single failure and resilver properly.
 
Maxnix said:
Why it is using scfb?
Which GPU do you have? Are the graphic drivers correctly loaded?
Click to expand...
$ pciconf -lv | grep -B4 VGA
vgapci0@pci0:1:5:0: class=0x030000 rev=0x00 hdr=0x00 vendor=0x1002 device=0x9616 subvendor=0x1458 subdevice=0xd000
vendor = 'Advanced Micro Devices, Inc. [AMD/ATI]'
device = 'RS780L [Radeon 3000]'
class = display
subclass = VGA

$ cat /dev/sndstat
Installed devices:
pcm0: <ATI RS690/780 (HDMI)> (play)
pcm1: <Realtek ALC887 (Rear Analog)> (play/rec)
pcm2: <Realtek ALC887 (Front Analog)> (play/rec) default
pcm3: <Realtek ALC887 (Onboard Digital)> (play)
No devices installed from userspace.
 
That happened to me once in a dying disk and lost everything in there.
It's already mentioned here that there's no fsck for zfs but I got your idea, I did exactly the same that time.
Try to boot with a USB installer pen and then go to the console/shell.
Then try to recover your zpool data
There are commands like
zpool status
and
zpool scrub zpool_name
that may help
I remember that didn't work to me but who knows it works for you
You may want to take a look at those commands.
Good luck and I hope you got a zpool backup
 
You left the system for 10 minutes, was it doing a pkg upgrade or something?
Not knowing what caused that doesn't seem good and I would start to check your disk for errors if I were you.
Great it's working, take your time for backups.
 
rmomota said:
You left the system for 10 minutes, was it doing a pkg upgrade or something?
Not knowing what caused that doesn't seem good and I would start to check your disk for errors if I were you.
Great it's working, take your time for backups.
Click to expand...

The system was just idle. I was not running anything. The only thing that ran, as far as I know, is the timeout where you logged off, and have to log back in again.
I did do a pkg upgrade, maybe an hour earlier.
I know that sometimes, system changes do not take effect you are logged out, or until you reboot.
Maybe a file was removed during the upgrade, but nothing happened until I was logged out?
I am just guessing. I would also like to know, for sure, what happened.
 
walterbyrd said:
Dillo is a browser ... I have since got my X11 to work, and I have figured out how to start the chromium browser,
Click to expand...
Are we trying to diagnose a failed disk drive (disk crash) or a corrupted file system, or are we playing with a computer?

If you are interested in fixing the disk / file system / mount problems, you don't need a GUI, you don't need X, you don't need any browsers. You are much better off working from the console, initially in single user mode only.

It seems to me that most of your problems can be explained by interesting misconfigurations of your complex user environment. Except the fact that ZFS doesn't mount is a bit puzzling.
 
You must log in or register to reply here.
Back
Top