UFS Unable to fsck

balanga · May 11, 2019

I have a system which has been running FreeBSD 12.0-RELEASE amd64 for several months but all of a sudden it will not boot up. After running fsck -y many many times I can't a clean filesystem. I have a 1TB disk and FreeBSD is installed on a 250GB UFS partition. It is partitioned using GPT and has 670GB free.

I've found that I can run misc/mc and have not encountered any problems looking round the system.

What can I do to clean the filesystem?

Beastie · May 11, 2019

There's not much to it. Boot into single-user mode run fsck. Personally I prefer to run it manually on each partition (e.g. # fsck -y /dev/ada0s1a).

If you still have problems after that, run SMART diagnostics.

ralphbsz · May 12, 2019

balanga said:
...it will not boot up.

OK. You are saying the file system is so broken that the system will not boot, which I would take to mean: you don't get to a prompt of a shell, and the system doesn't work.

After running fsck -y many many times ...

Strange. A moment ago you said the system didn't boot. Now you are saying that the system boots well enough to get you to a shell prompt, and you can run fsck from the prompt. Contradiction.

...I can't a clean filesystem.

Sorry, but just saying that you ran fsck many times, and then the file system is still not clean, does not help us debug it. You need to tell us exactly how fsck failed. What were the messages? Did you run it in single-user mode, or multi-user? On the root file system? Did it run to completion?

I've found that I can run misc/mc and have not encountered any problems looking round the system.

OK, now you are saying that the system boots perfectly, well enough for a user-space application to start, and that there are no problems with the file system. That contradicts pretty much everything you said above.

Please explain the problem. I can not understand what you wrote.

balanga · May 12, 2019

ralphbsz said:
Please explain the problem. I can not understand what you wrote.

When I said the system does not boot up, I meant it does not boot up to the command prompt. It drops into single user mode because the file system is dirty. After running both fsck -y /dev/ada0p2 and

 fsck -n /dev/ada0p2

maybe twenty times or more the filesystem remained dirty. However I did find that I could run misc/mc and after looking around the filesystem couldn't find anything obviously wrong.

In order to try and log the problem, I hooked up the disk via a USB enclosure and booted from another disk, and ran fsck -y /dev/da01p2 and to my amazement it resulted in a clean filesystem. Since that was unexpected and didn't have a chance to log it and can't show exactly what was wrong, and have no idea how the filesystem got fixed. I am now booting again normally from the original disk and it seems to work normally.

Beastie · May 12, 2019

Then it could simply be a loose cable and nothing to do with the disk itself. Disconnecting and later reconnecting the disk as you did may have fixed the issue.

But I'll repeat myself: do run SMART diagnostics. Something may always come up.

mfaridi · May 12, 2019

Do you have enough free space on your system?
check your free space.

ralphbsz · May 12, 2019

Ah, now I get it. Your system came up in single-user mode, and you got the shell prompt there. And then you ran fsck on the root file system, and it didn't work. But when you take the same disk drive and connect it as an extra file system to a computer (where it is not root), then fsck works.

I think this is perfectly expected and normal: When booting in single user mode, the root file system is mounted read-only for safety. ~~And fsck can not fix things in read-only mode, because it can not modify the root file system. But when you took the disk and didn't mount it read-only, fsck was suddenly capable of working.~~

An interesting question is this: Did fsck print warning messages?

P.S. Fixed this post after PMc's correction below. Sorry, I got confused.

PMc · May 12, 2019

ralphbsz said:
I think this is perfectly expected and normal: When booting in single user mode, the root file system is mounted read-only for safety. And fsck can not fix things in read-only mode, because it can not modify the root file system. But when you took the disk and didn't mount it read-only, fsck was suddenly capable of working.

Little correction seems necessary here. fsck can repair filesystem only when in read-only mode - thats why the root fs comes up read-only in single-user, so that one can just run fsck.