UFS UFS Superblock Error after Extending Partition - fs->fs_fsbtodb (0) != ILOG2(fs->fs_fsize / sectorsize)

andshrew · Dec 11, 2023

Hello, this is a slightly unusual request for help. I'd appreciate any advice that can be offered.

I have a UFS partition that I am cloning and extending from a 60GB HDD to a 1TB SSD. I'm using a powerpc64 build of FreeBSD to do this (as an emulated guest using qemu-system-ppc64 from a Linux host), as the original partition originates from a big-endian PlayStation 3 (PS3) system. I'm using FreeBSD for this as - so far - it has been the easiest to get running as a VM, and also being the closest I can get to using what the PS3 OS is supposedly forked from.

The steps I have taken to do this:- I am using one of the pre-built VM images.

Code:

root@:~ # uname -a
FreeBSD  15.0-CURRENT FreeBSD 15.0-CURRENT #0 d0941ed: Wed Nov  8 13:53:34 UTC 2023   
root@FreeBSD-main-powerpc64-build.jail.ci.FreeBSD.org:/usr/obj/usr/src/powerpc.powerpc64/sys/GENERIC64 powerpc

Used dd to clone the drives

Used growfs /dev/vtbd1 to extend the partition to the maximum size

Code:

It's strongly recommended to make a backup before growing the file system.
OK to grow filesystem on /dev/vtbd1 from 54GB to 929GB? [yes/no] yes
super-block backups (for fsck_ffs -b #) at:
112919360, 113510560, 114101760, 114692960, 115284160, 115875360, 116466560, 117057760, 117648960, 118240160,
...
1946821760, 1947412960, 1948004160, 1948595360

Ran fsck_ufs -y /dev/vtbd1

Code:

root@:~ # fsck_ufs -y /dev/vtbd1
** /dev/vtbd1
** Last Mounted on
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames

UPDATE FILESYSTEM TO TRACK DIRECTORY DEPTH? yes

** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
66 files, 128564 used, 235823179 free (31 frags, 58955787 blocks, 0.0% fragmentation)

***** FILE SYSTEM IS CLEAN *****

***** FILE SYSTEM WAS MODIFIED *****

Mounted the partition mount /dev/vtbd1 /cell_mw_cfs/

Code:

root@:~ # mount /dev/vtbd1 /cell_mw_cfs/
root@:~ # df /dev/vtbd1
Filesystem 1K-blocks   Used     Avail Capacity  Mounted on
/dev/vtbd1 943806972 514256 867788160     0%    /cell_mw_cfs

Everything at this point appears to be working fine; so I return the drive to the PS3 where everything is working as expected there too with it seeing all of the newly extended space. Excellent... except I can no longer mount the partition on the FreeBSD system.

FreeBSD now gives the following error:

Code:

root@:~ # mount /dev/vtbd1 /cell_mw_cfs/
UFS2 superblock failed: fs->fs_fsbtodb (0) != ILOG2(fs->fs_fsize / sectorsize) (3)
mount: /dev/vtbd1: Invalid fstype: Invalid argument

I can mount the partition (read-only) under a Linux system, and the PS3 continues to be able to mount it perfectly fine - so it appears the partition and data are fine.

But "something" has happened to the partition after the PS3 mounted it that is now specifically making FreeBSD unable to read it.

I've repeated this a couple of times and the same thing continues to happen.

Does anyone have any insights or suggestions as to what can be done to try and fix this partition? Or is it just a bad idea to even try and do this - that is extending the UFS file system using a current version of FreeBSD and then expecting it to work on a BSD variant from ~2006? I would try this with a release of FreeBSD from the era that the PS3 is from, but so far I haven't been able to get earlier versions to boot in qemu.

I've shared some additional command output in the spoiler sections below should that offer any insight as to what has changed (presumably in the superblock?).

Thanks

Original 60GB HDD:

Code:

root@:~ # file -s /dev/vtbd2
/dev/vtbd2: Unix Fast File system [v2] (big-endian) last mounted on /cell_mw_cfs,
last written at Sun Jan  1 00:11:14 2012,
clean flag 1,
readonly flag 0,
number of blocks 14061452,
number of data blocks 13618311,
number of cylinder groups 191,
block size 16384,
fragment size 4096,
average file size 16384,
average number of files in dir 64,
pending blocks to free 0,
pending inodes to free 0,
system-wide uuid 0,
minimum percentage of free blocks 8,
TIME optimization

1TB SSD after growfs has been run:

Code:

root@:~ # file -s /dev/vtbd1
/dev/vtbd1: Unix Fast File system [v2] (big-endian) last mounted on ,
last written at Mon Dec 11 14:03:12 2023,
clean flag 1,
readonly flag 0,
number of blocks 243600816,
number of data blocks 235951743,
number of cylinder groups 3297,
block size 16384,
fragment size 4096,
average file size 16384,
average number of files in dir 64,
pending blocks to free 0,
pending inodes to free 0,
system-wide uuid 0,
minimum percentage of free blocks 8,
TIME optimization

1TB SSD after UFS2 superblock failed error (unchanged as far as I can see):

Code:

root@:~ # file -s /dev/vtbd1
/dev/vtbd1: Unix Fast File system [v2] (big-endian) last mounted on /cell_mw_cfs,
last written at Mon Dec 11 14:28:52 2023,
clean flag 1,
readonly flag 0,
number of blocks 243600816,
number of data blocks 235951743,
number of cylinder groups 3297,
block size 16384,
fragment size 4096,
average file size 16384,
average number of files in dir 64,
pending blocks to free 0,
pending inodes to free 0,
system-wide uuid 0,
minimum percentage of free blocks 8,
TIME optimization

Unsure if these are the full superblocks, so if it isn't I can re-run these commands...

Original 60GB HDD:

Code:

root@:~ # dd if=/dev/vtbd2 bs=512 skip=128 count=4 status=none | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 14 00 00 00 18  |................|
00000010  00 00 00 1c 00 00 09 24  00 00 00 00 00 00 00 00  |.......$........|
00000020  00 00 00 00 00 00 00 00  00 00 78 10 00 00 00 bf  |..........x.....|
00000030  00 00 40 00 00 00 10 00  00 00 00 04 00 00 00 08  |..@.............|
00000040  00 00 00 00 00 00 00 00  ff ff c0 00 ff ff f0 00  |................|
00000050  00 00 00 0e 00 00 00 0c  00 00 00 08 00 00 08 00  |................|
00000060  00 00 00 02 00 00 00 03  00 00 10 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 08 00  00 00 00 40 00 00 00 00  |...........@....|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 f5 35 bd 07  00 00 00 00 00 00 10 00  |.....5..........|
000000a0  00 00 40 00 00 00 00 00  00 00 00 00 00 00 00 00  |..@.............|
000000b0  00 00 00 00 00 00 00 00  00 00 90 80 00 01 20 ac  |.............. .|
000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000d0  00 01 00 80 2f 63 65 6c  6c 5f 6d 77 5f 63 66 73  |..../cell_mw_cfs|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000002d0  00 00 00 00 00 00 00 70  00 00 00 00 00 00 00 00  |.......p........|
000002e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000330  00 00 00 00 00 00 00 00  80 00 00 02 00 16 12 fc  |................|
00000340  80 00 00 02 00 16 00 00  80 00 00 02 00 16 10 00  |................|
00000350  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|
00000360  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000003e0  00 00 00 00 00 00 00 00  00 00 00 00 00 01 00 00  |................|
000003f0  00 00 00 00 00 00 00 25  00 00 00 00 00 33 75 8d  |.......%.....3u.|
00000400  00 00 00 00 00 6b cf 3c  00 00 00 00 00 00 00 1f  |.....k.<........|
00000410  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000430  00 00 00 00 4e ff a4 a2  00 00 00 00 00 d6 8f 8c  |....N...........|
00000440  00 00 00 00 00 cf cc 87  00 00 00 00 00 00 09 24  |...............$|
00000450  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000004a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|
000004b0  00 00 00 40 00 00 00 00  00 00 00 00 00 00 00 00  |...@............|
000004c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000520  00 00 00 02 00 00 00 08  00 00 00 78 00 00 00 00  |...........x....|
00000530  00 00 80 10 02 02 ff ff  00 00 00 00 00 00 3f ff  |..............?.|
00000540  00 00 00 00 00 00 0f ff  00 00 00 00 00 00 00 00  |................|
00000550  00 00 00 00 00 00 00 00  00 00 00 00 19 54 01 19  |.............T..|
00000560  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000800

1TB SSD after growfs has been run:

Code:

root@:~ # dd if=/dev/vtbd1 bs=512 skip=128 count=4 status=none | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 14 00 00 00 18  |................|
00000010  00 00 00 1c 00 00 09 24  00 00 00 00 00 00 00 00  |.......$........|
00000020  00 00 00 00 0e 85 0d b0  0e 10 56 7f 00 00 0c e1  |..........V.....|
00000030  00 00 40 00 00 00 10 00  00 00 00 04 00 00 00 08  |..@.............|
00000040  00 00 00 00 00 00 00 00  ff ff c0 00 ff ff f0 00  |................|
00000050  00 00 00 0e 00 00 00 0c  00 00 00 08 00 00 08 00  |................|
00000060  00 00 00 02 00 00 00 03  00 00 10 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 08 00  00 00 00 40 00 00 00 00  |...........@....|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 f5 35 bd 07  00 00 00 00 00 00 d0 00  |.....5..........|
000000a0  00 00 40 00 00 00 00 00  00 00 00 00 00 00 00 00  |..@.............|
000000b0  00 00 00 00 00 00 00 00  00 00 90 80 00 01 20 ac  |.............. .|
000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000d0  00 01 00 80 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000330  00 00 00 00 00 00 00 00  80 00 00 02 00 16 12 fc  |................|
00000340  80 00 00 02 00 16 00 00  80 00 00 02 00 16 10 00  |................|
00000350  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|
00000360  00 00 00 00 00 00 00 00  00 00 00 00 0e 85 0d b0  |................|
00000370  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000003e0  00 00 00 00 00 01 00 00  00 00 00 00 00 01 00 00  |................|
000003f0  00 00 00 00 00 00 00 25  00 00 00 00 03 83 98 0b  |.......%........|
00000400  00 00 00 00 07 45 00 3c  00 00 00 00 00 00 00 1f  |.....E.<........|
00000410  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000430  00 00 00 00 65 77 16 a0  00 00 00 00 0e 85 0d b0  |....ew..........|
00000440  00 00 00 00 0e 10 56 7f  00 00 00 00 00 d7 69 78  |......V.......ix|
00000450  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000004a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|
000004b0  00 00 00 40 00 00 00 00  00 00 00 00 00 00 00 00  |...@............|
000004c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000520  00 00 00 02 00 00 00 08  00 00 00 78 00 00 00 00  |...........x....|
00000530  00 00 80 10 02 02 ff ff  00 00 00 00 00 00 3f ff  |..............?.|
00000540  00 00 00 00 00 00 0f ff  00 00 00 00 00 00 00 00  |................|
00000550  00 00 00 00 00 00 00 00  00 00 00 00 19 54 01 19  |.............T..|
00000560  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000800

1TB SSD after UFS2 superblock failed error:

Code:

root@:~ # dd if=/dev/vtbd1 bs=512 skip=128 count=4 status=none | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 14 00 00 00 18  |................|
00000010  00 00 00 1c 00 00 09 24  00 00 00 00 00 00 00 00  |.......$........|
00000020  00 00 00 00 0e 85 0d b0  0e 10 56 7f 00 00 0c e1  |..........V.....|
00000030  00 00 40 00 00 00 10 00  00 00 00 04 00 00 00 08  |..@.............|
00000040  00 00 00 00 00 00 00 00  ff ff c0 00 ff ff f0 00  |................|
00000050  00 00 00 0e 00 00 00 0c  00 00 00 08 00 00 08 00  |................|
00000060  00 00 00 02 00 00 00 00  00 00 10 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 08 00  00 00 00 40 00 00 00 00  |...........@....|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 f5 35 bd 07  00 00 00 00 00 00 d0 00  |.....5..........|
000000a0  00 00 40 00 00 00 00 00  00 00 00 00 00 00 00 00  |..@.............|
000000b0  00 00 00 00 00 00 00 00  00 00 90 80 00 01 20 ac  |.............. .|
000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000d0  00 01 00 80 2f 63 65 6c  6c 5f 6d 77 5f 63 66 73  |..../cell_mw_cfs|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000330  00 00 00 00 00 00 00 00  80 00 00 00 00 5a 33 84  |.............Z3.|
00000340  80 00 00 02 00 16 00 00  80 00 00 00 00 5a 00 00  |.............Z..|
00000350  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|
00000360  00 00 00 00 00 00 00 00  00 00 00 00 0e 85 0d b0  |................|
00000370  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000003e0  00 00 00 00 00 01 00 00  00 00 00 00 00 01 00 00  |................|
000003f0  00 00 00 00 00 00 00 48  00 00 00 00 03 83 97 fb  |.......H........|
00000400  00 00 00 00 07 45 00 05  00 00 00 00 00 00 00 20  |.....E......... |
00000410  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000430  00 00 00 00 65 77 1c a4  00 00 00 00 0e 85 0d b0  |....ew..........|
00000440  00 00 00 00 0e 10 56 7f  00 00 00 00 00 d7 69 78  |......V.......ix|
00000450  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000004a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|
000004b0  00 00 00 40 00 00 00 00  00 00 00 00 65 77 1a 58  |...@........ew.X|
000004c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000520  00 00 00 02 00 00 00 08  00 00 00 78 00 00 00 00  |...........x....|
00000530  00 00 80 10 02 02 ff ff  00 00 00 00 00 00 3f ff  |..............?.|
00000540  00 00 00 00 00 00 0f ff  00 00 00 00 00 00 00 00  |................|
00000550  00 00 00 00 00 00 00 00  00 00 00 00 19 54 01 19  |.............T..|
00000560  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000800

Code:

root@:~ # fsck_ufs -y -d /dev/vtbd1
UFS2 superblock failed: fs->fs_fsbtodb (0) != ILOG2(fs->fs_fsize / sectorsize) (3)
Attempted recovery for standard superblock: failed
Attempted extraction of recovery data from standard superblock: failed
Attempt to find boot zone recovery data.
Finding an alternate superblock failed.
Check for only non-critical errors in standard superblock
UFS2 superblock failed: fs->fs_fsbtodb (0) != ILOG2(fs->fs_fsize / sectorsize) (3)
Failed, superblock has critical errors
SEARCH FOR ALTERNATE SUPER-BLOCK FAILED. YOU MUST USE THE
-b OPTION TO FSCK TO SPECIFY THE LOCATION OF AN ALTERNATE
SUPER-BLOCK TO SUPPLY NEEDED INFORMATION; SEE fsck_ffs(8).
setup: missing superblock

Trying to specify an alternative superblock:

Code:

root@:~ # fsck_ufs -y -d -b 1948595360 /dev/vtbd1
Alternate super block location: 1948595360
UFS2 superblock failed: fs->fs_fsbtodb (0) != ILOG2(fs->fs_fsize / sectorsize) (3)
1948595360 is not a file system superblock
setup: missing superblock

covacat · Dec 11, 2023

iirc freebsd ufs is endian dependent so you cant mount big on little or viceversa
netbsd ufs is endian agnostic so should work

andshrew · Dec 11, 2023

covacat said:
iirc freebsd ufs is endian dependent so you cant mount big on little or viceversa
netbsd ufs is endian agnostic so should work

Thank you, that's a good tip.

I've quickly configured an amd64 NetBSD VM to try this out, and it can indeed mount the big-endian partition.

Running fsck_ffs via that does actually say it's fixing some incorrect block count issues (screenshot attached), but despite that FreeBSD is still unable to mount it due to the same error as above.

But knowing NetBSD works gives me more options to explore now, so thank you for that suggestion.

andshrew · Dec 16, 2023

Just for interest, I think I was able to figure out what the problem is.

Using NetBSD and dumpfs I was able to compare the superblock of a partition I could mount in FreeBSD vs one that I get the above error with.

On the partition which I could not mount I noticed that the fsbtodb is set to 0. From the research I've done this seemingly means the file system is configured with a 4096 byte sector size - when it should be 512bytes - and so FreeBSD is unable to mount it.

If I manually edit the superblock to set this back to 3 then FreeBSD can mount the partition again. However as soon as I return the drive to the PS3 this value is reset back to 0, and becomes unmountable in FreeBSD again.

~~So it seems to be there is a bug/limitation in the PS3's UFS implementation which is causing this to be erroneously set on partitions over a certain size.~~
Edit: I've now realised the issue seems to be that the drive I am using is actually 4096 sector size, but when I've been connecting it to my machine via USB to SATA (and when passing into a VM) it is appearing as 512 sector size.

Disk directly connected to SATA

Code:

Disk /dev/sdd: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk connected to USB to SATA or passed into a VM

Code:

Disk /dev/sdd: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: 500SSD1    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

So the actual problem is this mismatch. The PS3 is correctly setting the superblock fsbtodb to 4096, and then FreeBSD cannot mount the partition because this value doesn't match the sector size that it is seeing the drive as (512). But manually editing this value in the superblock to 512 does allow FreeBSD to mount it, and everything works because presumably all of the actual 4096 to 512 translation is happening transparently behind the scenes.

UFS UFS Superblock Error after Extending Partition - fs->fs_fsbtodb (0) != ILOG2(fs->fs_fsize / sectorsize)

andshrew

covacat

andshrew

Attachments

andshrew