can't mount Dangerously Dedicated USB stick

Mjölnir

Daemon

Reaction score: 1,510
Messages: 2,114

According to your explanation above, the UFS (either V1 or 2) filesystem starts at the beginning of the device, but reserves space for the boot blocks. I.e. when copying the device, no blocks must be skipped; fsck(8) will know that it has to skip the 1st blocks because they are reserved for the boot blocks and are not used by the filesystem anyway. FreeBSD 7 has UFS2 already, correct? I guess that the OP did not skip any block on the initial copy (with dd(1)?). That's why I suggest to copy with recoverdisk(1) instead; because it seems there might have been something going wrong when copying. I recommended to check that the image is ok (verify img(dd) == img(recoverdisk)); but the OP did not reply... I recommended to fsck(8) the device & the image before mounting, but the OP did not reply...
 
OP
O

oneev

New Member

Reaction score: 1
Messages: 15

I used recoverdisk a few times to create an image using different options. I ended up with the same image. I checked in all cases and the images are identical to the original one. So I am confident that my image is identical to the USB stick.

I did the procedure suggested by Mjölnir and I get the same message:

Code:
root@NomadBSD:/data # fsck /dev/md2
Can't open /dev/md2: No such file or directory

Unfortunately fsck does not let me access the device neither if I use the full image
Code:
root@NomadBSD:~ # fsck /dev/md1
Can't open /dev/md1: No such file or directory

(and same result with fsck /dev/da1)
Alas I get the same error with dumpfs

Code:
root@NomadBSD:~ # dumpfs /dev/md1
dumpfs: /dev/md1: No such file or directory

I get this searching for the offset 0x1055c:
Code:
root@NomadBSD:/data # dd if=lttleowl.img count=131 status=none | hd -s 0x1055c -n 4
0001055c  00 00 00 00                                       |....|
00010560
So it appears those those four bytes are not there. What other offsets should I try besides 0x1055c?
 

Mjölnir

Daemon

Reaction score: 1,510
Messages: 2,114

Please look for ls -ltr /dev/md*. Please post the output of freebsd-version and uname -a. Are you doing this inside a virtual machine?

mdconfig(8) automagically uses a fresh number for it's devices: /dev/md0, /dev/md1, .... Please do exactly the command: fsck /dev/`mdconfig -f lttleowl.img` When finished with all work, release the memory device: mdconfig -lf lttleowl.img shows all md devices backed by that file, -lv lists all. mdconfig -du N releases md device #N.
 
OP
O

oneev

New Member

Reaction score: 1
Messages: 15

Here the commands:

Code:
root@NomadBSD:/data # ls -ltr /dev/md*
crw-rw----  1 root  operator   0xb Jan 31 18:43 /dev/mdctl
crw-rw----  1 root  operator  0xb6 Jan 31 18:43 /dev/md0
crw-rw----  1 root  operator  0xb9 Jan 31 18:43 /dev/md0.uzip
crw-rw----  1 root  operator  0xc3 Feb  2 19:47 /dev/md1
root@NomadBSD:/data # mdconfig -lv
md0    vnode     1981M    /uzip/usr.local.uzip   
md1    vnode     7648M    /data/lttleowl.img
root@NomadBSD:/data # freebsd-version
12.2-RELEASE-p2
root@NomadBSD:/data # uname -a
FreeBSD NomadBSD 12.2-RELEASE-p2 FreeBSD 12.2-RELEASE-p2 r369076 NOMADBSD  i386
I am using Nomad BSD on an other USB stick as I can't wipe the disk on this test PC. My goal is to install FreeBSD 8 for further tests. Could this make a difference? But I need to save the data on this disk before.
Code:
root@NomadBSD:/data # fsck /dev/`mdconfig -f lttleowl.img`
Can't open /dev/md2: No such file or directory
root@NomadBSD:/data # mdconfig -lf lttleowl.img
md1 md2 
root@NomadBSD:/data # mdconfig -lv
md0    vnode     1981M    /uzip/usr.local.uzip    
md1    vnode     7648M    /data/lttleowl.img    
md2    vnode     7648M    /data/lttleowl.img    
root@NomadBSD:/data # mdconfig -du 2
 

olli@

Daemon
Developer

Reaction score: 1,262
Messages: 1,140

The error code ENOENT (“No such file or directory”) comes from the function ffs_sbget in /sys/ufs/ffs/ffs_subr.c, and it means that no usable known superblock was found. Mount, fsck and dumpfs all use that function (via libufs(3)), so they all fail in the same way.

And this confirms that there is no superblock at the usual location, so this explains why you get “No such file or directory”:
I get this searching for the offset 0x1055c:
Code:
root@NomadBSD:/data # dd if=lttleowl.img count=131 status=none | hd -s 0x1055c -n 4
0001055c  00 00 00 00                                       |....|
00010560
So it appears those those four bytes are not there. What other offsets should I try besides 0x1055c?

The following shell command will search the whole image for the “magic” signature of UFS2 superblocks:
Code:
hd lttleowl.img | awk '$1 ~ /50$/ && /19 01 54 19/ {print (("0x" $1) - 1360) / 512}'
It might run for a while if the image is large. It might also produce long output, because the superblock is replicated in every cylinder group.
This is a sample output for a small UFS2 image (100 MB) that has four cylinder groups, so you have a total of five superblocks:
Code:
128
192
51456
102720
153984
Then run fsck_ufs(8) with the -b option and provide the block number of one of the superblock locations that you have found, for example:
Code:
fsck_ufs -b 128 /dev/md1
Good luck!
 
OP
O

oneev

New Member

Reaction score: 1
Messages: 15

Interesting. Here is what I get. It does take long time to run indeed.
Code:
root@NomadBSD:/data # hd lttleowl.img | awk '$1 ~ /50$/ && /19 01 54 19/ {print (("0x" $1) - 1360) / 512}'
31
66
80
95
144
162
176
191
223
12095
317315
317316
317388
317535
376479
752735
1128991
1505247
1881503
2257759
2634015

Unfortunately fsck has the same problem as before
Code:
root@NomadBSD:/data # fsck_ufs -b 31 /dev/md1
Alternate super block location: 31
Can't open /dev/md1: No such file or directory
I tried all values from above, including +- 1.

I tried to take a chunk of 100Mb of the USB I'm now booted on (/dev/da0)
Code:
root@NomadBSD:/data # dd if=/dev/da0 of=/mnt/da0.img bs=512 count=200000
200000+0 records in
200000+0 records out
102400000 bytes transferred in 154.870278 secs (661199 bytes/sec)
root@NomadBSD:/data # hd /mnt/da0.img | awk '$1 ~ /50$/ && /19 01 54 19/ {print (("0x" $1) - 1360) / 512}'
Doesn't return anything. I'm might be doing something wrong.
 

Mjölnir

Daemon

Reaction score: 1,510
Messages: 2,114

oneev Insert the stick and try fsck'ing the device in readonly mode & debug messages: fsck_ufs -dnR /dev/daN (replace N with the # of the device, usually 0 if that's the only USB mass storage device).
 
OP
O

oneev

New Member

Reaction score: 1
Messages: 15

I get the following. I tried also a few other potential super block positions
Code:
root@NomadBSD:/data # fsck_ufs -dnR /dev/da1
Can't open /dev/da1: No such file or directory
root@NomadBSD:/data # fsck_ufs -b 31 -dnR /dev/da1
Alternate super block location: 31
Can't open /dev/da1: No such file or directory
root@NomadBSD:/data # fsck_ufs -b 66 -dnR /dev/da1
Alternate super block location: 66
Can't open /dev/da1: No such file or directory
root@NomadBSD:/data # fsck_ufs -b 317316 -dnR /dev/da1
Alternate super block location: 317316
Can't open /dev/da1: No such file or directory
root@NomadBSD:/data # fsck_ufs -b 30 -dnR /dev/da1
Alternate super block location: 30
Can't open /dev/da1: No such file or directory
root@NomadBSD:/data # fsck_ufs -b 32 -dnR /dev/da1
Alternate super block location: 32
Can't open /dev/da1: No such file or directory
root@NomadBSD:/data #
 

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,681
Messages: 2,539

Check if the device is actually created. This looks like da1 is not created at all.
 

olli@

Daemon
Developer

Reaction score: 1,262
Messages: 1,140

Check if the device is actually created. This looks like da1 is not created at all.
It is. As he explained above, he can run ls(1), file(1) and dd(1) on it.
The error message comes from the ffs_sbget function in /sys/ufs/ffs/ffs_subr.c which is used by mount, fsck and dumpfs.
See my post above for all the details.
 

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,681
Messages: 2,539

Sorry, I skipped some of the thread.

One really stupid reason might be - is it actually the stick wanted or maybe it's the same make/color?

And remember, you don't look behind bushes you didn't hide behind before yourself. ;)
 
OP
O

oneev

New Member

Reaction score: 1
Messages: 15

Yes those bloopers happen. Not this time though. I'm positive.
It was happily running till last week. I don't know what caused all the corruption. I know one should not use a usb stick to run an operating system; however, to alleviate this problem I mounted /tmp and /var in memory and disabled swap. The installation was minimalist and ran only a few services. So I don't think the stick was damaged by read/write wear.
Last week the hardware crashed a multiple times and rebooted without issue (this board has a known bug that creates rare random crashes during prolonged IO). It always booted fine and I never had errors with fsck on the disks. I think I had soft updates activated on the stick.
But now something is wrong with the mainboard and it doesn't reach the bios and doesn't boot anymore.
Maybe the last crash created corruptions on the stick. Or the multiple crashes in a short time lead to the corruption.
 

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,681
Messages: 2,539

I lost one stick to overheating. That might be it. Or not. :/
 

Mjölnir

Daemon

Reaction score: 1,510
Messages: 2,114

[...] I know one should not use a usb stick to run an operating system; however, to alleviate this problem I mounted /tmp and /var in memory and disabled swap. The installation was minimalist and ran only a few services. So I don't think the stick was damaged by read/write wear. [...]
Well, in theory these precautionary measures should be sufficient, but unfortunately in practice many (most?) software is poorly written, in that it doen't respect the suggested path's (hier(7)). Plus, either the maintainers are too busy or too lazy to fix the ports(7)' poor defaults -- that can be very time consuming -- or some ports don't even have a maintainer and thus are built with the poor & "wrong" default settings. You can easily prove the effect by mounting the / root filesystem (by default including /usr & /usr/local) readonly. Many/most ports(7) will not run as expected... i.e. many do heavily write to /usr/local.

This means your USB stick might very well be worn out after a few years, even when the OS was only run occasionally. If you want to run an OS from an USB stick, you must mount it read-only, and only remount read-write when updating the configuration files or packages/ports/OS. You can produce a long list of bug reports by doing this... ;)
 
Last edited:
D

Deleted member 66267

Guest


Don't blame me please I'm just trying to help.

Please try to mount your image on a live Linux system. Just grab the most popular one, currently MX Linux.

Linux has some magics I don't know it could mount almost all variant of UFS from any BSDs albeit just read-only.

This is the last resort I usually try and always success before I go for all ZFS.

You could find tutorials about how to mount disk image on Linux easily, they are plenty on the internet.

Hope it helps.
 

olli@

Daemon
Developer

Reaction score: 1,262
Messages: 1,140

Given the information about the stick, I think it is likely that it is worn out.

The controllers of flash media implement so-called wear-leveling algorithms that aim to distribute the wear evenly across the flash cells, even if some blocks are overwritten much more often than others. Different brands of media have different quality of wear-leveling algorithms, but usually USB sticks are inferior to SSDs. Another problem is that – unlike SSDs – most USB sticks don’t support SMART, so you cannot look at the current wear status and expected remaining flash life.

I managed to wear out several high-quality USB sticks within just a few weeks by using them with an appliance that seems to have the bad habit of overwriting the same sector every few seconds. Finally I switched to using an external USB SSD (a Samsung T5), and it is now happily running for two years 24/7.

Most importantly: If you put valuable data on any medium (flash stick, hard disk, SSD, whatever), be sure to have a backup copy somewhere.
 
OP
O

oneev

New Member

Reaction score: 1
Messages: 15

As suggested by failure I tried mounting the stick from Linux.... and it WORKED!
I tried with an offset of 32256 (63 x 512).
Here the command I used:
Code:
mount -r -t ufs -o ufstype=ufs2,offset=32256 /dev/sdc /mnt
And it mounted fine.
I was able to get the precious 64 byte geli key and now I'm retrieving the data from the encrypted disks.

I then tried again on FreeBSD to use the same offset. However I don’t know how to set an offset to mount or mdconfig. So I created a new dd image with iseek=32256

And I could then mount the image:
Code:
root@NomadBSD:/mnt2 # dd if=image.dd of=test.img bs=512 iseek=63
15663041+0 records in
15663041+0 records out
8019476992 bytes transferred in 493.758550 secs (16241697 bytes/sec)
root@NomadBSD:/mnt2 # mdconfig -a -t vnode -f test.img -u 1
root@NomadBSD:/mnt2 # fsck_ufs /dev/md1
** /dev/md1

SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] y

ADD CYLINDER GROUP CHECK-HASH PROTECTION? [yn] y

** Last Mounted on /
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
UNREF FILE I=212161  OWNER=root MODE=100640
SIZE=63 MTIME=Jan 28 01:00 2021 
CLEAR? [yn] y

UNREF FILE I=212162  OWNER=root MODE=100644
SIZE=3741214 MTIME=Jan 28 12:42 2021 
CLEAR? [yn] y

LINK COUNT FILE I=235541  OWNER=operator MODE=100400
SIZE=4096 MTIME=Jan 29 10:22 2021  COUNT 2 SHOULD BE 1
ADJUST? [yn] y

UNREF FILE  I=239044  OWNER=operator MODE=100400
SIZE=4096 MTIME=Jan 29 10:55 2021 
RECONNECT? [yn] y

SORRY. NO SPACE IN lost+found DIRECTORY
UNEXPECTED SOFT UPDATE INCONSISTENCY
CLEAR? [yn] y

UNREF FILE  I=240981  OWNER=operator MODE=100400
SIZE=0 MTIME=Jan 29 10:55 2021 
RECONNECT? [yn] y

SORRY. NO SPACE IN lost+found DIRECTORY
UNEXPECTED SOFT UPDATE INCONSISTENCY
CLEAR? [yn] y

LINK COUNT FILE I=247301  OWNER=operator MODE=100400
SIZE=4096 MTIME=Jan 29 09:44 2021  COUNT 2 SHOULD BE 1
ADJUST? [yn] y

LINK COUNT FILE I=247312  OWNER=operator MODE=100400
SIZE=4096 MTIME=Jan 29 09:55 2021  COUNT 2 SHOULD BE 1
ADJUST? [yn] y

** Phase 5 - Check Cyl groups
FREE BLK COUNT(S) WRONG IN SUPERBLK

SALVAGE? [yn] y

346266 files, 2296025 used, 1495363 free (13883 frags, 185185 blocks, 0.4% fragmentation)

***** FILE SYSTEM MARKED CLEAN *****

***** FILE SYSTEM WAS MODIFIED *****
root@NomadBSD:/mnt2 #
No partition found. But I can mount it fine.

Code:
root@NomadBSD:/mnt2 # gpart show /dev/md1
gpart: No such geom: /dev/md1.
root@NomadBSD:/mnt2 # file -s /dev/md1
/dev/md1: Unix Fast File system [v2] (little-endian) last mounted on /, last written at Thu Feb 11 13:29:14 2021, clean flag 1, readonly flag 0, number of blocks 3915749, number of data blocks 3791388, number of cylinder groups 42, block size 16384, fragment size 2048, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 8, TIME optimization
root@NomadBSD:/mnt2 # mount /dev/md1 /mnt3
root@NomadBSD:/mnt3 # ls /mnt3
.cshrc        .snap         bin           dev           etc           libexec       mnt           proc          sbin          tmp
.profile      COPYRIGHT     boot          dhclient.core home          lost+found    net           rescue        sleepyowl     usr
.rnd          backup        compat        entropy       lib           media         private       root          sys           var
root@NomadBSD:/mnt3 # 
root@NomadBSD:/mnt3 # ll root/gelikey 
-rw-------  1 root  wheel  64 Apr 23  2011 root/gelikey
root@NomadBSD:/mnt3 #
To be honest I still don’t know what went wrong or is wrong. To me the filesystem on the key seems file, at leat the data seems fine. gpart doesn’t see a partition.

So I’m very happy to have rescued my geli key to read the disks.
Yes in the future I’ll be more careful to backup root and in particular the geli key. However, I’ll only use a passphrase in the future, as the key it too risky for me.


olli@ I also use a Samsung T5, it’s great, but overkill for the minimal setup I had.
 

olli@

Daemon
Developer

Reaction score: 1,262
Messages: 1,140

Good to hear that you were able to recover your data.

Apparently, there must have been a partitioning scheme on that stick at some point in the past (MBR and/or disklabel) that had one UFS partition starting at sector 63. That is, the stick was not “dangerously dedicated” to begin with. But somehow, that partition scheme was lost, so GEOM (gpart) wasn’t able to recognize it anymore. Maybe a foreign OS tried to “repair” the stick.
 
D

Deleted member 66267

Guest


As suggested by failure I tried mounting the stick from Linux.... and it WORKED!
I tried with an offset of 32256 (63 x 512).
Here the command I used:
Code:
mount -r -t ufs -o ufstype=ufs2,offset=32256 /dev/sdc /mnt
And it mounted fine.
I was able to get the precious 64 byte geli key and now I'm retrieving the data from the encrypted disks.
Glad that it worked for you.
 
D

Deleted member 66267

Guest


Apparently, there must have been a partitioning scheme on that stick at some point in the past (MBR and/or disklabel) that had one UFS partition starting at sector 63. That is, the stick was not “dangerously dedicated” to begin with. But somehow, that partition scheme was lost, so GEOM (gpart) wasn’t able to recognize it anymore. Maybe a foreign OS tried to “repair” the stick.
Maybe just running newfs on the whole device is better as it doesn't have any partitioning scheme, just the FS alone? Is it possible at all? Or the disk must be partitioned? Please correct me if my idea is wrong.
 

olli@

Daemon
Developer

Reaction score: 1,262
Messages: 1,140

Maybe just running newfs on the whole device is better as it doesn't have any partitioning scheme, just the FS alone? Is it possible at all? Or the disk must be partitioned? Please correct me if my idea is wrong.
You can run newfs on any raw device. So, yes, of course you can do that without partitions. Then it is like a huge floppy disk (those weren’t partitioned either). I explained that in another reply above.

However, such a stick will probably not be recognized by certain other operating systems. This can lead to damage.
 
Top