UFS Best way to recover (very) recently deleted files on UFS?

[RevennaFox]

I have a situation where a user accidentally deleted all of the photos in a directory and its sub-directories, ironically to save some space in preparation for a backup. (Helpful hint: Wise Duplicate File Remover apparently removes the originals along with the duplicates). The photos in question are stored on a FreeBSD server that the user was accessing over Samba as a network drive from Windows.

The first thing I did was immediately unmount the drive, remount as read only, and take a full image with dd. What do we know? There were probably somewhere around 2,000-5,000 pictures in the deleted directories, we don't have the exact file names, but most of them were taken on an iPhone, a Panasonic Lumix, and a Kyocera phone, so all of the filenames were in the format IMG_xxxx.JPG, P10xxxxx.JPG, and KIMGxxxx.jpg respectively.

So, right now I'm running a pass with recoverjpeg, which is in the ports collection. It's taking forever because it's a 1TB disk and there was a lot of other stuff on there. It's a little over halfway done and it's found over 77,000 files. Not sure how many are duplicates, but the same developer that created recoverjpeg also has a tool to deal with that. From a cursory glance it looks like most of the deleted files are being recovered, but with some issues. Most of the recovered images have very specific defects: either a horizontal line about a fourth of the way down the image, or multiple images weirdly mosaiced together. None of the images have their original filename or attributes, but most still have their EXIF data.



Tomorrow I'm going to take a crack at it with Sleuthkit Autopsy, which is actually a digital forensics tool, but its PhotoRec Carver module looks promising. I'll report back here on how well that works (or doesn't).

If you have any ideas on how to best recover deleted files from a UFS filesystem, please do feel free to post it. It would be excellent if there was a way to recover just the files that were deleted from that one directory tree, and even better if there was a way to do that while preserving the original attributes like the filenames and date modified. Both free and paid software suggestions are welcome.

 

[SirDice]

The photos in question are stored on a FreeBSD server that the user was accessing over Samba as a network drive from Windows.

There are two types of people in the world, those who diligently backup and those that haven't lost anything yet. Besides recovery I hope you're also planning some sort of backup strategy to prevent future disasters. Because these things can and will happen.

Anyway, give sysutils/testdisk and photorec(8) a go. But don't get your hopes up.

 

[Charlie_]

sysutils/magicrescue

Magic Rescue scans a block device for file types it knows how to recover and
calls an external program to extract them. It looks at "magic bytes" in file
contents, so it can be used both as an undelete utility and for recovering a
corrupted drive or partition. As long as the file data is there, it will
find it.

It works on any file system, but on very fragmented file systems it can only
recover the first chunk of each file. Practical experience shows, however, that
chunks of 30-50MB are not uncommon.

 

[RevennaFox]

I played around with it in Autopsy a little today and it managed to find some deleted files, none of the ones the user was missing though. Weirdly, it did find a ton of much older files, from around 2017. The ones I'm looking for are from 2019-2023. I'm not really familiar with this program yet, so I could be doing it wrong.

This evening I'll give the ones you guys suggested a try.

There are two types of people in the world, those who diligently backup and those that haven't lost anything yet. Besides recovery I hope you're also planning some sort of backup strategy to prevent future disasters. Because these things can and will happen.

Oh lord tell me about it. I'm sure if you've ever worked as a consultant you know users don't ever listen to IT people. This particular customer thought routine backups were too expensive, but a used backup drive and handful of tapes would have cost a lot less than what I'm going to have to charge them for this amount of work.

 

[SirDice]

Weirdly, it did find a ton of much older files, from around 2017. The ones I'm looking for are from 2019-2023. I'm not really familiar with this program yet, so I could be doing it wrong.

Careful what you might uncover. Certain files could have been deleted for, ehm, dodgy reasons. And some deleted files have a tendency to linger around a lot longer than you might expect. That's how a lot of criminals get convicted too.

This particular customer thought routine backups were too expensive,

There's still no hardware capable of dealing with human stupidity

 

[cy@]

There are two types of people in the world, those who diligently backup and those that haven't lost anything yet. Besides recovery I hope you're also planning some sort of backup strategy to prevent future disasters. Because these things can and will happen.

Anyway, give sysutils/testdisk and photorec(8) a go. But don't get your hopes up.

Unlike FAT filesystem, one cannot "undelete" files on virtually any other filesystem except for maybe NTFS. The reason for this is that FAT filesystem maintains file allocation table pointers (I use the term pointers liberally) whereas UFS, EXT* and ZFS maintain actual disk addresses in inode structures. Once a file's inode is gone there is no "undeleting" files.

We've been spoiled by the simplicity of FAT filesystems, which only delete the pointer to the first cluster, while the file allocation table still maintains which clusters and their order belonged (past tense) to a deleted file. Back in the MS-DOS days if no writes were performed one could reestablish the pointer to the first cluster and the file would magically reappear. Again, UFS and other filesystems have no such simplistic construct. Once a file is deleted, it's gone.

One might be able to visually identify bits and pieces of a file using some kind of disk scanning tool but what if you stitched the wrong extents "back together?" It's risky and likely to fail.

I have stitched files back together on IBM mainframe VTOC disks. But in that case I restored a copy of the disk to free disk and examined the differences, using that to stitch files back together. In that case I did manage to recover three VSAM catalogues but unfortunately could not stitch a database back together due to I not knowing the internal structure of the database files. It's hit and miss.

 

[_martin]

one cannot "undelete" files on virtually any other filesystem except for maybe NTFS.

You can. If by "undelete" we mean attempt to find inode and follow its structures to recover a file. Inode's metadata is being removed, data itself stays there. Deep analysis/heuristics with some magic and luck is needed. It's not 100% but better than nothing. And it's definitely not impossible.

I was saved by ext4magic many times before. OP here is also able to recover some files.

 

[cy@]

You can. If by "undelete" we mean attempt to find inode and follow its structures to recover a file. Inode's metadata is being removed, data itself stays there. Deep analysis/heuristics with some magic and luck is needed. It's not 100% but better than nothing. And it's definitely not impossible.

I was saved by ext4magic many times before. OP here is also able to recover some files.

Good luck with that. I as a mainframe kernel developer spent an all nighter stitching files together using a binary editor (IMASPZAP: super zap). Modern tools that understand the underlying fs may help but one needs to have a solid background to navigate through that maze.