Unable to delete some files

If you already asked on irc and didn't get much attention I'd suggest opening the PR or asking on mailing list (though I'd personally prefer the former).

With only commands from base you could check the inum of the .viminfo by ls -lai /root/ (leftmost column is the inode) and use it in conjunction with find: find /root -xdev -type f -inum $inum -exec cat {} \; where $inum is the actual inum of the .viminfo. Most likely you get the same error as before.

While it's annoying to be in this situation there could be a bug that could do more harm next time.

Looking as a specific dnode (zfs inode): zdb -AA -ddddd $zfs_fileset $dnode. For example:
Code:
tbsd01:(~)# ls -lai .cshrc
131270 -rw-r--r--  1 root  wheel  1089 Oct 23  2020 .cshrc
tbsd01:(~)#

tbsd01:(~)# df -m .
Filesystem      1M-blocks Used Avail Capacity  Mounted on
rpool/ROOT/13.0     23759 4672 19087    20%    /
tbsd01:(~)#

tbsd01:(~)# zdb -AA -ddddd rpool/ROOT/13.0 131270
Dataset rpool/ROOT/13.0 [ZPL], ID 134, cr_txg 2867566, 4.56G, 236469 objects, rootbp DVA[0]=<0:2147d7000:1000> DVA[1]=<0:5d1772000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique double size=800L/800P birth=7959118L/7959118P fill=236469 cksum=bcf813f93:e50cb1bbcab:abc520bd56894:5dfc62710b949ad

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
    131270    1   128K  1.50K     4K     512  1.50K  100.00  ZFS plain file
                                               168   bonus  System attributes
    dnode flags: USED_BYTES USERUSED_ACCOUNTED
    dnode maxblkid: 0
    path    /root/.cshrc
    uid     0
    gid     0
    atime    Wed Apr 14 22:10:25 2021
    mtime    Fri Oct 23 07:32:42 2020
    ctime    Wed Apr 14 22:10:25 2021
    crtime    Fri Oct 23 07:32:42 2020
    gen    2867815
    mode    100644
    size    1089
    parent    399
    links    1
    pflags    40800000004
Indirect blocks:
               0 L0 0:146422000:1000 600L/600P F=1 B=2867815/2867815 cksum=571c182773:53aa538d94f5:2c446ce14b2b2b:10d0419ba49cd37e

        segment [0000000000000000, 0000000000000600) size 1.50K

tbsd01:(~)#
If I were you I'd really like to hear devs opinion on the situation.
 
Code:
root@XXXX:~ # file ./.viminfo
./.viminfo: cannot open `./.viminfo' (No such file or directory), ASCII text
root@XXXX:~ #

When I press TAB while typing the command it does autocomplete .viminfo and the output is as above.

Thanks. Here, with my intentionally mis-named file:

Code:
% ls -hiln
total 1
48648 -rw-------  1 1002  1002   933B  4 Feb 23:29 ?.viminfo
47897 -rw-r--r--  1 1002  1002   6.3K  4 Feb 08:41 out.bin
48035 -rw-r--r--  1 1002  1002   670B  4 Feb 23:19 out.zip
% ls -hiln *.viminfo
48648 -rw-------  1 1002  1002   933B  4 Feb 23:29 ?.viminfo
% file
^K.viminfo  out.bin     out.zip  
% file .viminfo
.viminfo: cannot open `.viminfo' (No such file or directory)
% sh
$ find . -maxdepth 1 -inum 48648 -exec mv {} .viminfo \;
$ file .viminfo
.viminfo: ASCII text
$ exit

potzilov please, what's the serial number of the file in your case?

ls -hiln *.viminfo

Postscript

Deleted a post from page 1. Now, I remember using -ddddd years ago, maybe with Zevo.

Code:
root@mowa219-gjp4-8570p-freebsd:~ # zdb -ddddd august/usr/home 48648
Dataset august/usr/home [ZPL], ID 287, cr_txg 14, 139G, 860330 objects, rootbp DVA[0]=<0:582aa3a000:1000> DVA[1]=<0:5e2871f000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique double size=1000L/1000P birth=2282432L/2282432P fill=860330 cksum=10966ad5ae:2e00b3d802b7:440d9e25e4c8ac:470a35eaa17697da

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
     48648    1   128K     1K     4K     512     1K  100.00  ZFS plain file
                                               168   bonus  System attributes
        dnode flags: USED_BYTES USERUSED_ACCOUNTED USEROBJUSED_ACCOUNTED
        dnode maxblkid: 0
        path    /grahamperrin/Documents/IT/ZFS/OpenZFS/84000/
                                                             .viminfo
        uid     1002
        gid     1002
        atime   Fri Feb  4 23:29:39 2022
        mtime   Fri Feb  4 23:29:39 2022
        ctime   Fri Feb  4 23:45:27 2022
        crtime  Fri Feb  4 23:29:39 2022
        gen     2230775
        mode    100600
        size    933
        parent  48163
        links   1
        pflags  40800000004
Indirect blocks:
               0 L0 0:32c5458000:1000 400L/400P F=1 B=2230775/2230775 cksum=43e094ba5c:2669cb7d7774:d3e9e3b55bc75:3644d9eb2782884

                segment [0000000000000000, 0000000000000400) size    1K

root@mowa219-gjp4-8570p-freebsd:~ #
 
Last edited:
grahamperrin out.bin is just an arbitrary name for an output covacat chose in his post when he was using readdir(3) to get the files from a directory. From there you can see he's saving dirent(5) structure to the file.

Deeper question is, as I'm curious myself, how can this happen on a COW FS (this being a ghost entry in directory referring to a non-existent dnode and this being an observation from forum post, not a fact).
 
Thanks, I mean, what command did you (or can you) run on the provided binary, to 'see' its content?

Incidentally, I am aware that my artificial example lacks the ASCII text aspect when file(1) does not find the file.
 
The out.bin file is nothing but a binary dump of all the "struct dirent" that are returned by a series of readdir() calls. I showed what one can find in them above: Each contains the name of a thing in the directory (where thing = typically file or subdirectory, with special cases for bizarre things like devices and FIFOs, and with "." and ".." being like subdirectories too), the type of the thing, the inode number of the thing, and that's pretty much it. The way we did this here is that the OP and Covecat ran out.bin through hexdump -Cv, and then I looked at the hexdump; instructions are above.

Word of warning: If you do "man dirent", it shows you an ancient version of the dirent structure (which has two 8-bit numbers, and then a 32-bit pad word). The version of dirent that is really in use can be found by reading /usr/include/sys/dirent.h; in there the file type and file name length are de-facto 16-bit numbers (expressed in the header as an 8-bit number and an 8-bit pad byte).
 
potzilov please, what's the serial number of the file in your case?

ls -hiln *.viminfo

Not sure that a "ls -il" command will return that, since it may run into a brick wall trying to do a stat() call on the file, since the inode does not seem to exist. A "ls -i" command might work, or might not (don't know how ls is implemented). Anyway, I posted the file's inode number above (by reading it from the hexdump of the directory content), and it is 852495.

And as martin already wrote above: I don't think that an object with that inode number can be found, since the problem does seem to be that it does not exist. But to verify that, one could do "find / -inum 852495 -print", but my hunch is that it will return nothing. Finding the inode or verifying its absence will require going into ZFS internals, perhaps looking at older version of the inode storage (in older blocks, remember ZFS doesn't overwrite things on disk for as long as possible, it appends internally).
 
Thanks,

… I showed what one can find in them above: …

Your Code fragments in <https://forums.freebsd.org/posts/554604> above were taken from output from what command, or utility?

Sorry for leaning towards ELI5, but this is where the dyslexic side of me really needs to understand from the outset 'cause if I learn wrong, I might never properly un-learn.

(Decades ago, I used HexEdit on Macs. Probably laughable now. Not that I require a GUI now, just, I need a pointer on what best to use to 'see' the content of the .bin file that was provided (zipped) on page 1.)
 
Your Code fragments in <https://forums.freebsd.org/posts/554604> above were taken from output from what command, or utility?
Take the a.c program that Covecat posted, compile it into /tmp/ad. Run /tmp/ad (with the correct argument), save the output in a file called out.bin. Then run "hexdump -Cv out.bin", and you get the hex dump that I cut and pasted from. Or eliminate the middleman of saving it into a file, and run "/tmp/ad /root | hexdump -Cv", and the hexdump is right there.

Last night at midnight I was thinking that I should take Covecat's nice program, and make it prettier, so instead of just dumping the data in binary, it formats it nicely. By that time it was after midnight, so I didn't. And tonight I have lots of work to do.
 
Hi guys,

Recently I've discovered that there are some files that I'm unable to delete. See the screenshot below from mc (in /root directory).

View attachment 12836

A simple ls does show the file:

View attachment 12839

ls with other switches, e.g. ls -la does not show the file.

It's difficult for me to say what caused this, perhaps a power outage. What I do see is that there are additional files which are shown in mc as starting with the question mark (if it is indeed a question mark and not some other random symbol being shown as a question mark).

Another, more worrying example of the same is:

View attachment 12837

This leads to cron not being able to start. The same is true for wlan0 and some other files and daemons.

My system is
Code:
 FreeBSD XXXX 13.0-RELEASE-p6 FreeBSD 13.0-RELEASE-p6 #0 releng/13.0-b0c8bc5d9: Wed Jan 26 13:24:54 CET 2022     xxxx@xxxx:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64

Filesystem is ZFS on SSD and zfs scrub did not find any errors.

I did try the following suggestions from #freebsd on Libera.chat but without success:
  • Regular rm \?.viminfo - No such file or directory
  • Moving everything out of /root directory delete the directory with rm -rf /root - Directory not empty
  • Using rm -fv -- \?.viminfo
  • Using ls -alo - the file does not appear in the output
  • Using printf "%s" *.viminfo gives "no matches found *.viminfo"
  • Tried to find inode of the file in order to delete it that way - wasn't able to find the inode.
So now I'm stuck and hope you guys are able to help me.

Hi guys,

Recently I've discovered that there are some files that I'm unable to delete. See the screenshot below from mc (in /root directory).

View attachment 12836

Did you try
Code:
rm -i \.*
? Had the same issue with a directory that contained unicode characters.
 
Can someone find a ZDB command for that, please?
Sure:
zdb -dddd dataset inum
E.g.:
Code:
# zdb -dddd pond/usr/devel 5876111
Dataset pond/usr/devel [ZPL], ID 13311, cr_txg 7374, 43.1G, 3248754 objects, rootbp DVA[0]=<0:224007d4000:200> DVA[1]=<0:1ac03fc7400:200> [L0 DMU objset] fletcher4 lz4 unencrypted LE contiguous unique double size=800L/200P birth=81735298L/81735298P fill=3248754 cksum=11ebe42940:5e14a8496a6:10daa419cd79d:22ac6a867e6de9

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
   5876111    1   128K    59K    26K     512    59K  100.00  ZFS plain file
                                               168   bonus  System attributes
        dnode flags: USED_BYTES USERUSED_ACCOUNTED
        dnode maxblkid: 0
        uid     1001
        gid     0
        atime   Wed Feb  3 20:54:51 2021
        mtime   Wed Feb  3 20:54:51 2021
        ctime   Wed Feb  3 20:54:51 2021
        crtime  Wed Feb  3 20:54:51 2021
        gen     74765151
        mode    100644
        size    60219
        parent  5875861
        links   1
        pflags  40800000004

You can vary a number of d-s to get more or less detailed information.
 
My own case, hopefully less peculiar. I plan to remove a directory, but not until I understand more of what's below.

For a strangely named empty directory at the root of a file system, an inode 32904 with two (2) links, is it strange that find(1) finds only one file?

find / -xdev -inum 32904 -ls

Code:
root@mowa219-gjp4-8570p-freebsd:~ # zdb -ddddd august/ROOT/n253343-9835900cb95-b 32904
Dataset august/ROOT/n253343-9835900cb95-b [ZPL], ID 11281, cr_txg 2542568, 49.6G, 2701309 objects, rootbp DVA[0]=<0:d92d5dc000:1000> DVA[1]=<0:c0157c7000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique double size=1000L/1000P birth=2573771L/2573771P fill=2701309 cksum=1191d9bb3f:329f1aa61da7:4e165d55c16d65:5501ef751bf3813b

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
     32904    1   128K    512      0     512    512  100.00  ZFS directory
                                               168   bonus  System attributes
        dnode flags: USED_BYTES USERUSED_ACCOUNTED USEROBJUSED_ACCOUNTED
        dnode maxblkid: 0
        path    /
        uid     0
        gid     0
        atime   Tue Feb 23 23:55:00 2021
        mtime   Tue Feb 23 23:55:00 2021
        ctime   Tue Feb 23 23:55:00 2021
        crtime  Tue Feb 23 23:55:00 2021
        gen     11323647
        mode    41777
        size    2
        parent  4
        links   2
        pflags  40800000144
        microzap: 512 bytes, 0 entries

Indirect blocks:
               0 L0 EMBEDDED et=0 200L/1bP B=2053

                segment [0000000000000000, 0000000000000200) size   512

root@mowa219-gjp4-8570p-freebsd:~ #

Code:
% ls -hiln / | head -n 3
total 1
  32904 drwxrwxrwt   2 0  0     2B 23 Feb  2021
      4 drwxr-xr-x   2 0  0     2B 26 Feb 04:05 august
% ls -Bhiln / | head -n 3
total 1
  32904 drwxrwxrwt   2 0  0     2B 23 Feb  2021 \011\011\011
      4 drwxr-xr-x   2 0  0     2B 26 Feb 04:05 august
% ls -hiln /
total 1
  32904 drwxrwxrwt   2 0  0     2B 23 Feb  2021 ???
      4 drwxr-xr-x   2 0  0     2B 26 Feb 04:05 august
     33 drwxr-xr-x   2 0  0    46B 23 Feb 01:45 bin
      …



Cross-reference: <https://old.reddit.com/r/freebsd/comments/t210w2/-/>
 
For a strangely named empty directory at the root of a file system, an inode 32904 with two (2) links, is it strange that find(1) finds only one file?

An empty directory usually has a link count of two: One is the link from the name of the directory, the second is the link from "." inside the directory. If the directory has subdirectories, they increase the link count, because of the ".." entries inside the subdirectory.

All the rest you show is a directory with a really weird name (three times the \011 character, which is the tab character, or Control-I), and weird permissions (the 0777 is strange enough, but then add the final "t" which I think means sticky). That directory should be perfectly usable and deletable.
 
… the second is the link from "." …

Thanks!

I never guessed (but could have compared with the empty /bootpool directory, which is amongst the three other items to be removed).

Removed:

Code:
root@mowa219-gjp4-8570p-freebsd:~ # bfs -xdev -inum 32904 / -ls
    32904      1 drwxrwxrwt   2 root     wheel           2 Feb 23  2021 /
^C
root@mowa219-gjp4-8570p-freebsd:~ # bfs -xdev -inum 32904 / -rm
^C
root@mowa219-gjp4-8570p-freebsd:~ # bfs -xdev -inum 32904 / -ls
^C
root@mowa219-gjp4-8570p-freebsd:~ #
 
Back
Top