Solved ZFS, Shells and Filenames with "#"

Datapanic

Datapanic

Hi everyone! First post, please be nice!

For a few months, I've had problems with some of my older archived BBS files, mainly games in .zip format with filenames that start with a "#" character disappearing (#1DUKE.ZIP, for example). I traced it down to the ZFS scrub which I set to run once every 7 days. I found that the # character in a filename is not allowed for ZFS file systems. But, after some experimenting, I can create files that have "#" in them with SMB shares on ZFS file systems, I can create files with "#" in the filename logged in as root using csh, but I can't create with "#" in the name logged in as root using bash.

Fortunately, I have backups of all those old games and didn't loose any data, but I wonder if there is a way to prevent super user or anyone else to create files with illegal characters on a ZFS file system.

I only have 37 of these files with "#" in the name and could rename them along with reconfiguring the BBS software (runs on a vm and accessible via telnet to a terminal server) to know the new filenames. But, shouldn't ZFS not allow illegal characters in a filename no matter what or how they are created?

-Dan
 
  • Thanks
Reactions: Oko
As far as I know the only limitation ZFS has regarding filenames is the use of '/' (directory delimiter) and '@' (snapshot delimiter). It shouldn't have an issue with '#'.

Code: 
dice@armitage:~/test % ls -al
total 9
drwxr-xr-x  2 dice  dice   2 May 28 08:20 .
drwxr-xr-x  8 dice  dice  24 Feb 24 17:20 ..
dice@armitage:~/test % touch test#name
dice@armitage:~/test % ls -al
total 10
drwxr-xr-x  2 dice  dice   3 May 28 08:20 .
drwxr-xr-x  8 dice  dice  24 Feb 24 17:20 ..
-rw-r--r--  1 dice  dice   0 May 28 08:20 test#name
dice@armitage:~/test % echo $SHELL
/bin/tcsh
dice@armitage:~/test % sh
$ pwd
/home/dice/test
$ ls -al
total 10
drwxr-xr-x  2 dice  dice   3 May 28 08:20 .
drwxr-xr-x  8 dice  dice  24 Feb 24 17:20 ..
-rw-r--r--  1 dice  dice   0 May 28 08:20 test#name
$ touch test2#name
$ ls -al
total 10
drwxr-xr-x  2 dice  dice   4 May 28 08:20 .
drwxr-xr-x  8 dice  dice  24 Feb 24 17:20 ..
-rw-r--r--  1 dice  dice   0 May 28 08:20 test#name
-rw-r--r--  1 dice  dice   0 May 28 08:20 test2#name
 
This might be an issue with bash, at least that would be my first guess.
 
'#' isn't illegal, but it's a special character to the shell (bash or other bourne-type shell) . Using bash you have to escape a leading hash in filenames or the rest of the line is taken as a comment.

Code: 
# touch #one
usage: touch [-A [-][[hh]mm]SS] [-achm] [-r file] [-t [[CC]YY]MMDDhhmm[.SS]]
       [-d YYYY-MM-DDThh:mm:SS[.frac][tz]] file ...
# touch \#one
# touch f#two
# ls -l
total 1
-rw-r--r--  1 root  wheel  0 May 28 10:18 #one
-rw-r--r--  1 root  wheel  0 May 28 10:18 f#two
 
The file system or shell will allow a "#" in the file name, but when a ZFS scrub is executed, the file will be removed.
 
Here's what I had in my daily run after the scrub:
Code: 
Cleaning disks:

/mnt/VD0/MISC/CDOR03/cdorbbs/CDORBBS/RBBS/DOWNLOAD/G/#1bio.zip
/mnt/VD0/MISC/CDOR03/cdorbbs/CDORBBS/RBBS/DOWNLOAD/G/#1dark.zip
/mnt/VD0/MISC/CDOR03/cdorbbs/CDORBBS/RBBS/DOWNLOAD/G/#1duke.zip
My zfs scrub was set to run weekly in periodic.conf:
Code: 
# 800.scrub-zfs
daily_scrub_zfs_enable="YES"
daily_scrub_zfs_pools=""  # empty string selects all pools
daily_scrub_zfs_default_threshold="7"  # days between scrubs
#daily_scrub_zfs_${poolname}_threshold="35"  # pool specific threshold
 
It's cleaned by periodic(8). Here are the I believe relevant defaults from /etc/defaults/periodic.conf:
Code: 
# 100.clean-disks
daily_clean_disks_enable="NO"                           # Delete files daily
daily_clean_disks_files="[#,]* .#* a.out *.core *.CKP .emacs_[0-9]*"
Notice anything? It will delete files starting with #. Did you set daily_clean_disks_enable="YES" in /etc/periodic.conf? If so you might want to consider setting daily_clean_disks_files to some other value.
 
As far as I know a scrub looks at data blocks and tests if they're still valid. It doesn't care what the files are called.
 
Personally I would just disable the clean_disks function. To me it seems a bit of a waste to scan the entire pool every week just on the off-chance you may have a few unneeded files around using up a couple of MB of space.

As mentioned scrub runs above the filesystem level on the pool itself, and checks each ZFS record directly. If it does find errors that it can't fix, it will at that point look up and report the associated file. It will never actually remove anything. If scrub does find errors, it's up to the admin to remove the damaged files, zpool clear the errors, then re-scrub (hopefully having it now complete with no errors).
 
It also doesn't make a lot of sense for a filesystem to accept 'illegal' characters and after a while silently remove those files during a check. I've never encountered a filesystem that did that.
 
tobik said:
It's cleaned by periodic(8). Here are the I believe relevant defaults from /etc/defaults/periodic.conf:
Code: 
# 100.clean-disks
daily_clean_disks_enable="NO"                           # Delete files daily
daily_clean_disks_files="[#,]* .#* a.out *.core *.CKP .emacs_[0-9]*"
Notice anything? It will delete files starting with #. Did you set daily_clean_disks_enable="YES" in /etc/periodic.conf? If so you might want to consider setting daily_clean_disks_files to some other value.
Click to expand...

That's exactly what caused the files to be removed. I did change daily_clean_disks_enable to "YES" at some point in time. I'm disabling it for now.
 
Ahh, good old emacs backup files. They start with "#", don't they? So that is why this cleaner exists.
 
  • Thanks
Reactions: Oko
You must log in or register to reply here.
Back
Top