rm -rf dirname/ && several times in a row deleted all my files (?)

Avery Freeman

Avery Freeman

Hi,

I just had something weird happen and I was wondering if someone could explain why

I use FreeBSD to record TV. I have to clear it out every once in a while because I record a lot of news, so today I tried running a lot of rm -rf <dirname> && commands, but it deleted everything after running for a while.

At first, I thought it must've been a rm -rf * accidentally put in there somewhere, but I don't see it in the command chain that I ran. Here's exactly what I ran:

Code: 
rm -rf New\ Day\ With\ Alisyn\ Camerota\ and\ John\ Berman/ && rm -rf Morning\ Joe/ && rm -rf The\ Late\ Show\ With\ Stephen\ Colbert/ && rm -rf PBS\ NewsHour * && rm -rf The\ Situation\ Room\ With\ Wolf\ Blitzer/ && rm -rf CNN\ Newsroom\ * && The\ Axe\ Files\ With\ David\ Axelrod/ && rm -rf The\ Mueller\ Report\ What\ You\ Need\ to\ Know/ && rm -rf CNN\ Debate\ * && America\ Under\ Assault\ The\ Gun\ Crisis\ Cuomo\ Prime\ Time\ Town\ Hall/ && rm -rf The\ Van\ Jones\ Show/ && rm -rf S.\ E.\ Cupp\ Unfiltered/ && rm -rf Saturday\ Night\ Politics/ && rm -rf Smerconish/ && rm -rf 2020\ Democratic\ Candidates\ Debate/ && rm -rf MSNBC\ Live* && rm -rf Reliable\ Sources\ with\ Brian\ Stelter/ && rm -rf State\ of\ the\ Union\ With\ Jake\ Tapper/ && rm -rf CNN\ Democratic\ Presidential\ Debate/ && rm -rf This\ Week\ With\ George\ Stephanopoulos/ && rm -rf Washington\ Week/ && rm -rf Matter\ of\ Fact\ With\ Soledad\ O\'Brien/ && rm -rf Meet\ the\ Press/ && rm -rf Anthony\ Bourdain\ Parts\ Unknown/ && rm -rf Kasie\ DC/ && rm -rf PoliticsNation/ && rm -rf Cuomo\ Prime\ Time/ && rm -rf PBS\ NewsHour* && rm -rf Face\ the\ Nation/ && rm -rf AM\ Joy/ && rm -rf Up\ with\ David\ Gura/ && rm -rf Weekends\ With\ Alex\ Witt/ && rm -rf The\ Beat\ With\ Ari\ Melber/ && rm -rf The\ Rachel\ Maddow\ Show/ && rm -rf Andrea\ Mitchell\ Reports/ && rm -rf CNN\ Right\ Now\ With\ Brianna\ Keilar/ && rm -rf MTP\ Daily/ && rm -rf The\ Lead\ With\ Jake\ Tapper/ && rm -rf All\ In\ With\ Chris\ Hayes/ && rm -rf Deadline\ White\ House/ && rm -rf Anderson\ Cooper\ 360/ && rm -rf CNN\ Tonight\ With\ Don\ Lemon/ && rm -rf Shepard\ Smith\ Reporting/

I monitored the progress in another terminal and it ran fine, deleting single dirs at a time, until somewhere around rm -rf The\ Beat\ With\ Ari\ Melber/. I don't see anything weird about the commands, though.

Does anyone see something in that I don't? Is there another reason that might explain why everything got deleted?

The version is FreeBSD 12.0-RELEASE-p8, and the zpool is a pool+dataset shared from another fileserver through NFS to ESXi and then presented as a local drive (ESXi NFS dataset).

Thanks!
 
Code: 
rm -rf PBS\ NewsHour *

You didn't escape the space before the * so you recursively removed both directory PBS NewsHour and * (i.e. everything else in the current directory).

Code: 
rm -rf CNN\ Debate\ *

Here you got it right instead, thanks to the backslash before the last space.
 
spectrum48 said:
Code: 
rm -rf PBS\ NewsHour *

You didn't escape the space before the * so you recursively removed both directory PBS NewsHour and * (i.e. everything else in the current directory).

Code: 
rm -rf CNN\ Debate\ *

Here you got it right instead, thanks to the backslash before the last space.
Click to expand...

Good lord. I thought that might have been it. Thanks for your help.

I didn't bother to take a snapshot before I ran it and now my girlfriend is pissed I deleted all her shows. Wish me luck! :confused:
 
Avery Freeman said:
Yeah, the TV recording software just does that by default. I'm right there with you when I have control over it.
Click to expand...

A small trick:
  • create a temporary directory called delete (no spaces or other special characters);
  • move the files there (if you make a mistake you can move them back);
  • double check that all files you moved there need deletion;
  • delete the temporary directory (double check the command before hitting enter).
This way the dangerous rm -rf command is short and easy to check
 
Avery Freeman said:
Good lord. I thought that might have been it. Thanks for your help.

I didn't bother to take a snapshot before I ran it and now my girlfriend is pissed I deleted all her shows. Wish me luck! :confused:
Click to expand...
There used to be a joke about storage: "There are two kinds of people. Those who religiously perform backups, and those who haven't lost data yet."

Your sad fate gives rise to a new version of the joke: "... and those aren't single yet." Well, good luck.
 
userxbw said:
that'd remove files within dir then remove the same dir. it's a little safer.
Click to expand...
This would be safer still: rm -rf /dir. No wildcard. No need to empty the directory beforehand too, the -r takes care of that.
 
userxbw said:
rm -rf /dir/* && rmdir /dir

that'd remove files within dir then remove the same dir. it's a little safer.
Click to expand...

I don't think it's safer, as SirDice noted.

Also, it will fail if there are hidden files since they are not matched by /dir/*:

Code: 
$ mkdir /tmp/dir
$ touch /tmp/dir/foo /tmp/dir/.bar
$ rm -rf /tmp/dir/* && rmdir /tmp/dir
rmdir: /tmp/dir: Directory not empty
$ ls -a /tmp/dir
./    ../   .bar
 
If you use zsh as your login shell, I strongly recommend to set the option RM_STAR_WAIT (also make sure that RM_STAR_SILENT is not set; this is the default, however).

The effect is that each time you use rm with a wildcard pattern that consists only of a star, like rm * or rm some_dir/* (but not rm part*), the shell will wait ten seconds and ignore anything typed within that time (except for Ctrl-C), then ask for confirmation. This would have saved your day in the above case.

If you don't want to wait, you can always press the Tab key. That will expand a wildcard pattern immediately, so you see exactly which files and directories will be affected, and then you can press Enter to actually execute the command (without having to wait).

If you don't use zsh as your login shell yet, you should give it a try. It has a lot of useful features. Also, it supports most of the things bash does (some have to be enabled explicitly), so you don't have to learn a lot of new stuff when switching from bash to zsh. You can start learning zsh's features step by step.
 
Another advice, independent from your shell:
If you have important files and directories that you want to keep indefinitely (like your girlfriend's TV shows), protect them by setting the “immutable” flag:
Code: 
chflags -R uchg Valuable_Directory
If you do that, rm -rf will not be able to remove it. You'll have to clear the immutable flag first, then you can remove it.

By the way: It's a very bad habit to always use the -f flag with rm. Don't get used to that, it's not necessary. For the same reason, I think it is bad to have an alias rm="rm -i" because it makes people always use -f to override it, making things worse. rm should not be an alias; the default behavior is fine.
 
With regards to the original question, I would recommend using quotes. It's really easy to forget or miss escaping a space.

So instead of rm -rf New\ Day\ With\ Alisyn\ Camerota\ and\ John\ Berman/ use rm -rf "New Day With Alisyn Camerota and John Berman/"
 
spectrum48 said:
I don't think it's safer, as SirDice noted.

Also, it will fail if there are hidden files since they are not matched by /dir/*:

Code: 
$ mkdir /tmp/dir
$ touch /tmp/dir/foo /tmp/dir/.bar
$ rm -rf /tmp/dir/* && rmdir /tmp/dir
rmdir: /tmp/dir: Directory not empty
$ ls -a /tmp/dir
./    ../   .bar
Click to expand...
well being that it is his (my) data, one should know what one is dealing with. If one needs a more elaborate algorithm, then so be it, that is just one of them basic examples in how the command(s) can be applied.


rm -rfv /dir/* as apposed to rm -rfv /dir which as pointed out already that if spaces resided within the structure then if no precautions are taken one can run into trouble, as already pointed out prior to my post.

run a script that puts everything within a more controlled file structure (perhaps).
Code: 
baseDIr="$HOME/MyStuff"

#To put stuff into and keep it sepertated.
source1="$baseDir/data1"
source2="$baseDir/data2"
#To remove everything including the baseDir.
#the mv command too could be put to use here.
rm -rf "$baseDir"
which should only effect everything within the set baseDir

man rm (FreeBSD)
Recursively remove all files contained within the foobar directory hierarchy:

$ rm -rf foobar
Click to expand...

source:

rm

www.freebsd.org www.freebsd.org
 
olli@ said:
By the way: It's a very bad habit to always use the -f flag with rm. Don't get used to that, it's not necessary. For the same reason, I think it is bad to have an alias rm="rm -i" because it makes people always use -f to override it, making things worse. rm should not be an alias; the default behavior is fine.
Click to expand...
Agreed with the first part, but not so much with the second ... AFAIK, -i actually stands for interactive, so it's designed to be used when typing commands. Whether this should be forced by an alias is a matter of taste -- the design of rm clearly supports this by making -i and -f override each other.

Whether this is dangerous or not depends a bit on your mindset. You should always see -f as a red flag, and I personally think the wish to interactively remove a whole tree of files without being bothered about every file deserves the usage of such a flag. Sure, if you put -i in your alias just to use -f always defeats the purpose :)

I personally don't have such safety aliases on my own systems, but I wouldn't say it's generally a bad idea ... I'm often logged into other systems that have these configured system-wide, and if an rm command starts bothering me, I hit Ctrl+C, maybe double-check my command, and throw an -f in...

As for captain hindsight: Another way to avoid this data loss would have been to make use of the autocompletion feature of your shell for typing the names of the directories :)
 
Zirias said:
Agreed with the first part, but not so much with the second ... AFAIK, -i actually stands for interactive, so it's designed to be used when typing commands. Whether this should be forced by an alias is a matter of taste -- the design of rm clearly supports this by making -i and -f override each other.
Click to expand...
The problem is, once you have that alias, there is no option to restore the default behaviour. You have to type something like /bin/rm or command rm in order to circumvent the alias, but nobody does that because it's too much to type. So everybody uses -f, but that's more dangerous.
Sure, if you put -i in your alias just to use -f always defeats the purpose :)
Click to expand...
Yeah, many people do exactly that.

By the way, FreeBSD supports an option -I (upper-case “i”) that is much more user-friendly, see the rm(1) manual page. If this helps people make less (ab)use of -f, I'd suggest using an alias with this option instead.
As for captain hindsight: Another way to avoid this data loss would have been to make use of the autocompletion feature of your shell for typing the names of the directories :)
Click to expand...
Definitely. Also see my comment about zsh’s RM_STAR_WAIT above.
 
userxbw said:
rm -rfv /dir/* as apposed to rm -rfv /dir which as pointed out already that if spaces resided within the structure then if no precautions are taken one can run into trouble, as already pointed out prior to my post.
Click to expand...

No, there is absolutely no problem in file with spaces in their name within the structure. Problems can arise when explicitly typing these files on the command line since an error in escaping or quoting can lead to disaster.
 
On my Linux box, I am using following strategy:
1. Put following script (saferm.sh) in my /usr/bin/
pastebin.com

zetro - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
pastebin.com pastebin.com
2. Created a softlink in my home-dir with name "trash" pointing towards: ~/.local/share/Trash/files/
3. Created an alias: alias rm='saferm.sh'

Now, whenever I use rm command, instead of permanently deleting my files, it moves them to Trash, which can be emptied later.

I haven't checked it on my FreeBSD, but I believe it will also work there.
Thanks
 
Also remember that any "dangerous" command can be replaced with echo if you want to see what the arguments do before shooting yourself in the foot...
 
You could also use find with the appropriate matching pattern and/or other search conditions. First print then remove.

Bash: 
cd /dir/where/recordings/are
find . -type d -name <pattern>
# once OKed then
find . -type d -name <pattern> -exec rm -rf {} \;
 
spectrum48 said:
A small trick:
  • create a temporary directory called delete (no spaces or other special characters);
  • move the files there (if you make a mistake you can move them back);
  • double check that all files you moved there need deletion;
  • delete the temporary directory (double check the command before hitting enter).
This way the dangerous rm -rf command is short and easy to check
Click to expand...
They're huge files, so they take a long time to move. I can just take a snapshot like I should have in the first place.

But I will probably copy-paste all the dir names from # du -hd 1 | sort -n output instead of using any wildcards from now on ;)

It's a bunch of stuff I record regularly so I'll put it in a script...
 
_martin said:
You could also use find with the appropriate matching pattern and/or other search conditions. First print then remove.

Bash: 
cd /dir/where/recordings/are
find . -type d -name <pattern>
# once OKed then
find . -type d -name <pattern> -exec rm -rf {} \;
Click to expand...

That's a great idea! Thanks!
 
Avery Freeman said:
They're huge files, so they take a long time to move.
Click to expand...
If the destination directory is within the same filesystem, moving files takes near zero time, no matter how huge they are, because only the directory entry is moved. No actual data is moved.
I can just take a snapshot like I should have in the first place.
Click to expand...
That's a good idea.

If you use ZFS, snapshots are rather “cheap” (compared to UFS snapshots), so you might even consider creating snapshots regularly via cron, like nightly or even hourly (depending on the amount of write activity and free space), and removing them automatically after a while. That way you have a simple short-term backup without having to think about it. Remember that snapshots themselves take zero space initially because they only store data that has changed.

Slightly off-topic: The HAMMER filesystem of DragonFly BSD has even better support for fine-grained point-in-time recovery. This is really cool – Once you get used to it, you miss something like that on FreeBSD. Quote: “All non-temporary HAMMER filesystems in DragonFly by default automatically maintain 60 days worth of 1-day snapshots and 1-day worth of fine-grained (30-second) snapshots. These options can be further tuned to meet one's needs.”

By the way, contrary to popular opinion, I think that spaces in file names are perfectly fine, and also non-ASCII characters are no problem. Thanks to the features of modern shells, handling such files is not a big deal. POSIX semantics for filenames supports UTF-8, the only characters that are disallowed in filenames are the slash / and the NUL byte (0x00). The only things that – although allowed – should be avoided are non-printable characters and control codes, like newline, backspace, escape codes, zero-width space and similar.
For example, I have files like this: WALL·E – Der Letzte räumt die Erde auf (Trailer).mkv (contains three UTF-8 characters and several spaces). To watch the video, I simply type mplayer WALL[Tab][Enter].
 
You must log in or register to reply here.
Back
Top