C confused by unlink(2) man page

Bill Evans at Mariposa

Bill Evans at Mariposa

According to the unlink(2) man page, if one tries to unlink() a directory, one gets the EISDIR error. But one also gets the EPERM error. Obviously only one can be true, and the one that seems to be true is EPERM; that's apparently what POSIX specifies. So are both errors listed in the man page to alert us about ... something? (As a Linux [which uses EISDIR] expatriate, I'm confused by this.)
 
Last edited by a moderator:
The FreeBSD man page for unlink is indeed confusing: It lists both EPERM and EISDIR as possible error codes for unlinking a directory (EPERM is listed first, if that matters). The Posix standard (look up 1003.1 on the web) is clear though: The correct error code to return for attempting to unlink a directory is EPERM. The Posix page for unlink never even mentions EISDIR.

I just checked, and sadly you are correct: Linux does return EISDIR. Personally, I don't care; whether Linux is 100% Posix-compliant or not is really not my biggest problem in life. A sensibly-written user program shouldn't care either, and handle any error, even those that can't happen. A (former) colleague when I used to work on supercomputers describes it this way: on a sufficiently large system, anything that's unlikely will happen all the time; anything that's impossible will happen occasionally. So if you get ENOTTY (meaning "not a typewriter", my favorite errno) when trying to delete a directory, your code needs to not freak out too badly. After all, the directory really isn't a typewriter :)
 
ralphbsz said:
A (former) colleague when I used to work on supercomputers describes it this way: on a sufficiently large system, anything that's unlikely will happen all the time; anything that's impossible will happen occasionally.
Click to expand...
Agreed. That's been pretty much my guiding principle.
ralphbsz said:
whether Linux is 100% Posix-compliant or not is really not my biggest problem in life. A sensibly-written user program shouldn't care either, and handle any error, even those that can't happen.
Click to expand...
The particular program I'm porting from Linux does indeed handle any error. The way I want it to handle unanticipated errors in this case is to come to a "screeching" (figuratively speaking) "halt" (figuratively speaking, with modern operating systems) after "printing" (figuratively speaking; I no longer have to replace TTY ribbons occasionally) the line number, the error number, and briefly what the program is trying to do. And that's just what this program did. The reason this is a reasonable response is that in a black swan situation, I want to dive in and figure out what's going on and then recompile and rerun.
ralphbsz said:
So if you get ENOTTY (meaning "not a typewriter", my favorite errno) when trying to delete a directory, your code needs to not freak out too badly. After all, the directory really isn't a typewriter :)
Click to expand...
Back in the day, we used Model 33 Teletypes with rolls of yellow paper and no lower case; watching the meaning of ENOTTY gradually change so that now it more likely than not refers to an xterm window was rather like being in a pot of gradually warming cooking water, except it was more intriguing and not really painful. (Oh. And. ASCII smilies rule. :) )
 
For us in userland, the answer lies in making our code POSIX compliant. We all know who (*coughPoetteringcough*) doesn't think much of keeping code POSIX compliant, don't we? That's not the reason I left Linux, but it's closely related.
 
Although, the level of POSIX compliance FreeBSD has can be said way more than many other OSes (make a guess) around, still, FreeBSD is not 100% POSIX compliant.

POSIX standard suggests one should use rmdir(2) instead of unlink(2) while attempting to remove a directory. Apparently, returning EISDIR from an unlink(2) call is not POSIX compliant at all (correct me if I am wrong).

With my limited familiarity with kernel data structures and macros, I interpret the code snippet from the implementation of kern_unlinkat() below as "unlink(2) returns EPERM in an attempt to remove a directory and this error code is used for POSIX compliance reasons."

Code: 
if (vp->v_type == VDIR && oldinum == 0) {
error = EPERM;      /* POSIX */
Bill Evans at Mariposa, have you tested the return value and see EISDIR is actually returned at all?
 
fnoyanisi said:
Bill Evans at Mariposa, have you tested the return value and see EISDIR is actually returned at all?
Click to expand...

My original code running under Linux, upon wanting to remove something which was either a regular file or a directory, simply tried an unlink(2); upon success, it was done. Upon getting EISDIR, it did an rmdir(2) (and tested the result, of course). Under Linux, that worked smoothly; the directory got removed via the rmdir(2). But if the original unlink(2) failed with some error other than EISDIR, the program would die after displaying the error code; this case never happened under Linux. Under FreeBSD, this is exactly what happened when the program tried to unlink(2) a directory; it died, displaying the EPERM error code.

To make the code more portable (runnable under both operating systems), I now check whether the object is a directory, and then do unlink(2) or rmdir(2), as appropriate.

I don't know whether that answers your question.
 
Bill Evans at Mariposa said:
To make the code more portable (runnable under both operating systems), I now check whether the object is a directory, and then do unlink(2) or rmdir(2), as appropriate.
Click to expand...
I think this is the most portable/POSIX way of doing it (I would assume you used stat(2) or lstat(2) and then decide which system call to use).

From what you mentioned, I understand that EISDIR is not returned by unlink(2) at all. I will see if I can get EISDIR returned in some-way, otherwise we should get the unlink(2) manpage updated. With its current form
  • It is misleading
  • not POSIX compliant
Bill Evans at Mariposa said:
My original code running under Linux
Click to expand...
By the way, I think this is called "linuxism" and leads people to write less portable code (no offence here, just what the case is)
 
Bill Evans at Mariposa said:
To make the code more portable (runnable under both operating systems), I now check whether the object is a directory, and then do unlink(2) or rmdir(2), as appropriate.
Click to expand...
And that implementation has a race condition window. Imagine that a name "foo" exists, and at the time you do the stat(2) call, it is a file. Then you call unlink(2). But in the meantime, behind your back, someone has removed that foo and created a new foo, which is at a directory. The unlink call will fail.

By the way, I don't have a concrete suggestion for improving on this. You could retry in case of failure ... but that only makes the problem less likely (a sufficiently byzantine adversary could foil your retry attempts). You could go with the pythonic EAFP way (easier to ask forgiveness than permission): just try the unlink, and if it fails, try rmdir. The problem with that is that there is no unique error code that is returned by unlink which clearly indicates that the problem is that the object is a directory ... EPERM can also be caused by other things. Plus that suggestion also has a race condition window. You could use locking, but you'd have to lock the parent directory to prevent the byzantine attacks, and with multiple instances of your program running, there could be deadlocks.

And this is yet another example for why I think that Posix is a bad thing. Not because I hate standardization, on the contrary. But because Posix had to simply standardize the existing Unix interfaces, which in many aspects are just very badly designed. Error handling and absence of reasonable atomic operations are among my bigger complaints.
 
I have submitted a bug report to remove the statement about EISDIR from unlink(2) manual page. You can refer bug report 221331 for more details.

Another thing to note is that according to a more up to date version of POSIX standards page for unlink(), returning EISDIR is LSB only and does not comply with POSIX.

Thanks Bill Evans at Mariposa
Code: 
<fnoyanisi> hi there. the unlink(2) man page states EISDIR returned if you attempt to delete a directory. The same man page also says EPERM is returned.
<fnoyanisi> If I am not mistaken, unlink(2) calls kern_unlinkat(), which does not return EISDIR in any case
<fnoyanisi> only EPERM is returned if the vnode is a directory
<fnoyanisi> http://bxr.su/FreeBSD/sys/kern/vfs_syscalls.c#1821
<kaktus> good catch, can you produce patch for unlink.2 and fill the bug report on bugs.freebsd.org?
<fnoyanisi> thanks. in fact, another user on the forum spotted this, I just digged a little bit
<fnoyanisi> https://forums.freebsd.org/threads/61901/
<fnoyanisi> I will file a bug report
<kaktus> thanks
<fnoyanisi> kaktus : I think unlink.2 should not mention EISDIR at all
<kaktus> after a quick look at kern_unlinkat i thinks so
<kaktus> didn't check all the vfs_ calls that can produce the error
<kaktus> but anyway, there should be one and only one error code for given problem
<fnoyanisi> here is the bug report
<fnoyanisi> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221331
<fnoyanisi> with the relevant patch attached to it
 
fnoyanisi said:
By the way, I think this is called "linuxism" and leads people to write less portable code (no offence here, just what the case is)
Click to expand...
... and no offense taken. In the migration from Linux, I spend a significant amount of time correcting code written (a) when I was a much less mature coder, and (b) when it seemed a safe assumption that I'd be with Linux.
 
Several decades ago, when people still used printed documentation (remember VMS and the blue wall, orange wall and gray wall?), I was working on software that needed to be run on both Linux and HP-UX (and we had hopes for SunOS and AIX too). I decided that instead of using the vendor documentation (which in the case of Linux meant: read the source code), I would buy the O'Reilly Posix books and use them as a guide. So I got the Posix.1 book, and Bill Gallmeister's Posix.4 book (for threads, locks, and real-time stuff like async IO). By the way, Bill is a great guy, fellow high school band parent, engaged in the community, but I digress. Any way, I developed my software following the Posix standard "to the letter". On HP-UX and the other commercial OSes this worked really well, and made portability a snap (the only remaining problems were makefile stuff and directory names, stuff like /usr/lpp versus /opt). On Linux, it didn't work. Even though Linux pretended to be Posix compliant (and had all the header files, and you could set all the compiler defines), stuff didn't actually work. My favorite example: RedHat had implemented the aio... calls in order to be Posix compliant, but they were actually not the slightest bit asynchronous, and were simply executed in the caller's thread synchronously in the foreground. I did some digging, and the story is that some software engineer at RedHat working on libc and librt got into a spat with a kernel developer about the correct interfaces for async IO (this was the time when the kernel was still developed by beer-drinking college students, this was around kernel version 2.0 and 2.2), so they decided to "fake" async IO in userspace, with threads. And because multi-threading wasn't working well yet (things tended to lock up when you had more than a dozen threads), the default number of worker threads for async IO was 1. That kind of problem can really only be dissolved in alcohol.
 
ralphbsz said:
RedHat had implemented the aio... calls in order to be Posix compliant, but they were actually not the slightest bit asynchronous, and were simply executed in the caller's thread synchronously in the foreground.
Click to expand...
This, of course, is criminal.
ralphbsz said:
That kind of problem can really only be dissolved in alcohol.
Click to expand...
Agreed.

As long as we're talking about paper documentation and POSIX and threads, two books covering POSIX threads are wildly different, and it's the only time I've been disappointed with an O'Reilly book. That book, Pthreads Programming by Nichols, Buttlar & Farrell, does cover all the wrinkles. But there's more to the subject than that. The Addison-Wesley book, Programming with POSIX Threads by David R. Butenhof, not only gives you all the wrinkles, but pulls you deeply into the subject and makes you think about what's going on, so you can avoid the traps. (As I'm sure you're aware, trying to find a bug which reflects a shallow understanding of threads can be a gargantuan task.) The O'Reilly book is adequate; because it's only adequate about POSIX threads, it's outrageously dangerous. It's like learning how to shoot a gun just by being able to take it apart, clean it, and put it back together again. The O'Reilly book scares me.
 
ralphbsz said:
And because multi-threading wasn't working well yet (things tended to lock up when you had more than a dozen threads), the default number of worker threads for async IO was 1.That kind of problem can really only be dissolved in alcohol.
Click to expand...
This is a good one :)
 
I think I have one of those Pthreads books in a bookshelf somewhere, from 15 years ago. Don't remember which one. The difference you describe is the difference between a manual and a guide; both have value.

On the other hand, writing parallel code is hard, and requires being very careful about things like locking, scheduling, IO, and so on. Multiple threads is the most common example, although distributed systems such as MPI have the same issues. That stuff is just difficult, requires exquisite care and attention to detail, and lots of training. And your gun example is very apt: Knowing how to disassemble and lubricate a gun neither teaches you how to be safe with it, nor how to shoot accurately. Knowing how to call pthread_mutex_lock(3) doesn't teach you how to write deadlock-free code, and even less how to be productive writing code (meaning get lots of problems solved per day or work) that is both correct (deadlock-free is a good starting point!) and optimized (meaning utilizes the CPU, IO, network and memory of your machine(s) efficiently).
 
You must log in or register to reply here.
Back
Top