gnupg, yubikey and openfile limits

[freezmi]

OK, so I have this 14.3 RELEASE OS that is using quarterly packages in 99% of cases.

I decided to use my old yubikey 4 nano in conjunction with gnupg in order to sign and decode files.
it was a minefield to set up, but let's jump ahead to when gpg is able to use the device.

sometimes there's a cornercase in which gpg commands that involve using the yubikey seem to hang. looking closer, here's a trace:

prodan@sunspire ~ $ truss -s1000 -o/tmp/gpg -f gpg --decrypt test.asc
gpg: encrypted with rsa4096 key, ID 0xB28F87A6E90628CD, created 2016-01-08
"email censored"
gpg: using "844E7F30" as default secret key for signing
[ long delay, see trace below ]
gpg: public key decryption failed: Timeout
gpg: decryption failed: Timeout
[ process is hung, needs kill gpg-agent ]

less /tmp/gpg

35757: fork() = 35872 (0x8c20)
35872: <new process>
35872: thr_self(0xaeb13042008) = 0 (0x0)
35872: munmap(0xaeb13063000,32768) = 0 (0x0)
35872: openat(AT_FDCWD,"/dev/null",O_RDONLY,00) = 7 (0x7)
35872: openat(AT_FDCWD,"/dev/null",O_WRONLY,00) = 8 (0x8)
35872: openat(AT_FDCWD,"/dev/null",O_WRONLY,00) = 9 (0x9)
35872: dup2(7,0) = 0 (0x0)
35872: dup2(8,1) = 1 (0x1)
35872: dup2(9,2) = 2 (0x2)
35872: getrlimit(RLIMIT_NOFILE,{ cur=940266,max=940266 }) = 0 (0x0)
35872: close(3) = 0 (0x0)
35872: close(4) = 0 (0x0)
35872: close(5) = 0 (0x0)
35872: close(6) = 0 (0x0)
35872: close(7) = 0 (0x0)
35872: close(8) = 0 (0x0)
35872: close(9) = 0 (0x0)
35872: close(10) ERR#9 'Bad file descriptor'
35872: close(11) ERR#9 'Bad file descriptor'
35872: close(12) ERR#9 'Bad file descriptor'
[..]
35872: close(716780) ERR#9 'Bad file descriptor'
[..]
35872: close(940265) ERR#9 'Bad file descriptor'
35872: execve("/usr/local/bin/gpg-agent",0xaeb13413240,0x8204fc3f0) EJUSTRETURN
35872: mmap(0x0,135168,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 1127546355712 (0x10687000000)
35872: mprotect(0x116092703000,8192,PROT_READ) = 0 (0x0)
[..]
35872: getrlimit(RLIMIT_NOFILE,{ cur=940266,max=940266 }) = 0 (0x0)
35872: fstat(0,{ mode=crw-rw-rw- ,inode=20,size=0,blksize=4096 }) = 0 (0x0)
35872: fstat(1,{ mode=crw-rw-rw- ,inode=20,size=0,blksize=4096 }) = 0 (0x0)
35872: fstat(2,{ mode=crw-rw-rw- ,inode=20,size=0,blksize=4096 }) = 0 (0x0)
35872: fstat(3,0x8208f2930) ERR#9 'Bad file descriptor'
35872: fstat(4,0x8208f2930) ERR#9 'Bad file descriptor'
[..]
35872: fstat(940264,0x8208f2930) ERR#9 'Bad file descriptor'
35872: fstat(940265,0x8208f2930) ERR#9 'Bad file descriptor'
35872: sigprocmask(SIG_UNBLOCK,0x0,{ }) = 0 (0x0)
35872: issetugid() = 0 (0x0)

this hugely long close() x 940266 and fstat() x 940266 loops multiple times, gpg decryption simply times out and then hangs.

this does not happen in Linux, because of clever use of /proc, see exechelp-posix.c:98

first of all, how come the FreeBSD default limit for open files is 940266 compared to 1024 in Linux? In my 25 years of linux administration I needed to set a slightly bigger limit (maybe 2048) twice when using some exotic daemons probably written by monkeys on crack. I'm in no way bashing the FreeBSD limit, but out of curiosity when are these many open files (per process!) required? setting 'limits -n 128' before my command cures the problem, but I'd prefer not doing that.

is there a preferred way in FreeBSD to close all open file descriptors, or get the highest number fd that is currently open by a process? mounting the fake procfs does not count.

thank you.
peter

 

[rbranco]

OK, so I have this 14.3 RELEASE OS that is using quarterly packages in 99% of cases.

I decided to use my old yubikey 4 nano in conjunction with gnupg in order to sign and decode files.
it was a minefield to set up, but let's jump ahead to when gpg is able to use the device.

sometimes there's a cornercase in which gpg commands that involve using the yubikey seem to hang. looking closer, here's a trace:



less /tmp/gpg


this hugely long close() x 940266 and fstat() x 940266 loops multiple times, gpg decryption simply times out and then hangs.

this does not happen in Linux, because of clever use of /proc, see exechelp-posix.c:98

This can be done in FreeBSD as well. I added support for /proc/self but FreeBSD may not mount /proc by default.

first of all, how come the FreeBSD default limit for open files is 940266 compared to 1024 in Linux? In my 25 years of linux administration I needed to set a slightly bigger limit (maybe 2048) twice when using some exotic daemons probably written by monkeys on crack. I'm in no way bashing the FreeBSD limit, but out of curiosity when are these many open files (per process!) required? setting 'limits -n 128' before my command cures the problem, but I'd prefer not doing that.

is there a preferred way in FreeBSD to close all open file descriptors, or get the highest number fd that is currently open by a process? mounting the fake procfs does not count.

You can call closefrom(2) or close_range(2). NetBSD has the F_CLOSEM fcntl for getting the highest numbered descriptor. They only have closefrom().

This would be the better approach.

 

[ralphbsz]

man 2 closefrom; there are other options linked in there, such as close_range

But I think the problem is worse. Why in the world is gpg opening roughly a million files, each on /dev/null? Is that a bug, or an intentional action that happens to be cheap on Linux and can be deadly slow on other OSes?

 

[freezmi]

But I think the problem is worse. Why in the world is gpg opening roughly a million files, each on /dev/null? Is that a bug, or an intentional action that happens to be cheap on Linux and can be deadly slow on other OSes?

It's doing close() and fstat() on fds that were never opened, it's merely opening a handful of files.

 

[ralphbsz]

As far as I can see, the only use of the loop over fstat is to see what is open. Which is sort of silly, could keep track of it instead. And the FreeBSD closefrom or close_range call then take care of closing everything before fork/exec. Might require an hour or two of coding though. And having the FreeBSD version diverge from upstream is tricky.