Cannot get core of jailed httpd

[Bartlomiej Butyn]

Hello,
I have issues with httpd in jail. Sometimes it happen that it stuck in infinite loop takeing 100% of processor. I want to examine coredump of such process to find out where is a problem. So I wrote a script that detects such a process and kills it, I get:

root@vh5:~ # dmesg | grep httpd | tail -3
pid 93785 (httpd), uid 1049: exited on signal 3
pid 3018 (httpd), uid 1049: exited on signal 3
pid 3306 (httpd), uid 1049: exited on signal 3

but there is no core file left (I also tried to kill with SIGSEGV with no joy). If it matters I send kill signal from outside a jail.

My sysctls:

root@vh5:~ # sysctl -a | grep core
kern.corefile: /tmp/core.%N.%U.%P.core
kern.coredump_devctl: 0
kern.nodump_coredump: 0
kern.coredump: 1
kern.capmode_coredump: 0
kern.sugid_coredump: 1
kern.coredump_pack_vmmapinfo: 1
kern.coredump_pack_fileinfo: 1
debug.ncores: 5
debug.elf32_legacy_coredump: 0
debug.elf64_legacy_coredump: 0
hw.ixl.core_debug_mask: 0

I also have :coredumpsize=unlimited:\ in default class in /etc/login.conf

What else should I set up to get core file?

 

[Snurg]

Maybe use gdb and use option that httpd doesnt daemonize, and then hit ctrl-c when it "hangs" to find where this occurs?

 

[leebrown66]

gcore(1) will let you grab a live coredump, maybe try that instead of killing it.

I would also suggest killing it from within the Jail and check if both the host and jail can coredump (I don't know, but I would imagine hope that host coredump settings trump jail settings).

 

[SirDice]

truss(1) can also be quite useful, so you can see what it's doing without having to kill it.

truss -p <pid of process>

 

[Bartlomiej Butyn]

... use option that httpd doesnt daemonize, and then hit ctrl-c ...

I cannot do that.

truss(1) can also be quite useful, so you can see what it's doing without having to kill it.

Yes, but truss shows nothing. Like in 'while (true) do nothing()' loop.

gcore(1) will let you grab a live coredump, maybe try that instead of killing it.

Thank you for gcore(1). I didn't knew that tool.
I changed my watchdog to do gcore(1) before kill(1) and I'm waiting for results.
BTW gcore ignores login.conf entry coredumpsize

I would also suggest killing it from within the Jail and check if both the host and jail can coredump (I don't know, but I would imagine hope that host coredump settings trump jail settings).

And that sugestion allowed me to find out what probably happened.
For others having same issue:
In host /etc/login.conf I had unlimited coredumpsize, but in jailed /etc/login.conf httpd was running under other login class, and that class had coredumpsize set to 0 (in jail only).