Build port and all its dependencies with debug info

AKuchin · Jan 19, 2023

Good day!

I have apache24 process crashing several times a day. I rebuilt it with DEBUG info and also php5 (yes, it is old and software does not work on anything but php5) with debug.

# file /usr/local/sbin/httpd
/usr/local/sbin/httpd: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.1 (1301510), FreeBSD-style, with debug_info, not stripped

So, it's not stripped.

Bow when i try to examine core i get his:

gdb /usr/local/sbin/httpd -c 00_httpd_10_18095_81467.core

Reading symbols from /usr/local/sbin/httpd...
[New LWP 100720]
Core was generated by `/usr/local/sbin/httpd -k start'.
Program terminated with signal SIGBUS, Bus error.
Sent by kill() from pid 81467 and user 18095.
#0 0x00000008277c5081 in ?? ()
(gdb) bt full
#0 0x00000008277c5081 in ?? ()
No symbol table info available.
#1 0x00000008208c3e60 in ?? ()
No symbol table info available.
#2 0x0000000000000000 in ?? ()
No symbol table info available.

I understand that i can be inside anything but httpd. So, i need to rebuild world with debug info (i know how to do it) and i need to rebuild apache and everything it depends on with debug. I hate to track it all manually. Is there any way to force debug info for all ports?

Another thing is that if it is inside php so file then how gdb will load symbol table for it? This gets lost.

SirDice · Jan 19, 2023

An option that's always defined but not always visible, build with WITH_DEBUG set.

covacat · Jan 19, 2023

the stack looks corrupted to me 0x000....000 does not look ok

AKuchin · Jan 19, 2023

SirDice said:
An option that's always defined but not always visible, build with WITH_DEBUG set.

Yes, but i will then need to set
WITH_DEBUG_PORTS=...
to list of all needed ports in make.conf . Right?

Or can i just use
portupgrade -Rf --make-args WITH_DEBUG=1 apache24 ?

-R
--upward-recursive Act on all those packages required by the given
packages as well. (When specified with -F, fetch
recursively, including the brand new, uninstalled
ports that an upgraded port requires)

AKuchin · Jan 19, 2023

covacat said:
the stack looks corrupted to me 0x000....000 does not look ok

I check several core dumps. It is always like this. Also, why there are only 3 entries on stack? I think it should be big. What does it mean?

cracauer@ · Jan 19, 2023

Even without debug info you would get the names of functions in shared libs in the backtrace. Just no source code info.

I agree with covacat that this looks like a corrupted stack.

AKuchin · Jan 19, 2023

cracauer@ said:
I agree with covacat that this looks like a corrupted stack.

Does this mean that debugging information will not help me ?

covacat · Jan 19, 2023

Code:

Core was generated by `/usr/local/sbin/httpd -DNOHTTPACCEPT'.
Program terminated with signal SIGABRT, Aborted.
Sent by kill() from pid 64259 and user 80.
#0  0x000000080078157a in _kevent () from /lib/libc.so.7
(gdb) bt
#0  0x000000080078157a in _kevent () from /lib/libc.so.7
#1  0x0000000800637d13 in ?? () from /lib/libthr.so.3
#2  0x00000008005ef350 in ?? () from /usr/local/lib/libapr-1.so.0
#3  0x0000000800abe9bf in ?? () from /usr/local/libexec/apache24/mod_mpm_prefork.so
#4  0x0000000800abe51d in ?? () from /usr/local/libexec/apache24/mod_mpm_prefork.so
#5  0x0000000800abd7bb in ?? () from /usr/local/libexec/apache24/mod_mpm_prefork.so
#6  0x000000000027009b in ap_run_mpm ()
#7  0x000000000025d26c in main ()

i killed an idle httpd process
i have no debug syms on anything

cracauer@ · Jan 19, 2023

covacat said:
i have no debug syms on anything

Hm. Well, you can gather them with `nm` on the shared lib and the offset in the memory map

I thought gdb does that. Now I'm curious why I thought that.

AKuchin · Jan 19, 2023

covacat said:
i killed an idle httpd process

Did the same. httpd compiles with debug info.

Code:

# kill -s ABRT 52660
# gdb -c 00_httpd_6_18050_52660.core

Code:

Core was generated by `/usr/local/sbin/httpd -f /hoster/sites/confs/illectrix.ru -k start'.
Program terminated with signal SIGABRT, Aborted.
Sent by kill() from pid 52660 and user 18050.
#0  0x00000008258a3dfa in ?? ()
(gdb) bt
#0  0x00000008258a3dfa in ?? ()
#1  0x0000000824937ec2 in ?? ()
#2  0x0000000820394a10 in ?? ()
#3  0x00000000002af500 in ?? ()
#4  0x0000000829079028 in ?? ()
#5  0x000000082904b028 in ?? ()
#6  0x0000000820394b70 in ?? ()
#7  0x0000000829075110 in ?? ()
#8  0x0000000820394a30 in ?? ()
#9  0x0000000822f1c188 in ?? ()
#10 0x0000000000000001 in ?? ()
#11 0x0000000000000000 in ?? ()

if i load binary:

Code:

# gdb /usr/local/sbin/httpd -c 00_httpd_6_18050_52660.core
Reading symbols from /usr/local/sbin/httpd...
[New LWP 100393]
Core was generated by `/usr/local/sbin/httpd -f /hoster/sites/confs/illectrix.ru -k start'.
Program terminated with signal SIGABRT, Aborted.
Sent by kill() from pid 52660 and user 18050.
#0  0x00000008258a3dfa in ?? ()
(gdb) bt
#0  0x00000008258a3dfa in ?? ()
#1  0x0000000824937ec2 in ?? ()
#2  0x0000000820394a10 in ?? ()
#3  0x00000000002af500 in strcmplex (str1=0x820394b70 "\377\377\377\377\b", str2=0x82904b028 "(\220\004)\b") at util_expr_eval.c:337
#4  0x0000000822f1c188 in ?? ()
#5  0x0000000000000001 in ?? ()
#6  0x0000000000000000 in ?? ()

See, stack end with 00000 always. And output is always only numbers and ??. why...

Build port and all its dependencies with debug info

Administrator