Apache or php problem (i think) (http.core dump)

billgates · #1 November 10th, 2010, 15:03

Hi all !

I have been running some old php websites without problem until now. I dont know what the problem is exactly just that sometimes the box grinds to a halt under huge load (20+). Ssh barely responds.
It is a vmware esx 4.1 guest running 8.1 with php52 + apache22 + mysql in JAIL. I have disabled suhosin and multibyte.

httpd.conf:
CoreDumpDirectory /tmp/apache2-dump

Code:

# gdb /usr/local/sbin/httpd  httpd.core
....
Loaded symbols for /usr/local/lib/libpthread-stubs.so.0
Reading symbols from /usr/lib/librpcsvc.so.4...done.
Loaded symbols for /usr/lib/librpcsvc.so.4
Reading symbols from /usr/local/lib/php/20060613/mysqli.so...done.
Loaded symbols for /usr/local/lib/php/20060613/mysqli.so
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0000000804d11bb0 in _zend_mm_free_int () from /usr/local/libexec/apache22/libphp5.so
[New Thread 8018041c0 (LWP 100219)]
(gdb) bt
#0  0x0000000804d11bb0 in _zend_mm_free_int () from /usr/local/libexec/apache22/libphp5.so
#1  0x0000000804d1f0e5 in free_zend_constant () from /usr/local/libexec/apache22/libphp5.so
#2  0x0000000804d378f2 in zend_hash_apply_deleter () from /usr/local/libexec/apache22/libphp5.so
#3  0x0000000804d379f9 in zend_hash_reverse_apply () from /usr/local/libexec/apache22/libphp5.so
#4  0x0000000804d209b1 in shutdown_executor () from /usr/local/libexec/apache22/libphp5.so
#5  0x0000000804d2cdff in zend_deactivate () from /usr/local/libexec/apache22/libphp5.so
#6  0x0000000804ceae2b in php_request_shutdown () from /usr/local/libexec/apache22/libphp5.so
#7  0x0000000804db1815 in php_handler () from /usr/local/libexec/apache22/libphp5.so
#8  0x0000000000435e4a in ap_run_handler (r=0x8057464c0) at config.c:157
#9  0x0000000000439082 in ap_invoke_handler (r=0x8057464c0) at config.c:376
#10 0x00000000004435aa in ap_internal_redirect (new_uri=Variable "new_uri" is not available.
) at http_request.c:549
#11 0x00000008049bcc60 in handler_redirect (r=0x80574c0a0) at mod_rewrite.c:4831
#12 0x0000000000435e4a in ap_run_handler (r=0x80574c0a0) at config.c:157
#13 0x0000000000439082 in ap_invoke_handler (r=0x80574c0a0) at config.c:376
#14 0x000000000044371e in ap_process_request (r=0x80574c0a0) at http_request.c:282
#15 0x0000000000440a58 in ap_process_http_connection (c=0x805731290) at http_core.c:190
#16 0x000000000043cd42 in ap_run_process_connection (c=0x805731290) at connection.c:43
#17 0x00000000004479a1 in child_main (child_num_arg=Variable "child_num_arg" is not available.
) at prefork.c:662
#18 0x0000000000447c44 in make_child (s=0x80181c708, slot=13) at prefork.c:763
#19 0x00000000004484d4 in ap_mpm_run (_pconf=Variable "_pconf" is not available.
) at prefork.c:898
#20 0x000000000042374a in main (argc=2, argv=0x7fffffffead8) at main.c:739
(gdb) quit

Suggestions ?

mievmk · #2 November 10th, 2010, 18:22

Saw it nany times. Not curable, IMHO.

Alt · #3 November 10th, 2010, 21:58

What means not curable? o O
billgates i think you should try to debug by turning off virtualhosts one by one.. With huge load do you have cpu idle?

mievmk · #4 November 11th, 2010, 08:30

Not curable means not curable. PHP (in this case - mod_php5, but this situation is typical for cgi, fast-cgi and fpm too) catches segfault on exit, when request was processed and client recieved its result. On some kind of freeing memory (_zend_mm_free_int, look gogle for many bugs). I saw it on very different virtual hosts, site engines, different php code.

billgates · #5 November 13th, 2010, 17:32

@Alt, what do you mean "with huge load do you have cpu idle" ? Is this possible?

@mievmk, my httpd processes do not die, is this your experience too ?

Alt · #6 November 13th, 2010, 21:10

Quote:

Originally Posted by billgates

@Alt, what do you mean "with huge load do you have cpu idle" ? Is this possible?

Sure, cpu load is cpu load. While load averages is number of waitng processes. So if 100 processes waiting disk i/o, LA will be 100 while cpu load is low

billgates · #7 November 17th, 2010, 12:38

Ok, so i ran truss -p httpdpid on a pid that i think "hangs" and i see :

Code:

....
gettimeofday({1289992042.474568 },0x0)           = 0 (0x0)
gettimeofday({1289992042.485542 },0x0)           = 0 (0x0)
gettimeofday({1289992042.490529 },0x0)           = 0 (0x0)
gettimeofday({1289992042.497070 },0x0)           = 0 (0x0)
gettimeofday({1289992042.504518 },0x0)           = 0 (0x0)
gettimeofday({1289992042.507055 },0x0)           = 0 (0x0)
gettimeofday({1289992042.514551 },0x0)           = 0 (0x0)
gettimeofday({1289992042.522065 },0x0)           = 0 (0x0)
gettimeofday({1289992042.543116 },0x0)           = 0 (0x0)
gettimeofday({1289992042.553166 },0x0)           = 0 (0x0)
gettimeofday({1289992042.558047 },0x0)           = 0 (0x0)
gettimeofday({1289992042.563117 },0x0)           = 0 (0x0)
gettimeofday({1289992042.568216 },0x0)           = 0 (0x0)
gettimeofday({1289992042.570552 },0x0)           = 0 (0x0)
gettimeofday({1289992042.578012 },0x0)           = 0 (0x0)
gettimeofday({1289992042.597010 },0x0)           = 0 (0x0)
gettimeofday({1289992042.607025 },0x0)           = 0 (0x0)
gettimeofday({1289992042.613119 },0x0)           = 0 (0x0)
gettimeofday({1289992042.618052 },0x0)           = 0 (0x0)
gettimeofday({1289992042.623148 },0x0)           = 0 (0x0)
gettimeofday({1289992042.628184 },0x0)           = 0 (0x0)
gettimeofday({1289992042.633143 },0x0)           = 0 (0x0)
gettimeofday({1289992042.635553 },0x0)           = 0 (0x0)
gettimeofday({1289992042.640519 },0x0)           = 0 (0x0)
gettimeofday({1289992042.640639 },0x0)           = 0 (0x0)
gettimeofday({1289992042.645606 },0x0)           = 0 (0x0)
gettimeofday({1289992042.648172 },0x0)           = 0 (0x0)
gettimeofday({1289992042.653071 },0x0)           = 0 (0x0)
....LOT more of gettimeofday....

Is this normal ? (this is on vmware esx in a Jail)

Best regards

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
CYRUS IMAP cyradm core dump problem	davidwhytcross	Web & Network Services	0	September 21st, 2010 06:32
pgsql.so causes php cli core dump	lashmish	Installation and Maintenance of FreeBSD Ports or Packages	4	December 12th, 2009 23:52
PHP fail (core dump)	pimaxc	Installation and Maintenance of FreeBSD Ports or Packages	1	November 6th, 2009 16:17
[Solved] another problem Apache+PHP	zeiz	Web & Network Services	6	November 4th, 2009 23:30
apache 2.0.63 crash on restart ... php related (very old problem)	edhunter	Web & Network Services	17	February 5th, 2009 16:29