1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Apache or php problem (i think) (http.core dump)

Discussion in 'Web & Network Services' started by billgates, Nov 10, 2010.

  1. billgates

    billgates New Member

    Messages:
    15
    Likes Received:
    0
    Hi all !

    I have been running some old php websites without problem until now. I dont know what the problem is exactly just that sometimes the box grinds to a halt under huge load (20+). Ssh barely responds.
    It is a vmware esx 4.1 guest running 8.1 with php52 + apache22 + mysql in JAIL. I have disabled suhosin and multibyte.

    httpd.conf:
    CoreDumpDirectory /tmp/apache2-dump
    Code:
    # gdb /usr/local/sbin/httpd  httpd.core
    ....
    Loaded symbols for /usr/local/lib/libpthread-stubs.so.0
    Reading symbols from /usr/lib/librpcsvc.so.4...done.
    Loaded symbols for /usr/lib/librpcsvc.so.4
    Reading symbols from /usr/local/lib/php/20060613/mysqli.so...done.
    Loaded symbols for /usr/local/lib/php/20060613/mysqli.so
    Reading symbols from /libexec/ld-elf.so.1...done.
    Loaded symbols for /libexec/ld-elf.so.1
    #0  0x0000000804d11bb0 in _zend_mm_free_int () from /usr/local/libexec/apache22/libphp5.so
    [New Thread 8018041c0 (LWP 100219)]
    (gdb) bt
    #0  0x0000000804d11bb0 in _zend_mm_free_int () from /usr/local/libexec/apache22/libphp5.so
    #1  0x0000000804d1f0e5 in free_zend_constant () from /usr/local/libexec/apache22/libphp5.so
    #2  0x0000000804d378f2 in zend_hash_apply_deleter () from /usr/local/libexec/apache22/libphp5.so
    #3  0x0000000804d379f9 in zend_hash_reverse_apply () from /usr/local/libexec/apache22/libphp5.so
    #4  0x0000000804d209b1 in shutdown_executor () from /usr/local/libexec/apache22/libphp5.so
    #5  0x0000000804d2cdff in zend_deactivate () from /usr/local/libexec/apache22/libphp5.so
    #6  0x0000000804ceae2b in php_request_shutdown () from /usr/local/libexec/apache22/libphp5.so
    #7  0x0000000804db1815 in php_handler () from /usr/local/libexec/apache22/libphp5.so
    #8  0x0000000000435e4a in ap_run_handler (r=0x8057464c0) at config.c:157
    #9  0x0000000000439082 in ap_invoke_handler (r=0x8057464c0) at config.c:376
    #10 0x00000000004435aa in ap_internal_redirect (new_uri=Variable "new_uri" is not available.
    ) at http_request.c:549
    #11 0x00000008049bcc60 in handler_redirect (r=0x80574c0a0) at mod_rewrite.c:4831
    #12 0x0000000000435e4a in ap_run_handler (r=0x80574c0a0) at config.c:157
    #13 0x0000000000439082 in ap_invoke_handler (r=0x80574c0a0) at config.c:376
    #14 0x000000000044371e in ap_process_request (r=0x80574c0a0) at http_request.c:282
    #15 0x0000000000440a58 in ap_process_http_connection (c=0x805731290) at http_core.c:190
    #16 0x000000000043cd42 in ap_run_process_connection (c=0x805731290) at connection.c:43
    #17 0x00000000004479a1 in child_main (child_num_arg=Variable "child_num_arg" is not available.
    ) at prefork.c:662
    #18 0x0000000000447c44 in make_child (s=0x80181c708, slot=13) at prefork.c:763
    #19 0x00000000004484d4 in ap_mpm_run (_pconf=Variable "_pconf" is not available.
    ) at prefork.c:898
    #20 0x000000000042374a in main (argc=2, argv=0x7fffffffead8) at main.c:739
    (gdb) quit
    
    Suggestions ?
     
  2. mievmk

    mievmk New Member

    Messages:
    11
    Likes Received:
    0
    Saw it nany times. Not curable, IMHO.
     
  3. Alt

    Alt New Member

    Messages:
    726
    Likes Received:
    0
    What means not curable? o O
    billgates i think you should try to debug by turning off virtualhosts one by one.. With huge load do you have cpu idle?
     
  4. mievmk

    mievmk New Member

    Messages:
    11
    Likes Received:
    0
    Not curable means not curable. PHP (in this case - mod_php5, but this situation is typical for cgi, fast-cgi and fpm too) catches segfault on exit, when request was processed and client recieved its result. On some kind of freeing memory (_zend_mm_free_int, look gogle for many bugs). I saw it on very different virtual hosts, site engines, different php code.
     
  5. billgates

    billgates New Member

    Messages:
    15
    Likes Received:
    0
    @Alt, what do you mean "with huge load do you have cpu idle" ? Is this possible?

    @mievmk, my httpd processes do not die, is this your experience too ?
     
  6. Alt

    Alt New Member

    Messages:
    726
    Likes Received:
    0
    Sure, cpu load is cpu load. While load averages is number of waitng processes. So if 100 processes waiting disk i/o, LA will be 100 while cpu load is low
     
  7. billgates

    billgates New Member

    Messages:
    15
    Likes Received:
    0
    Ok, so i ran truss -p httpdpid on a pid that i think "hangs" and i see :
    Code:
    ....
    gettimeofday({1289992042.474568 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.485542 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.490529 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.497070 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.504518 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.507055 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.514551 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.522065 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.543116 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.553166 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.558047 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.563117 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.568216 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.570552 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.578012 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.597010 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.607025 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.613119 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.618052 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.623148 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.628184 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.633143 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.635553 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.640519 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.640639 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.645606 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.648172 },0x0)           = 0 (0x0)
    gettimeofday({1289992042.653071 },0x0)           = 0 (0x0)
    ....LOT more of gettimeofday....
    
    Is this normal ? (this is on vmware esx in a Jail)

    Best regards