PHP 5.4 dies executing this code on FreeBSD PHP 5.4, and 5.3 depending on compiler

[turbo]

Howdy all!

Quite a long title, but let me explain this issue I have been figthing with for a while now. There's a piece of PHP code that causes PHP processs to die with a segmentation fault, however this same code runs fine in other platforms / situations, so I don't think that the problem is a PHP bug, but rather a problem on how PHP is compiled on FreeBSD.

The code in question is in a pastie, as takes 8kb: PHP test code.

To test this code, I use [cmd=]# php test.php[/cmd]. Expected output is:

Before preg_replace_callback: 7996
After preg_replace_callback: 17

Faulty output is:

Before preg_replace_callback: 7996
Segmentation fault (core dumped)

I'm using a clean base intall of FreeBSD 8.3-RELEASE amd64, with PACKAGESITE set to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8-stable/Latest/ for the tests.

Running the test with:

• PHP 5.3.17 pre-compiled package: works OK.
• PHP 5.4.7 pre-compiled package: Segmentation Fault.
• PHP 5.3.23 built from ports with default GCC 4.2.2: Segmentation Fault.
• PHP 5.3.23 built from ports with GCC 4.4.7: works OK.

I know that GCC 4.4 is deprecated, but some time ago I also tried with 4.6 and 4.7, and those were causing the same issue. I will re-run the tests with these versions again.

Also note that modifying the string in the text code by removing some of the "x", makes the code work in the otherwise faulty PHP versions.

The questions:

1. As the test code runs fine or segfaults in PHP 5.3 depending on how it is compiled, what is the ideal way to compile PHP from the ports?
   
2. PHP 5.4 does a segmentation fault even with the precompiled package, however the code runs fine using the PHP 5.4 binaries for Windows. Any suggestions on where to go from here? I don't think it's a PHP bug, but maybe it's platform dependent. Should I report it somewhere specific?
   
3. If any of you guys have a FreBSD box running PHP 5.4, can you give a try to the code and let me know if it works without segfaulting?

Thanks for any suggestions.

 

[Anonymous]

I checked your code against two of my installations of PHP. One on Mac OS X 10.8.3 running the readily installed PHP 5.3.15, and the other on FreeBSD 9.1-RELEASE having PHP 5.4.12 compiled from the ports. Of which only the second test resulted in a segmentation fault.

I think, that there is nothing good we users can do about it, other than submitting a PR.

I loaded the core dump into gdb, in oder to obtain more information about the cause, it appears that the segmentation fault exactly happens within /usr/local/lib/libpcre.so.3:

# gdb /usr/local/bin/php php.core

GNU gdb (GDB) 7.5.1 [GDB v7.5.1 for FreeBSD]
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd9.1".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/bin/php...(no debugging symbols found)...done.
[New process 100153]
[New Thread 802007400 (LWP 100153)]
Core was generated by `php'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000800d5ab5d in match () from /usr/local/lib/libpcre.so.3

 

[turbo]

Thanks for testing it and for providing the problem report link. I'll submit PR there.

 

[derekschrock]

It might be better to explain what you're trying to do with that regex. libpcre is calling match() ~8000 times. Are you trying to remove HTML comments over multiple lines that might have !'s in them?

 

[Anonymous]

It might be better to explain what you're trying to do with that regex. libpcre is calling match() ~8000 times.
Are you trying to remove HTML comments over multiple lines that might have !'s in them?

Of course, it is always good to optimize your code, and ask yourself whether it can be done better in a different way.

However, this is definitely not better, compared to helping (by sending a PR) resolving a crasher in libpcre. If libpcre finds it too boring to call 8000 times match(), then it should stop with a human readable error message and not with a segmentation fault, which btw. may constitute a vector for remote attacks.

The crasher should be fixed. Code optimization is another story.

 

[turbo]

@derekschrock, I couldn't explain it better than @rolfheinrich. I certainly could approach the comment removal in a different way, but this code is just an example to reproduce the issue.

It took me some time to identify what was causing the PHP 5.4 process to die in relation to the executed PHP code as it was a rare occasional crash, and it is not just that particular regular expression, but other calls to preg_replace_callback in different scripts that run in a particular server which work perfectly fine with PHP 5.3.

I have been playing a bit more with the code that produces the error, and achieved 2 versions. One that causes a segfault, and another that works ok with PHP 5.4. The only difference between both of them is a single "x" character in the $code string, and it does not matter from which position it is removed.

Code that fails (pastie):

# fetch -o test_fail.php 'http://pastie.org/pastes/7110430/download?key=tjggwrfdbqfv7natvsajg'
test_fail.php                                 100% of 5894  B   15 MBps
# php -n ./test_fail.php

Before preg_replace_callback: 5445
Segmentation fault (core dumped)

Code that works (pastie):

# fetch -o test_ok.php 'http://pastie.org/pastes/7110448/download?key=msom7gp61dzbvrikynygqw'
test_ok.php                                   100% of 5896  B   15 MBps
# php -n ./test_ok.php

Before preg_replace_callback: 5444
After preg_replace: 17

I'm submitting the PR just now

 

[turbo]

Just a note out of curiosity. It does not matter how many "!" characters are inside the $code string. The main problem here with that particular regex seems to be the string length.

Fails: (pastie)

Works: (pastie)

 

[turbo]

I submitted the PR, however it has been closed because the maintainer cannot reproduce the issue. I have realized that the length of the string causing the segmentation fault is not the same for different systems. On a second system I tried, the length needed to be increased slightly to cause the seg fault.

I would appreciate if other people can test their PHP 5.4 installation so I can report this properly.

You can use this one-liner, and change the str_repeat value to a higher number if it does not crash.

# php -n -r 'echo var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'

Thank you!

 

[Anonymous]

... I would appreciate if other people can test their PHP 5.4 installation so I can report this properly.

# php -n -r 'echo var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'

The one-liner as is crashes my php.

# php --version

PHP 5.4.13 (cli) (built: Mar 25 2013 16:26:04) 
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies

# pkg_version -v | grep pcre

pcre-8.32                           =   up-to-date with port

Please inform the PR number, and perhaps it helps the maintainer to accept the PR if others confirm your findings.

 

[turbo]

The PR number is 177375. I replied back to the PR status change email with additional info to reproduce the error.

If there's no reply and more people confirm the issue, I'll try to report it again.

 

[derekschrock]

I truly believe you're barking up the wrong tree here:

[CMD="man"]pcrestack[/CMD]

This man page clearly states that some patterns (with long strings) can run out of stack.

 

[turbo]

Thanks for pointing out the pcrestack man entry, that was an interesting read. You are right, it can go out of stack, however PHP provides the setting pcre.recursion_limit which defaults to 100,000. This code crashes PHP 5.4 before reaching this limit, and PHP 5.3 does not. I would expect the same behavior in both versions.

# php -v
PHP 5.3.23 with Suhosin-Patch (cli) (built: Mar 23 2013 10:19:12)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2013 Zend Technologies

# php -n -r 'var_dump(ini_get("pcre.recursion_limit"));'
string(6) "100000"

# php -n -r 'var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'
string(0) ""

# php -n -r 'ini_set("pcre.recursion_limit", "999");var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'
NULL

# php -n -r 'ini_set("pcre.recursion_limit", "10999");var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'
string(0) ""

# php -v
PHP 5.4.13 (cli) (built: Mar 22 2013 16:06:41)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies

# php -n -r 'var_dump(ini_get("pcre.recursion_limit"));'
string(6) "100000"

# php -n -r 'var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'
Segmentation fault (core dumped)

# php -n -r 'ini_set("pcre.recursion_limit", "999");var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'
NULL

# php -n -r 'ini_set("pcre.recursion_limit", "10999");var_dump(preg_replace("/(<!)(?!\\s*(?:\\[if [^\\]]+]|<!|>))((?:(?!>).)*[^\\]])(-->)/is", "", "<!" . str_repeat("x", 9999) . "-->"));'
Segmentation fault (core dumped)

The maintainer was finally able to reproduce this issue and the PR 177375 has been reopened.

 

[kpa]

The maximum number of recursions is just one variable in a crash scenario, you also have to know how much stack each recursive call will consume on average. This can vary from version to version.