confusion in reasons for big difference of the php-gd on FreeBSD / Linux

P

pelmen

Hi and marry christmas.

Working on a problem on FreeBSD hosting i see a big difference in the processing through the php5-gd extension. As i browse source code of php5-gd - nothing Linux-specified code.

To demonstrate, I specifically created a large jpeg file and upload to http://81.176.66.218/test1.jpg (8 MB)

Both station (FreeBSD and Linux) is 64-bit, php5-gd compiling identically (t1lib, truetype support..)

On the Linux host:

$ ls
imgresize.php php.ini
$ grep ^memory_limit php.ini
Code: 
memory_limit = 32M

$ php -v
Code: 
PHP 5.3.3-1ubuntu9.1 with Suhosin-Patch (cli) (built: Oct 15 2010 14:00:18) 
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
linux:/tmp/test$ wget http://81.176.66.218/test1.jpg

$ strace -o /tmp/straceme.log php -c ./php.ini imgresize.php test1.jpg
Code: 
crop: Memory usage: 679752
Memory usage: 675784
Memory peak usage: 767856

$ uname -rm
Code: 
2.6.35-24-generic x86_64

$ cat /etc/lsb-release
Code: 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.10
DISTRIB_CODENAME=maverick
DISTRIB_DESCRIPTION="Ubuntu 10.10"

Result: All perfect - Linux take 700 Kbytes only.


Ok, FreeBSD host now:

# grep ^memory php.ini
Code: 
memory_limit = 288M
# truss -o /tmp/trussme.log php -c ./php.ini imgresize.php test1.jpg
Code: 
PHP Fatal error:  Allowed memory size of 301989888 bytes exhausted (tried to allocate 18892 bytes) in /tmp/test/imgresize.php on line 49

Result: Fatal error, not enough memory ;(

examining the system calls, see only that both machines use the mmap(2) (Linux - mmap2(2)). But FreeBSD is not asked getrlimit(2) limits and does not use brk(2). On the other hand - the file has only 8 megabytes - that's not enough for the projection of a memory or a problem elsewhere?

Linux syscall: http://81.176.66.218/straceme.log
FreeBSD syscall: http://81.176.66.218/trussme.log

PS: I try three FreeBSD hosts for test: - 7.4 amd64, 8.2 amd64 and 9.0-current (snap from 10/26/2010). Different versions of PHP in BSD also try: php4 + php4-gd, php52 + php52-gd, php5-php5-gd - ports on the 26.10.2010. But with no difference.

PS2: Content of imgresize.php
Code: 
<?php
function resize($img, $preset, $recache=false) {
    $imginfo = pathinfo($img);
    //Check if GD extension is loaded
    if (!extension_loaded('gd') && !extension_loaded('gd2')){
        trigger_error("GD is not loaded", E_USER_WARNING);
        return false;
    }
        $newfilename='';
        $preset['addpath'] = isset($preset['addpath'])?$preset['addpath']:'';
        $newfilename = '/tmp/'.$imginfo['basename'];
    //Get Image size info
    list($width_orig, $height_orig, $image_type) = getimagesize($img);

    switch ($image_type) {
        case 1: $im = imagecreatefromgif($img); break;
        case 2: $im = imagecreatefromjpeg($img); break;
        case 3: $im = imagecreatefrompng($img); break;
        default:
            $err = 'Unsupported filetype: ' . $img;
            trigger_error($err, E_USER_WARNING); 
            throw new Exception($err);
        break;
    }
    if (!$im) {
        return false;
    }
    if ($preset['height'] == 'auto') {
                $aspect_ratio = (float) $width_orig / $preset['width'];
        $preset['height'] = ceil($height_orig / $aspect_ratio);
    }
    if (isset($preset['maxheight']) && $preset['height'] > $preset['maxheight']) {
        $preset['height'] = $preset['maxheight'];
    }
        if (isset($preset['maxwidth']) && $preset['width'] > $preset['maxwidth']) {
        $preset['width'] = $preset['maxwidth'];
    }

    $ratio = array($width_orig / $height_orig, $preset['width'] / $preset['height']);

    if ($ratio[0] != $ratio[1] && isset($preset['crop']) && $preset['crop'] == 1)
        {
        $scale = min((float)($width_orig / $preset['width']), (float)($height_orig / $preset['height']));
                $cropX = (float)($width_orig - ($scale * $preset['width']));
                $cropY = (float)($height_orig - ($scale * $preset['height']));
        // cropped image size
        $cropW = (float)($width_orig - $cropX); 
        $cropH = (float)($height_orig - $cropY);
        $crop = ImageCreateTrueColor($cropW, $cropH); 
        // crop the middle part of the image to fit proportions
        ImageCopy($crop, $im, 0, 0, (int)($cropX / 2), (int)($cropY / 2), $cropW, $cropH);
echo "crop: Memory usage: " . memory_get_usage() . "\n";
    }
    $newImg = imagecreatetruecolor($preset['width'], $preset['height']);
    /* Check if this image is PNG or GIF, then set if Transparent*/
    if(($image_type == 1) OR ($image_type==3))
    {
        imagealphablending($newImg, false);
        imagesavealpha($newImg,true);
        $transparent = imagecolorallocatealpha($newImg, 255, 255, 255, 127);
        imagefilledrectangle($newImg, 0, 0, $preset['width'], $preset['height'], $transparent);
    }
    if (isset($crop)) { // been cropped 
        ImageCopyResampled($newImg, $crop, 0, 0, 0, 0, $preset['width'], $preset['height'], $cropW, $cropH); 
        ImageDestroy($crop); 
    } else {
        imagecopyresampled($newImg, $im, 0, 0, 0, 0, $preset['width'], $preset['height'], $width_orig, $height_orig);
    }
    //Generate the file, and rename it to $newfilename
    switch ($image_type)
    {
        case 1: $result = imagegif($newImg,$newfilename); break;
        case 2: $result = imagejpeg($newImg,$newfilename); break;
        case 3: $result = imagepng($newImg,$newfilename); break;
        default:  trigger_error('Failed resize image!', E_USER_WARNING);  break;
    }
    if (!@$result) {
        //trigger_error("cacher error saving image", E_USER_WARNING);
        return false;
    }

    //cacher::imageServersUploadQueue($newfilename);

    return $newfilename;
}

$preset = array(
    'width'             =>      168,
    'height'    =>      'auto',
    'maxheight' =>      300,
    'crop'              =>      1,
    'addpath'   =>      'upic168',
    'name'              =>      'avatar'
);

if (!file_exists($argv[1])) {
    echo "Gimme filename\n";
    die();
}

resize($argv[1], $preset, true);

echo "Memory usage: " . memory_get_usage() . "\n";
echo "Memory peak usage: " . memory_get_peak_usage() . "\n";
 
Hello ,

Merry Christmas to you too

Here are some tests from me with your scripts & image :

Code: 
8.0-STABLE amd64 ( PHP 5.2.11 ) - allowed memory size of 134217728 bytes exhausted (tried to allocate 4743 bytes)

8.1-STABLE amd64 ( PHP 5.2.15 ) - crop: Memory usage: 317143944 ( here I have a limit of 512 MB ) 

7.2-STABLE i386 ( PHP 5.2.10 ) - Memory usage: 76360, Memory peak usage: 521228

7.2-STABLE amd64 ( PHP 5.2.8 ) - Memory usage: 249392, Memory peak usage: 613272

9.0-CURRENT i386 8 Dec ( PHP 5.3.4 ) - Memory usage: 351616, Memory peak usage: 796220 

7.0-RC1 i386 ( PHP 5.3.4 ) - Memory usage: 344912, Memory peak usage: 789412

7.0-RC1 i386 ( PHP 5.2.9 ) - Memory usage: 74296, Memory peak usage: 498964

8.1-RELEASE i386 ( PHP 5.3.2 ) - Allowed memory size of 134217728 bytes exhausted (tried to allocate 18972 bytes )

Somewhere some dependency ...

EDIT:

Did you try to upgrade your ports tree and reinstall all php related ports and its dependencies ?
For example with something like
Code: 
portupgrade -rR php5
- I did it on -CURRENT , I had php 5.3.3 ( your script allocated more than 300MB memory ) , after upgrading to 5.3.4 with the above command your script allocates ~ 350 KB memory
Even PHP 5.3 is more slow than 5.2 my box beats your Linux :) , as you can see on my 7.2-STABLE amd64 php 5.2.8 your scripts allocates about 250KB memory .
 
Thanks You for reply and your time. and pls forgive my google.translate accent )

quintessence said:
Did you try to upgrade your ports tree and reinstall all php related ports and its dependencies ?
Click to expand...

Yes. Even tried to recompile with debug. And no effect. I think the problem lies outside of the php / php-gd

# file `which php` && ldd `which php`
Code: 
/usr/local/bin/php: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.0 (900029), 
not stripped
/usr/local/bin/php:
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x800a18000)
        libpcre.so.0 => /usr/local/lib/libpcre.so.0 (0x800b31000)
        librt.so.1 => /usr/lib/librt.so.1 (0x800c6e000)
        libm.so.5 => /lib/libm.so.5 (0x800d74000)
        libxml2.so.5 => /usr/local/lib/libxml2.so.5 (0x800e94000)
        libz.so.6 => /lib/libz.so.6 (0x8010d8000)
        libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x8011ee000)
        libthr.so.3 => /lib/libthr.so.3 (0x8013e8000)
        libc.so.7 => /lib/libc.so.7 (0x80150a000)

# php -i | grep -i ^gd
Code: 
gd
GD Support => enabled
GD Version => bundled (2.0.34 compatible)
gd.jpeg_ignore_warning => 0 => 0

With debug php, ive see place of evil:
Code: 
PHP Fatal error:  Allowed memory size of 301989888 bytes exhausted at /usr/ports/graphics/php5-gd/work/php-5.3.4/ext/gd/libgd/gd.c:201 (tried 
to allocate 18892 bytes) in /usr/imgresize.php on line 49

201 string is
Code: 
im->tpixels[i] = (int *) gdCalloc(sx, sizeof(int));


gdCalloc is Zend ecalloc wrapper and in the end this is the usual malloc(2)

I try to play with some "Double/halve the size" parameters by
# ln -s <FLAGS> /etc/malloc.conf

But with no happy.

And still question about brk

&quot said:
D Use sbrk(2) to acquire memory in the data storage segment (DSS).
This option is enabled by default. See the ``M'' option for
related information and interactions.
Click to expand...

I can see *brk in system call unlike Linux. Where is difference with Linux? *alloc function? Block size or something else..
With Linux box I have never met a Fatal error on this code, although memory_limit there is a 64 Mb. On FreeBSD with the same PHP code - I see a Fatal error on different custom picture when memory_limit 256 Mb. Possible problem of floating - as some great pictures are handled well.
 
UPDATE: Today i install FreeBSD 7.3-RELEASE i386 in Virtual Box and scripts working fine! Also i find some information about this:
http://old.nabble.com/Why-process-memory-starts-so-high-up-in-virtual-space-with-FreeBSD-malloc--td20780943.html But playing with MALLOC_OPTIONS not changes anything.

Giorgos Keramidas wrote:
"The 'D' and 'M' options set what malloc() will _prefer_, they do not
force malloc() to use _only_ the particular type of memory space."

type of memory space - what the space? kmem ? I read that FreeBSD excellent work with the memory subsystem. In practice, in my case turned out the opposite - for me it's one big mystery)
 
You must log in or register to reply here.
Back
Top