Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The Situation

I am having an issue with a PHP script getting the following error message:

Fatal error: Out of memory (allocated 359923712) (tried to allocate 72 bytes) in /path/to/piwik/core/DataTable.php on line 969

The script I'm running is: /path/to/piwik/misc/cron/archive.sh

I am assuming the numbers are Bytes, which means that total is approximately 360MB.

For all intents and purposes, I have increased the memory limits on the server well above 360MB, yet this is the number (give or take a byte) it consistently errors out at.

Please note: This question is not about fixing a memory leak in the script, nor about why the script itself is using so much memory. The script is part of the Piwik archiving process, so I cannot just fix any memory leaks, etc. For more info on this script and why I am increasing the memory limit, see "How to setup auto archiving"

The question

Given that the script is attempting to use over 360MB of memory, which I cannot change, why does it not seem possible for me to increase the amount of memory available to php on my server?

Update 23rd June: See "What I've tried" > "Increasing the per process memory limits of Linux" below for background... If I set ulimit -v 1024000, then check it with ulimit -v I get the correct value of '1024000'. If I run the script again, it will progress much further, but will eventually error out with the same memory limit (~360MB) having been reached. If I immediately check ulimit -v, it has reset to the original value of '524288'. This appears as though it may be the root cause of the issue.


What I've tried

Increasing PHP's memory_limit

Given the php.ini file:

php -i | grep php.ini
Configuration File (php.ini) Path => /usr/local/lib
Loaded Configuration File => /usr/local/lib/php.ini

I have edited that file, so the memory_limit directive reads;

memory_limit = -1

Restart Apache, and check the new value has stuck;

$ php -i | grep memory_limit
memory_limit => -1 => -1

Run the script, and get the same error.

I've also tried 1G, 768M, etc, all to the same result (ie; no change).

Update 22nd June: Based on Vangel's help, I have attempted to set post_max_size to 20M in combination with setting memory_limit. Again, this has no effect.

Update 23rd June: Based on olefebvre's help, I can confirm that the user running the script has read/write permission to the php.ini file containing the settings of memory_limit.

Removing the memory limit on child processes of Apache

I have found and edited the httpd.conf file to make sure there is no RLimitMEM directive.

I then used WHM's Apache Configuration > Memory Usage Restrictions to generate a restriction, which it claimed was at 1000M (and confirmed by checking httpd.conf).

Both of these resulted in no change to the script erroring at 360MB.

Increasing the per process memory limits of Linux

The current limits set on the system:

$ ulimit -m
524288

$ ulimit -v
524288

I have attempted to set both of these to unlimited:

$ ulimit -m unlimited
$ ulimit -v unlimited

$ ulimit -m
unlimited

$ ulimit -v
unlimited

Once again, this has resulted in absolutely no improvement in my problem.

Update 23rd June: I have run into a related issue here. If I set ulimit -v 1024000, then check it with ulimit -v I get the correct value of '1024000'. If I run the script again, it will progress much further, but will eventually error out with the same memory limit having been reached. If I immediately check ulimit -v, it has reset to the original value of '524288'. This appears as though it may be the root cause of the issue.


My setup

$ cat /etc/redhat-release
CentOS release 5.5 (Final)

$ uname -a
Linux example.com 2.6.18-164.15.1.el5 #1 SMP Wed Mar 17 11:30:06 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

$ php -i | grep "PHP Version"
PHP Version => 5.2.9

$ httpd -V
Server version: Apache/2.0.63
Server built:   Feb  2 2011 01:25:12
Cpanel::Easy::Apache v3.2.0 rev5291
Server's Module Magic Number: 20020903:13
Server loaded:  APR 0.9.17, APR-UTIL 0.9.15
Compiled using: APR 0.9.17, APR-UTIL 0.9.15
Architecture:   64-bit
Server compiled with....
 -D APACHE_MPM_DIR="server/mpm/prefork"
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D HTTPD_ROOT="/usr/local/apache"
 -D SUEXEC_BIN="/usr/local/apache/bin/suexec"
 -D DEFAULT_PIDLOG="logs/httpd.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_LOCKFILE="logs/accept.lock"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"

Output of $ php -i: http://pastebin.com/EiRut6Nm

share|improve this question
    
I think you edited the wrong php.ini. usually the php.ini is in /etc/php.ini. I'm not sure that it might be there, but on all my RHEL server and CentOS server, it always was there. So why don't give it a try ;) –  Anarko_Bizounours Jun 22 '11 at 6:38
    
Thanks for the tip. My server doesn't have a /etc/php.ini file, so definitely not there :) As per the question, I can confirm that after editing the /usr/local/lib/php.ini file and restarting apache that the settings stick. –  Jess Telford Jun 22 '11 at 13:13
    
I am curious as to why the question received a down-vote? Is it not a good question? If not, how can I improve it? –  Jess Telford Jun 22 '11 at 13:17

7 Answers 7

Background:

I just grabbed the current version of Piwik -- In version 1.5, line 969 appears as:

public function addRowsFromSerializedArray( $stringSerialized )
{
    $serialized = unserialize($stringSerialized);
    if($serialized === false)
    {
        throw new Exception("The unserialization has failed!");
    }
    $this->addRowsFromArray($serialized);
}

and is specifically $serialized = unserialize($stringSerialized);. The call to unserialize can be incredibly memory intensive. There is an excellent post here.

As you've noted, this clearly isn't a bug in your script and is a valid Out of Memory.

Suggestion: In the config file, as noted above in my comment:

/.../piwik/config/global.ini.php

I think you may need to increase one of these limits:

# during archiving, Piwik will limit the number of results recorded, for performance reasons
# maximum number of rows for any of the Referers tables (keywords, search engines, campaigns, etc.)
# this limit will also be applied to the Custom Variables names and values reports
datatable_archiving_maximum_rows_referers = 1000
# maximum number of rows for any of the Referers subtable (search engines by keyword, keyword by campaign, etc.)
datatable_archiving_maximum_rows_subtable_referers = 50

# maximum number of rows for any of the Actions tables (pages, downloads, outlinks)
datatable_archiving_maximum_rows_actions = 500
# maximum number of rows for pages in categories (sub pages, when clicking on the + for a page category)
# note: should not exceed the display limit in Piwik_Actions_Controller::ACTIONS_REPORT_ROWS_DISPLAY
#       because each subdirectory doesn't have paging at the bottom, so all data should be displayed if possible.
datatable_archiving_maximum_rows_subtable_actions = 100

# maximum number of rows for other tables (Providers, User settings configurations)
datatable_archiving_maximum_rows_standard = 500

where I changed the semi-colons to # signs just to make sf's autocolor readable.

You might also try adding:

CMD_TO_CHECK_SETTINGS="$PHP_BIN -i > /tmp/piwik-php-env.out"
$CMD_TO_CHECK_SETTINGS

to archive.sh to determine if there are other settings afoot which override the php.ini file.

share|improve this answer
    
Thank you for the answer, however it points out what I already know as stated in the question. Since this answer doesn't help with the actual question asked, I would like to down-vote (but don't have enough rep to do so). –  Jess Telford Jun 22 '11 at 5:52
    
A-ha! Progress! I upped memory_minimum_limit to 1024, and increased the variables you listed by a factor of 10 each, and set ulimit -m and ulimit -v to 'unlimited'. The error message I get now is SQLSTATE[08S01]: Communication link failure: 1153 Got a packet bigger than 'max_allowed_packet' bytes Further investigation lead me to increase mysqld's max_allowed_packet=32M (was previously at 1M). Ran the script perfectly! I will narrow it down and report back as to the exact solution :) –  Jess Telford Jun 22 '11 at 8:07
    
Final solution; 1. Edit /etc/my.cnf and set max_allowed_packet=32M. 2. Edit /usr/local/lib/php.ini and set memory_limit = 512M. 3. Set ulimit -v 1048576. Not sure why, but changing those settings you mentioned in piwik's config file seem to have induced the second SQL error stated above, which lead me to setting max_allowed_packet. Thanks heaps for your help! –  Jess Telford Jun 22 '11 at 8:49
1  
Without knowing precisely what/where the error message is, my initial suspicion would be that a function call is issuing a command under another user which does not have the generous ulimit settings you've specified. –  M. Tibbits Jun 22 '11 at 14:00
1  
@M.Tibbits let us continue this discussion in chat –  Jess Telford Jun 23 '11 at 0:08
up vote 2 down vote accepted

M. Tibbits answer alludes to allowing Piwik to use more resources during the archiving process.

There is a limitation on how large an SQL command can be executed, which defaults to just 1MB. For more info; Packet too large

To increase the size limitation in MySQL, edit /etc/my.cnf and set max_allowed_packet=32M

Ensure the PHP memory limit is set to be high enough per forked process by editing /usr/local/lib/php.ini and set memory_limit = 512M

Finally, ensure all processes have at least a 1G hard limit before the system shuts them off by executing ulimit -v 1048576 on the command line.

Update

ulimit -v 1048576 will only raise the soft limit. If the hard limit is not high enough, the system will automatically reset the soft limit to that of the hard limit.

To set the hard limit, add the -H switch:

ulimit -vH 1048576

followed by increasing the soft limit up to this value:

ulimit -vS 1048576
share|improve this answer
    
I may have spoken too soon - it appears this solution does not work. –  Jess Telford Jun 22 '11 at 13:10
1  
Need to up the hard limit followed by the soft limit. This was the final solution. –  Jess Telford Jun 29 '11 at 1:06

this is definitely a script bug. It seems that there is a memory leak so no matter how much you increase the memory limit in PHP it will hit it.

I use Piwik on a basic level and dont remember ever using an archiving script directly. Piwik is a bit buggy still but the latest version is 1.4 and seems to be much less buggy that previous ones.

I recommend not to screw around with system or PHP settings for the sake of accommodating an erroneous script. Hitting the piwik support forum might be a better idea

share|improve this answer
    
Thank you for your answer. As stated in the question; this is not about fixing an error in Piwik. The strange behavior is: Given that a script uses over 360MB of memory, why can I not seem to allow it to use more? –  Jess Telford Jun 16 '11 at 3:53
    
restarted the webserver? –  Vangel Jun 16 '11 at 4:04
    
I have restarted Apache, if that's what you mean. It's not feasible for me to restart the entire operating system. –  Jess Telford Jun 16 '11 at 5:01
    
yes just web server. so what did you get? Also if it hasnt worked out it would be helpful to look at the phpinfo. I suspect its not picking up the ini from the place where you are modifying it? p.s. we are way off timezones. PM me your skype id if you can and maybe we can just chat to figure this out, then post the solution here –  Vangel Jun 17 '11 at 4:16
1  
I read your response above on Tibbits answer. I was going to suggest (i know quite late) to look at limits. If ulimit is your problem you can make the change permanent with either with kernel tune parameter or i you OS has this file /etc/security/limits.conf and add memlock limit to unlimited for the user that runs php/web. –  Vangel Jun 23 '11 at 10:52

Have you tried using the define command line option ? php -d memory_limit=512M test.php

btw have you checked the php.ini is readable by the user running php from the commandline ?

share|improve this answer
    
I can confirm that the php.ini is readable by the user running php, as I am logged in as that user when editing the values within. php -d memory_limit=512M produces the same result as setting the value within the php.ini file. Thank you for the suggestions :) –  Jess Telford Jun 23 '11 at 0:33
1  
FYI: I tried calling php -d memory_limit=2G /path/to/test.php with test.php containing: <?php $str = ""; while(true) $str .= "M";?> which resulted in the same error message: Fatal error: Out of memory (allocated 362545152) (tried to allocate 362282961 bytes) in /path/to/test.php on line 1 –  Jess Telford Jun 23 '11 at 7:10

The Linux per proc/user limit has nothing to do with this; the error you are getting is specifically tied to PHP.

On that note, I suspect there is ini_set directive somewhere that overwrites your limit; even if you put ini_set on top level file, the lower level file will overwrite it.

Simply run `grep -r -i "ini_set" /path/to/piwiki" and see if they are manually overwriting the limit.

share|improve this answer
    
Thanks for the tip. The output of `ack "ini_set" /path/to/piwik/ is here: pastebin.com/kDJHLC3e - You can see on line 47 of the output that Piwik is messing with memory_limit. After further investigation, it is simply trying to raise the value if it is too low. Hence, it is actually helping not hindering :( Good idea, though! –  Jess Telford Jun 23 '11 at 7:02
1  
Well it could be raising it too low ;) If anything var_dump($minimumMemoryLimit) to see what it's raising it to. –  Aleksey Korzun Jun 23 '11 at 15:59
    
It was raising it to the expected value I had set in httpd.conf - thanks for the thought though :) –  Jess Telford Jun 29 '11 at 1:07

Can you confirm which config file you modified the memory limits in? Note that there are two different configs: one for PHP executed via the webserver and the second for PHP executed via the CLI.

~$ ls /etc/php5/
apache2  cli  conf.d
~$ ls /etc/php5/apache2/
conf.d  php.ini
~$ ls /etc/php5/cli/    
conf.d  php.ini

Also, if this being executed via a browser, in the same directory as where the actual script lies, create a temp.php file with the following contents:

<?php phpinfo(); ?>

Check what the memory_limit shows up as when you browse to that file.

share|improve this answer
    
The script was being run from the terminal. As stated in the question, the output of php -i confirms that I have set memory_limit in the correct place. Thank you for the input. –  Jess Telford Jun 29 '11 at 1:22

I have this problem last week, my solutiosn in CentOS rehel is

nano /etc/httpd/conf/hpptd.conf

search RLimitMEM directive delete or comented no uses this directives or if use allow 2000000 value

  • save this file
  • restart the service
  • service httpd restart
share|improve this answer
1  
As noted in the question, I have already tried this with no success. –  Jess Telford Feb 13 '12 at 0:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.