I have a PHP script that uses cURL to load a file from another server.

The file is about 24MB. I understand why the script itself loading the file would take a bit to execute, however any request to the site while the script it running will hault until the script finishes.

This did not happen on our old shared server with the same script. The new server is a cloud server. I took it up to 10 nodes (6Ghz dedicated cpu, 3760MB ram, 2500GB bandwidth) and it had no effect on this issue.

I don't mind the script itself taking a long time to execute since it is going to be an automated task for a data feed. I can't have the whole site locking up while it runs though.

Any ideas why this could be happening?

Update it looks like this is only happening locally. If I try to load the site on a separate computer while the script is running it works as expected.

  • What's PHP running on? – Shane Madden Aug 10 '11 at 17:52
  • @Shane it is a LAMP stack (Ubuntu lucid 10.04 x64, Apache 2) on a vps.net cloud server. – JD Isaacks Aug 10 '11 at 18:24
  • Which Apache MPM, and what module are you using for PHP (mod_php, fcgi, or other)? – Shane Madden Aug 10 '11 at 18:33
  • @Shane, not sure, how can I find that out? – JD Isaacks Aug 10 '11 at 18:39
  • apache2 -V will give you the MPM. For the module, check what's in /etc/apache2/mods-enabled - if both are in there, you'll need to look at your config to see which is used. – Shane Madden Aug 10 '11 at 18:53

You might be able to "ignore_user_abort(true)" and then use an HTTP redirect on the page to another page.

PHP 5 also has decent threading, that might help as well.


Now, the key question is: what else does your script do than fetches that file? Processes it and inserts the data to database? Perhaps in your old environment you had MySQL using InnoDB tables and in your new environment MySQL with MyISAM tables.

MyISAM requires full table lock during write operations instead of row-level locks InnoDB has, so that might cause your site to hang if your actual site is using same tables than the curl script.

  • Thanks, thats a good idea, the old site did process and save to a db like you said, currently the new one isn't doing that yet. Also the old environment was MyISAM and the new one is InnoDB. – JD Isaacks Aug 11 '11 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.