Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We run a meta search engine (price comparison) where each search spawns a number of real-time searches on various websites and presents the merged and sorted search results on our site.

We use PHP/MySQL/Apache on a Linux Debian server in a quite straight-forward setup, but the background processing of searching multiple sites in parallel is handled by Tomcat and a Java servlet. However, because of some problems with this setup (Tomcat), I am investigation new approaches to the background searching.

One way that looks promising is also very simple: From the main search page, each individual search script is run as PHP CLI using exec():

exec("nohup /usr/bin/php search.php '$params' &> /dev/null &");

Using the ampersand in the end sends the PHP CLI scripts directly to the background and the main page can continue without waiting. There are about 20 scripts running for each search.

It seems to perform well enough, but when I run a stress test using Apache Benchmark with concurrent requests, problems start to show up. What happens is that MySQL reports this error:

Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (11)

It seems to be a variation of "Too many connections" but it does not help to raise the limits in my.cnf. But what really puzzles me is that these errors are present even if I call a small dummy script that does nothing, and certainly does not access MySQL. So, it looks like just running PHP CLI puts load on MySQL even if it is not used.

There are at most about 700 instances of PHP running during my tests, and the CPU and memory load is lower than when using Tomcat for calling the PHP scripts, so from that point of view it looks feasable.

Does someone have a clue about the MySQL problem? And of course, I am open for new ideas of how to handling the background tasks!

Regards, Martin

share|improve this question

3 Answers 3

Sorry for the late reply. I have been down with the flu.

Thanks for your feedback. There is a lot to think about!

I have done some more experimenting with calling the scripts with wget instead of as CLI scripts, and furthermore having Lighttpd serve these special search scripts on port 81 - to keep the burden off the main Apache server.

So far it looks promising, but I will perhaps ask again when I have done a bit more testing.

Edit: I am the original poster, don't know why it looks like somebody else's post. Perhaps something with my OpenID...

share|improve this answer

I'd say the spawning of jobs is a little too simple, and certainly open to a starvation of resources situation.

To avoid that, I'd use some form of queue - either through a database table that lists what is required, and is then checked regularly, or a job-queue system, something like Gearman or Beanstalkd. You put a message into the queue - what you want to search for - and a background task, which is already running, that picks the jobs up as they come in and it has the capability, performs the appropriate tasks and passed the information back, either via the DB, or some in-memory system like Memcached. The front-end system can refresh (or use Ajax) to call back and see if any info has been found yet, displaying it as appropriate.

That separates the front and back-end, and avoids having to close all the connections your script has open (see the warning on exec)

share|improve this answer

Martin,

I am definitely not an expert on this, but a couple of things come to mind.

I think that php loads support for mysql whether you use it or not in your script, but it shouldn't start a connection unless you are using Persistent connections.I wouldn't think that the cli scripts would inherit the mysql connection from the Apache child, but apparently it must based on your testing.

You might also be running into to too many files limit on mysql. There is also reference to running out of RAM/resources being a cause of the mysql error. You said that it is showing less load than your old setup, but could it be too much? When these scripts are running, is there an opportunity to start a connection to the mysql server using the GUI (mysql administrator)? If so, you could potentially see it run out of resources...

It might be more efficient from a database point of view to have your 20 searches only be in one script, rather than spawning multiple scripts. This would allow you to use one mysql connection for all 20 searches and should increase your connection capacity accordingly. In other words, Web Script ---- spawns -----> CLI script in the background which loops through all of your searches.
Not knowing your code, or exactly what you are doing, this is just a guess as a coding fix.

Good luck

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.