Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I'm currently trying to process a bunch of data with apache and php,
I estimate it will take a few hours and I'd like to reduce that time

In my webmin I see following information

CPU usage   25% user, 0% kernel, 19% IO, 55% idle

is there a way i can dedicate more cpu usage to the process temporarily? (even if it means changing configs and restarting apache twice)

it's basically a lot of mysql selects/inserts/updates and regex

The processor is Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz, 4 cores

Thanks for any tips on how to improve the speed

edit The CPU usage list shows this

ID      Owner       CPU     Command   
26841   mysql       38.6 %  /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file ...
5627    www-data    0.3 %   /usr/sbin/apache2 -k start
share|improve this question

2 Answers 2

Given that you've got a quad core CPU and you're using exactly a quarter of it, I'd be inclined to say that you're running all your data processing in one single-threaded job. This will only use one CPU core. Rewrite your code to execute several tasks in parallel (either through threads or multiple independent processes) and it will use all of your CPU cores correctly. This is not a trivial operation.

Note that questions regarding parallelising your code should be directed to Stack Overflow.

share|improve this answer
    
it seems to be mostly dependant on MySQL, but is it possible to have multiple mysql processes? –  Moak Jul 8 '11 at 9:26
    
If you mean mysqld on different ports, then no. I am more than sure, it is not possible, as you would do sharing violation within database files. –  John Jul 8 '11 at 9:53
    
No, you only run one MySQL server, but that's multithreaded so it'll use multiple CPU cores if they're available. But if you're only issuing one query at a time, it can only do one thing at a time. Again, parallelise your data processing to better utilise your system resources. –  womble Jul 8 '11 at 10:16

MySQL from version 5.0 is already multi-threaded and optimized for running on multi-core systems. Is your disk subsystem running well ? Maybe the disk is representing the bottleneck.

Otherwise, you have to recode your PHP scripts in order to run as multiple parallel processes as stated above. PHP does not fully support threading within one instance, or it is struggle to use it.

@edit: Or you can divide data to some blocks if it is possible and process it by running multiple different PHP scripts.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.