Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I need to execute a dataset processing task. It’s a simple php script to warm up memcached machines. The dataset contains the keys (around 24 million)

If I were to execute the warm up script from one machine it will take me nearly 3 days to complete. But if I were to “split” and run 6 differences process it will take me a day.

If I know the ip address of all the other machines in my network and has ssh access, is there a way to parallelize this task?

share|improve this question

1 Answer 1

Take a look at pssh, cluster ssh, ...

  1. setting up ssh public key authentication
  2. creating a list of your machine:

    ip1:port1 user1
    ip2:port2 user2
    ip3:port3 user3
    
  3. distributing your script to all the servers and parallelize:

    $ pssh -h <host_file> -P /path/to/php/script
    
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.