1

I need to execute a dataset processing task. It’s a simple php script to warm up memcached machines. The dataset contains the keys (around 24 million)

If I were to execute the warm up script from one machine it will take me nearly 3 days to complete. But if I were to “split” and run 6 differences process it will take me a day.

If I know the ip address of all the other machines in my network and has ssh access, is there a way to parallelize this task?

Improve this question

1 Answer 1

Reset to default
3

Take a look at pssh, cluster ssh, ...

  1. setting up ssh public key authentication

  2. creating a list of your machine:

    ip1:port1 user1
ip2:port2 user2
ip3:port3 user3

  3. distributing your script to all the servers and parallelize:

    $ pssh -h <host_file> -P /path/to/php/script
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .