3

I have a few terabytes of data that are processed by a tool. The output of the tool should be sent to two other machines simultaneously (simultaneously, because there's downtime while the tool runs and I want to limit the downtime).

If I just pipe from one machine then it's easy:

tool terabyte.txt | ssh [email protected] /sbin/process-input

but how can I send the data to multiple machines simultaneously?

I do not mind if the situation involves other software or scripts. it doesn't have to be a "pure ssh" solution.

3 Answers 3

3

You can try with ClutterSSH :

With ClusterSSH it is possible to make a SSH connection to multiple servers or machines to perform tasks from one single command window, without any scripting. The ‘cssh’ command lets you connect to any server specified as a command line argument, or to groups of servers (or cluster nodes) defined in a configuration file.

0
3

You could try using tee

tool terabyte.txt | tee >(ssh [email protected] /sbin/process-input) | ssh [email protected] /sbin/process-input
3

Perhaps, pee from moreutils might be even better:

pee is like tee but for pipes. Each command is run and fed a copy of the standard input. The output of all commands is sent to stdout.

So,you can run

tool terabyte.txt |pee 'ssh [email protected] /sbin/process-input' 'ssh [email protected] /sbin/process-input'

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .