I am looking for a project which would simplify EC2 on-demand instance management for script execution. Here is my scenario.
- script waits for new job
- my script reads execution parameters
- it executes external python script for long-term processing
- script handles output from python script
- loops to the start (goto 10)
Instead of executing cpu intensive python script on the same server, i'd like to delegate it onto an on-demand EC2 instance. I am looking for a project to help me with the following:
- provide script built on top of ec2 for instance management
- i suppose, once instance is up, it might ssh into it and set up necessary environment, such as upload data file
- perform application execution in this new instance
- collect results form instance and terminate instance
I know that this can be all done through some insane scripting, but I'd prefer to use existing software for this. Please suggest a software packages / libraries or tools I can use.