So, i'm currently writing my bachelor thesis and my job is to cloudify a webservice which recommends a soundtrack for images. The main part of the recommendation process is a lookup on a ~40gb index file. The index file is read only and reading it should be as fast as possible. I also want to automatically launch additional instances based on demand. I did some research and have 3 possible ways to do it
- Increase the root partition of the ec2 instance (to ~50gb), store the index file on the root partition and create an AMI. The benefit of this approach is, that it is very easy to start a new instance since everything is included in the AMI. But i also read that the speed of the root partition is very slow.
- Store the data on a EBS volume, create a snapshot of it and whenever i launch a new instance, i create a new EBS Volume from the snapshot an attach it to the instance. I suppose this is the best way to do it but launching a new instance is a tad more complex
- Storing the index file on S3 and whenever a new ec2 instance is launched, i download the file to the ephemeral storage of the instance. Problem with that approach is that it takes more time until the new instance is operable as well as the the cost for traffic.
I was also thinking about doing a Raid0 setup to improve the read performance on the index file. So which approach should i use?