I am currently building a simple webapp architecture on AWS, as follow: Elastic Load Balancer - multiple front ends - RDS database.
I have also set up an NFS server to store all images uploaded by users through the web app, such as profile pics mainly (I can't use S3 for this, I am only following the specs I get...).
The front ends VMs are spawned by an autoscaling group. When a VM is spawned, it downloads the web app sources from an S3 bucket. The problem is, I have some huge files within those sources (static files mainly, such as mp4 or images) I am thinking about storing those big static files on the NFS server as well, because it takes too much time to download from S3.
So My question is: When is it relevant to store a file on the NFS, and when is it be better to let it on each front end? e.g if I store mp4 on the NFS, why not storing the whole web application on this NFS?
note: I don't want to build an AMI containing those files, as it is not flexible enough when modifying them