I need to set up some VPSs for serving static content (many small files). I plan on using Nginx for this, and would like to set it up so that we are able to scale out relatively easily. The requirements are:
- Many files (at least hundreds of thousands).
- Small file sizes (less than 10KB).
- Files are being added constantly by an application on neighbouring servers.
- New files must be immediately available to all Nginx servers.
My current plan is:
- Have a 'master' server with an NFS share containing all of the files.
- Application producing new files interacts only with master.
- Have multiple Nginx servers mounting this NFS share.
- Load balance across Nginx instances.
One obvious problem with this is that the 'master' server is a single point of failure (any remedy for this?). Are there other potential issues which I have overlooked? Are there elements here which will not scale well in this way? Would anyone suggest an alternative approach?
Regarding memory requirements, I'm assuming I should give each Nginx server as much as possible so that hot files can be cached (by OS? Nginx?) an not have to be reqested form the NFS share constantly.
Lastly, am I crazy not to use a CDN?