I have a web application, A, that returns a number of files, stored on disk and in memory, based on client device requests. This web application has to scale for a huge amount of users. I am using a cluster to handle all the requests. I also have another web application, B, that deploys the files to application A that clients download. I want to know what is the ideal configuration for B to deploy to all A's. There is a single instance of B running at any time, possibly on the same machine as one of the A's. Also, A's have no state. All requests carry all the data required to handle them. They are only concerned with having the most up to date files from B.
The solution I have right now is for B to periodically do an HTTP sync with my A servers who then refresh their cache (this doesn't happen very often, maybe a few times a week). So all A's have copies of all the most recent files deployed by B. When you need an extra A, you simply register it with B and they can sync just fine.
I had also thought of having B deploy files to a single repository where a Master A is running and where Slave A's would sync at given intervals. But I no longer want there to be a distinction between master and slave servers. This way I can have a single configuration for everything.
Is there a better way to do this with the information I have given you?