[I asked this on stackoverflow.com, but they thought that this list would be better]
I have a slowly evolving dynamic website served from J2EE. The response time and load capacity of the server are inadequate for client needs. Moreover, ad hoc requests can unexpectedly affect other services running on the same application server/database. I know the reasons and can't address them in the short term. I understand HTTP caching hints (expiry, etags....) and for the purpose of this question, please assume that I have maxed out the opportunities to reduce load.
I am thinking of doing a brute force traversal of all URLs in the system to prime a cache and then copying the cache contents to geodispersed cache servers near the clients. I'm thinking of Squid or Apache HTTPD mod_disk_cache. I want to prime one copy and (manually) replicate the cache contents. I don't need a federation or intelligence amongst the slaves. When the data changes, invalidating the cache, I will refresh my master cache and update the slave versions, probably once a night.
Has anyone setup a http cache and then replicated it? Is it a good idea? Are there other technologies that I should investigate? I can program this, but I would prefer a configuration of open source technologies solution.
ps context: The root problem is certainly:
- Database query load on your DB server.
- Business logic load on your web/application server.
The response time is often dozens of seconds (please, don't ask). As mentioned, I cannot address them in the short term (or rather, I am addressing them, but there are a great deal of them, and they are not JSP based and ....). I have clients with USA, European and Asian users, so I would very much like to replicate the cache once I have primed it. For internal corporate users, Akamai-like is not appropriate. I'd like to tar, zip the cache and FTP it back to the slaves. In other cases, the cache server, but not the app needs to be on a DMZ