I have a client who send out a weekly newsletter to around 3,000 recipients. The newsletter is built using around 50-100 images depending on the template they use (which I recognize is horribly efficient, given the design of the newsletter).
However until recently, the send out caused their web server to become non-responsive since it was a single web server handling the requests for both static assets and PHP, causing the web server to run out of apache children and not be able to serve fast enough. Measuring the web server right after a send out we observed the following:
- Sample time of about 15 minutes
- Consistently above 300 requests per second (the vast majority being static assets, very few web site visitors)
- Peaked around 800 requests per second
- Approx. 100 recorded newsletter views
- Approx 50 recorded newsletter clicks (the majority leading to the website itself)
We fixed the issue temporarily with a secondary web server dedicated to serving static assets, with a much leaner apache configuration. This resulted in a newsletter send out that the web site itself barely flinched at.
Our confusion revolves around how such a massive surge of requests for static assets does not match the number of newsletter views and clicks, is this something anybody else has seen, and are there any other ways to deal with such traffic? I imagine reducing the number of static assets and a separate static assets web server should get us most of the way, however I'd really like to be able to explain this phenomenon.
Thanks for any advice!