0

I have a client who send out a weekly newsletter to around 3,000 recipients. The newsletter is built using around 50-100 images depending on the template they use (which I recognize is horribly efficient, given the design of the newsletter).

However until recently, the send out caused their web server to become non-responsive since it was a single web server handling the requests for both static assets and PHP, causing the web server to run out of apache children and not be able to serve fast enough. Measuring the web server right after a send out we observed the following:

  • Sample time of about 15 minutes
  • Consistently above 300 requests per second (the vast majority being static assets, very few web site visitors)
  • Peaked around 800 requests per second
  • Approx. 100 recorded newsletter views
  • Approx 50 recorded newsletter clicks (the majority leading to the website itself)

We fixed the issue temporarily with a secondary web server dedicated to serving static assets, with a much leaner apache configuration. This resulted in a newsletter send out that the web site itself barely flinched at.

Our confusion revolves around how such a massive surge of requests for static assets does not match the number of newsletter views and clicks, is this something anybody else has seen, and are there any other ways to deal with such traffic? I imagine reducing the number of static assets and a separate static assets web server should get us most of the way, however I'd really like to be able to explain this phenomenon.

Thanks for any advice!

Improve this question
4
  • This all sounds very cache'able or CDN'able - is that right?
    – Chopper3
    Jul 9, 2012 at 10:07
  • 2
    What phenomenon? People open your spam (triggering asset loads), decide they don't want it, and don't visit your website (or buy from you ever again). Seems pretty straightforward to me.
    – womble ♦
    Jul 9, 2012 at 10:21
  • @Chopper3 it's very CDNable, yes, but I doubt the client is willing to pay for the service. I suspect we'll end up running 2 webservers, 1 to catch static asset requests, and then pass on page requests to the 2nd web server.
    – DanH
    Jul 11, 2012 at 3:12
  • @womble Mailchimp is only recording 100 newsletter views, which would surely be triggering if people were viewing the newsletters, whether in spam or not. Unrelated: The site sells no product, newsletter signup is only done directly via the subscribe form, there's no tricking people to subscribe here.
    – DanH
    Jul 11, 2012 at 3:12

1 Answer 1

Reset to default
1

Approx. 100 recorded newsletter views

And how is that recorded? I would count the views by embedding a non-cacheable image in the newsletter - but that's obviously not the case here. You certainly can't rely on SMTP DSNs nor javascript.

But I would still expect that a number of the 'viewers' will be malware scanners and compressing proxies (you should be able to determine from the source ip / user agent).

(BTW wouldn't it have been cheaper to stagger the email delivery or use a CDN rather than implement a new server?)

Improve this answer
2
  • The views and clicks are reported by Mailchimp, which I'm fairly sure inserts a non-cacheable image like you say. The reason another server was used is because it was handy to make use of in a short time frame.
    – DanH
    Jul 10, 2012 at 4:35
  • 1
    I expect you'll find (with a little bit of digging) that you have prima-facia evidence that mailchimp are not getting an accurate count of email views.
    – symcbean
    Jul 10, 2012 at 8:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .