Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Help!

Here's the scenario:

I'm hosting and serving display ads. These consist of some media creatives and some javascript files to assemble and display the ad. The files are hosted on a CDN.

We did a small test run with 1 ad over the weekend. It burned about 10,000 impressions. When reviewing my CDN logs, I notice that one single IP, was requesting a single media file, on average about 2-3 times per second to the tune of over 1.6 million hits on the file. All in within a 24 hour period!

Now this is a big problem because I'm charged for bandwidth, and currently we've transferred over a TB for no apparent reason.

Why is this happening?? What can I do to prevent anything from accessing files directly like this? They should only be accessed when the javascript calls them into an ad placement.

Lines from CDN log:

#Fields: timestamp time-taken c-ip filesize s-ip s-port sc-status sc-bytes cs-method cs-uri-stem - rs-duration rs-bytes c-referrer c-user-agent customer-id x-ec_custom-1
1305405902 116 XX.XX.XX.XX 1281559 XX.XX.XX.XX 80 TCP_HIT/200 327990 GET http://XXXXXXXXXX.ogg - 0 557 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729)" 10629 "-" 
1305405902 89 XX.XX.XX.XX 1281559 XX.XX.XX.XX 80 TCP_HIT/200 655670 GET http://XXXXXXXXXX.ogg - 0 557 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729)" 10629 "-"
1305405902 86 XX.XX.XX.XX 1281559 XX.XX.XX.XX 80 TCP_HIT/200 453386 GET http://XXXXXXXXXX.ogg - 0 557 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729)" 10629 "-"
1305405902 7 XX.XX.XX.XX 1281559 XX.XX.XX.XX 80 TCP_HIT/200 1281869 GET http://XXXXXXXXXX.ogg - 0 557 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729)" 10629 "-"
1305405903 86 XX.XX.XX.XX 1281559 XX.XX.XX.XX 80 TCP_HIT/200 786742 GET http://XXXXXXXXXX.ogg - 0 557 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729)" 10629 "-" 
share|improve this question
1  
Are you sure this isn't due to a bug in your javascript that only occurs on some platforms? A browser issue, perhaps? –  Steve314 May 17 '11 at 20:48
add comment

migrated from stackoverflow.com May 17 '11 at 20:50

This question came from our site for professional and enthusiast programmers.

2 Answers

You could...

Set the expiration header on the http response...

Since you don't mention the CDN I don't know how they will let you influence that. They might be making big bucks off poor client by not allowing it :) In that case, run, don't walk, to a CDN that will allow it

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.21

Edit A single IP? You did run whois yet... Perhaps you can contact the abuse contact from there

share|improve this answer
    
I've submitted this recommendation to my CDN's customer support. Also, WHOIS returns the IP as block of IPs belonging to the State of Ohio. –  Anonymous May 17 '11 at 20:40
    
FROM MY CDN SUPPORT: The default max-age is set for 7 days, if you would like to set a different value for the max-age we can do it. Not sure how it will prevent the access to you files, as its just controls for how long the file will be cached on the Edge and on the web browser. –  nuecaster May 18 '11 at 1:13
    
Sry for late reply. Obviously, the guess was that the CDN wasn't providing expiration headers at all. 7 days is enough (it really depends on your use case), but at least enough to prevent accidental hit-to-death by a regular browser. –  sehe May 25 '11 at 7:58
add comment

I've seen this before (ironically enough, with an ad network I was once contracted to) -- they had a bug in their Javascript that (under certain, rare-but-not-unknown circumstances) caused the client to head into an infinite loop and continuously attempt to retrieve ads. They were running pretty close to the limits of their hosting infrastructure already, and doing 80+million impressions a day, so it only took a small percentage of misbehaving clients to cause some serious damage to their uptime -- it was effectively a self-imposed DDoS. The devs fixed that bug real quick, but it took a couple of days for the last misbehaving client to go away (people leave their browsers open for a lot longer than I would have expected).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.