Hot answers tagged

40

I've hit a few pages in Google where you can set the Header in S3 for individual objects. That's really not a productive way to do it specially since in my case we are talking of several objects. Well, "productive" or not, that is how it actually is designed to work. CloudFront does not add Cache-Control: headers. CloudFront passes-through (and also ...


18

The curl client isn't caching files, but the remote server network might well be. Try adding an arbitrary query string variable to the URL to see if you can reproduce it.


11

In addition to what answered by @Gabriel-Talavera, I'd add a couple of notes: Network routing, as well as Geographical Load Balancing, is totally unrelated to "data synchronization" between different servers. They are two problems addressed with plenty of very different technologies. As the title of your question seems to be focused on the networking side, ...


9

Ok it seems I can now give a rough answer to my own question. Anurag Bhatia says that there exist two methods how a CDN works: DNS Have DNS to do the magic i.e when users from network ISP A lookup for cdn.website.com, they should get a unicast IP address of Cache A in return, similarly for users coming from ISP B network, Cache B’s unicast IP ...


9

(Updated for future reference) Let's say your CloudFront distribution is in account 123456789012 with logging configured to a bucket your-logging-bucket in a different account. Create a S3 Bucket Policy that gives the CloudFront account 123456789012 permissions to do s3:GetBucketAcl and s3:PutBucketAcl on your-logging-bucket. This is the required Bucket ...


7

Newer CDNs (Cloudflare, MaxCDN, fast.ly) use anycast for both DNS and actual content servers. This is somewhat better than trying to use the source IP of a DNS query and an ever changing mapping database. In theory using anycast for both the DNS and content servers allows the network itself to find the "closest" servers to the client. In practice, this is ...


7

This behavior likely results from the fact that by default CloudFront sets the Host: HTTP request header to the origin hostname, in this case elb.example.com. The application then presumably generates links based on that hostname. If, instead, you configure CloudFront to whitelist that header for forwarding to the origin, the Host header sent by the browser ...


6

Belatedly, try: curl -v -H "Cache-Control: no-cache" That will tell the web server to not cache. Doesn't stop layers below caching unless it's coded to obey the headers.


6

While every provider is different and may have different reasons for making the same choices, one common reason this is done is because of Cookies. If your website makes use of cookies and your cookies may need to be used for multiple subdomains, then you will end up sending cookies along with every CDN request. This causes two issues: If you have a lot of ...


5

CloudFront requests come from the documented IP ranges as well as with a User-Agent string that includes Amazon CloudFront. You can block either, but with AWS's IP ranges expanding fairly frequently I'd go with the User-Agent block.


5

There are two basic mechanisms to route requests to topologically proximate servers: Split-view DNS… like you said. Anycast routing: BGP is used to tell an ISP's router to route traffic for an IP address to an optimally located host. Of the two techniques, split-view DNS is simpler to implement. However, anycast routing has the advantage that it works ...


5

Do I really need cdn for video delivery? "It depends". It depends on where your users are, how many there are of them, what's their concurrency needs, if your infrastructure can deliver what's needed on its own and how much you want to spend on delivery. You really need to understand these factors before deciding if a CDN works for you. As an example say ...


4

After a quick check with some resources, it appears that you can rent a botnet capable of that much throughput at somewhere about 25-50$ USD per hour. For two weeks that means that they paid anywhere from 8400$ USD to 16,800$ for the attack. It sounds like you only have circumstantial evidence pointing to your competitors, and that seems to be why the ...


4

Quite possibly you might want one anyway. They tend to be good at not being vulnerable to DDoS, and make it more difficult to find and attack your real servers in many cases. Not all of the Internet's topology is geographic. If you want to use one, select one which is well-peered to ISPs which serve your target market.


4

This has nothing to do with your site's size or anything. Wordpress is almost completely blocked in China. The government blocks nearly all sites like that, or makes them so damned slow that nobody wants to use them. I've lived in China for nearly 5 years, and in order to do anything on the web outside of China's control, I've gotta use a VPN. Includes other ...


4

The easiest way to Cache Everything on a given endpoint in CloudFlare is to use a Cache Everything Page Rule, an asterisk will match a wildcard result. So in your first example we can do the following: Cache Everything can only be enabled using a Page Rule, so they above example will best in the vast majority of cases, but it is also possible to set a ...


4

Do it in Akamai. The closer to the edge your redirect takes place, the faster the request response time will be for your users. https://blogs.akamai.com/2016/02/redirecting-on-the-edge.html If you're confident that you can serve your entire domain over HTTPS now and into the future then you should also look into enabling HSTS. This will mean only first ...


4

I think it would be easier to simply set reasonable Cache-Control: max-age=... headers on the origin responses, than to manually force the caches to refresh certain content. Then, on the nginx caches, if you have proxy_cache_revalidate on on, the caches will only check the origin for particular content once every max-age period. If the content on the origin ...


4

Yes. You need to implement your geo blocking at the CDN level. This usually means paying for the CDN as this is typically not included in free tiers (they'll give you space for a few rules, but the kinds of rules you need for geo blocking will normally exceed the free allocation). You can geo-block based on the X-Forwarded-For header but as you've correctly ...


4

Put your server behind a CDN / DDOS protection vendor such as CloudFlare / AWS CloudFront + WAF and ensure that your server only accepts direct connections from your vendor and your own static IP's. The general steps for this would be: Sign up for the service and subscribe to the plan you require Set up DNS on your service to point at your host. Make sure ...


3

Neal, IMO I would go with AWS, utilizing S3+cloudfront. S3 is extremely cheap storage, and cloudfront is very inexpensive as well. I suggest checking out the AWS Simple Monthly Calculator to figure out your expenses: http://calculator.s3.amazonaws.com/index.html Depending on your needs, it might be worth checking out the AWS Free Tier, as if this is your ...


3

The problems with geographically distributing WordPress, as you'll see, all revolve around the MySQL database. There are basically two ways to architect this: Read replicas In this architecture, you run your LAMP frontend with WordPress in several geographic locations, each of which has a read-only replica (slave) of the master MySQL database. You will ...


3

The price, based on my researching less legitimate sites, would be around 200 USD for 24 hours. Now considering you have about two weeks would make me think discounts would be approved so leaving you at about maybe 180 USD / day. So that would make 14*180 USD = 2520 USD. Prices are quite volatile, but in recent years everyone has started offering these type ...


3

Yes, that's exactly how most CDNs work. In addition, many have servers that are co-located in ISPs' datacenters for direct access.


3

First, I'm assuming you have a custom origin, and that it's a web server like nginx that serves the files with a setting like "expires 1w". Also know that the amount of traffic you receive could also effect the ratio of hits to misses. For starters, you need to see how Cloudfront sees your objects as they are served from your origin. I use Chrome Devtools ...


3

You can give selective DNS responses based on location with BIND Views if you are using BIND as your external DNS server. The Technical Preview of the new version of Windows Server also has a feature called DNS Policies which looks very promising. To serve content based on client location and other criteria such as User Agent or schedules, F5 has an ...


3

Don't bother micromanaging and overanalyzing. Just use the keys and cookies now because that's good practice to protect your assets and bandwidth anyway. It's an easy decision to make, and you'll not have to spend four times the effort to track abusers.


3

Most websites using a CDN make use of a subdomain (e.g. cdn.yourwebsite.com) in order to deliver static assets from. This means that in order to deliver CDN assets over SSL you will need a certificate for the "cdn" subdomain. If you already have an SSL certificate installed on your origin, its likely that it is valid for yourwebsite.com and www.yourwebsite....


3

I will start from the end of your question and dig my way up. So first of all, there is no way for proxy (Cloudflare acts like one) to distinguish between static content and the one generated on fly, the responses are virtually identical. The only difference could be extension of the requested file and according cache rule send by your server. The Cloudflare ...


Only top voted, non community-wiki answers of a minimum length are eligible