So I got a unique problem. Our app is designed to server large amount of files using a special link that is one time use. We have been playing with ELB but notice an interesting "bug", if you start to get a lot of traffic or load it looks like ELB will try to scale out and retry the connection. This in turns causes our system to invalidate the link before the download is done.

So quickly I put together a nginx proxy server to test the issue and in that setup the connection does not get invalided but I am seeing performance drop by about 2/3 to 1/2. Seeing as I cant throw more bandwidth at the instance as I am locked to what AWS gives me what alternatives do I have? Maybe there is a configuration with the instance/nginx I am not doing correctly or do I need to move to something like RR DNS and pay the price with that.

  • Do you have the option to switch from one time links to another design solution? Single use links seem like they are a good answer to certain problems but are often not. You've found one reason why, but there are more. – Eric Hammond Sep 20 '11 at 1:19
  • @EricHammond Currently no. The way our app is setup this is one of the many layers we provide for security of the link. We can look in other options but right now the feeling is lets modify the infrastructure vs our code. – Zuhaib Sep 20 '11 at 2:39

If www.example.com is being load balanced to server1.example.com and server2.example.com, have your one-time link explicitly link to server1/2.example.com (or the current instance's AWS CNAME, if your instances change frequently enough that your DNS isn't necessarily up-to-date).

  • Actually what you suggested is the route we did to get around the ELB issue. – Zuhaib Jan 16 '12 at 8:38

I'm not sure what the requirements are for your application, but since you're already using AWS, you might consider using a feature of S3 that allows you to give out a signed URL that expires at a specific point in the future (e.g., in two minutes). This would your user retry failed connections but would not let them pass it around for other people to use past the expiration time.

If S3 works for your app, it could also remove a huge burden of having to run the download servers yourself as S3 takes care of all the work.

  • Our system has some secret sauce so the data stored in S3 is not readable without our app. So passing the S3 link would not work. – Zuhaib Sep 22 '11 at 17:45
  • Zuhaib: Got it. Perhaps you could implement a time-based expiration URL in your own app instead of single-use. – Eric Hammond Sep 23 '11 at 22:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.