2

Has anyone experienced duplicate requests sent to multiple EC2 instances using application load balancer?

We have an API on EC2 fronted by ALB and recently noticed that some requests are being sent to multiple instances and causing an issue as certain actions like payments are happening more than once.

The API is built around .net core, but don't think this is important. The issue started recently and it does not happen all the time.

Has anyone seen something like this. I have seen some older issues from around 2011/2012 but nothing more recent.

3

Enable ALB access logs (they are delivered to S3) and analyse them.

More likely then not you'll find that it's actually your clients sending the request multiple times - either through user interaction (e.g. user reloads the page while waiting for the payment processing) or caused by your client side app (JS times out waiting for a response and re-sends the request). The logs will tell you.

Hope that helps :)

  • I believe you are right, as the front end is a ReactJS app using ajax. The request is sent upon button click. But frontend team says as button is clicked the UI is blocked therefore doubleclick cannot happen apparently testing team tried clicking button a few times but find it gets blocked after first click. This therefore is going back and forth as there is no evidence its the ajax. – AliK Jul 23 at 4:00
  • 1
    @AliK as of now, you also have no evidence that it is the balancer. The ALB logs include fields like request_creation_time and trace_id (which is also injected into the request headers as X-Amzn-Trace-Id, so your server can capture it for correlation of app logs and balancer logs) that can be used to help pinpoint the nature of the issue you're experiencing. – Michael - sqlbot Jul 23 at 10:02
0

We also have observed the same thing with a classic ELB (as opposed to your ALB). We have evidence that occasionally (a dozen of times per day in a traffic-heavy environment) 2 instances receive the exact same request, and only one of those instances is in the ELB access log.

Also, by our logs, this has been occurring for at least 3 months (and perhaps forever), but it may have gotten worse since mid-July.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.