I am using an Nginx proxy as a front-end to a web application that tends to get identical duplicate requests milliseconds apart, causing duplicate items in my database.

I thought I might be able to fix this by letting Nginx cache content for 1s, so that duplicate requests would not hit my app at all

I configured it with

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=prevent_duplicates:10m max_size=1g inactive=1s use_temp_path=off;

And then added proxy_cache prevent_duplicates; to my root location section, where I have proxy_pass

But my application gets hit on all requests anyway - is it a timing thing? Multiple threads all starting at once for each request all thinking they are first?

  • 1
    That sounds more like bad app/DB programming. – tater Sep 1 at 14:02
  • Unfortunately I have no control over the client in this case - the client will always make 2 calls to the application in a row. I can fix it app-side but I thought this would be a cleaner solution – user1456632 Sep 1 at 14:09
  • Eh? Fixing the app is the clean solution. This is just a workaround. – Michael Hampton Sep 1 at 18:57
  • Fixing it in which way? A client over which I have no control performs duplicate requests as a bug in some cases - this is not in the scope of the app. – user1456632 Sep 1 at 18:58
  • It is a bug in the application and the bug needs to be fixed. Report it to the application developer. Implementing any kind of workaround on server side adds unnecessary complexity and introduces more chances of other failures. – Tero Kilkanen Sep 2 at 5:37

Maybe rate limiting can be the solution?

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;
server {
    location /login/ {
        limit_req zone=mylimit;
        proxy_pass http://my_upstream;

Source: https://www.nginx.com/blog/rate-limiting-nginx/

The rate=1r/s can be tweaked, if you put 2r/s NGINX will allow 1 request every 500 milliseconds...

| improve this answer | |
  • I ended up adding a local cache to filter duplicates - it cannot scale since instances do not share a cache but it works for now – user1456632 Sep 23 at 13:33
  • Great, happy that you found a solution. This will probably be the last thing I ever write on Stack Exchange Sites as the communities around here are very prickish and trollish. Wish you all the best. – michacassola Sep 24 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.