0

We have some super heavy APIs, say /api/heavy, it will slow down our system when peak time, we did some benchmark, if the simultaneous request count is greater than 200, the system becomes slow, and if greater than 600, our system becomes unavailable.

We are unable to add redis or memcached layer right now, because it needs to update the source code which is not possible right now for some contract problem.

So we are thinking to put some cache server in front of our API server, and cache these APIs for 10 seconds when the request rate > 500 and 3 seconds when the rate > 150.

How can we do this using Nginx or Varnish? Or other solutions? CDN?

3
  • Pretty sure you can do this with CloudFlare, but you'd need one of the higher subscription plans. It sounds like something that should be possible with Nginx as well.
    – Tim
    May 9, 2018 at 2:10
  • Does this API returns the same answer for all clients?
    – Alexey Ten
    May 9, 2018 at 6:41
  • @AlexeyTen no, it differs
    – Sato
    May 11, 2018 at 1:24

2 Answers 2

1

You can achieve this by nuster cache server

# cache /heavy for 100 seconds if be_conn greater than 10
acl heavypage path /heavy
acl tooFast be_conn ge 100
nuster rule heavy ttl 100 if heavypage tooFast

I don't know if the API is private or shared? In case it's private, meaning the result of the API differs per user, you can also cache /api/heavy per user like this:

nuster rule heavy key method.scheme.host.uri.cookie_sessionID ttl 100 if heavypage tooFast

Hope this helps

1
  • Works perfectly
    – Sato
    May 22, 2018 at 2:22
0

Possible approach using Varnish: during vcl_recv use Redis VMOD (disclaimer: I'm the author) or some throttling VMOD (e.g. vsthrottle included in https://github.com/varnish/varnish-modules) to check / update request rate to the API end-point. If not over the limit skip the cache simply doing a pass. Otherwise do a hash and cache as preferred during vcl_backend_response. This can be easily generalized if more than one limit is needed.

vsthrottle is way simpler than the Redis approach, but obviously is local to each Varnish instance. Usually that's ok for a throttling scenario, so no need to add Redis to your stack in most cases.

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .