2

I have something I'll call a "microservice" running on AWS Lambda (using node.js).

Basically it serves up condensed summaries drawn from a few hundred megabytes of binary blob. There are a lot of possible outputs and pre-generating all possibilities isn't an option, and it needs to be reasonably responsive (sub-second at the worst, say) as it's accessed (via API Gateway) from interactive webpages which allow parameters to be changed rapidly. Access patterns in the blob are essentially random, although any summary produced will typically only have accessed ~0.1-1% of the total data. The data and access patterns aren't very compatible with storing the data in a database (although see mention of DynamoDB below).

My current approach is to have the big binary blob hosted on S3, and have the Lambda handlers cache the blob locally between Lambda invocations (just as a buffer in the javascript code, with scope outside the hander function; obviously the Lambda's memory is configured sufficiently large). Handler instances seem to be persistent enough that, once a server is up and running it works well and is very responsive. However there are at least a couple of downsides:

  • The initial fetch of the data from S3 seems to be at around 50-60MByte/s (seems to be in agreement with other reports on S3 bandwidth I've seen) so there can be an annoying multi-second delay on first access.

  • Related to the previous point, if the client is very active and/or user load increases, more server instances get spun up and users may find their requests routed to instances which are stalled on fetching the data blob, which leads to annoying glitches in an otherwise smoothly functioning client.

I freely admit I'm probably expecting too much from what's really intended to be a "stateless" service by having it actually have a big chunk of state in it (the binary blob), but I'm wondering if anything can be done to improve the situation. Note that the data is not particularly compressible (it might be possible to take 1/3 off it, but that's not the sort of order-of-magnitude I'm looking for, or at least it's just part of the solution at best).

Any suggestions how to get the data into the Lambda faster? The sort of thing I'm imagining is:

  • Pull your data from somewhere else that Lambdas have much higher bandwidth to... but what? DynamoDB (split into as many 400k binary records as needed)? ElastiCache? Something else on the AWS "menu" I haven't noticed.

  • Use some cunning trick (what?) to "pre-warm" lambda instances.

  • You're using completely the wrong tool for the job; use... instead? (I do really like the Lambda model though; no need to worry about all that instance provisioning and auto-scaling, just concentrate on functionality).

If Google or Microsoft's recently announced Lambda-alike offerings (about which I know little) have any attributes which would assist with this use-case better, that'd be very interesting information too.

One option I have contemplated is baking the binary data into a "deployment package" but the 250MByte limit on that is too low for some anticipated use-cases (even if the blob was compressed).

3
  • 1
    Are you using Lambda VPC access and an S3 endpoint for your VPC?
    – Jukka
    May 16, 2016 at 20:12
  • @Jukka. Not yet. I'm vaguely aware of the VPC option for Lambda and have used VPC for some EC2 stuff (t2.micro instances are VPC only) but I never looked at I/O performance there. Should I expect significantly higher performance accessing S3 from VPC via an S3 endpoint? Does that use different infrastructure or bypass some "public internet" gateways to stay on AWS' private net? Certainly sounds intriguing... this is the sort of info/ideas I was hoping to get thanks... any numbers?
    – timday
    May 16, 2016 at 20:51
  • 1
    No numbers, but using the endpoint, S3 traffic does get routed internally. Try it out.
    – Jukka
    May 17, 2016 at 5:42

1 Answer 1

1

If the binary blob is only a few hundred megabytes, you can just include it with your function as a "dependency". You can add it as a file alongside your code and reference it accordingly.

Another option would be to have two lambda functions. One function does nothing but serve up the blob (which you create as above by sending the blob with the function) and then you can use a timer (cron basically) to "tickle" that function every minute to keep it active. Then your second lambda is the one that does the work, and the first thing it does on startup is call the first lambda to get the blob. Lambda to lambda calls are high bandwidth so the startup time shouldn't be a problem.

The ideal solution would be to figure out a way to summarize the data and store it in DynamoDB, but it sounds like you tried that route and it didn't make sense for you.

1
  • As a general rule, trying to keep something warm (or pre-warm it) is likely to be problematic. You're not in control of the underlying implementation, so if your lambda moves to a different physical server or something, then it may or may not need re-warming. Tickling it is as good a way to mitigate this as any, but you trade off frequency against cost of execution. Aug 1, 2019 at 14:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .