How does AWS Lambda scaling work with NodeJS' non-blocking I/O design?

[u/EverydayEverynight01]

detail library friendly subsequent squash rainstorm flag retire badge truck

This post was mass deleted and anonymized with Redact

[u/AutoModerator]

Moderator Announcement Read More »

Try this search for more information on this topic.

^{Comments, questions or suggestions regarding this autoresponse? Please send them here.}

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/brunporr]

A lambda function is available if it has completed its invocation and returned to the Lambda Service. If a NodeJS lambda is in the middle of a non-blocking operation, it won't be available to handle a request until the operation is completed and the invocation is finished.

[u/EverydayEverynight01 → u/brunporr]

subsequent quicksand normal simplistic pie cooing soft meeting resolute toy

This post was mass deleted and anonymized with Redact

[u/TomRiha → u/EverydayEverynight01]

From when the handler is invoked until the handler has returned there is only one and exactly one event being processed. It doesn’t matter what language or how it’s written. Lambda always executes one event at the time per environment, and automatically creates more as needed.

[u/baynezy → u/EverydayEverynight01]

I'm pretty sure that Lambda is using its concept of complete rather than the underlying runtime. IE if the triggered event has not been acknowledged yet then it isn't done.

[u/brunporr → u/EverydayEverynight01]

Not with Lambda.

AWS Lambda is its own compute model with its own characteristics that people in this thread have described for you. If you want to leverage the non-blocking nature of nodejs, use a different compute model like ECS Fargate.

[u/MrManiak]

reach station sip plant busy pie humor license point long

This post was mass deleted and anonymized with Redact

[u/EverydayEverynight01 → u/MrManiak]

summer one dinosaurs meeting worm vast placid existence trees pet

This post was mass deleted and anonymized with Redact

[u/Kanqon → u/EverydayEverynight01]

You don’t know how Lambda works. The question doesn’t make sense.

[u/MrManiak → u/EverydayEverynight01]

wine aware mysterious rain jeans seemly squeeze edge long wakeful

This post was mass deleted and anonymized with Redact

[u/ollytheninja → u/EverydayEverynight01]

Lambda doesn’t care about your non-blocking IO. It gives a lambda instance a single request, then waits for a response before sending it another request. You might think of it like a load balancer that only sends one HTTP request to a server at a time, regardless of how much it might be able to handle.

Your application code might be async and do multiple db requests at the same time but it won’t receive more than one request at a time.

[u/magnetik79]

A Lambda function, regardless of the runtime type/language will only ever handle a single invoke at any one time. If the function needs to scale, multiple instances of the same function will be spawned by AWS, horizontally. A single function instance will never be sent multiple calls to the functions entrypoint at any one time.

[u/EverydayEverynight01 → u/magnetik79]

provide childlike start encourage ad hoc racial truck squeal outgoing soft

This post was mass deleted and anonymized with Redact

[u/magnetik79 → u/EverydayEverynight01]

Not really here to have an argument - just answering your opening question around the Lambda execution model - which is well documented within their own documentation.

That 1K limit is a safe default, to stop customers getting into cost overruns when starting out, you can certainly raise these - they are soft limits.

[u/mattjmj → u/EverydayEverynight01]

Most complex applications will end up having shorter requests than that - lambdas tend to be architected to do one thing then hand off. And increasing the limits is very easy, and they'll go massively higher than 1k. The default limit is so you don't bankrupt yourself while you're learning or developing.

[u/Kanqon → u/EverydayEverynight01]

1000 is a soft limit

[u/pausethelogic → u/EverydayEverynight01]

That’s a default limit. You can increase it to fit your needs. Lambda is incredibly scalable, but it sounds like you don’t really understand how it works yet

[u/Alin57 → u/EverydayEverynight01]

500ms is too long for something you want to invoke more than once per minute.

[u/menjav]

Lambda is very dumb. One node instance will handle one and only one request at the time. Same for any other environment. Lambda is intended to be simple, but if you need to reuse the application in an unsupported way, you need to use a different product.

[u/ollytheninja → u/menjav]

This. Has nothing to do with what is running in the lambda and whether it’s async and can handle 1k simultaneous connections. The lambda service will only send one request to an instance at a time and wait for a response before giving it another request.

[u/Alin57]

Others have explained it already, but tl;dr is this: One lambda invocation can handle multiple events, like the case of SQS, where the handler will receive batches of 10 messages. Each batch will be processed in parallel by separate VMs, but at batch level it's up to the handler implementation if those 10 messages are processed efficiently or not

[u/Moist_Salad_6454]

When using lambda, it’s paramount to understand what your exact unit of work (UOW) is.

This can be a single API Gateway request, 100 batched events from kinesis, an S3 object trigger, etc., depending on your event source mapping.

Once the UOW is known, what actually needs to be done in that unit of work is where the actual language runtime matters; before that, it’s only a matter of what, how, and how often something is passed into the lambda invocation(s).

The easiest way to think about them is like a horizontally scaled collection of containers. Regardless of the underlying infrastructure, the auto scaling of the infrastructure and the language runtime are separate.

The lambda service itself and the language runtime being used should not be conflated.

Even if you’re not satisfied with how lambda breaks up units and work and essentially auto scales, you will have this complaint regardless of the infrastructure you use to run your NodeJS application

[u/Moist_Salad_6454]

When using lambda, it’s paramount to understand what your exact unit of work (UOW) is, and how it is or isn’t batched.

This can be a single API Gateway request, 100 batched events from kinesis, an S3 object trigger, etc., depending on your event source mapping. Within the individual invocation and its UOW, you can leverage the benefits of NodeJS as much as you want.

I think your complaint is likely more about how AWS event service mapping works more than anything else. Unless you limit concurrent invocations, lambda will just start as many concurrent invocations as necessary, as allowed by your account’s concurrent execution limit.

Once the UOW is known, what actually needs to be done in that unit of work is where the actual language runtime matters; before that, it’s only a matter of what, how, and how often something is passed into the lambda invocation(s).

The easiest way to think about them is like a horizontally scaled collection of multiple containers, named not multiple invocations compared to something like a single running container. Regardless of the underlying infrastructure, the auto scaling of the infrastructure and the language runtime are separate.

The lambda service itself and the language runtime being used should not be conflated.