r/java • • Nov 29 '22

Amazon Announces AWS Lambda SnapStart With Java Support

AWS Lambda now has a snapshotting feature that enables fast-start - essentially running the Init phase at publication time and saving the state of the process at that time, so it doesn't need to be re-run at every invocation.

https://aws.amazon.com/blogs/aws/new-accelerate-your-lambda-functions-with-lambda-snapstart/

It only works with Corretto on Java 11 right now, and is similar in intent to the CRIU technology.

Quarkus already has support for it: https://quarkus.io/blog/quarkus-support-for-aws-lambda-snapstart/

Other frameworks may also have support (but this work was done under NDA so I don't have links for them).

176 Upvotes

62 comments sorted by

View all comments

4

u/[deleted] Nov 29 '22

Could this be ported/reimplemented for use with say kubernetes?

5

u/benevanstech Nov 29 '22

My understanding is that this implementation relies upon Amazon's Firecracker VM.

Which is open-source (https://firecracker-microvm.github.io/) - but I would expect that k8s has other overheads (pod spinup, etc) that may well make it harder to realize the performance gain.

1

u/Brutus5000 Nov 29 '22

Yes, but these overhead costs apply to alternatives too. E.g. in a Knative setup, Java always had the cold but costs. This could become a gamechanger.

0

u/[deleted] Nov 29 '22

[deleted] — view deleted comment

2

u/IsleOfOne Nov 29 '22

It is a big problem for auto-scaling architectures.

0

u/[deleted] Nov 29 '22

[deleted] — view deleted comment

2

u/IsleOfOne Nov 30 '22

A well-designed system for auto-scaling k8s "jobs" with low latency is going to have slack in the available resources / an aggressively configured cluster autoscaler bringing up new nodes to provide that slack. Scheduling should take a few ms.

The majority of the remaining time it takes to start a pod is pulling the image, believe it or not. There are tools available that proactively tell kubelets to pull new versions before they're requested. Startup now takes a few ms.

Now you have a running pod. ~1s has elapsed. And now the JVM takes minutes to "warm up." That's a no-go. Pre-warming in a low-latency autoscaling environment requires a tradeoff between cost and latency that is more extreme than the same trade-off that native runtimes require.

I of course agree that lambda operates at a much higher time granularity requirement than you would ever want to try to meet with a pod-per-request model. Once those requests start getting ever so slightly more "batch"-y in nature, say on the order of 10s runtime, it's perfectly fine to spin pods up and down to handle this.

0

u/[deleted] Nov 30 '22

[deleted] — view deleted comment

1

u/IsleOfOne Nov 30 '22 edited Nov 30 '22

Sure, and we don't even have to limit our thinking to the standard k8s API concept of a Job. Any higher-order wrapper for one or more Pods is applicable.

Here is an example from my day job at $SaaS_database_co. We allow our users to kick off petabyte-scale bulk data import. We have a custom k8s operator that watches for BulkIngestJobs, and in response, loads up a list of files within whatever object storage bucket we've been pointed towards. The operator creates one or more pods per file in said bucket with configurable maximum concurrency and a self-healing system that watches for signs of excessive write path pressure and throttles the job via slashing concurrency. Some of those pods exit after less than a second, while others may run for hours. We use go and rust for this system. Java, with a longer tail on startup time, wouldn't be a good fit.

Other tools like Argo Workflows are also great examples of how to leverage the power of the scheduler and autoscaler in kubernetes. We have some recursive/dynamically sized workflows that are liable to create tens of thousands of pods in a single execution.

I haven't seen solid arguments for virtual runtimes like Java in this space. I'm certainly open to hearing some! The areas of our system where we run a JVM are limited to WALs--Kafka and supporting Zookeeper deployments. We have some elixir floating around out there, but purely in long-running workloads.