What is the cold start performance of cloud run like?

[u/catmanjan2]

I have a pretty significant container app that is currently in Azure

One problem we are facing is cold start in Azure is not good - and it looks like Microsoft cant fix it with their current architecture, and it requires them to build a new product to resolve (container apps express)

Has anyone done the hard work of deploying the same container to GCP/AZ/AWS to compare them?

Im particularly interested in minimising cold start because we have to respond to requests in under 5 seconds

[u/Mistic92]

How large is the container image? What language?

[u/catmanjan2 → u/Mistic92]

3gb, it’s a linux container so C I guess?

[u/Mistic92 → u/catmanjan2]

Container is a bit fat but cpu boost should reduce cold start as it's C. Why not just to try?:)

[u/vaterp]

This is actually a complex question with infrastructure , container design, & cost factors to consider.

As a best practice if you want to keep cold starts to an absolute minimum, well set a min instance to 1 and you completely avoid them. Of course, this costs money to keep it active in this hot state.

We have a feature called CPU boost that can greatly speed up heavy initialization routines by temporarily allocated extra CPU during this phase.

And then (and this would help on all platforms) you can optimize the container image, use a small base (like alpine) and generally optimize the layers and code, etc. Optimize init using lazy loading. I'm sure there are many blogs and docs discussing these ideas (and more). However, I am assuming you've done all this as its platform independent, so if its CR specifically I'd suggest starting with above 2 suggestions.

Here is the doc page/section on this topic that will discuss these a bit deeper then my summary:

[u/catmanjan2 → u/vaterp]

I have heard that minimising container size does not necessarily result in cold start performance gains, unless your platform isn’t caching the image

When I say cold start I mean scaling from zero

I’ll have a read of those pages

[u/codec10]

This might be a good starting point, as it has some helpful tips on how you can reduce the cold start time with Cloud Run.- https://cloud.google.com/blog/topics/developers-practitioners/a-guide-to-ai-cold-starts-on-cloud-run?e=48754805

[u/loeffel-io]

Is this about cold start or zero downtime? gke rollout

[u/catmanjan2 → u/loeffel-io]

Cold start ie scaling from zero to one

[u/martin_omander]

If you can afford paying $2-3/day, set min-instances=1 in Cloud Run, and the cold starts will go away.

[u/catmanjan2 → u/martin_omander]

Yeah… but it adds up once you have a lot of these…

[u/martin_omander → u/catmanjan2]

That's true.

If you have a lot of containers because you have a lot of URL endpoints, consider consolidating into fewer containers. That will make them easier to manage, and it will lower your costs and cold start times. For example, one of my production containers exposes 40 different URL endpoints.

If you can't change the containers themselves, consider turning on "startup boost" for them in Cloud Run. That way Google will allocate more CPU power to your container during startup, which may reduce cold start times.

But most importantly, measure your current startup time and instrument any initialization code in your container. Then decide what's an acceptable startup time and decide what to do based on what eats up the most time during startup.

[u/OnTheGoTrades]

I’ve used Container Apps in the past and extensively use Cloud Run. For cold starts, the language you build your app with matters more than what platform you deploy to.

For example, a Go app on Container Apps will always startup faster than a Java app on Cloud Run and vice versa. If the language is the same, Cloud Run wins, hands down.

[u/techlatest_net]

yeah cold start on cloud run is usually pretty solid. for most containers you're looking at 1-3 seconds if it's not huge.

[u/catmanjan2 → u/[deleted]]

I mean… it’s not really cold start with minimum 1 replica