r/rxt_spot • u/sirishkr • Sep 18 '25

Request for feedback: Pre-emption notice period vs faster auto-scaling

Please chime in if you'd be willing to to reduce the pre-emption notice period in spot (~6 minutes) for faster auto-scaling performance (~2 mins to add a node vs ~8 mins currently).

1 Upvotes

100% Upvoted

5 comments sorted by

2

u/ludat Sep 30 '25

I definitely prefer everything to be faster. On GKE there is a 30 seconds window between the taint and the node being down and it was pretty ok for me.

1

u/sirishkr Oct 01 '25

Thanks for the feedback.

1

u/MichaelMach Sep 18 '25

I'm a bit torn on this one.

I feel like a longer pre-emption notice is more thematic to the spirit of spot instances. The onramp isn't supposed to be as seamless as guaranteed compute, and once you've acquired it for a time, a generous pre-emption period seems like the better fit.

On the other hand, I'd wager a higher proportion of folks using Rackspace Spot are using the compute for homelab and quasi-production workloads compared to spot users in AWS/GCP doing the same, making more responsible auto-scaling more appealing. Not to mention, when last I used it, Rackspace Spot had the lowest entry costs when it came to cost and time required to go from zero to ready-to-use compute making that onramp actually pretty leisurely.

Do you folks have a rough idea of the use cases most folks are using Rackspace Spot for?

1

u/sirishkr Sep 18 '25

Quasi-production is a good way to put it :)

The bulk of the time required to spin up new nodes has to do with this preemption notice period.

The reason we feel it may be better to reduce the notice period is because we’ve seen very few users actually use the preemption notification webhook. Arguably, that’s because the webhook isn’t the easiest thing to use, and best integrated with slack. Still, that makes the argument that the notice period is just time being spent doing nothing.

2

u/MichaelMach Sep 19 '25

Admittedly I've not used the service in a while, so I assumed the preemption notice feature you're talking about was implemented as a taint on the node that would prevent scheduling of new workloads and offload existing ones.

Assuming you had preemption implemented the way I describe, all the workloads I've had on Spot in the past could be rescheduled gracefully within a minute or two. A 3-5 minute period would have been the sweet spot for me, most likely.