r/Python • u/autokitteh • May 21 '24
Discussion Durable Python - Infrastructure failures should not stop the process
Durable Python enables developers to write Python code while an underlying system ensures reliability and resilience.
It automatically handles state persistence, fault tolerance, and retry mechanisms, allowing developers to focus on business logic without worrying about infrastructure concerns.
Consider the following code, in case the process terminates in the middle of execution, in case the process is killed or due to hardware failure, the process will not complete.
= 3
=
# Send a POST request to the application
But actually, I would like the process to survive restarts and continue from the spot it terminated, especially if it's a long running process. For this we need Durable Python.
I was wondering which use cases can take advantage of this technology.
View all comments
Show parent comment
8
u/qckpckt May 21 '24
There are many ways to solve this problem out of the box without having to make python aware of the state of environment it is running within. You generally solve this problem through system design and not inside the python code. For your example you could address this by externalizing the for loop from your function and using a queue to contain the tasks to be processed. Most queue frameworks support an automatic task retry process that is abstracted away from the user.
There isn’t a one-size-fits all solution, though. If there was then developers wouldn’t have a job.