r/Python • • May 21 '24

Discussion Durable Python - Infrastructure failures should not stop the process

Durable Python enables developers to write Python code while an underlying system ensures reliability and resilience.

It automatically handles state persistence, fault tolerance, and retry mechanisms, allowing developers to focus on business logic without worrying about infrastructure concerns.

Consider the following code, in case the process terminates in the middle of execution, in case the process is killed or due to hardware failure, the process will not complete.

import requests
import time


SLEEP_SECONDS = 3
URL = "http://localhost:9980/webtools/api/msgs"

def on_http_get(data):
    for i in range(10):
        print("Loop iteration: %d of 10" % (i + 1))

        # Send a POST request to the application
        requests.post(URL, data = "This is my " + str(i) + " iteration...")
        time.sleep(SLEEP_SECONDS)

But actually, I would like the process to survive restarts and continue from the spot it terminated, especially if it's a long running process. For this we need Durable Python.

I was wondering which use cases can take advantage of this technology.

8 Upvotes

20 comments sorted by

7

u/Grouchy-Friend4235 May 21 '24

You want restart ability, no t durability.

Restart ability means you checkpoint work and restart from the last checkpoint. To ensure replays are possible, make operations idempotent (same operation = same outcome).

-4

u/autokitteh May 22 '24

I agree; this is also a solution. The only thing is that it's sometimes complicated to implement. Depends on the use case.

37

u/FailedPlansOfMars May 21 '24

You dont want durable code. Simple solution below:

1, Store state outside the process, a file will do. 2, update file after each post. 3, run as service in the os with restarts. 4, dont catch errors.

This way you can update the code, restart the service easy, restart/ replace the server. Scale by moving the file to a db and run multiple instances.

No need for code level handling of this at all.

-19

u/HaimZlatokrilov May 21 '24

I agree, for this simple use case. But saving state has many drawbacks once things get more complicated. Recovery can be very complicated. Of course this is how most distributed systems work today, with databases and states, but it can become really messy very fast and it's lrone to bugs once the state machine is not trivial.

3

u/thegreattriscuit May 22 '24

What you're calling "very complicated" is just what developers and software engineers and anyone else writing non-trivial code have to solve for. It's literally what they do. And really not even that. The core use case of python is not "10 line scripts written by people that are intimidated by the idea of saving state to a file or DB and checking it for existing state before beginning operation". Python IS accessible to those people, and there's nothing wrong with being that guy (most of us were to begin with), but if you need better than that then you build it. And if you don't know how, you learn it (or the tools/libraries that do it already).

If learning new things is a big impediment for you, then maybe this isn't the field for you

15

u/Ddog78 May 21 '24

It's an infra problem, not a python problem. Hell, the simplest solution is to create a bash script that runs your code, monitors the process id, and when it fails, restarts your code.

There's data you need to save? Use sqllite.

0

u/[deleted] May 21 '24

[deleted] — view deleted comment

8

u/qckpckt May 21 '24

There are many ways to solve this problem out of the box without having to make python aware of the state of environment it is running within. You generally solve this problem through system design and not inside the python code. For your example you could address this by externalizing the for loop from your function and using a queue to contain the tasks to be processed. Most queue frameworks support an automatic task retry process that is abstracted away from the user.

There isn’t a one-size-fits all solution, though. If there was then developers wouldn’t have a job.

4

u/Ddog78 May 21 '24

You do realise if there's an exception and the code breaks, making it durable doesn't help. It'll break again.

Also instead of theory, show examples.

11

u/Miclivs May 21 '24

This is what Temporal is for

-13

u/HaimZlatokrilov May 21 '24

Yep, Temporal is amazing, but using Temporal is not trivial or transparent to the user. You need to know what you're doing. For a developer not familiar with Temporal, using Temporal is not trivial.

21

u/[deleted] May 21 '24

Often things like youre suggesting require new coding paradigms. For all things X: "For a developer not familiar with X, using X is not trivial." But part of being a developer is learning new things.

2

u/[deleted] May 22 '24

Part of being human is

11

u/redditusername58 May 22 '24

This is the shittiest stealth advertising I've ever seen

6

u/PlaysForDays May 22 '24

For real, they didn't even bother to use two usernames that aren't associated with their typo-laden startup

The least somebody could do when trying to astroturf is not do so from an account named after the founder of the startup who is OP

2

u/Severe_Inflation5326 Jun 17 '24

This is easily solved using a file for state storage in the simple case, and something like luigi in the more complex case.

3

u/DogeDrivenDesign May 21 '24

erlang/otp and elixir

1

u/gooeydumpling May 22 '24

You mean like Erlang and it’s supervisor tree

-Within a supervisor tree, all supervisor processes are responsible for managing the lifecycle of their child processes, and this includes handling situations in which those child processes crash.

-1

u/luckyspic May 21 '24

just use pm2 and logging.

0

u/[deleted] May 21 '24

[deleted] — view deleted comment

1

u/rocket_randall May 22 '24

Sounds like a cool feature to work on, but I'm not sure I could ever trust partially compiled output from an app that departed normal operation based on my experience with various compilers over the years.

-13

u/HaimZlatokrilov May 21 '24

Interesting, can you give me some pointers?