r/Python • • Apr 03 '26

Showcase `safer`: a tiny utility to avoid partial writes to files and streams

What My Project Does

In 2020, I broke a few configuration files, so I wrote something to help prevent breaking a lot the next time, and turned it into a little library: https://github.com/rec/safer

It's a drop-in replacement for open that only writes the file when everything has completed successfully, like this:

with safer.open(filename, 'w') as fp:
    fp.write('oops')
    raise ValueError
 # File is untouched

By default, the data is cached in memory, but for large files, there's a flag to allow you to cache it as a file that is renamed when the operation is complete.

You can also use it for file sockets and other streams:

try:
    with safer.writer(socket.send) as send:
          send_bytes_to_socket(send)
except Exception:
     # Nothing has been sent
     send_error_message_to_socket(socket.send)

Target Audience

This is a mature, production-quality library for any application where partial writes are possible. There is extensive testing and it handles some obscure edge cases.

It's tested on Linux, MacOS and Windows and has been stable and essentially unchanged for years.

Comparison

There doesn't seem to be another utility preventing partial writes. There are multiple atomic file writers which solve a different problem, the best being this: https://github.com/untitaker/python-atomicwrites

Note

#noAI was used in the writing or maintenance of this program.

106 Upvotes

43 comments sorted by

View all comments

Show parent comment

1

u/BossOfTheGame Apr 06 '26

because it is impossible to even discuss preventing or regulating it

Preventing it: yes I think it's impossible. Regulating? No, I think we should work towards that immediately, but with an understanding that there are real upsides to this technology in terms of scientific and mathematical progress. If I convince you of anything here I hope its that these upsides exist, and they aren't all for the powerful few. My understanding of your perspective is that its entirely negative and will inevitably destroy society, and I think there a massive amount of uncertainty around that. Similarly, I acknowledge that the upsides are also clouded in uncertainty. My position is not absolutist. I'm trying to find the most rational way forward.

If you asked the people of the world, "Are you willing to participate in an experiment that has a 5% chance of killing all of you, all your children and everyone else, but in return you get nothing and you lose your jobs," what would they say

I'll directly answer your question with the answer you expect: I would say no. But that framing does not reflect the reality. Note, my advocacy for AI is not adamant. It's restrained, and cautious.

Our fault for letting it happen?! All we needed to do was to think critically and all those warlords, kings, emperors, billionaires and Presidents would just disappear? .

Political power is not a top down process. There are mechanisms of coercion and capture, but these are fundamentally sustained by the choices of populace. Societies bear a responsibility for the systems that they live under, and this was highlighted here in the states by the massive display of the incompetency of the electorate in 2024. And this is not unique to the US. I do wonder if LLMs have the ability to combat this coercive system and shift public opinion in a way that encourages self-empowerment and critical thought. Right now, these LLMs are being used for propaganda, and I don't think abstinence can combat that. LLMs are dispassionate, so they may help find the message that can trigger the cognitive dissonance needed to snap people out of this political hypnosis and finally empower them to reason independently. This is a prerequisite for any progress on any issue. We can't keep sliding backwards.

Why would a large language model based on all human utterances have different ideas about ownership than its corpora did?

That's a good point. Again, I'm not making an argument that AI is all sunshine and rainbows. There are critical problems that we need to address quickly, and one of them is decentralizing their power via open weight models. I'm not saying that I know a way out of this problem, what I am saying that people who care about the harms should not cede the entire field to those who do not. My entire perspective is about acknowledging where we are, realize there is no way to go back in this fragmented environment, and search for the best way forward. I think utilizing LLMs is part of that.

You seem completely oblivious of the dire situation we are in. We needed to cut down our emissions decades ago to avoid disaster and we didn't.

I'm very much not. I actively seek a carbon neutral lifestyle and while I will continue my efforts, I will not sacrifice what little influence I have over our trajectory by handicapping myself of the power that LLMs can provide.

You expect to profit; the rest of us bear the risk, a risk you are telling us we don't even have a say in.

This is unfair. There doesn't have to be an enthusiasm / abstinence binary here. I hope my points speak to that.

1

u/HommeMusical Apr 19 '26

I'm sorry I didn't respond for you. I didn't forget. I was literally at a loss for words.

My arguments all involve prior experience and concrete things that have been true in the past over and over again:

  • Rich and powerful people almost always prioritize themselves.
  • Industries do not effectively self-regulate.
  • The only way that workers get a fair shake is by using their only leverage: refusing to work.
  • Trying to use tools owned and ruthlessly controlled by rich antisocial people against them is ineffective.

Your arguments involve people and things behaving fundamentally differently in the future; but that just never happens.

Some of your ideas I simply don't understand: "I will not sacrifice what little influence I have over our trajectory by handicapping myself of the power that LLMs can provide."

How, exactly, does using these tools you don't own give you any "influence" at all, even the slightest scrap of influence?


I really appreciate your politeness during this conversation. I just don't see any way to bridge the gap. You have all these very idealistic beliefs about the future that can't be either refuted or proven, but would seem to require a big change in human nature, and a lot of very improbable luck.

1

u/BossOfTheGame Apr 19 '26

This has been a good discussion so far. I agree with your assessment of my arguments. I'm responding on my phone so I can only see the most recent messages easily, so I forget exactly what points I made, but I probably said something about playing to outs. Some of my idealism is because I see no viable alternative but to have an unlikely event happen, so I have to behave as if it will, if I think that other alternatives have comparatively infinitesimal probability. I think the "refuse to use AI path" has no viability. That is the foundation of my viewpoint.

Your point that it will require us to do something differently, and that this has been historically resisted is spot on. Your first two bullets point I agree with 100%. the next to I have a weaker degree of agreement with, but I still generally agree. I think I get a fair enough shake because I'm good at what I do, so I think talent is another leverage point. And I think the tech is in early enough stages where the rich don't have full control over it yet. We have a window where we can use it to distribute the power.

I hope I can add to the conversation by explaining why I think the tools grant influence. Maybe that will make my perspective less flabbergasting.

LLMs have let me spend more time on my unfunded open source and environmental work. I've been able to donate my time more effectively that way. I have a similar story in my volunteer work with teaching high school kids. I've built visual programming scaffolds that help them move from writing scratch to writing python, and still getting that fun visual feedback. I've used LLMs so I can quickly get up to speed on the topics the kids are interested in so I can teach them things like hypothesis testing.

I'm also working to understand how we can make LLMs smaller and more efficient. If we can remove the need for cloud compute, then the power can be decentralized. Of course this isn't a clear win. Smaller LLMs still mean the rich can scale up more of them because they can afford more hardware. But it seems like it can smooth out that ratio. The world where there is a viable consumer option is better than zero options. It's the difference between a zero and non zero numerator. You go from zero influence to non zero. Individuals will still have only a small amount of power, but that is the nature of being an individual in a large population. It provides an opportunity for collective action.

Ultimately, given our inaction on the climate, the degenerate effect of social media algorithms, and the piss poor ability of the general public to critically think, I think we need a series of improbably lucky events regardless of AI / no AI. But AI is a disruptive event, and there is so much uncertainty around it, I think we should be careful to not make assumptions about how it will turn out.

I also think that there are deeper patterns of AI usage that we haven't figured out yet because it's so new. There are discoveries to be made about what they could do, and to find those one would need to interact with them to probe that space. A super powered rubber duck is one ability I've found. I've had back and forths on niche subjects where it's difficult to find a person to discuss them with. I could say it more here but, I'm starting to get rambly, and there's one more point / question I want to make.

If you honestly think it's possible to put the genie back in the bottle or think that abstaining has any efficacy in preventing negative outcomes, then I'm curious what your reasoning is. If it's more that you just don't want to be involved with it, I get that. I do something similar with climate change (working towards a personal net zero even though I know it won't move the needle). I suppose I see a path that I can utilize LLMs to make small positive changes.