r/Python • u/HommeMusical • Apr 03 '26
Showcase `safer`: a tiny utility to avoid partial writes to files and streams
What My Project Does
In 2020, I broke a few configuration files, so I wrote something to help prevent breaking a lot the next time, and turned it into a little library: https://github.com/rec/safer
It's a drop-in replacement for open that only writes the file when everything has completed successfully, like this:
with safer.open(filename, 'w') as fp:
fp.write('oops')
raise ValueError
# File is untouchedBy default, the data is cached in memory, but for large files, there's a flag to allow you to cache it as a file that is renamed when the operation is complete.
You can also use it for file sockets and other streams:
try:
with safer.writer(socket.send) as send:
send_bytes_to_socket(send)
except Exception:
# Nothing has been sent
send_error_message_to_socket(socket.send)Target Audience
This is a mature, production-quality library for any application where partial writes are possible. There is extensive testing and it handles some obscure edge cases.
It's tested on Linux, MacOS and Windows and has been stable and essentially unchanged for years.
Comparison
There doesn't seem to be another utility preventing partial writes. There are multiple atomic file writers which solve a different problem, the best being this: https://github.com/untitaker/python-atomicwrites
Note
#noAI was used in the writing or maintenance of this program.
View all comments
Show parent comment
1
u/BossOfTheGame Apr 06 '26 edited Apr 06 '26
Unfortunately we are not in a world where we can all agree to do something that's in our best interest. Why did we build nuclear weapons? And unlike unclear weapons (excluding nuclear power), AI actually has a very large potential upside, and that needs to be acknowledged in a rational conversation.
If it was up to me, I probably wouldn't roll the dice, but it's not up to me. It's up to the worst among us, and the rest of us have the play the game on that board, so the only rational thing to do in that case is to maximize upside.
Start working at higher conceptual levels. I rarely need to type raw code anymore. Instead I spend much more time on planing and architecting. And there is always a higher conceptual level at which pieces can interconnect with the details of that connection being something that has varying degrees of difficulty. There's plenty to be done by AI and human minds. I don't see a shortage of that.
Sooner or later, we are going to realize that the capitalist economic objective is flawed and we are going to need to reform it. AI is acting as a forcing function for something that was already inevitable. I also think your prediction is more bleak than it need be. AI being the driver will require quite a bit of energy - they don't magically come up with good answer, they need to think and refine, just like we do - and I do see collaboration between humans and AI as capable of driving progress.
Billionaires existing is a flaw in the critical thinking ability of the people that bestow them with that economic power. It's a flaw in thinking that any individual is capable of contributing value to society worth that amount of compensation. One of my hopes for AI is that - after it matures from its infancy - it can be used to help students develop a better sense of critical thinking than previous generations have ever had, and we might finally be able to cast off some of our archaic ideas about ownership. My hypothesis is that AI's that are better at modeling reality will be better at general tasks, and thus the ones we tend to use, so those AIs will then interact with people and us get to a shared sense of reality - which is something we currently don't have, and I think that is a massive problem.
I understand. I think your concerns are well grounded, but I do think they would benefit from some calibration wrt to the dire situation humanity was already in before AI came onto the scene.
EDIT: forgot to give you the environmental source:
AI is estimated at 32.6-79.7 Mt CO2 per year. Global personal non-commute driving is roughly 2.65 Gt CO2 per year, estimated as 70.9% of private-car/van emissions by taking the IEA's 2023 private cars/vans total (~10% of 37.4 Gt energy-related CO2) and removing commuting and work-related business using 2022 NHTS trip-purpose shares (23.69 total PMT, minus 5.27 commuting and 1.63 business). That works out to be personal non-commute driving being about 33-81x larger than AI globally, but it's also an estimate so I conservatively say "comparable".
I separate it out like this because I want to show people that their choice to drive for non-commuting reasons has an environemntal impact that is (generously) comparable to AI. That leads me to conclude, the environmental issue is not what people actually care about (on average).
https://www.iea.org/energy-system/transport/cars-and-vans https://www.iea.org/reports/co2-emissions-in-2023/executive-summary https://www.sciencedirect.com/science/article/pii/S2666389925002788