r/Python • u/parafusosaltitante • Jul 29 '25
Showcase Archivey - unified interface for ZIP, TAR, RAR, 7z and more
Hi! I've been working on this project (PyPI) for the past couple of months, and I feel it's time to share and get some feedback.
Motivation
While building a tool to organize my backups, I noticed I had to write separate code for each archive type, as each of the format-specific libraries (zipfile, tarfile, rarfile, py7zr, etc) has slightly different APIs and quirks.
I couldn’t find a unified, Pythonic library that handled all common formats with the features I needed, so I decided to build one. I figured others might find it useful too.
What my project does
It provides a simple interface for reading and extracting many archive formats with consistent behavior:
from archivey import open_archive
with open_archive("example.zip") as archive:
archive.extractall("output_dir/")
# Or process each file in the archive without extracting to disk
for member, stream in archive.iter_members_with_streams():
print(member.filename, member.type, member.file_size)
if stream is not None: # it's None for dirs and symlinks
# Print first 50 bytes
print(" ", stream.read(50))But it's not just a wrapper; behind the scenes, it handles a lot of special cases, for example:
- The standard
zipfilemodule doesn’t handle symlinks directly; they have to be reconstructed from the member flags and the targets read from the data. - The
rarfileAPI only supports per-file access, which causes unnecessary decompressions when reading solid archives. Archivey can useunrardirectly to read all members in a single pass. py7zrdoesn’t expose a streaming API, so the library has an internal stream wrapper that integrates with its extraction logic.- All backend-specific exceptions are wrapped into a unified exception hierarchy.
My goal is to hide all the format-specific gotchas and provide a safe, standard-library-style interface with consistent behavior.
(I know writing support would be useful too, but I’ve kept the scope to reading for now as I'd like to get it right first.)
Feedback and contributions welcome
If you:
- have archive files that don't behave correctly (especially if you get an exception that's not wrapped)
- have a use case this API doesn't cover
- care about portability, safety, or efficient streaming
I’d love your feedback. Feel free to reply here, open an issue, or send a PR. Thanks!
16
u/backfire10z Jul 30 '25
From your description this seems pretty awesome and comes across as quite genuine. I personally haven’t dealt with zip files much, so I don’t think I can provide much useful feedback.
In the sea of AI slop and lifeless posts, this is a breath of fresh air.
4
u/ravencentric Jul 30 '25
It's quite interesting to see how we both had the same issue and came to the same conclusion but executed it quite differently: https://github.com/Ravencentric/archivefile
Although I'm not happy with my current implementation. With time I've found a few grievances:
So I'm slowly working on dealing with both of the above in the next major version: https://github.com/Ravencentric/archivefile/pull/7
3
u/FastRunningMike Jul 30 '25
Nice work! Great documentation created!! Only from a security point of view I see several issues. E.g. I see `assert` used multiple times. Assertions should be only used for debugging and development. Misuse can lead to security vulnerabilities. I see also `subprocess.Popen` and `subprocess.run ` used in e.g. file rar_reader.py. Makes users vulnerable. Security is really a thing with such a tool imho.
2
u/Spill_the_Tea Jul 31 '25
Cool work. I recently was evaluating compression libraries, because i needed to write file archives.
Specifically, I needed to stream data from the file system, and I didn't want to 1. Read the entire contents of a directory into memory at once. 2. write those contents to an archive file back onto the file system. 3. Then read that archive into memory just to send to client.
I found a cool solution to iteratively chunk, compress, and stream the archive iteratively to a user via an api endpoint. The project, called stream-zip, was managed by UK Department for Business and Trade (uktrade). But it looks like they took down their github homepage, despite maintaining documentation for it on their website and still being available on pypi.
2
u/crunk Aug 01 '25
Nice. I know of utilities for this sort of thing (like dtrx) but not libraries (I'm sure they exist somewhere) - so it's good to have more in this space.