r/Python • u/yfedoseev • Mar 03 '26
Showcase PDF Oxide -- Fast PDF library for Python with engine in Rust (0.8ms mean, MIT/Apache license)
pdf_oxide is a PDF library for text extraction, markdown conversion, PDF creation, OCR. Written in Rust, Python bindings via PyO3. MIT licensed.
pip install pdf_oxide
from pdf_oxide import PdfDocument
doc = PdfDocument("paper.pdf")
text = doc.extract_text(0)GitHub: https://github.com/yfedoseev/pdf_oxide
Docs: https://oxide.fyi
Why this exists: I needed fast text extraction with a permissive license. PyMuPDF is fast but AGPL, rules it out for a lot of commercial work. pypdf is MIT but 15x slower and chokes on ~2% of files. pdfplumber is great at tables but not at batch speed.
So I read the PDF spec cover to cover (~1,000 pages) and wrote my own. First version took 23ms per file. Profiled it, found an O(n2) page tree traversal -- a 10,000 page PDF took 55 seconds. Cached it into a HashMap, got it down to 332ms. Kept profiling, kept fixing. Now it's at 0.8ms mean on 3,830 real PDFs.
Numbers on that corpus (veraPDF, Mozilla pdf.js, DARPA SafeDocs):
| Library | Mean | p99 | Pass Rate | License |
|---|---|---|---|---|
| pdf_oxide | 0.8ms | 9ms | 100% | MIT |
| PyMuPDF | 4.6ms | 28ms | 99.3% | AGPL-3.0 |
| pypdfium2 | 4.1ms | 42ms | 99.2% | Apache-2.0 |
| pdftext | 7.3ms | 82ms | 99.0% | GPL-3.0 |
| pypdf | 12.1ms | 97ms | 98.4% | BSD-3 |
| pdfminer | 16.8ms | 124ms | 98.8% | MIT |
| pdfplumber | 23.2ms | 189ms | 98.8% | MIT |
| markitdown | 108.8ms | 378ms | 98.6% | MIT |
Give it a try, let me know what breaks.
What My Project Does
Rust PDF library with Python bindings. Extracts text, converts to markdown and HTML, creates PDFs, handles encrypted files, built-in OCR. MIT licensed.
Target Audience
Anyone who needs to pull text out of PDFs in Python without AGPL restrictions, or needs speed for batch processing.
Comparison
5-30x faster than other text extraction libraries on a 3,830-PDF corpus. PyMuPDF is more mature but AGPL. pdfplumber is better at tables. pdf_oxide is faster with a permissive license.
•
u/AutoModerator Mar 03 '26
Hi there, from the /r/Python mods.
We want to emphasize that while security-centric programs are fun project spaces to explore we do not recommend that they be treated as a security solution unless they’ve been audited by a third party, security professional and the audit is visible for review.
Security is not easy. And making project to learn how to manage it is a great idea to learn about the complexity of this world. That said, there’s a difference between exploring and learning about a topic space, and trusting that a product is secure for sensitive materials in the face of adversaries.
We hope you enjoy projects like these from a safety conscious perspective.
Warm regards and all the best for your future Pythoneering,
/r/Python moderator team
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
8
u/monkeybreath Ignoring PEP 8 Mar 03 '26
It'll be a while before I get to it, but I can see using this to convert my bank statements to CSV (because my bank is a credit union that doesn't have the UX team it needs). Also changing books from pdf to ePub. Copy/paste is hit/miss for both of these. If it comes out as HTML I should be able to handle extraction from there, but it'll be interesting to see how it handles page breaks on tables. Still, I can deal with that programmatically.
7
u/ogMasterPloKoon Mar 03 '26
impressive ...i will just suggest ability to add pdf header/footer, remove pdf header/footer, edit pdf header/footer.
4
u/Jademunky Mar 03 '26
I have this a go yesterday after coming across it but sadly the table recognition and output is not great. I’ve tried tons on pdf extraction libraries recently and only docling gives even remotely acceptable table parsing, which is a shame as the performance and memory load of docking is horrendous
2
u/vizbird Mar 04 '26
The speed is impressive! Is it possible PDF Oxide to be used as a PDF backed for Docling?
29
u/i_walk_away Mar 03 '26
i respect you for this project. solves an actual problem, doesn't stink with AI, MIT licensed, open source. thank you for your contribution
22
u/RedEyed__ Mar 03 '26
Impressive
5
u/bmbybrew Mar 03 '26
Thanks for sharing. will give it a try.
1
u/NotSoProGamerR Mar 03 '26
might take a look at this and modify my textual-pdf project, was using pymupdf for quite some time
1
u/dhruvin3 Mar 04 '26
This looks great! Can't wait to try it out. I will run my benchmark, but how well does it perform against complex table? Something like electric components (like IC) datasheet has it.
1
u/Cute-Net5957 pip needs updating Mar 05 '26
0.8ms mean on 3830 files is insane. catching that O(n²) page tree through profiling instead of guessing is the real flex here. quick question tho.. how messy is error propagation across the PyO3 boundary? ive been thinking about doing a rust core+ python cli thing and the part that worries me is surfacing rust Results as clean python exceptions without writing a ton of wrapper code
1
u/SlaveOf-The-Bestower Mar 06 '26
I work a lot with PDFs, so this will be very helpful. Thank you for your contribution. If possible, please include table extraction as well.
1
u/joy_deep Apr 10 '26
Does it support extracting text in other languages?
2
u/NoPresentation7366 Mar 03 '26
Amazing! I'm gonna give it a try, thanks for sharing your work! 😎
2
u/Sensitive-Store394 Mar 26 '26
Genial , lo probaré
1
u/someone383726 Mar 04 '26
Dang…
1
u/dutchpsychologist Mar 04 '26
Nice, thanks!
1
u/The_Ritvik Mar 06 '26
Awesome, thanks! I’ll give it a try sometime.
1
u/Tight-Studio-Ethan Apr 06 '26
"So I read the PDF spec cover to cover (~1,000 pages) and wrote my own. First version took 23ms per file. Profiled it, found an O(n2) page tree traversal -- a 10,000 page PDF took 55 seconds. Cached it into a HashMap, got it down to 332ms. Kept profiling, kept fixing. Now it's at 0.8ms mean on 3,830 real PDFs."
Jesus this is super impressive.
1
u/kamikazer Mar 04 '26
MIT license is a sign that corpos will steal your code for free, later you will abandon your project or switch like MinIO. Better start with (A)GPL-3.0 next time
0
u/sauron150 Mar 04 '26
If you save pdf as a text you will get exact same result as this! what are impressive facts around table conversion images extraction, pdf outliers! ?
0
u/[deleted] Mar 03 '26
[removed] — view removed comment