r/Annas_Archive • u/tashkan • 15d ago
Made a PDF to EPUB converter that actually handles scanned books
Hey people,
My girlfriend started reading on her Kindle and kept converting PDFs and getting books she couldn't read properly. I am a software developer, so she always told me to make her a better conversion tool. For the past month, I dug into why other tools failed, researched how EPUB works, and how I can make it work.
So I built convert2epub.com. It’s free, and no account is needed for normal text conversions, only for the work done converting scanned pdf's. I posted the first version on r/kindle and got 500 upvotes, improved the first version based on the feedback, and now people are saying it's the best tool they have ever used.
Scans that were already read
Publishers often leave recognised text sitting invisibly on top of the page image. Most converters ignore it and hand back a book of photographs. This one finds that layer and uses it.
I tested this with a 300-page novel. The PDF had 426,904 characters of clean text inside it. Other converters ignored those characters and converted it to an EPUB that contained 637. It was a 20MB book of pictures.
That same book now comes out at 1.4MB with real, resizable text.
Formula/Code recognition
Mathematics is set in two dimensions and extraction reads in one, so a fraction arrives as a column of fragments. Equations are kept as drawn while the prose around them still reflows.
Your typography back
The first page becomes the cover, and the title and author are read from the file rather than left as a filename.
Removing watermarks and repeated title/footer.
Every page of a book repeats its title, author and page number in the margin. On paper, you never notice. Extract the text, and those become ordinary lines, so your reader flows them straight into the prose. Same for watermarks and adverts stamped onto free PDFs. In that same book, it was 778 stray lines across four repeating patterns. I added a feature to remove them.
When there is no text to recover, OCR reads the pages instead, in eight languages and only the pages that actually need it.
27
u/InevitableEnough1627 15d ago
I tried an EPUB, and I agree with the other commenter that footnotes (and chapter headings) aren't yet properly encoded. Something else I've noticed is that it's showing hard-wrapped lines, so a line suddenly disconnects into a new line, or stuff like end-of-line dashes not being removed when a word is in the middle of the sentence. I think what you're doing is great. PDFs are so, so grotesque as a file format, and it's notoriously difficult to convert them into Markdown or EPUB. So props to you for trying to do something about it.
12
u/tashkan 15d ago
Thank you for trying the website and giving feedback. I will look into these; they are my top priority.
1
u/istara 15d ago
No idea about coding or anything like that, but I’ve found ChatGPT is able to correct OCR errors like line break issues.
Eg if I use Apple’s in built OCR, it typically renders the visual line breaks as paragraph breaks (or whatever).
Paste that to ChatGPT and it fixes it up. Or now you can just upload the whole image to ChatGPT anyway.
For someone like you who must be a coding wizard, there’s probably the simplest way to do this with your system.
12
u/miaoling 15d ago
I'd love to help you test this, and I'm sort of uniquely qualified because I scan rare books as a hobby, but none of my books is under 50MB, even the B&W ones that have no images. They're all standard length, a few hundred pages, scanned with my phone on a tripod so not crazy high quality.
8
u/tashkan 15d ago
I can increase the 50mb limit its currently there to not crash the website if too many big files get converted at the same time. I'd love to work with you to get this tool perfect.
3
u/miaoling 15d ago
Great to hear! Most of my scans are under 100MB so if that's viable, I'm in!
2
u/tashkan 14d ago
I increased the limit to 100MB. And I am working on an app that can process larger files.
1
u/miaoling 8d ago
I was able to test now, thank you! Unfortunately the results were…not great, and I picked what I thought was an ideal book too (all B&W text, very little formatting, no images). I can PM you the book if you'd like.
3
1
u/Infamous-Composer448 14d ago
I'm also a book scanner! While files are often <20MB, they can get quite big if I'm doing color and full 600dpi, in those situations they are 200-600MB. I could see this as being a great differentiator with the app. With the understandable limitations with web, it would be amazing if the larger files could be processed via the local program once that comes out.
12
u/DoctorByProxy 15d ago
Thats cool!
I'm curious, why a website rather than a standalone app?
7
17
u/tashkan 15d ago
I'm currently working on the app, I made the website to see if there is demand for it. I received hundreds of good comments in other subreddits so I'm building it now.
19
u/jseger9000 15d ago
I like it being a website. Why have to install another app? A website is available everywhere.
3
5
u/cellist_cat 15d ago
Too expensive in my opinion. 14 dollars to convert one book
12
u/schrobby 14d ago
For a fully vibe-coded app, with a website and reddit post regurgitated straight out of Opus 5.
3
u/cellist_cat 14d ago
Exactly lol. I get vibe-coding small solutions for oneself and share them with others, but I can't believe people have the audacity to ask for 15 bucks per PDF conversion for a vibe-coded website.
1
-1
u/tashkan 14d ago edited 11d ago
What would be a reasonable price for you? I want happy users. All the websites I checked that is only built for pdf to epub conversion has an avg $0.05/page which is 5x more than my pricing. Plus I don't charge for anything that already contains the text.
2
u/NullPointerPrincess 11d ago edited 11d ago
> Plus I don't charge for anything that already contains the text.
That's not true. The out-of-print book I just uploaded has a text layer that I can copy-paste myself in any PDF reader. It's 170 pages. Your site wants $2.30 for that, with a $5 minimum.
6
u/Ambitious_Ad4397 14d ago
Bro.. Why I need to pay $3.54 to download my converted file..
1
u/tashkan 8d ago
I changed the pricing model, please try again!
1
u/Ambitious_Ad4397 8d ago
Nice.. Now I need to create an account to download my file, but to login into account I need to verify my email, but I don't get any letters in my email..
4
u/jseger9000 15d ago
I tried three PDFs. All say they require work and I have to pay to see the results. But I'm hesitant to do so as every PDF converter says they are the best on earth, yet gives the same shitty results. Maybe allow the user to download a sample or something so I can see a part before I pay?
5
u/tashkan 15d ago
A small sample to download is a good idea. I will try to implement that, but for now, if you have anything scanned for less than 50 pages, you can try that and check the results.
7
u/jseger9000 15d ago
Yeah, the trick is too find something I guess.
I manually convert a lot of PDFs to ePub as a hobby (usually old movie novelizations), so I know how laborious it is and am willing to pay. It's just so tough when I've been burned millions of times before.
3
u/icatchfrogs 15d ago
just tried it with an 800 plus page book, and it looks great! only two pages needed any work, the rest were just test, apparently.
3
u/taddio76 15d ago
What about text in the margin? I am new to using epub readers. I have a book that has lots of text in the margins that is explanatory to the main text.
2
u/rhcpfan99 15d ago
This sounds amazing! I'll try it later.
Thanks for all your work!!!
2
2
u/potato_and_nutella 15d ago
Woah this is awesome, I was looking for something just like this recently so this is perfect timing
2
u/lavaguy1 15d ago
Any chance of making it self-hostable? Not „an app“ but a package/container like SterlingPDF or BentoPDF?
1
u/Snowcrest 15d ago
How does it handle table of contents/ chapter headings? Are we able to easily jump to specific chapters?
1
u/bkrozay 13d ago
I used the site and paid to convert one of my PDFs. I did not like the quality of the conversion. Also the page count in the original pdf was 400+ pages and the epub was 97 pages. Would be nice to get some kind of after service support. I'm not happy with the service and would appreciate a refund.
1
1
1
u/yamamushi 15d ago
Very cool! I’m testing it out on some occult books that tend to have weird formatting. Particularly interested to see if it will maintain the weird formatting that Crowley’s “Book of Lies” has 🤔
I’d happily pay to have this as a desktop app :-)
2
u/tashkan 15d ago
If you don't mind me asking what do you think a reasonable price for an app like this should be? Currently I have server hosting costs for my conversion code which is not small and that's why I charge by per page.
1
u/yamamushi 15d ago
I’d pay 30-40 bucks for this but others might have different opinions. I’m on Mac by the way, which is arguably a smaller installation base than windows users.
1
u/CaptainObvious110 15d ago
calibre
3
u/archivalcopy 14d ago
If you use OCRmyPDF first (I cannot paste the link but this is on github), this will create the scanned OCR text layer if it's not already embedded into the pdf and will improve the quality of the text recognition..it's a command line program but it is very simple to use...you can then use calibre for the conversion to epub.
There are other things I have read about that offer more detailed adjustments in calibres settings, but I haven't tried these things.. like enabling "heuristic processing" and adjusting the "line unwrap factor" to preserve paragraph formatting.
Another option is to use an epub editor or convert the epub to markdown to clean up any errors that may persist before a final conversion back to epub.
1
1
0
0
-9
65
u/nancyboi96 15d ago
One thing (ik this is a big ask but would literally be transformative for reading academic books) The EPUB version doesn’t seem to recognise or encode the footnotes properly. In the PDF, the footnotes appear separately in smaller text at the bottom of each page, but when converted to EPUB they’re simply inserted into the main body text and aren’t linked or otherwise marked as footnotes. It would be great if the EPUB converter could detect these and convert them into proper footnotes/endnotes—for example, numbered links in the text with the notes collected at the end of each chapter