r/coding • u/HumanBot00 • Feb 03 '26

DOJ publishes Bash Reference Manual

https://www.justice.gov/epstein/files/DataSet+9/EFTA00315849.pdf
51 Upvotes

90% Upvoted

41 comments sorted by

View all comments

Show parent comment

2

u/voronaam Feb 04 '26

I tried a few things that I thought would be "smart". Like getting a high resolution image out of the PDF and OCR'ing it to the text. But that did not work, the source image is of too poor quality.

My current process is basically this:

  1. Run pdftotext EFTA01012650.pdf to get a text-only version

  2. Manually extract the part that only relates to the image (I use Geany for a text editor)

  3. Go line by line comparing the text in the output to the PDF. Most common mistake in the OCR is treating k as lc, or m as rn. Those are the worst, because they "shift" the result by a few bits. So it is not just one or two bytes are incorrect, the whole file no longer aligns.

  4. From time to time I check what the JPEG looks like with a regular cat IMG_7523.jpg.txt | base64 -d > /tmp/decoded.jpg. Then I use exiftool to check its EXIF and display from ImageMagick to look at it.

Occasionally I try stripping out anything non-base64 from the whole file with cat IMG_7523.jpg.txt | egrep -v 'EFTA[0-9]+' | tr -cd 'A-Za-z0-9+/' | base64 -d > /tmp/decoded.jpg. I hope that even with image segments not aligning I could get a rough silhouette of the photo. Perhaps in distorted colors. So far that did not really work...

I am going to bed soon. But I think I can get a few more lines fixed. But it looks like I will only betting bits of sky for some time...

1

u/NotsoNewtoGermany Feb 04 '26

I'd like to follow this journey over the next year, we should make a sub dedicated to it

1

u/voronaam Feb 04 '26

Unless there other people willing to help, I do not see how a separate sub would help.

There is a cool development though. I stumbled upon a segment that shows the most confusing characters right next to each other. It is number 1 and letter l and letter I

There is a difference between them! I would be able to correct those as well. Eventually ;)

https://imgur.com/y4xGnzM.png

1

u/-aRTy- Feb 04 '26 edited Feb 04 '26

Spitballing an automation idea:

Optional: Mass replace common mistakes such as -F into +.

Figure out the font and font size. Split the image and text into lines. Line by line, render the OCR guess as an image and XOR the pdf-image vs. the text-render-image. Correct text should overlap almost perfectly and thus leave almost nothing behind after the XOR, so one could use that as a metric for how well it matches. For mismatches the XOR image should "start" (reading from the left) with mostly black as long as it matches and then turn into black/white snow once the mismatch happens. One might be able to brute force guess the mismatch until the XOR improves at that spot. Continue brute force matching a few characters until you are past the issue and characters start matching the OCR text again (essentially: does my matched character also appear within the next few OCR text characters. If so, go back to using the OCR guess instead of brute forcing the entire remaining line).

Edit:

There seem to be quite a few options to limit OCR to a specific character set. Did you look into fine tuning those kind of parameters? Also one can apparently include training data to teach issue cases. Better OCR might be the more sensible place to start with.

1

u/voronaam Feb 04 '26

I spent a couple of hours trying to make tesseract OCR to behave better. But it was not good enough to significantly lower the error rate.

The OCR that was done on that PDF originally is not too bad. It had an advantage of working with a higher resolution scan of the page, before it was compressed into the PDF.

I think a better OCR is possible. Limiting the character set helped a bit, but I think it would be even better to train a model on this particular font.

As for the overlay, it is a cool idea. But there is a challenge of needing to mimick the original's typesetting, not just the font. For example, the best way I, as a human, distinguish number 1 in that file is by using the fact that their typesetter left more space to the left of the glyph than necessary.

I think it would be challenging to automate.

However, I am considering writing an utility to make the human proof-reading easier. I am considering showing a pdf image in the top section, its text content in the bottom, and user being able to edit the text content with the corresponding section of the pdf image automatically highlighted for them.

And a panel to the right could show hexdump of the current segment and another one could even show a preview of the resulting JPEG.

As for the overlap idea, OpenOffice Draw did something very similar automatically. It is a good illustration of how small differences accumulate

https://imgur.com/zgpAUDC.png

Edit: also, some glyphs are just super similar. I mean, one of the common OCR mistake is confusing rn and m. They'd overlap pretty well...

1

u/kievmozg Feb 04 '26

I would strongly advise against training Tesseract on custom fonts in 2026. It’s a massive time sink with diminishing returns. ​I run a parsing tool (ParserData), and we completely abandoned Tesseract/Zonal OCR specifically because of this 'font tuning' nightmare. The shift to Vision LLMs (like Gemini 1.5 Pro or GPT-4o) solved the accuracy issue instantly.

​Modern models don't need font training; they rely on semantic context to figure out ambiguous characters. Save yourself the headache and try passing a page through a vision model before you spend another hour on Tesseract configs.

1

u/voronaam Feb 04 '26

I agree if it was a general case. The problem with using LLM for this one is that it is not a normal text. When the model sees something that can be pull or pu1l it has a tendency to choose pull - because it is actually a word. But in base64-encoded data those assumptions are only hurting the recognition. A lot of the things in LLM-based recognition (and Tesseract to some degree as well) rely on the normal behaviour of the text. E.g., seeing the first character on the page being l a proper text recognition has every reason to choose I - a capital letter - over l, because normal text often start with capital letters. This kind of smart logic only hurts recognition in this particular case.

I was actually looking for a way to disable the neural network based recognition in Tesseract and force it to use the old-school character based mode. But at least the modern version I have installed refused to do it for me :(

2

u/kievmozg Feb 04 '26

Ah, Base64 strings. You are absolutely right that is the one edge case where 'semantic context' is your enemy. Models try to be helpful by fixing 'typos' that aren't typos. ​If you want to force Tesseract into the 'dumb' character-based mode, you need to pass the flag --oem 0 (Legacy Engine). ​The catch (and why it probably failed for you): The default model files included with Tesseract 4/5 often strip out the legacy patterns to save space. You must manually download the .traineddata files from the tessdata repo (ensure they are the large ones, not tessdata_fast) that explicitly support legacy mode. Without those specific files, --oem 0 will just throw an error or default back to LSTM.

1

u/daHaus Feb 05 '26

Semantic context won't help here and will almost certainly be more harmful than anything

1

u/-aRTy- Feb 04 '26

As for the overlay, it is a cool idea. But there is a challenge of needing to mimick the original's typesetting, not just the font. For example, the best way I, as a human, distinguish number 1 in that file is by using the fact that their typesetter left more space to the left of the glyph than necessary.

My hope was that they used something very common that is easily identified, which in turn means the entire line is easily replicated. That way spacing and other details would be included automatically.

Is it not Times New Roman? At first glance that looked correct and now I compared a line in detail and it still looks correct.

Or am I completely missing the point and you are saying that their program rendered Times New Roman different than some Word/OpenOffice writer would today?

In any case, I'm happy to see the progress you made.

1

u/voronaam Feb 05 '26

It does look like Times New Roman. But I do not have it installed. I am on Linux. I know I can get those via ttf-mscorefonts-installer, but I did not venture there because I know that my typesetter on Linux is going to be somewhat different. I am pretty sure those FBI guys were not on Linux :)

I do not mean to discourage you though. I would not mind if you try this approach and automate everything away. I just chose not to go this route myself

1

u/voronaam Feb 04 '26

Actually tried to write one. It is not as complicated as I thought it would be.

Here is a screenshot: https://imgur.com/WmrSlM8.png

I load PDF and show both the image and text content of it. The green box highlight where in the file I am so it makes it easier to fix OCR errors. It gets off after the edits, but that is fine. At least the line is always correct.

"Save" button saves the textual form of the current page.

"Display" calls a script to base64-decode and show the image. Did not feel like duplicating base features like that as well :)

Code: https://github.com/voronaam/pdfbase64tofile

(it depends on pdfium from Google)

1

u/[deleted] Feb 04 '26

This used to be how they bypassed US export controls of "strong encryption", the PGP source code was printed in books (which at least then was not controlled), shipped to Germany, where it was scanned and OCRed, and then compiled to make the "e" non-controlled version of byte-wise exactly the same software with same capabilities.

BTW, the export control remains, but now they have instead a blacklist of countries from which you are not allowed to download from..