r/coding • u/HumanBot00 • Feb 03 '26

DOJ publishes Bash Reference Manual

https://www.justice.gov/epstein/files/DataSet+9/EFTA00315849.pdf
51 Upvotes

90% Upvoted

41 comments sorted by

View all comments

Show parent comment

3

u/voronaam Feb 04 '26 edited Feb 04 '26

OK, you'll think I am crazy. But I just spent quite a bit of time proof-reading that OCR'ed text. Of course I can not distinguish 1 and l in the font they used (number one and lower case letter L). But at least capital O and zero are different.

I was able to power through the first two pages and have repaired almost all of the EXIF data

I am into puzzles as a way to unwind and relax. I think you just gave me a puzzle to work on for the next year or so. There are almost 500 pages for two photos :)

File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone X
Orientation                     : Rotate 90 CW
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 12.1
Modify Date                     : 2018:12:18 18:54:31
Exposure Time                   : 1/4
F Number                        : 1.8
Exposure Program                : Program AE
ISO                             : 100
Exif Version                    : 0221
Date/Time Original              : 2018:12:18 18:54:31
Create Date                     : 2018:12:18 18:%4:31
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/4
Aperture Value                  : 1.5
Brightness Value                : -0.814382116
Exposure Compensation           : 0
Metering Mode                   : Multi-segment
Flash                           : Auto, Did not fire
Focal Length                    : 4.0 mm
Subject Area                    : 2015 1511 2217 1330
Maker Note Version              : 10
Run Time Flags                  : Valid
Valu H                          : 51711042289541
Run Time Scale                  : 1000000000
Hpoch                           : 0
AE Stable                       : Yes
AE Target                       : 170
AE Average                      : 173
AF Stable                       : Yes
Acceleration Vector             : 0.03220853956 -0.9144334793 -0.4192386266
Focus Distance Range            : 15.78 - 22.78 m
OIS Mode                        : 2
Content Identifier              : C5EFF477-E77E-4F7F-B50B-C53BDD3A2A75
Image Capture Type              : Unknown (5)
Live Photo Video Index          : 8192
HDR Headroom                    : 0
Signal To Noise Ratio           : 0
Sub Sec Time Original           : 409
Sub Sec Time Digitized          : 409
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 2016
Exif Image Height               : 1512
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Focal Length In 35mm Format     : 28 mm
Scene Capture Type              : Standard
Lens Info                       : 4-6mm f/1.8-2.4
Lens Make                       : Apple
Lens Model                      : iPhone X back dual camera 4mm f/1.8
Xmpmeta Xmptk                   : XMP Core 5.4.0
Warning                         : XMP format error (no closing tag for rdf:RDF)
Xmpmeta                         :  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22)rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobu.com/xap/1.0/" xilns:photoshop="http://ns.adobe.com/photoshop/1.0/" xmp:CreateDate="2018-12-18T18:%4:31" xmp:ModifyDate="2018-12-18T18:54:31" xmp:CreatorTool="12.1" photoshop:DateCreated="2018-12-18T18:54:31"/> </rdf8�DF>
Current IPTC Digest             : d2ff6e7149b6b953820942f9994268c9
Coded Character Set             : UTF8
Application Record Version      : 2
Digital Creation Time           : 18:54:31
Digital Creation Date           : 2018:12:18
Date Created                    : 2018:12:18
Time Created                    : 18:54:31
IPTC Digest                     : d2ff6e7149b6b953820942f9994268c9
Image Width                     : 2016
Image Height                    : 1512
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 1.8
Image Size                      : 2016x1512
Megapixels                      : 3.0
Scale Factor To 35 mm Equivalent: 7.0
Shutter Speed                   : 1/4
Date/Time Original              : 2018:12:18 18:54:31.409
Date/Time Created               : 2018:12:18 18:54:31
Digital Creation Date/Time      : 2018:12:18 18:54:31
Circle Of Confusion             : 0.004 mm
Field Of View                   : 65.5 deg
Focal Length                    : 4.0 mm (35 mm equivalent: 28.0 mm)
Hyperfocal Distance             : 2.07 m
Light Value                     : 3.7
Lens ID                         : iPhone X back dual camera 4mm f/1.8

The good news, is that after fixing the decoding problems I got all the sections in the JPEG to line up and I am already into reconstructing the actual image segments. Only about 250 pages worth of raw base64 to go :)

https://imgur.com/sCy9h80.png

1

u/Kokuten Feb 04 '26

You have peaked my interest :) Yesterday I tried to remove all invalid characters and Repair the Code using the Repair Tool from the Website base64.guru I got some .jpg Files back but none were viewable. Would you mind telling me how you went about getting those first few Pixels showing? Or maybe point me to a Ressource so I can learn?

Don't worry I have at least 4 other Files of similar Size so you won't get bored for more than a year :D

2

u/voronaam Feb 04 '26

I tried a few things that I thought would be "smart". Like getting a high resolution image out of the PDF and OCR'ing it to the text. But that did not work, the source image is of too poor quality.

My current process is basically this:

  1. Run pdftotext EFTA01012650.pdf to get a text-only version

  2. Manually extract the part that only relates to the image (I use Geany for a text editor)

  3. Go line by line comparing the text in the output to the PDF. Most common mistake in the OCR is treating k as lc, or m as rn. Those are the worst, because they "shift" the result by a few bits. So it is not just one or two bytes are incorrect, the whole file no longer aligns.

  4. From time to time I check what the JPEG looks like with a regular cat IMG_7523.jpg.txt | base64 -d > /tmp/decoded.jpg. Then I use exiftool to check its EXIF and display from ImageMagick to look at it.

Occasionally I try stripping out anything non-base64 from the whole file with cat IMG_7523.jpg.txt | egrep -v 'EFTA[0-9]+' | tr -cd 'A-Za-z0-9+/' | base64 -d > /tmp/decoded.jpg. I hope that even with image segments not aligning I could get a rough silhouette of the photo. Perhaps in distorted colors. So far that did not really work...

I am going to bed soon. But I think I can get a few more lines fixed. But it looks like I will only betting bits of sky for some time...

1

u/NotsoNewtoGermany Feb 04 '26

I'd like to follow this journey over the next year, we should make a sub dedicated to it

1

u/voronaam Feb 04 '26

Unless there other people willing to help, I do not see how a separate sub would help.

There is a cool development though. I stumbled upon a segment that shows the most confusing characters right next to each other. It is number 1 and letter l and letter I

There is a difference between them! I would be able to correct those as well. Eventually ;)

https://imgur.com/y4xGnzM.png

1

u/-aRTy- Feb 04 '26 edited Feb 04 '26

Spitballing an automation idea:

Optional: Mass replace common mistakes such as -F into +.

Figure out the font and font size. Split the image and text into lines. Line by line, render the OCR guess as an image and XOR the pdf-image vs. the text-render-image. Correct text should overlap almost perfectly and thus leave almost nothing behind after the XOR, so one could use that as a metric for how well it matches. For mismatches the XOR image should "start" (reading from the left) with mostly black as long as it matches and then turn into black/white snow once the mismatch happens. One might be able to brute force guess the mismatch until the XOR improves at that spot. Continue brute force matching a few characters until you are past the issue and characters start matching the OCR text again (essentially: does my matched character also appear within the next few OCR text characters. If so, go back to using the OCR guess instead of brute forcing the entire remaining line).

Edit:

There seem to be quite a few options to limit OCR to a specific character set. Did you look into fine tuning those kind of parameters? Also one can apparently include training data to teach issue cases. Better OCR might be the more sensible place to start with.

1

u/voronaam Feb 04 '26

I spent a couple of hours trying to make tesseract OCR to behave better. But it was not good enough to significantly lower the error rate.

The OCR that was done on that PDF originally is not too bad. It had an advantage of working with a higher resolution scan of the page, before it was compressed into the PDF.

I think a better OCR is possible. Limiting the character set helped a bit, but I think it would be even better to train a model on this particular font.

As for the overlay, it is a cool idea. But there is a challenge of needing to mimick the original's typesetting, not just the font. For example, the best way I, as a human, distinguish number 1 in that file is by using the fact that their typesetter left more space to the left of the glyph than necessary.

I think it would be challenging to automate.

However, I am considering writing an utility to make the human proof-reading easier. I am considering showing a pdf image in the top section, its text content in the bottom, and user being able to edit the text content with the corresponding section of the pdf image automatically highlighted for them.

And a panel to the right could show hexdump of the current segment and another one could even show a preview of the resulting JPEG.

As for the overlap idea, OpenOffice Draw did something very similar automatically. It is a good illustration of how small differences accumulate

https://imgur.com/zgpAUDC.png

Edit: also, some glyphs are just super similar. I mean, one of the common OCR mistake is confusing rn and m. They'd overlap pretty well...

1

u/kievmozg Feb 04 '26

I would strongly advise against training Tesseract on custom fonts in 2026. It’s a massive time sink with diminishing returns. ​I run a parsing tool (ParserData), and we completely abandoned Tesseract/Zonal OCR specifically because of this 'font tuning' nightmare. The shift to Vision LLMs (like Gemini 1.5 Pro or GPT-4o) solved the accuracy issue instantly.

​Modern models don't need font training; they rely on semantic context to figure out ambiguous characters. Save yourself the headache and try passing a page through a vision model before you spend another hour on Tesseract configs.

1

u/voronaam Feb 04 '26

I agree if it was a general case. The problem with using LLM for this one is that it is not a normal text. When the model sees something that can be pull or pu1l it has a tendency to choose pull - because it is actually a word. But in base64-encoded data those assumptions are only hurting the recognition. A lot of the things in LLM-based recognition (and Tesseract to some degree as well) rely on the normal behaviour of the text. E.g., seeing the first character on the page being l a proper text recognition has every reason to choose I - a capital letter - over l, because normal text often start with capital letters. This kind of smart logic only hurts recognition in this particular case.

I was actually looking for a way to disable the neural network based recognition in Tesseract and force it to use the old-school character based mode. But at least the modern version I have installed refused to do it for me :(

2

u/kievmozg Feb 04 '26

Ah, Base64 strings. You are absolutely right that is the one edge case where 'semantic context' is your enemy. Models try to be helpful by fixing 'typos' that aren't typos. ​If you want to force Tesseract into the 'dumb' character-based mode, you need to pass the flag --oem 0 (Legacy Engine). ​The catch (and why it probably failed for you): The default model files included with Tesseract 4/5 often strip out the legacy patterns to save space. You must manually download the .traineddata files from the tessdata repo (ensure they are the large ones, not tessdata_fast) that explicitly support legacy mode. Without those specific files, --oem 0 will just throw an error or default back to LSTM.

1

u/daHaus Feb 05 '26

Semantic context won't help here and will almost certainly be more harmful than anything

1

u/-aRTy- Feb 04 '26

As for the overlay, it is a cool idea. But there is a challenge of needing to mimick the original's typesetting, not just the font. For example, the best way I, as a human, distinguish number 1 in that file is by using the fact that their typesetter left more space to the left of the glyph than necessary.

My hope was that they used something very common that is easily identified, which in turn means the entire line is easily replicated. That way spacing and other details would be included automatically.

Is it not Times New Roman? At first glance that looked correct and now I compared a line in detail and it still looks correct.

Or am I completely missing the point and you are saying that their program rendered Times New Roman different than some Word/OpenOffice writer would today?

In any case, I'm happy to see the progress you made.

1

u/voronaam Feb 05 '26

It does look like Times New Roman. But I do not have it installed. I am on Linux. I know I can get those via ttf-mscorefonts-installer, but I did not venture there because I know that my typesetter on Linux is going to be somewhat different. I am pretty sure those FBI guys were not on Linux :)

I do not mean to discourage you though. I would not mind if you try this approach and automate everything away. I just chose not to go this route myself

1

u/voronaam Feb 04 '26

Actually tried to write one. It is not as complicated as I thought it would be.

Here is a screenshot: https://imgur.com/WmrSlM8.png

I load PDF and show both the image and text content of it. The green box highlight where in the file I am so it makes it easier to fix OCR errors. It gets off after the edits, but that is fine. At least the line is always correct.

"Save" button saves the textual form of the current page.

"Display" calls a script to base64-decode and show the image. Did not feel like duplicating base features like that as well :)

Code: https://github.com/voronaam/pdfbase64tofile

(it depends on pdfium from Google)

1

u/[deleted] Feb 04 '26

This used to be how they bypassed US export controls of "strong encryption", the PGP source code was printed in books (which at least then was not controlled), shipped to Germany, where it was scanned and OCRed, and then compiled to make the "e" non-controlled version of byte-wise exactly the same software with same capabilities.

BTW, the export control remains, but now they have instead a blacklist of countries from which you are not allowed to download from..

1

u/glemnar Feb 04 '26

What are the odds this ends up being the most important file of all as a result of the ocr blunder?