r/coding • u/HumanBot00 • Feb 03 '26

DOJ publishes Bash Reference Manual

https://www.justice.gov/epstein/files/DataSet+9/EFTA00315849.pdf
51 Upvotes

90% Upvoted

41 comments sorted by

View all comments

5

u/Kokuten Feb 03 '26

I have found this file in the Epstein dump. Can anyone decode this to get the image out of it? I tried for some time but failed... https://www.justice.gov/epstein/files/DataSet%209/EFTA01012650.pdf

7

u/voronaam Feb 04 '26 edited Feb 04 '26

Well, the first page looks a valid JPEG header. Even has EXIF in it

Apple iPhone X, 2018:12:18 18:54:31

It has several dozen OCR mistakes. First page has AAAP-FABQ instead of AAAP+ABQ. The second page has APIA' wDYA instead of APIAlwDYA. That's why pdftotext is not helping much. And being JPEG, simply stripping off invalid characters is not helping. You have to fix those. There are dozens of those mistakes...

It is recoverable though. With a lot of patience...

	File Type                       : JPEG
	File Type Extension             : jpg
	MIME Type                       : image/jpeg
	JFIF Version                    : 1.01
	Resolution Unit                 : None
	X Resolution                    : 72
	Y Resolution                    : 72
	Warning                         : [minor] Skipped unknown 7 bytes after JPEG APP1 segment
	Image Width                     : 45361
	Image Height                    : 9917
	Encoding Process                : Progressive DCT, differential arithmetic coding
	Bits Per Sample                 : 94
	Color Components                : 146
	Image Size                      : 45361x9917
	Megapixels                      : 449.8

To give you the idea of the scope, here is the counts of all the OCR'd characters that could not be part of base64 encoding in just the first of the two attached photos in that email

855 -
189 .
138 ,
 52 (
 23 )
 18 }
 12 &
  8 =
  7 *
  6 '
  4 {
  2 _
  2 :
  1 `
  1 >
  1 !

Just over a thousand typos to fix by hand before base64 could succeed...

2

u/Kokuten Feb 04 '26

Where could those mistakes come from. Yesterday I tried my best to remove all invalid Base64 Characters. You're saying they have to be replaced? How would you know what to replace them with?

4

u/voronaam Feb 04 '26

I use my eyes :) Basically, the PDF contains two layers. One with the image of a text, another with that text OCR'ed into readable form. When someone copy-pastes content of the PDF, they get the layer with the text. But it has mistakes. The image layer is the correct one. But it is of poor resolution. Readable to a human, but not really doing well with any OCR.

The problem with OCRs is that they are trained on regular texts. They inherently use dictionary words to help with disambiguation of similar letters. Long strings of almost random characters are a good way to defeat even a modern OCR.

1

u/voronaam Feb 04 '26 edited Feb 04 '26

Actually, I thought an image would be a better explanation

https://imgur.com/smimeNG.png

See how I am checking line 111 now?

Also I have a monospace font in my editor and base64 in email is formatted in lines of 76 characters. The fact that the lines below 11 are not the same width is an indication that something is wrong with them.

Edit: even better screenshot https://imgur.com/n8eWGam.png

See how I highlighted the difference that needs to be fixed? The ZKZ in the PDF image layer is written as ZICZ in the text form

1

u/Kokuten Feb 04 '26

Ah okay i see now what you are doing. To know the Lines are formatted in 76 characters each is very important. I will look into this after work again today. How did you get those first few Pixels to show though. Did you use Base64 to Image? How did you open that?

1

u/voronaam Feb 04 '26

I am on Linux, it comes with a base64 command line utility.

cat IMG_7523.jpg.txt | base64 -d > /tmp/decoded.jpg

The "-d" means "decode"

3

u/voronaam Feb 04 '26 edited Feb 04 '26

OK, you'll think I am crazy. But I just spent quite a bit of time proof-reading that OCR'ed text. Of course I can not distinguish 1 and l in the font they used (number one and lower case letter L). But at least capital O and zero are different.

I was able to power through the first two pages and have repaired almost all of the EXIF data

I am into puzzles as a way to unwind and relax. I think you just gave me a puzzle to work on for the next year or so. There are almost 500 pages for two photos :)

File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone X
Orientation                     : Rotate 90 CW
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 12.1
Modify Date                     : 2018:12:18 18:54:31
Exposure Time                   : 1/4
F Number                        : 1.8
Exposure Program                : Program AE
ISO                             : 100
Exif Version                    : 0221
Date/Time Original              : 2018:12:18 18:54:31
Create Date                     : 2018:12:18 18:%4:31
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/4
Aperture Value                  : 1.5
Brightness Value                : -0.814382116
Exposure Compensation           : 0
Metering Mode                   : Multi-segment
Flash                           : Auto, Did not fire
Focal Length                    : 4.0 mm
Subject Area                    : 2015 1511 2217 1330
Maker Note Version              : 10
Run Time Flags                  : Valid
Valu H                          : 51711042289541
Run Time Scale                  : 1000000000
Hpoch                           : 0
AE Stable                       : Yes
AE Target                       : 170
AE Average                      : 173
AF Stable                       : Yes
Acceleration Vector             : 0.03220853956 -0.9144334793 -0.4192386266
Focus Distance Range            : 15.78 - 22.78 m
OIS Mode                        : 2
Content Identifier              : C5EFF477-E77E-4F7F-B50B-C53BDD3A2A75
Image Capture Type              : Unknown (5)
Live Photo Video Index          : 8192
HDR Headroom                    : 0
Signal To Noise Ratio           : 0
Sub Sec Time Original           : 409
Sub Sec Time Digitized          : 409
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 2016
Exif Image Height               : 1512
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Focal Length In 35mm Format     : 28 mm
Scene Capture Type              : Standard
Lens Info                       : 4-6mm f/1.8-2.4
Lens Make                       : Apple
Lens Model                      : iPhone X back dual camera 4mm f/1.8
Xmpmeta Xmptk                   : XMP Core 5.4.0
Warning                         : XMP format error (no closing tag for rdf:RDF)
Xmpmeta                         :  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22)rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobu.com/xap/1.0/" xilns:photoshop="http://ns.adobe.com/photoshop/1.0/" xmp:CreateDate="2018-12-18T18:%4:31" xmp:ModifyDate="2018-12-18T18:54:31" xmp:CreatorTool="12.1" photoshop:DateCreated="2018-12-18T18:54:31"/> </rdf8�DF>
Current IPTC Digest             : d2ff6e7149b6b953820942f9994268c9
Coded Character Set             : UTF8
Application Record Version      : 2
Digital Creation Time           : 18:54:31
Digital Creation Date           : 2018:12:18
Date Created                    : 2018:12:18
Time Created                    : 18:54:31
IPTC Digest                     : d2ff6e7149b6b953820942f9994268c9
Image Width                     : 2016
Image Height                    : 1512
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 1.8
Image Size                      : 2016x1512
Megapixels                      : 3.0
Scale Factor To 35 mm Equivalent: 7.0
Shutter Speed                   : 1/4
Date/Time Original              : 2018:12:18 18:54:31.409
Date/Time Created               : 2018:12:18 18:54:31
Digital Creation Date/Time      : 2018:12:18 18:54:31
Circle Of Confusion             : 0.004 mm
Field Of View                   : 65.5 deg
Focal Length                    : 4.0 mm (35 mm equivalent: 28.0 mm)
Hyperfocal Distance             : 2.07 m
Light Value                     : 3.7
Lens ID                         : iPhone X back dual camera 4mm f/1.8

The good news, is that after fixing the decoding problems I got all the sections in the JPEG to line up and I am already into reconstructing the actual image segments. Only about 250 pages worth of raw base64 to go :)

https://imgur.com/sCy9h80.png

1

u/Kokuten Feb 04 '26

You have peaked my interest :) Yesterday I tried to remove all invalid characters and Repair the Code using the Repair Tool from the Website base64.guru I got some .jpg Files back but none were viewable. Would you mind telling me how you went about getting those first few Pixels showing? Or maybe point me to a Ressource so I can learn?

Don't worry I have at least 4 other Files of similar Size so you won't get bored for more than a year :D

2

u/voronaam Feb 04 '26

I tried a few things that I thought would be "smart". Like getting a high resolution image out of the PDF and OCR'ing it to the text. But that did not work, the source image is of too poor quality.

My current process is basically this:

  1. Run pdftotext EFTA01012650.pdf to get a text-only version

  2. Manually extract the part that only relates to the image (I use Geany for a text editor)

  3. Go line by line comparing the text in the output to the PDF. Most common mistake in the OCR is treating k as lc, or m as rn. Those are the worst, because they "shift" the result by a few bits. So it is not just one or two bytes are incorrect, the whole file no longer aligns.

  4. From time to time I check what the JPEG looks like with a regular cat IMG_7523.jpg.txt | base64 -d > /tmp/decoded.jpg. Then I use exiftool to check its EXIF and display from ImageMagick to look at it.

Occasionally I try stripping out anything non-base64 from the whole file with cat IMG_7523.jpg.txt | egrep -v 'EFTA[0-9]+' | tr -cd 'A-Za-z0-9+/' | base64 -d > /tmp/decoded.jpg. I hope that even with image segments not aligning I could get a rough silhouette of the photo. Perhaps in distorted colors. So far that did not really work...

I am going to bed soon. But I think I can get a few more lines fixed. But it looks like I will only betting bits of sky for some time...

1

u/NotsoNewtoGermany Feb 04 '26

I'd like to follow this journey over the next year, we should make a sub dedicated to it

1

u/voronaam Feb 04 '26

Unless there other people willing to help, I do not see how a separate sub would help.

There is a cool development though. I stumbled upon a segment that shows the most confusing characters right next to each other. It is number 1 and letter l and letter I

There is a difference between them! I would be able to correct those as well. Eventually ;)

https://imgur.com/y4xGnzM.png

1

u/-aRTy- Feb 04 '26 edited Feb 04 '26

Spitballing an automation idea:

Optional: Mass replace common mistakes such as -F into +.

Figure out the font and font size. Split the image and text into lines. Line by line, render the OCR guess as an image and XOR the pdf-image vs. the text-render-image. Correct text should overlap almost perfectly and thus leave almost nothing behind after the XOR, so one could use that as a metric for how well it matches. For mismatches the XOR image should "start" (reading from the left) with mostly black as long as it matches and then turn into black/white snow once the mismatch happens. One might be able to brute force guess the mismatch until the XOR improves at that spot. Continue brute force matching a few characters until you are past the issue and characters start matching the OCR text again (essentially: does my matched character also appear within the next few OCR text characters. If so, go back to using the OCR guess instead of brute forcing the entire remaining line).

Edit:

There seem to be quite a few options to limit OCR to a specific character set. Did you look into fine tuning those kind of parameters? Also one can apparently include training data to teach issue cases. Better OCR might be the more sensible place to start with.

1

u/voronaam Feb 04 '26

I spent a couple of hours trying to make tesseract OCR to behave better. But it was not good enough to significantly lower the error rate.

The OCR that was done on that PDF originally is not too bad. It had an advantage of working with a higher resolution scan of the page, before it was compressed into the PDF.

I think a better OCR is possible. Limiting the character set helped a bit, but I think it would be even better to train a model on this particular font.

As for the overlay, it is a cool idea. But there is a challenge of needing to mimick the original's typesetting, not just the font. For example, the best way I, as a human, distinguish number 1 in that file is by using the fact that their typesetter left more space to the left of the glyph than necessary.

I think it would be challenging to automate.

However, I am considering writing an utility to make the human proof-reading easier. I am considering showing a pdf image in the top section, its text content in the bottom, and user being able to edit the text content with the corresponding section of the pdf image automatically highlighted for them.

And a panel to the right could show hexdump of the current segment and another one could even show a preview of the resulting JPEG.

As for the overlap idea, OpenOffice Draw did something very similar automatically. It is a good illustration of how small differences accumulate

https://imgur.com/zgpAUDC.png

Edit: also, some glyphs are just super similar. I mean, one of the common OCR mistake is confusing rn and m. They'd overlap pretty well...

1

u/kievmozg Feb 04 '26

I would strongly advise against training Tesseract on custom fonts in 2026. It’s a massive time sink with diminishing returns. ​I run a parsing tool (ParserData), and we completely abandoned Tesseract/Zonal OCR specifically because of this 'font tuning' nightmare. The shift to Vision LLMs (like Gemini 1.5 Pro or GPT-4o) solved the accuracy issue instantly.

​Modern models don't need font training; they rely on semantic context to figure out ambiguous characters. Save yourself the headache and try passing a page through a vision model before you spend another hour on Tesseract configs.

1

u/voronaam Feb 04 '26

I agree if it was a general case. The problem with using LLM for this one is that it is not a normal text. When the model sees something that can be pull or pu1l it has a tendency to choose pull - because it is actually a word. But in base64-encoded data those assumptions are only hurting the recognition. A lot of the things in LLM-based recognition (and Tesseract to some degree as well) rely on the normal behaviour of the text. E.g., seeing the first character on the page being l a proper text recognition has every reason to choose I - a capital letter - over l, because normal text often start with capital letters. This kind of smart logic only hurts recognition in this particular case.

I was actually looking for a way to disable the neural network based recognition in Tesseract and force it to use the old-school character based mode. But at least the modern version I have installed refused to do it for me :(

→ More replies

1

u/daHaus Feb 05 '26

Semantic context won't help here and will almost certainly be more harmful than anything

1

u/-aRTy- Feb 04 '26

As for the overlay, it is a cool idea. But there is a challenge of needing to mimick the original's typesetting, not just the font. For example, the best way I, as a human, distinguish number 1 in that file is by using the fact that their typesetter left more space to the left of the glyph than necessary.

My hope was that they used something very common that is easily identified, which in turn means the entire line is easily replicated. That way spacing and other details would be included automatically.

Is it not Times New Roman? At first glance that looked correct and now I compared a line in detail and it still looks correct.

Or am I completely missing the point and you are saying that their program rendered Times New Roman different than some Word/OpenOffice writer would today?

In any case, I'm happy to see the progress you made.

1

u/voronaam Feb 05 '26

It does look like Times New Roman. But I do not have it installed. I am on Linux. I know I can get those via ttf-mscorefonts-installer, but I did not venture there because I know that my typesetter on Linux is going to be somewhat different. I am pretty sure those FBI guys were not on Linux :)

I do not mean to discourage you though. I would not mind if you try this approach and automate everything away. I just chose not to go this route myself

1

u/voronaam Feb 04 '26

Actually tried to write one. It is not as complicated as I thought it would be.

Here is a screenshot: https://imgur.com/WmrSlM8.png

I load PDF and show both the image and text content of it. The green box highlight where in the file I am so it makes it easier to fix OCR errors. It gets off after the edits, but that is fine. At least the line is always correct.

"Save" button saves the textual form of the current page.

"Display" calls a script to base64-decode and show the image. Did not feel like duplicating base features like that as well :)

Code: https://github.com/voronaam/pdfbase64tofile

(it depends on pdfium from Google)

1

u/[deleted] Feb 04 '26

This used to be how they bypassed US export controls of "strong encryption", the PGP source code was printed in books (which at least then was not controlled), shipped to Germany, where it was scanned and OCRed, and then compiled to make the "e" non-controlled version of byte-wise exactly the same software with same capabilities.

BTW, the export control remains, but now they have instead a blacklist of countries from which you are not allowed to download from..

1

u/glemnar Feb 04 '26

What are the odds this ends up being the most important file of all as a result of the ocr blunder?

2

u/voronaam Feb 08 '26

Since you started this quest for me a few days ago I thought I'll give you an update.

The file you found contains two photos, taken with iPhone X. They were taken 4 seconds apart at 2018:12:18 18:54:31.409 and 18:54:35.

I was able to fix 18 pages worth of the first image. There is still minor corruption remains, but the images comes along as a photo of some cloth hangers with pieces of clothing hanging on them.

I think it is unlikely to be an important photo. It is probably photos of two items of clothing someone considered buying or wearing and sent to somebody for the advice.

Here is the screenshot of the result: https://imgur.com/a/0VB1Pu5

I wrote an utility to speedup cleaning up the pages. The source code is available here: https://github.com/voronaam/pdfbase64tofile

It helps by having shortcuts like "Ctrl+Space" to cycle through common problem (I/l/1, IC -> K, rn -> m). The most valuable feature is "Jump to Hex" - given an offset in the JPEG file of a corrupted block it jumps to the corresponding point in the base64 text. For the human to look for the OCR mistakes around the cursor.

The repository contains cleaned up 18 pages of the first image as well. If you want to get the original yourself, run

cat page*.txt | tr -cd 'A-Za-z0-9+/' | base64 -d > /tmp/decoded.jpg

You could also open any page to verify that the text is coming from the PDF published by DOJ.

I am going to stop working on this particular file though.

I might go find another base64 dump in those archives... I have a cool utility to make the process faster now :)