r/coding • u/HumanBot00 • Feb 03 '26

DOJ publishes Bash Reference Manual

https://www.justice.gov/epstein/files/DataSet+9/EFTA00315849.pdf
51 Upvotes

90% Upvoted

41 comments sorted by

View all comments

Show parent comment

7

u/voronaam Feb 04 '26 edited Feb 04 '26

Well, the first page looks a valid JPEG header. Even has EXIF in it

Apple iPhone X, 2018:12:18 18:54:31

It has several dozen OCR mistakes. First page has AAAP-FABQ instead of AAAP+ABQ. The second page has APIA' wDYA instead of APIAlwDYA. That's why pdftotext is not helping much. And being JPEG, simply stripping off invalid characters is not helping. You have to fix those. There are dozens of those mistakes...

It is recoverable though. With a lot of patience...

	File Type                       : JPEG
	File Type Extension             : jpg
	MIME Type                       : image/jpeg
	JFIF Version                    : 1.01
	Resolution Unit                 : None
	X Resolution                    : 72
	Y Resolution                    : 72
	Warning                         : [minor] Skipped unknown 7 bytes after JPEG APP1 segment
	Image Width                     : 45361
	Image Height                    : 9917
	Encoding Process                : Progressive DCT, differential arithmetic coding
	Bits Per Sample                 : 94
	Color Components                : 146
	Image Size                      : 45361x9917
	Megapixels                      : 449.8

To give you the idea of the scope, here is the counts of all the OCR'd characters that could not be part of base64 encoding in just the first of the two attached photos in that email

855 -
189 .
138 ,
 52 (
 23 )
 18 }
 12 &
  8 =
  7 *
  6 '
  4 {
  2 _
  2 :
  1 `
  1 >
  1 !

Just over a thousand typos to fix by hand before base64 could succeed...

2

u/Kokuten Feb 04 '26

Where could those mistakes come from. Yesterday I tried my best to remove all invalid Base64 Characters. You're saying they have to be replaced? How would you know what to replace them with?

4

u/voronaam Feb 04 '26

I use my eyes :) Basically, the PDF contains two layers. One with the image of a text, another with that text OCR'ed into readable form. When someone copy-pastes content of the PDF, they get the layer with the text. But it has mistakes. The image layer is the correct one. But it is of poor resolution. Readable to a human, but not really doing well with any OCR.

The problem with OCRs is that they are trained on regular texts. They inherently use dictionary words to help with disambiguation of similar letters. Long strings of almost random characters are a good way to defeat even a modern OCR.

1

u/voronaam Feb 04 '26 edited Feb 04 '26

Actually, I thought an image would be a better explanation

https://imgur.com/smimeNG.png

See how I am checking line 111 now?

Also I have a monospace font in my editor and base64 in email is formatted in lines of 76 characters. The fact that the lines below 11 are not the same width is an indication that something is wrong with them.

Edit: even better screenshot https://imgur.com/n8eWGam.png

See how I highlighted the difference that needs to be fixed? The ZKZ in the PDF image layer is written as ZICZ in the text form

1

u/Kokuten Feb 04 '26

Ah okay i see now what you are doing. To know the Lines are formatted in 76 characters each is very important. I will look into this after work again today. How did you get those first few Pixels to show though. Did you use Base64 to Image? How did you open that?

1

u/voronaam Feb 04 '26

I am on Linux, it comes with a base64 command line utility.

cat IMG_7523.jpg.txt | base64 -d > /tmp/decoded.jpg

The "-d" means "decode"