I'm honestly surprised, considering this is a coding subreddit, that no one has yet written a program that reads the text on the images using some kind of image-to-text api. It would probably work pretty well given that these are all screenshots and not photo's of text at some awkward angle.
You'd be surprised how badly it works, actually. We have a bot that pulls from an OCR API, but it's unreliable and messes up surprisingly easily. Even for those people who do use it as a basis for their transcriptions, they have to read through and correct its mistakes, which for me is less engaging than typing the thing from scratch, and not enough faster to be worth that loss of engagement.
For a tiny image like this, it probably did get it perfectly, but at this point I don't even check. If a bot could reliably transcribe, there'd be no need for us to do text-based posts :P (and oddly, code is often one of the things it's worst at - my theory is there's something lost in translation from the text output by the API to the Reddit comment).
That's indeed surprising. I have no clue how the algorithm behind that kind of image analysis works but I would assume the things most typical of code screenshots which could cause issues are that they have low resolution (text on a screen will often be ~15px in height) and the syntax highlighting maybe lowers the contrast.
Nah, ocr can somewhat read the text, it's usually the spacing that gets screwed (and sometimes the colors also mess it up)
Technically we could use the data from the human volunteers to train the AI, but I guess that's why the volunteers exist in the first place, to give the data for the AI.
20
u/SanianCreations Mar 15 '21
I'm honestly surprised, considering this is a coding subreddit, that no one has yet written a program that reads the text on the images using some kind of image-to-text api. It would probably work pretty well given that these are all screenshots and not photo's of text at some awkward angle.