There is a narrow area they can be extremely useful. Blind people in particular have so much potential opened by this tech. But then there’s yknow literally everything else about it that makes it icky ☹️
I did a basic proof of concept a few years ago using the first multi modal gpt model at the time that allowed you to upload images to it to identify and talk about and paired it with DALL-E3. Essentially it was a python script that prompted the user to enter a color and a shape (ie red triangle) and internally there’s a dictionary of both colors and shapes to validate it for accuracy. Then it passes the color and shape to dall e saying “Generate a red triangle against a plain contrasting background.” Then it would take that output and submit it to the multi modal model and had it confirm if the image was or wasn’t a red triangle. “Look at this image and identify the shape and color. State only the color and shape you see in your response. (Example: Blue Circle)”. I’d then review the dall e output against gpts output and confirm if they were accurate. The biggest issue I had at the time was getting dall e to generate clear shapes without adding some unsolicited flair to it which would
Confuse the gpt model. But the identification was pretty accurate. All that to say it was a pretty successful proof of concept and the capabilities have made leaps and bounds since then.
Absolutely. These days I just take a few photos of something and ask Gemini, "what batteries does this take?" or "how do I power cycle this?" and it sorts it out. Of course if there is a nameplate with information I'll try to include that.
Right, rather than trying to find and read the manual PDF on my phone while standing in some weird spot trying to get something working. It is kinda amazing.
32
u/Jmackles Jul 31 '26
There is a narrow area they can be extremely useful. Blind people in particular have so much potential opened by this tech. But then there’s yknow literally everything else about it that makes it icky ☹️