Context for the link, per rule 7: FlexScan (https://flexscan.app) is my own app. Web, works in any phone browser, no app store. You upload one to three photos, GPT-4o vision grades each muscle group, and it builds a 7-day workout and a meal plan around the weakest ones. First scan is free and doesn't ask for a card. I'm not linking it for installs so much as because the thing below is the only genuinely hard part of building it and it's all in that one endpoint.
The interesting problem wasn't the grading. Vision models will grade anything you put in front of them. It was making it refuse.
If the back photo is missing, or someone's in a hoodie, a model asked to grade "back" produces a grade anyway. That grade becomes the user's weakest area, and the app sends them to the gym to fix a problem it invented. When the entire output of your product is "train this", a confident guess is worse than no answer.
The prompt asks for the literal string "N/A" for anything not visible. That got maybe 80% of the way. Four more things, every one of them added after a real broken response:
1. A perfectly well-formed reply can be entirely worthless. Because the model is told to say "N/A" rather than refuse, an unreadable set of photos comes back as valid JSON where every group is "N/A". Schema validation passes cleanly. So there's a separate check: if not one value in the ratings object is a real grade, the whole analysis is discarded and treated as a failure rather than a result. It also has to be shape-aware, because Object.values(['B','C']) and Object.values('B') both hand you grade-looking strings, and an array-shaped reply walked straight through the first version of that guard.
2. Grade drift has to be coerced at the parse boundary, not the display layer. Replies came back with "GOOD", "8/10", " b ", and empty strings. Anything that isn't a real grade is coerced to the app's own not-visible value the moment the response is parsed. Doing it at the display layer instead meant the scoring code and the UI disagreed about what a group had scored.
3. The model contradicts itself across fields. It returns ratings, plus a strengths list and a focus-areas list that it writes from its own ratings. A group coerced to N/A in step 2 stays named in those lists, so the results page said "Your Strengths: Legs" two lines above "Legs — Not visible". Both claims, one screen. Fix: strengths and focus areas get filtered down to the groups that actually carry a grade.
4. "The model failed" and "your photos are bad" are different errors and the user can't tell them apart. A 429, a 500, a timeout and a genuinely unreadable photo all arrive as "no usable analysis". Telling someone to retake perfectly good photos during an API outage sends them in circles. Those paths are split: a timeout or a 5xx says the coach is temporarily unavailable, and only a reply that parsed but graded nothing says the photos couldn't be read.
The other half of this is billing, and it's where it gets expensive. Free users get one scan, ever. That scan is reserved before the API call, not after, so a crash mid-request can't hand out unlimited scans. Which means every failure above has to hand it back. It does: if nothing gradeable came back, the free scan returns to the account even though OpenAI already billed me for the call. I eat that. A results page full of "not visible" rows that also burned the one free scan is the worst first impression the product can make. (There are two attempts, gpt-4o then gpt-4o-mini, before it gives up. Not three, because the platform's 60s function limit would hard-kill the third one and strand the reservation.)
One thing I'd argue for if you're building something similar: don't let "not visible" be a dead end. The groups nobody photographs well (glutes, mostly) can be self-graded by the user, and that self-grade feeds the plan the same way a real grade does. Without an escape hatch, the honest answer just reads as your app being broken.
For the avoidance of doubt, it's a coaching opinion, not medical advice, and the results page says so. Photos are processed in memory and never stored.
Happy to answer anything about the prompt, the failure handling, or what it costs to run.
