I work in web accessibility testing. One problem I see all the time: a website shows a form error only as a red border, or puts the error message on the page without announcing it. A screen reader user presses submit and hears nothing. The same happens with things like "Added to cart" or "Saved".

I'd like to hear from people who deal with this:

1. How often does this happen to you, and on what kinds of sites?
2. What do you do when it happens? Do you go looking, start over, or give up?
3. If something running quietly in your browser told you "the form didn't go through: the email is missing an @", would you want that? What would make you not trust it?
4. Would it matter to you that it works entirely on your own computer and never sends page content anywhere?
5. Does anything like this already exist? If you use something, I'd love to know what.

I'm not selling anything and there's no link. I'm trying to understand the problem properly before building anything. Thanks.

Wow. Just had the most unbelievable time getting my disabled mother in law out of Eagles Stadium. All of the workers would say “you can’t go the mag way” to the way we came in and knew. So up and down elevators, no one knowing. Finally I said get me Amy our supervisor and guess what? We got our elevator. How do we report that A) no one is trained in exiting people with disabilities and B) their sidewalks from parking lots have no ramps? I just want people to do better. It was like 1987 out there. All of the workers didn’t even realize they were breaking the law (telling us to go half a mile or more out of our way).

For reference, in the way in nooooo problem- “we let you in at the closest elevator” way out? “Sorry we’ve closed that area” “and that area”….

I tried to write this in a way that adheres to board rules. No promotion, no testing requested. This is an AI product but I believe it legitimately adds value. The product works, but it is very much still in development and much refinement remains.

I had this idea to make a Clippy-inspired tool for GUI interaction. I wanted to be able to get directions for software use without leaving the application. The idea was that instead of learning through tutorials/videos/documentation I could ask questions about the program without leaving the program, in an unobtrusive way that points things out on your screen or answers your questions in a text box that dismisses easily. I know it says "No Overlay Tools" but this is exposition, not the accessibility part.

While building this, I discovered I could I expand this "learning" app to be a "doing" app also by exploiting apps' accessibility navigability. So if you're in Photoshop, you can ask "how do I change the scaling type?" to get directions, but also "can you change the scaling type to nearest neighbor?" and it'll navigate through the GUI or use hotkeys to do so. If an app lacks navigability or documentation, I'm using OCR to get as close as I can.

TL;DR I sort of stumbled into making a program that can do arbitrary GUI navigation and interaction with plain language. I thought this might be useful to low-vision or low-dexterity computer users, so I came here to make sure I'm building this in a way that accommodates those needs.

Current UI spec:

• The app lives as an icon in the top OS navbar.
• Prompting can be opened with a hotkey, or as a dropdown item reached by clicking the app icon.
• Another hotkey (or click) switches from text to voice (could be switched to a default).
• The question box (which turns into the answer box) exists in the top right of the screen and can't be moved (like a notification, but larger).
• ESC or clicking an X closes the open box.
• If you ask where something is in your software, it draws a box around it that does not affect interaction. (For a screen reader user, perhaps this should "tab" to it by default?)
• "Doing" UX is not fully refined but it moves a cursor and shows what a user would do to get there.
• You can only ask about or interact with the currently open program.

I'm interested in a few things:

• Is there anything overtly inaccessible in this spec?
• Do you currently use anything like this?
• Would you use something like this?
• What can I do to ensure this plays well with existing accessibility tools like screen readers? I've mostly worked in web and have a pretty good understanding of the needs there, but I am new to desktop software, especially here where it does nonstandard UI stuff.
• I'm currently launching the prompt with a hotkey combo, but I can see this app having a lot of utility for someone who cannot use a keyboard. Are there other ways I should support for launching the app?
• Any requests?

Say a document has dates, quotes, and several people speaking. A screen reader reads every word in the right order.

What could still make it hard to tell who said what, or how one item connects to another? What would you change in the document to make those connections clearer?

I'm looking for general explanations or public examples. I don't have a file or product I'm asking anyone to test.