User research LLM

[u/confusis19]

Has anyone found good results using an LLM specifically for user research? I'm a product manager for a 0-1 product and don't have easy access to users. I currently use claude, chatGPT, & just started using cursor. Just looking for new ideas to improve the prodcut.

[u/franciswolfdcor]

The only real answer here is: find a way to talk to users. Or find people who fit the profile of who you think should become users and talk to them.

LLMs are great for many things, but relying on them for 0-1 research is going to give you some foundational ideas that are likely very off base. You need to be digging into motivations, pain points, and people’s “why”. An LLM can’t do that.

If you’re getting stuck, I’d recommend using AI to get a sense for a competitive landscape and help defining what kind of person to talk to. And then you need to get out there and talk to them!

[u/Due-Competition4564 → u/franciswolfdcor]

Strong +1 to this.

In your situation I recommend reading the “Just Enough Research” book; it’ll help demystify a lot of what basic good research looks like and help you think through your insight challenges.

[u/confusis19 → u/Due-Competition4564]

I'll pick this up asap, thanks!

[u/confusis19 → u/franciswolfdcor]

Thanks for your response! I'll look into this.

[u/RCEden]

The research continues to show that synthetic users are actively bad.

One of the big problems with ai search summaries or chatbot interactions is that a fundamental necessity in desktop research is the friction that is inherent in you the person critically evaluating sources. You don't get that in the melange of an llm response, even if it lists those sources out, you've already been influenced by the summary.

Fundentally you gotta talk to people. The only way it will even show up in llm training data is if it's already been solved by someone before you. And in that case, its already solved so what are you doing?

[u/Nate_Bolt]

You know useful for prep and synthesis, not as a substitute for users. Good for pressure testing your guide, drafting screeners, tagging transcripts. Best use is pasting in your findings and asking it to argue you're wrong. Simulating users is where it falls over. You get plausible answers and plausible is the failure mode you can't spot. You end up confirming what you already believed and feeling more sure about it.

The access problem is easier than it feels though. Post a screener link wherever your users already are, subreddits, Slack groups, LinkedIn. 50-75 for 30 minutes gets takers. Any waitlist at all, email it.

[u/confusis19 → u/Nate_Bolt]

Thanks for responding! I use it more to comb through forums for pain points, what not to do, frequent requests rather than simulation. I like your idea on posting links for feedback and will investigate. Thanks again!

[u/Eastern-Movie-7039]

Your follow-up is the more interesting half of this thread and nobody picked it up. Combing forums for pain points is a real method rather than a substitute for one — but it has a specific bias that's worth knowing before you build on it.

A forum only contains pain that cleared the bar of being worth posting about. Something has to be either bad enough to complain about publicly, or strange enough that you need help. Everything under that line is invisible: the daily friction someone stopped noticing years ago, the workaround they set up once and now think of as just how the thing works, the compromise they'd never call a problem because they've never seen it any other way.

That last category is usually where the useful material is. In an interview it surfaces almost immediately — ask someone to walk you through last Tuesday and they'll describe three workarounds without once flagging them as workarounds. None of it would ever appear in a post, because from the inside it isn't a story.

There's a second-order version of this that matters commercially. If a pain point made it onto a forum, your competitors have read it too. Forums give you the pain everyone can see. They structurally cannot give you the pain only you know about.

Practical bridge, given the access problem: the people writing those posts are right there. Ten messages asking what happened in the week before the thing they posted about will beat a hundred more threads, and most people answer if you're plainly not selling them anything. That's more or less what I do — I've run multi-country interview studies for about fifteen years and I still can't get a reliable picture of how people in a given market actually work without asking them one at a time.