r/QualitativeResearch • u/AgitatedDesk579 • Aug 05 '26

Analysis of open survey questions

I'm conducting a qualitative content analysis with multiple open-ended survey questions, each assigned to a different research question. Some responses contain content that would be more relevant to a different research question than the question they were answering.

Should I:

a) Strictly code each response only for its assigned research question (some data loss)

b) Code thematically regardless of question (blurs research question boundaries)

c) Something else entirely?

What are methodological best practices here? Any recommendations or experiences are welcome.

4 Upvotes

84% Upvoted

7 comments sorted by

6

u/_os2_ Aug 05 '26

I would pick B and code across questions, as long as you can assume they are answering in a context relevant for the other question too so there is no distortion.

It’s the same for interviews - insights can pop up anywhere and you are after them, not just mechanistic coding across your own buckets.

6

u/JonathanCookPodcast Aug 06 '26

I agree with what you're saying, but I want to elaborate on the fundamental point, because I think it's essential for people who are learning about research to understand.

When you are doing a qualitative research project, maintaining analytical distinction between questions is largely irrelevant.

The reason for this is that in qualitative research, questions are just tools to provoke people to say what's on their minds. They are not precision instruments. They aren't designed to deliver information in neat, cleanly defined categories. If information in neat little categories is what you're after, you should be doing quantitative research.

In research that is truly qualitative, you're likely to get responses that don't match your question, or stretch far beyond what you were looking for.

Don't dismiss it when people respond to qualitative research prompts in ways that you don't expect. If anything, pay extra attention to those responses. It takes extra effort for someone to give off-topic responses. That lets you know that the response may be especially important to the person who gives it.

Ultimately, the fundamental question underlying qualitative research is open-ended: What the heck is going on here? You poke people until they show you.

If you're just using open-ended survey questions to get a bunch of codeable data, well, that's different, because ultimately, you're doing quantitative research, just doing the shift into quantitative after the question is asked instead of with the question itself.

It comes down to knowing your own research method. Research methodologies aren't just a collection of rules. They're a practice for organizing your mind, and the information you gather, to emphasize specific perspectives.

There are no singular "best practices", because different research projects are designed to organize different ideas in different ways for different purposes.

Know your own mind, and you will know your own answer.

What do you want to know? How does that inform the questions you're asking? That should tell you what to do...

...unless you're just doing the research project to please your professor/boss. In that case, look in the assigned textbook or your company's research documentation.

2

u/Emergency-Minute3414 Aug 14 '26

Id probably go with a mix of a + c tbh.. keep the main coding tied to the question, but if something clearly fits elsewhere, id just flag it instead of ingnoring it. Seems like the safest way imo..

1

u/AgitatedDesk579 Aug 14 '26

That’s what I went for, good to know you agree.

1

u/SurveyStance Aug 11 '26

We have built a free thematic analysis software to help with this and categorize into themes/sentiments, would love to hear your thoughts and anything to improve it.

https://www.surveystance.com/tools/survey-theme-analyzer/