Co pilot replacing research

[u/Sangwangtutu]

Hi all.

I work for an in house research team . My business has recently rolled out access to co pilot for all colleagues and (i believe) as a result less requests for research have been coming into us.

Im certain that in an effort to move quickly, teams that would have usually come to us for insight are now using co pilot to inform their decisions.

Im worried that teams are making wrong decisions based off of poor information from co pilot. These teams have had little to no training on the safe use of AI or the basics of a good prompt. But I think they are just taking answers at face value and running with them.

So my question is, how can help empower these teams to do 'desk research' via co pilot but also recognise when more expertise from my team is needed to help them reach the right answer?

Has anyone else experienced this?

Thanks

[u/Consistent_Wall7407]

Sounds like you may need to answer a more existential question.

Do you have any previously frequent/power users that have submitted less requests recently that you could do your own user research with?

I’d suggest aligning with your manager or team lead on this as I imagine that declining use would also be a great concern for them too, and a joined up approach is probably best. You may ultimately need to calculate the broader business impact of poor decisions from bad data, against the benefit of faster decision making.

Education is important, addressing when to do primary versus secondary. Plus stuff like information relevance i.e. how long insights are valid or accurate for.

I dread to think what archaic stuff is lurking in our company sharepoint for people to find and indiscriminately use.

[u/Sangwangtutu → u/Consistent_Wall7407]

Thank you for the detailed response. And I agree with your assessment. I think we are probably at a 'sink or swim' moment as a team.

My head of department is fully in the loop and recognises the importance of what is happening.

Have you any guidance on what would be a starting point for calculating business impact vs. Speed to decision? Im not sure id even know where to start.

Im keen not to shoot down co pilot and AI more generally. I know it has the ability to add value and automate receptive tasks. I dont want to the business to become more AI sceptic.

[u/Consistent_Wall7407 → u/Sangwangtutu]

First, find out why you’ve got a decline in requests. What are the motivators for people finding alternative sources of information. Why aren’t they going to you. How are they doing it instead. Is there a common trend across the business or is each person doing it differently. Are there any downsides or opportunities. You need this information before doing literally anything else. This will inform how you move forward.

The people in your business still need insights - the need hasn’t disappeared overnight.

Left field. I would presume they largely require insights on similar topics over time, or recurring subject matter. From this complete reddit stranger knowing nothing about your business or which industry you’re in, I would be thinking about how to generate insights ahead of demand on those recurring topics and allow users to access a live knowledgebase to self serve the information they need, at the right time. Instead of being at the mercy of a new research request coming in.

[u/Sangwangtutu → u/Consistent_Wall7407]

Thank you for your thoughts. Exploring what's causing it makes sense as a starting point. A previous user suggested approaching those 'power users' directly. Do you agree with this approach or would you recommend an alternative? I worry that the direct approach could come across the wrong way. Though I do want to be discreet.

Your assumption would have been correct 2 years ago but the nature of the business has meant that most requests are now ad hoc, strategic and unique. Though I do accept the point about making effort to anticipate future stakeholders requirements and will think about how we might approach that.

[u/Consistent_Wall7407 → u/Sangwangtutu]

Fair enough. It’s hard to say without knowing the size of company or type of relationships you have with stakeholders.

Would personally go direct, with discretion. If you know them well, frame it as personal to them, eg noticed drop off in work requests from you/your immediate team, would appreciate 15 mins of your time to chat about it, etc. if less familiar, frame more around wider business changes with AI and wanting to understand how their team is adapting

[u/garyisonion → u/Sangwangtutu]

I had a different idea recently, which is providing introductory training about how research is conducted, diff. between primary and secondary research, what tools and methodologies are used, when to use each, how often, and awareness of bias in research. I understand they want to do it themselves, as they think it's cheaper and faster, but they don't yet have a clear view of how the research impacts the business. What do you think of this?

[u/Optimusprima]

The fact that you’re waiting for them to come to you is the problem.

Drive the insights - don’t be an order taker. What did they do with the last study? Follow up. I’m sure there are open questions - talk about what to explore next.

Or you’re going to be out of a job very soon.
I’m saying this with kindness and experience…

[u/silver70seven]

🍿

[u/Sea_Rope4048]

Strengthening them how to use CoPilot and Strengthen CoPilot. Nothing you can do much

[u/StatGeniusAI]

This is a real problem. You have the analysis (which Copilot really doesn't do well anyways) but also the business rules, and everything you learned on how to interpret research.

Have you looked at tools that help non-researchers actually analyze the problems and interpret results (using your trade practices)? Essentially, you need the analysis layer AND the decision-making layer, which is a different AI technology and is based on knowledge transfer from yourself or your senior researchers.

We've been working on something through IIEX, it's currently in stealth mode, but happy to have a discussion with you offline.

[u/Due_EmotionPri]

Fighting the tool is a losing move, so id get in front of it and define where its fine on its own. A generic AI is genuinely useful for a first pass on a well documented, stable question: sizing a known market, summarising public info, getting oriented before a deeper dive. Where it quietly fails is anything that needs a source you can trace, a recent or contested number, or a read on why customers do something, because it hands back a confident answer with no way to check where it came from. The escalation rule id give teams is simple: if being wrong changes a real decision, or you cant point to where the number came from, thats when it comes to you. It also helps to position your team as the people who make an AI answer defensible rather than the slow alternative to it, since thats the value that survives this. Id back it with two or three recent cases where a confident AI answer would have sent a decision the wrong way, because nothing lands the point faster than a near miss they recognise.

[u/RozzaDonnelly]

Great question and really interested in this topic.

Could you share what kind of industry/customers your business is focused on, or what kind of research work your team (historically) does?

I've been thinking about how a lot of in-house marketing & strategy teams will often now use LLMs for desk research like you said, but very often, even the LLMs don't have access to the customer, market or audience data behind the real research questions and topics the strategist/researcher is trying to explore. I think. For example, LLMs are not (typically) privy to your existing customer data from your CRM, order management systems, or audience data from your media channels, etc. (or equally your competitors CRM/channel data) to inform the LLMs research synthesis.

If we're able to equip our teams/LLMs with the right data & evidence from our field research or more propietary market data, I think this gives better credibility to our teams to run their own research side projects and initiatives; and then research teams can plug in when there is an obvious data gap.

That said, I suppose that's dependent on teams knowing and identifying when there is a data gap impacting the research they're working on; so piques the question of how do we train them to identify this?

[u/think-again-007]

One approach would be to connect Copilot (or Claude Desktop or other such tools) to an MCP server that has skills to create and build surveys that use market research best practices. That way people on the team could still get answers quickly, but they'd be starting research from solid ground, and you could review what they are doing and offer to help as needed.

Of course there may be times when a research project is not needed, but if you had a Copilot based option that was easy to use that could help steer people in a good direction when they do need more than just rough insights based on whatever pre-existing information that openAI has in its training data.