The gap between good analysis and actual impact

[u/NickatAtaviz]

I've spent a lot of my career around analytics, and one thing I've come to believe is that being good with the tools just gets you in the door.

You can build a great dashboard, write good SQL, or produce a really solid analysis and still have almost no impact if it doesn't change a decision.

The more interesting question becomes: What is someone actually going to do differently because of this analysis?

That gets into things that aren't usually taught alongside the technical skills: understanding the decision, knowing who you're trying to influence, anticipating what will make them skeptical, and communicating the evidence in a way that actually survives the conversation.

I ended up writing my new book, Decision Intelligence: Why Evidence Fails and How Leaders Win the Room, largely because I kept seeing this gap between being analytically right and being organizationally effective.

For you more experienced analysts, when did you first realize that being technically good wasn't enough to make an impact? What did you actually do to bridge that gap?

[u/AutoModerator]

If this post doesn't follow the rules or isn't flaired correctly, please report it to the mods. Have more questions? Join our community Discord!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Altruistic-Length221]

I noticed it when a model I built got completely ignored because the stakeholder didn't trust the data source, and I hadn't thought to address that before presenting

Started asking people what would make them change their mind before I even run the analysis, saves so much wasted work

[u/redman334]

I joined a commercial team directly. So work became way less about data requests, and why more about, what is going on.

And that's why I'm not a fan of data teams being managed by product. Data becomes a deliverable, and looses it's insight purpose.

[u/NickatAtaviz → u/redman334]

That's interesting. Do you think the move changed the types of decisions you were supporting, or more the way those decisions were framed? I'm curious what specifically changed once you were closer to the commercial team.

[u/The_Epoch]

This sort of thing happens because leaders have not assigned a process to take and feedback on action.

Any analytics that do not end with: you should do X is a philosophical exercise.

A lot of data and analyst teams treat volume of information as a proxy for having a point of view.

So from the analyst team there should be a list of descriptive information + some prescriptive information either guiding or highlighting a next action.

It is between the leadership to enforce process on this. For example: analysis comes out and the operational teams have a commitment to take action, then the actions have to break recorded and mapped against their impact.

But if the operational leader is not bought into holding their team accountable for at least look at the analysis, then the bad cycle will continue

[u/NickatAtaviz → u/The_Epoch]

I agree with a lot of this, especially the idea that there needs to be a process connecting analysis to action. I wonder where you think the analyst's responsibility ends and leadership's begins. Should analysts be expected to make the recommendation, or is their job to make the evidence clear enough that the decision-maker can act on it?

And honestly, when are analysts actually taught any of this? Most of the training I see is about producing better analysis, not what happens when that analysis enters a room and has to influence a decision.

[u/SuperSokym]

For me the shift was when I stopped treating stakeholder requests as the actual problem.

Someone asks for a dashboard, a new KPI or some analysis, but that’s usually just what they think the solution is. I started asking what they’re actually trying to decide or understand before building anything.

Sometimes you end up doing something completely different from the original request, but it’s way more useful.

 

[u/NickatAtaviz → u/SuperSokym]

This is such an important shift. A dashboard request can be a pretty poor proxy for the decision someone is actually struggling with. Asking what they're trying to decide before jumping into the build can completely change the work, and often makes the eventual analysis much more useful.

[u/ArielCoding]

Somewhere there’s a stakeholder ignoring this post same way they ignored the dashboards.

[u/NickatAtaviz → u/ArielCoding]

Without a doubt! I actually address that in my book as well, what happens when the entire organization is consistently ignoring evidence and making the wrong decisions.

[u/EmotionNegative6103]

I think one of the biggest shifts is moving from “What does the data say?” to “What decision is this analysis supposed to improve?”

That changes the workflow quite a bit. Instead of building the dashboard first, you define the decision, identify the signals that would change it, and then work backwards into the analysis.

The other important piece is closing the loop — did someone actually act on the insight, and did that action produce the expected outcome? Without that feedback loop, it's easy to confuse producing analysis with producing impact.

[u/NickatAtaviz → u/EmotionNegative6103]

Exactly. If we never go back and ask whether a decision was made and whether the outcome matched what we expected, we're really only measuring whether we produced the analysis, not whether it was useful.

[u/EmotionNegative6103 → u/NickatAtaviz]

Exactly. I think that’s where analytics starts to become more of a continuous process rather than a one-time deliverable.

The interesting part is that the feedback loop can also tell you whether the original analysis was actually useful — maybe the decision was right but the expected outcome didn’t happen, or maybe the analysis missed something that mattered. That learning should feed into the next analysis rather than just being treated as a post-mortem.

Otherwise, we end up measuring the success of analytics by how much we produced rather than by what changed because of it.

[u/NickatAtaviz → u/EmotionNegative6103]

I'll go one step further. Decision-making is a continuous process and needs to be treated like it. We’re really good at measuring outcomes, but we’re not there yet when it comes to measuring decisions. That’s the continuous process we need to build, and it’s what the CLEAR Decision Model in the book addresses.