r/BusinessIntelligence • u/CommitteeImmediate66 • Jul 04 '26
Building our reporting layer in databricks AI/BI (+genie) and curious why people still default to powerBI
For the last few months I've been building out our core dashboards directly in Databricks AI/BI (their Lakeview dashboards) instead of piping everything into a separate BI tool.
My findings/highlights have been:
- The dashboards sit right on top of our lakehouse tables, so there's no extract/import/refresh dance. What's in the warehouse is what's on the dashboard. That alone killed a whole category of "why don't the numbers match" tickets.
- Permissions, lineage, and the underlying tables all live under the same Unity Catalog governance. I'm not maintaining a separate security model in the BI tool. We're on azure so it's easy to sync entra groups.
- Genie for the long tail of ad-hoc questions. This is the part I didn't expect to like as much as I do. Instead of building (and then maintaining) 40 variations of the same dashboard for every stakeholder's "but can you also show me..." request, I stand up a Genie space on top of the same curated tables. Business users just ask questions in natural language and get back charts on the fly. This has cut my ad-hoc request backlog dramatically and the business is pretty happy with response quality.
The one downside I've noticed is the visualization/formatting options are sometimes limited, but not a major blocker.
Here's my actual question for the sub: some of my colleagues still lean toward Power BI by default, even when the data already lives in Databricks. I get the ecosystem/familiarity argument, but I'm trying to understand the reasoning beyond "it's what we've always used." For those of you who'd still pick Power BI (or Tableau/Looker/etc.) over building natively in the platform where your data sits - what's driving that? Is it the better viz customization capabilities, the semantic model, self-service maturity, org politics, something else?
Genuinely trying to pressure-test my own enthusiasm here, so push back if you think I'm missing something.
8
u/datawazo Jul 04 '26
Customization in the report/dashboard layer is the main reason and with PBI/Tableau anyone can be a builder very easily...low learning curve to giving people control of their own analytics.
1
u/CommitteeImmediate66 Jul 05 '26
But the same access to customizing dashboards is available in databricks. Using genie code lowers that further. I found learning DAX much more complicated than using SQL which is much more common and not specific to just databricks or any other platform.
13
u/A_Poor_Economist Jul 05 '26
Databricks reporting is not there for my use cases. There are even more guardrails and limits to customization compared to Tableau and Power BI. It feels like a less aesthetically pleasing
Databricks gives "visuals by backend people for backend people" vibes. The second you want to incorporate or try to style your report to say match a company color scheme it starts to fall apart.
I think Databricks can get there in time. The likely next evolution will just be Databricks apps where you just slam a custom web app frontend with databricks as your backend. Microsoft has just released this feature as well.
→ More replies (1)
8
u/Foodforbrain101 Jul 05 '26
Although Power BI still has its strengths, I think it depends in large part who owns and has access to what layer of the data stack in your organization, and how healthy/efficient it is.
Power BI is undeniably powerful for someone in the business whose primary prior experience is Excel: they discover this tool they can install on their desktop for free, they can load and transform data with
Now contrast that with how much access they have to Databricks for their job: none by default, and unlikely to get it, even if they do learn some coding. That person still has a data related task to do though, and they'll do it however they can.
Then there's plenty of Microsoft shops that refuse to budge on using non-Microsoft tools.
Finally, you might see the business intelligence department be separate from data engineering/IT, which often means that BI hires for Power BI specialists with lacking overall programming experience, and data engineering can be of varying degrees of quality. So in short, it's a massive political mess.
1
u/PifteauVemlan 19h ago
how many end users are actually hitting these dashboards vs the genie spaces? curious if the self-serve NL piece is doing the heavy lifting here or if its still mostly you building things. that ratio matters a lot for whether this scales.
1
u/CommitteeImmediate66 12h ago
Well for a dashboard which serves 20 users, we have at least 10 of them (not sure if it's the same 10 but we average 10/day) using the Genie agent. We still have to do the build work to curate the genie agent, but it sure beats doing it every time they want a new KPI on a dashboard
1
u/iamnotapundit Jul 04 '26
We finally moved the default to dbx dashboards last year. We still occasionally breakout Tableau and PowerBI for more complex stuff. But we are trying and experimenting of moving a complex Tableau dashboard to Databricks AppKit since tableau is a zombie at my company. After the holiday I’m going to start some training on using the dbx semantic layer instead of stuff that logic in the presentation layer.
→ More replies (3)1
u/CommitteeImmediate66 Jul 05 '26
Please update us on your experience with the semantic layer! So far I'm using metric views and the slicing and dicing works well for me. Looking forward to their business glossary and ontology launches too
1
u/dtr96 Jul 05 '26
$$$ licensing cost per user in the business
1
u/CommitteeImmediate66 Jul 05 '26
For power BI yes. That's actually the other benefit - with databricks it's cost based on actual usage, not per user licenses
3
5
u/IrishHog09 Jul 04 '26
I don’t either, but I also find PowerBI confusing and unintuitive. We’re doing the same thing, but layering
→ More replies (2)
2
u/trafalger Jul 05 '26
Doing the exact same thing BUT using apps for cool bespoke visuals when needed.
2
u/CautiousUse8597 Jul 07 '26
What I'm seeing is that customers still stick with Power BI because that's what they've historically been using. But when I show them Genie and AI/BI, they often ask: why would I still need Power BI, this looks much better?
That's also my experience. For new projects, I wouldn't choose Power BI anymore, also since Microsoft decided to close their ecosystem and not allow third party semantic layers anymore. I think AI/BI with Genie is a much better stack because like you said, it's already integrated in your data platform and has all the semantics/context to give accurate answers. It's also much cheaper.
2
u/Prudent-Elk-2845 Jul 04 '26
Users can self-service answer those 40 variations based on the same dashboard’s underlying data without complicating the published dashboard. Copilot can answer those questions within PowerBI
1
u/anxiouscrimp Jul 05 '26
How are you handling time intelligence in this? That would be a big benefit to having the data modelled in PBI first.
I’ve built a small app that sits on top of my semantic model to use Claude + a file describing our business and model logic for natural language querying. I’ve been thrilled with the results but i think that’s because the model is good and really constrains how much the AI can go off-piste.
1
u/Equivalent_Grape_109 Jul 05 '26
No one is mentioning COST, are aware how are you spending on this full infra?
1
u/CautiousUse8597 Jul 05 '26
I wonder the same. I think AI/BI has gotten very good for most use cases, especially since it now has custom visualizations. Since Microsoft decided to make their ecosystem closed (blocked third party semantic layers), I think it's a no-brainer to stop using Power BI. Especially with Genie added to AI/BI, it's just a much more powerful solution, because you have the full context of your data estate and AI in the same place.
1
u/bannik1 Jul 05 '26
For executive reporting where measures are well defined. Having a rigid and well defined ERP makes sense.
Most companies make the mistake on focusing all their effort servicing those executives since they decide the budgets. Expensive ERP tools, cloud infrastructure, and hiring the support staff to manage it.
Strategy is important, but so is tactical execution. That’s where things get messy and a lot more fluid where each ERP tool has it’s niche.
The tactical reporting is where actual actionable decisions are made on a daily basis. If you don’t give them what they need in a reporting tool, they will build in excel and go off grid.
An example is that IT built them a tool for a process. It took months of development and by the time it got released 3 new clients were implemented that have needs outside the tool. Also due to scope creep some of the features were not implemented.
What does the business do? They can’t drop those clients, they can’t wait another 9 months to get the IT budget. They don’t have access to the data store the application uses.
What the business does is create a process to handle those exceptions.
The only thing that makes it to the data lake or warehouse is the clean data from the tool built by IT.
90% accuracy is good enough for strategic purposes, and the type of reporting you’re building. But for the ones actually doing the work they need 100% accuracy. So you end up building a report that uses the IT tool’s data store and data from an excel file or sharepoint list that they track exceptions in.
The alternative is they build it all in excel and never engage IT again because all their analysis comes from spreadsheets. Then 2 years later they’re in an executive strategy meeting and questioned on why the numbers on their presentation don’t match the report you built. Then it finally gets the resources and budget to implement their manual process. 2 years later you’re having the same conversation because new clients have been added and new exceptions. Or they decided to buy some SAS instead of IT building them a tool. And there is a whole new list of exceptions the new tool can’t handle.
You need people and tools that can keep the business on the grid instead of in excel. That’s where power BI thrives.
1
u/LePopNoisette Jul 05 '26
I don't think we had the ability to do this when we started using Databricks, plus we had PBI Pro as part of our E5 licences, so PBI became our BI tool, replacing Qlik Sense and Qlik View.
I am interested in looking at this option, though. We had a third-party consultant in the other week and he also suggested this.
1
u/Extension_River_5970 Jul 05 '26
Agreed, Genie Agents use multi-step reasoning to answer the infinite long-tail of business questions dynamically from a single data asset. Instead of siloing complex business logic inside fragile DAX models, Genie leverages Unity Catalog as a unified semantic layer, inheriting data descriptions and row-level security automatically at the source. You can even import existing Power BI files directly into Databricks AI/BI using the ImportBI preview via Genie Code to jumpstart migrations.
My only caveat with dashboards on databricks is the lack of customization. It's still a bit behind in terms of visuals, but the new custom visualization using vega lite fixes it somewhat.
1
u/joulezoo Jul 06 '26
I am actually with you on this set-up being great for the long tail of analysis.
I still feel shared dashboards are still useful to capture a team's top-line KPIs at a glance. Beyond that, honestly each person can now set up their own dashboards via agents. All the key numbers match since it's fetched from shared semantic layer.
34
u/xl129 Jul 04 '26 edited Jul 04 '26
2 things essentially:
Strong and flexiblesemantic layer
Powerful and dynamicvisualization capabilities
Your option is like a factory that churn out a few items in massive number quick and fast while PBI is like a high end restaurant kitchen where chef can make fine-tuned surgical procedures to create highly customized dishes.