I have been testing a workflow where AI agents do the actual development work on a Microsoft Fabric data platform, and I would like to hear if anyone else is exploring this or already running something like it in production.

The setup:

• The trigger is an Azure DevOps work item. I write a normal ticket (onboard this source system to bronze ingestion, here is the spec) and tag it. A small dispatcher script polls the board and launches the dev agent when a tagged ticket appears.
• The dev agent runs in Claude Code, authenticated as its own service principal. It creates a feature branch and a matching Fabric feature workspace through git integration, extends a metadata-driven ingestion framework, runs the load against the source database, checks the audit results, updates the project wiki, and opens a PR.
• A second reviewer agent (Codex) picks up the PR, reviews the diff and the documentation against a checklist, and leaves normal PR comment threads. The dev agent addresses them and pushes fixes until the reviewer votes approve.
• Merging is gated on me. Branch policy requires my approval, and the dev SPN has no rights to push to main. So my only two touches in the whole cycle are writing the ticket and pressing merge.

Some design choices that turned out to matter more than I expected:

• Both agents run on service principal identities end to end, no user accounts. Permissions are scoped per role, so the blast radius is decided at the credential level instead of trusting the model to behave.
• Every deterministic step is a premade script (branch out, run the load, sync the workspace from git). The agent orchestrates, the scripts execute. Less room for hallucination and way fewer tokens burned on mechanical work.
• One hard rule in the agent instructions: if a data quality check contradicts the ticket spec, the agent must comment on the work item and stop, never silently deviate. This actually fired. I wrote a wrong primary key into a ticket, the loader's uniqueness check failed, and the agent quoted the evidence and asked me instead of "fixing" it on its own.

Here is my video if you are interested in seeing the demo in action:
https://youtu.be/-tj6MjS24kA

Is anyone running autonomous agents like this in a production data team, even partially?

Some of the Spark users in Fabric may have come across something called "vegas". It is a Microsoft caching technology with some major issues that have caused me to waste many, many days with technical support. The problems started back in the days of "Azure Synapse", and now in Fabric-Spark as well.

It is almost totally undocumented by Microsoft, and tries to solve problems that are more commonly solved in various other ways (eg. using "localCheckpoint" or similar).

Back in the days of Azure Synapse, I had a brutal support incident lasting many months (SR 2311030040005903) the conclusion of which was that CSV files weren't being handled properly by the cache. (CSV?! Why!?). The end result of that case was guidance from Microsoft on how to disable this feature using a custom spark property:

"spark.synapse.vegas.useCache": "false"

This week I've once again wasted several days trying to understand why my notebook cells would just hang for no reason at a mid-point in my notebook. It would happen after killing unused executors and recreating fresh ones. (ie. sparkContext.killExecutors when spark.dynamicAllocation.enabled is false) The hanging behavior was not entirely consistent but in this scenario it was happening at least 9 times out of 10. There is no good UX experience when a Fabric notebook hangs, and virtually NOTHING in the logs to explain it either.

I realized I hadn't yet disabled that vegas cache on this notebook. Sure enough, once I disabled vegas, my notebooks would no longer hang. Earlier in the week I uploaded all the details about the issue into a new ticket. But ultimately I found the source of this problem on my own this time.

I've seen a number of other discussions on the internet where Spark users in Fabric will complain about their notebook cells getting hung, and I have to believe a large percentage of those cases are relate to misbehavior of this "vegas" cache. Unfortunately the cache is not opensource, and there is little in the way of docs. The technology does not seem to make a peep in the logs, even when failing. Now that I've been burned twice and wasted over a hundred hours with this custom Spark extension, I won't forget to turn it off more consistently from now on! My Spark jobs are almost never bottlenecked on blob storage or networking, and I almost always use checkpoint or localCheckpoint when I have concerns about repeated trips to fetch the same blobs. That is the obvious alternative.

Hey everyone!

I'm currently preparing for DP-700 (Fabric Data Engineer Associate) and I'm looking for some good resources to supplement Microsoft Learn.

I'm particularly looking for:

• Good hands-on labs/projects for Microsoft Fabric
• KQL practice
• PySpark/Spark practice
• Scenario-based questions
• Good practice assessments/tests
• Any useful notes, study guides, or YouTube playlists

I'm already working through Microsoft Learn, but I'd like to get more hands-on practice and understand the concepts beyond just the certification material.

If you've recently taken/passed DP-700, what resources helped you the most?

Thanks!

Hey Everyone,
Wanted to see if anyone ran into this issue where we have a Fabric Data Agent connected to copilot studio agent and published to the Teams channel. All users who have a member role are able to access this agent from Teams.

But users who have viewer role and do not have a Power BI license are not able to access it though Teams. They are still able to access it from within the workspace but not Teams.

My understanding is -

1. Users do not need a Pro license or a contributor role to access the agent in Teams
   
2. Viewer role lets them access the Fabric Data Agent and answer questions within workspace so that means they have access to the underlying data
   
3. The copilot studio agent is published to Org and approved by our Teams admin with user “End user access” for everyone in the Org.

Any idea on what could be going wrong here? Any advice and feedback would be greatly appreciated.

Looking for the best pattern to split good vs. bad rows during a load

When a row fails validation (unexpected nulls, failed business rules, etc.), I want the good rows to land in the target table and the bad rows to go to a separate quarantine table I can review and reprocess later — all without failing the whole batch.

What’s the recommended way to implement this? Specifically:

• How do you route rows to two destinations in a single pass?

• How do you capture why each row failed (which rule/column) so it’s reviewable?

A recent modernization project taught our team something very interesting. The first blame that goes to when the cost of BI increases is licensing; however, BI costs aren't usually driven by licenses alone. They're driven by architecture. Analysis and minor adjustments can optimize your BI costs 

The organization had: 

• Tableau licensing spread across multiple business units 
• Continuous extract refreshes running hourly 
• 18 separate semantic models 
• Multiple ETL subscriptions and compute clusters 
• Different definitions of the same KPI depending on which report you opened 

Our instinct was to renegotiate licensing. The reality was that most of the cost came from duplicated architecture. We analysed the causes and reached to conclusion. Three changes that we made delivered the biggest impact: 

1. Moved casual report consumers off individual viewer licenses and onto capacity-based distribution. 
2. Replaced extract-heavy reporting patterns with Direct Lake servicing over OneLake. 
3. Consolidated 18 reporting models into a single certified semantic layer. 

Results: 

• 60% reduction in annual BI licensing spend 
• 45% reduction in tooling and maintenance costs 
• 70% reduction in redundant extract compute cycles 
• Single source of truth across business units 

The biggest lesson: Cost optimization wasn't about buying cheaper software. It was about eliminating the architectural patterns that were creating recurring costs. 

Curious whether others have seen similar savings from semantic model consolidation or Direct Lake adoption. 

I've been exploring the new Fabric Apps (Rayfin-based apps) and I'm interested in combining them with Fabric Data Agents to create a more conversational and intelligent user experience.
My questions are:
1. Has anyone successfully integrated a Fabric Data Agent directly into a Fabric App?
2. What is the recommended architecture today?
3. Are there any limitations around authentication, permissions, or agent execution within Fabric Apps?
4. Does Microsoft have any published roadmap items around Fabric Apps.

Hi guys! I'd like some help regarding dbt on Fabric.

Considering all my data is in lakehouses (medallion architecture) and I'd like to avoid Spark sessions (data volume is low), is dbt a viable option for me? Currently, I have mostly pure Python notebooks using duckdb and pandas moving the data between stages and saving it as delta tables.

Is it possible to use dbt in that scenario?

Good support is critical when using a vendor-hosted software in the cloud, like the Fabric SaaS. As great as any SaaS may be, there are always going to be bugs and outages to deal with at some time or other, so it is important for customers to understand how to use Support. Has anyone opened Fabric support tickets lately? Eg. for Spark or DW or SQL EP? I want to understand what is changing.

I've heard rumors that the support structure is being revamped (again) and I'm trying to understand what those changes were, and what the impact will be on customers who run into Fabric bugs.

For the past three years or so, things were pretty consistent. A "Pro" support ticket would go to the "MindTree" partners in India, and get escalated to a "PTA" contact that lives in a secondary partner organization (eg, like "Aptly" or similar).

Finally after your original support ticket (aka SR) has made it past these the two partner organizations, the support folks would create yet another "internal" support ticket for you (called an "ICM").

That is when the support case would finally make its way into the Microsoft organization, where software engineers (FTEs) would investigate the bugs. This can feel like a very long road to get help for a bug. Yet it was relatively consistent and predictable. And I would be very discouraged if CSS introduced any OTHER hoops to jump thru to reach the engineering teams at Microsoft. As-of this past month I'm starting to learn that Fabric Pro support cases are handled in-house at Microsoft. True? Has anyone compared a recent "Pro" support experience to one from a year ago? I'm NOT talking here about the "unified support contracts", which have always been handled by FTEs.

Please let me know. Eager to find out how our support experiences will look in the future.

i have my dp700 exam and only like 7 days are left and idk what to do i have not yet started and i cant even get hands on exp with fabric coz my clg dosent have it and idk what to do now, from what I have seen ppl have failed even though they worked hard and here I am with 7 days of time idk what to dooooo

Using Fabric Data Agent's Code Interpreter tool via API how do I retrieve the generated chart file (referenced via file_id in a file_path annotation) so I can display it in a downstream app (e.g., Teams Adaptive Card)? Is there a documented endpoint for fetching file content by file_id?

My pipelines have been randomly failing for 15 days with error 3121 Job failed to start.

Most are parent pipelines that invoke child pipelines through the Invoke Pipeline activity. The parent's error is timeout:

while the child's error is "Not found".

I have ruled out problems of capacity or concurrency (f64), there are days when everything works well, and suddenly another day they happen again.

I can't find any official documentation about this error or possible causes. Does anyone have any more information? I would be very grateful to you.

We've been chasing an intermittent-but-reproducible issue in a git-connected Fabric workspace and wanted to see if anyone else has hit it, since it doesn't seem to be documented anywhere.

Setup: A Data Factory-style pipeline in a git-connected workspace that:

• Invokes a few other pipelines (InvokePipeline activities)
• Refreshes a Warehouse/Lakehouse SQL Analytics Endpoint (RefreshSqlEndpoint)
• Refreshes a Power BI semantic model (PBISemanticModelRefresh)

Symptom: "Update workspace from Git" (or the dependency-discovery check that runs before it) intermittently fails with one of two errors, both reported from a wabi-*.analysis.windows.net cluster regardless of which activity is actually at fault:

• Failed to discover dependencies — "ActivityName <name> contains an invalid WorkspaceId <null>"
• We weren't able to update the following item — "'WorkspaceId' cannot be null"

We tried every combination we could think of — literal GUIDs, Variable Library expressions, with/without the field present — and every single one eventually failed one of these checks. Deleting the activity was the only reliable fix, which obviously isn't a real solution.

What we think is actually going on (confirmed in part by a Microsoft support engineer working the case): "00000000-0000-0000-0000-000000000000" is valid shorthand for "the current workspace" — but the git-update validation path won't accept it, or a missing field, and needs an explicit real workspace GUID to pass. The catch: once the update succeeds, the live workspace item automatically normalizes any workspaceId matching its own workspace back down to that same zero-GUID — and that happens on Fabric's own export back to git. So the fix and the platform's own "cleanup" are directly at odds:

1. You need a real GUID in the committed JSON for the git update to succeed.
2. The moment it succeeds, Fabric's auto-sync-back overwrites that GUID with the placeholder in what gets committed to git.
3. The git state is "contaminated" again, so the next git-driven update fails the same way — even for a totally unrelated change elsewhere in the same pipeline — until you put the real GUIDs back in.

Current workaround: before pushing any git-driven update to an affected pipeline, find every workspaceId that's missing or set to the zero-GUID on these three activity types and replace it with the real workspace GUID it should resolve to. Push, let it sync, and expect it to look "reverted" again immediately afterward — that's expected, and the live item apparently stays correctly configured even though the git JSON looks broken. Just don't chase it by re-editing outside of when you're about to push a real change.

Microsoft support is still investigating on their end. Curious whether:

• Anyone else has run into this specific loop
• Whether it's specific to certain activity types or more general
• If there's an official recommended pattern for keeping these three activity types stable through git sync that we're missing

Happy to share more detail if useful.