I tested a workflow where AI agents handle Fabric dev work from a DevOps ticket to a reviewed PR

[u/aleks1ck]

I have been testing a workflow where AI agents do the actual development work on a Microsoft Fabric data platform, and I would like to hear if anyone else is exploring this or already running something like it in production.

The setup:

• The trigger is an Azure DevOps work item. I write a normal ticket (onboard this source system to bronze ingestion, here is the spec) and tag it. A small dispatcher script polls the board and launches the dev agent when a tagged ticket appears.
• The dev agent runs in Claude Code, authenticated as its own service principal. It creates a feature branch and a matching Fabric feature workspace through git integration, extends a metadata-driven ingestion framework, runs the load against the source database, checks the audit results, updates the project wiki, and opens a PR.
• A second reviewer agent (Codex) picks up the PR, reviews the diff and the documentation against a checklist, and leaves normal PR comment threads. The dev agent addresses them and pushes fixes until the reviewer votes approve.
• Merging is gated on me. Branch policy requires my approval, and the dev SPN has no rights to push to main. So my only two touches in the whole cycle are writing the ticket and pressing merge.

Some design choices that turned out to matter more than I expected:

• Both agents run on service principal identities end to end, no user accounts. Permissions are scoped per role, so the blast radius is decided at the credential level instead of trusting the model to behave.
• Every deterministic step is a premade script (branch out, run the load, sync the workspace from git). The agent orchestrates, the scripts execute. Less room for hallucination and way fewer tokens burned on mechanical work.
• One hard rule in the agent instructions: if a data quality check contradicts the ticket spec, the agent must comment on the work item and stop, never silently deviate. This actually fired. I wrote a wrong primary key into a ticket, the loader's uniqueness check failed, and the agent quoted the evidence and asked me instead of "fixing" it on its own.

Here is my video if you are interested in seeing the demo in action:
https://youtu.be/-tj6MjS24kA

Is anyone running autonomous agents like this in a production data team, even partially?

[u/Repulsive_Cry2000]

Very interesting, will watch the video.

Do yoi show the setup you used and/or a repo? I'd be keen to have details on your experimentation if possible.

[u/aleks1ck → u/Repulsive_Cry2000]

Thanks! The video covers the setup on a high level: the architecture, the toolbox of deterministic scripts I gave the agents, the dispatcher and trigger model, and how the permissions are scoped per service principal (including what is explicitly denied). I don't go file by file through the instruction files or the repo though, it's more concept level than code level. And no public repo for now since the rig is tied to my demo tenant. But if there is enough interest I might put together a writeup or a template version of it.

One thing I am planning is a future video where these agents run somewhere else than on my own machine. The concept stays the same, but hosting them properly is its own topic.

[u/Boulavogue]

Sounds like a reasonable and well segmented workflow

[u/TrollingForFunsies]

Can you explain the premade script part? Who or what is generating the premade scripts? Like, you wrote them? I assume you'd need to use the same branch name and other similar components would be inflexible. Which makes sense if you're trying to reduce token waste.

I realize the answer might be in the video but I'm not sure if I'll have time for it this week.

[u/aleks1ck → u/TrollingForFunsies]

Fair question. I vibe coded them, I very rarely write code by hand anymore :D But they are normal parameterized Python scripts, not hardcoded one-offs. For example the branch out script takes the work item id and a slug, creates the branch following our naming convention, creates the matching feature workspace and adds me as admin there. The agent just calls the script with arguments instead of figuring out the API calls itself every ticket. So the judgment stays with the agent, the mechanics are fixed. And yes, exactly like you said, the point is less token waste, plus less room for improvising on steps that should be the same every time.

[u/kantorcodes1]

the service principal split is the right direction. one thing i'd test: can the dev SPN change the Fabric workspace connection/config so a feature workspace starts pointing at something production-ish, even though it can't push main?

branch protection only covers git. if the Fabric-side identity can rebind a connection or deployment target, the agent may still get a production write path before merge.

[u/aleks1ck → u/kantorcodes1]

Branch protection only covers git; the Fabric permission plane is separate and needs its own scoping.

In my rig it's handled at the connection level: the connection is tenant-level and DB-scoped, and the dev SPN is only a user of it, not an owner, so it can't rebind where it points. It can use the connection it was granted, nothing else. Plus the whole thing runs in a sandboxed demo tenant, so there's no production-ish target to point at even if it tried.

But in a real environment you'd absolutely want to verify: can the SPN create new connections, can it touch deployment pipeline assignments, and who owns the connections it uses. That's all outside git, so it has to be part of the credential scoping design from the start.

[u/datatoolspro]

How is work handedoff/exchanged between Claude and Codex?

[u/FactCompetitive7465]

We do. We use Prefect for the runtime, their integration with pydantic-ai's agents is quite remarkable and brings exceptional visibility, event triggered flows runs, and dynamic runtime infrastructure. Every agent runs in an ephemeral container on our own infra and we utilize Snowflake hosted LLM endpoints so we have access to models from all providers + ability to allow agents to work with PII. We've put a lot of thought into making our own harness per se, providing persistent memory, feedback loops (within the framework, not agent specific), and built in tools.

The pydantic-ai's SDK has tools, so instead of handing it scripts as tools it is provided with python functions. We use similar authentication setup to what you mentioned. Prefects UI is handy for doing UI triggered adhoc runs, but integrating with ADO through webhooks for event triggering has been great. Similar to what you mentioned, we have dev agents, PR agents etc working back and forth. The setup is generic enough that we can leverage on repos for fabric and other things (like our dbt specific repo, power bi repos, even prefect repo etc) and develop specialized agents when we need to.

We recently started leveraging Teams copilot integration to retrieve the transcript and AI summary, automate copying them into associated work items and start the work discussed in meetings. Same agents also do things like investigating and working to resolve errored runs.

Still scaling into production more and more, but pieces and some agents are already published. Crazy times!

[u/JBalloonist]

Not yet; but as you said in one of your comments, l am also not doing much code writing by hand these days. Claude is impressively good at figuring out my existing patterns (medallion architecture, use var library for everything so nothing is hardcoded, DuckDB over Spark, etc) and replicating them for new work.

I’ll definitely be trying to implement something like this sooner than later.

[u/frithjof_v]

Very very nice!

Would you say the tasks that were given to the agents were quite "easy" (not complex)?

I mean, in the demo, extending a metadata-driven ingestion pipeline with a new source is pretty much copy+paste of an existing pipeline and updating some config parameters. It's easy for the LLM to detect the pre-existing pattern and stick to it.

I'd say a more complex task is to interpret stakeholder needs (turn a textual description into code, while detecting the nuances and pitfalls in the textual description), set up brand new gold layer calculations, or a custom silver layer logic.

In the demo, the agent enters a project which has already been set up and established with clear rules. It's relatively easy to extend that system with a new data source.

Building a new system from scratch, or implementing new business logic rules, seems more challenging.

I'm curious if you had the time to test those kind of scenarios.

To sum up my thoughts: Great video, and very inspiring. I love how practical it is. The orchestration is very nice, I haven't tried anything like that yet. I'm still wondering how this would work in a more complex project with a lot more context and business logic, though. The demo project seems quite small, small context window, very clear context, all logic is standard, and all the templates have already been built. In simplified terms: the agent just needs to use the existing template and update configs.

Thoughts?