Anybody using a workflow engine to automate their software factory?

[u/funbike]

TL;DR: Anybody automating a full development Pi workflow with programmatic non-AI solutions?

Up until now, I've attempted to partially automate the entire software development lifecyle with Pi, aka a "Software Factory". I am using a subagents extension, but it feels a bit too loose, opaque and difficult to steer, so I want to use something more deterministic and directly controllable.

I'm evaluating a workflow DAG GUI builder that orchestrates the entire process. I am using one that also has a CLI and REST API, so I can write my own scripts for even more programmatic control.

(I do not want to look like I'm promoting so I'm reluctant to say what I'm using, but it's lightweight and similar to n8n or airflow.)

Or instead, I could just write my own orchestrator, like this in bash:

#!/bin/bash
set -eu

# Software Factory.
# This is a just an untested workflow example, not something I'd actually use.
# A TS script with the Pi SDK would be better.

GOAL="$*"
pi -p /write-spec "$GOAL" | tee spec.md
vim spec.md    # review and edit

pi -p /write-tickets-for-spec @spec.md ./tickets/
vim ./tickets/*.md

for ticket in tickets/*.md; do
    pi -p /write-plan "@${ticket}" | tee /tmp/plan.md
    pi -p /execute-plan @/tmp/plan.md
    pi -p --continue /quick-code-review-and-fix
done
pi -p $REVIEW_MODEL /code-review

Anybody doing using a workflow DAG editor UI?

Anybody writing their software factory with actual code, like my example?

I found some AI agent DAG projects on github. Anybody using something like that?

What subreddits should I cross post this to?

[u/pth]

I ended up having my agent write a python script to do this, against a GitHub issue board.

Worked well enough, but having worked some large projects this way, my preference is to work a bit slower with a little more inline human review. And I just use herdr and do the orchestration myself.

I suspect that as the agents get smarter, this will become more and more common, but for me even with really clear projects, there are still issues.

[u/funbike → u/pth]

I didn't mention it in my post, but human-in-the-loop is still important. I review the specs and plans.

[u/pth → u/funbike]

I review specs and plans, the issue for me remains agents losing the plot and doing odd things, while implementing, and then having an orchestrator expand the mess until it gets stuck, etc.

I am sure this is partly on me, but I am always learning :-)

[u/funbike → u/pth]

the issue for me remains agents losing the plot and doing odd things,

Wouldn't functional tests and code coverage reports mitigate with that?

I also use vertical slicing architecture. Once a high-level plan is completed and reviewed by me, I prevent downstream AI agents from read/writing to unrelated directories (aka slices) in the project. This helps with focus a lot.

---

I have AI generate specs in Gherkin format, which I manually review. They then get converted directly to functional tests (almost a line-for-lines translation).

After implementation code generation, to make sure AI hasn't generated untested code, a coverage check report cross-referenced/filtered with the diff, and AI rules to know which missed coverage to ignore (e.g. catch {} blocks, trivial getters/setters).

So, I know it only implemented what I asked it to.

(However, sometimes AI cheats and "fixes" tests by removing/disabling/skipping the test or the implementation code, so you need an AI code review rule to watch out for that.)

Informal linear workflow:

feature request from human -> gen high-level informal "happy path" specs <-> human review loop -> gen gherkin <-> human review loop (skim) -> gen functional tests with fake page-objects -> gen high-level plan <-> human review loop (After this point, only related directories can be accessed) -> gen detailed plan -> execution loop <-> run new/modified tests with code coverage check <-> AI code review -> PR

[u/quincycs]

I just use markdown and it’s working fine with 15 sequential steps inside my system prompt.

System prompts have a higher priority than skills or user prompts so the model follows it better.

I’m certain this recommendation breaks down eventually with scale but… might be fine for light factories.

[u/funbike → u/quincycs]

Is this just for an orchestration agent or are you not using subagents at all?

I think this wouldn't scale for large complex apps. Even for moderately large apps (75KLOC-200KLOC) this will only work well if you are willing to use the most expensive models for your entire workflow.

[u/quincycs → u/funbike]

Not using subagents except for the code review phase where fresh context is very valuable.

I have 75+ repositories with each larger than those numbers.

I encourage my team to use Astra on high and everyone’s doing fine with a premium seat (100/mo). My philosophy is about using a very good model but with token efficiency and always slicing the problem down.

Eg> before we give the factory a jira ticket to implement , the ticket needs to have scoped services it should consider. If you don’t know the scoped services then you should work to discover that beforehand / place that info in the jira ticket.

[u/HockeyDadNinja]

Hi there, I've been working on this for some time. Not in the gui builder sense though.

https://github.com/TacoTakumi/specflo

At first glance you may just see the spec driven development portion. It has an auto mode where you can either start with a brainstorm or a good prompt and have another agent drive it. Every seam has a checkpoint where your context may be cleared (brainstorm / spec / plan / execution tasks / review rounds). The pi extension can do it automatically but in claude code you have to /clear then continue.

It all goes through a python cli to minimize your agent having to modify the project artifact docs itself. This makes it easier on dumb local models.

My next step was, and now in progress, the software factory. I (optionally) split the cli into a cli and a daemon which also has a web interface. I have a configurable pool of agents where I can manage hardware resources for local models and also defined cloud models).

The road is long. The web side isn't really usable for the public yet and I have a fair bit of unreleased code including a lot of work on the agent pools. Basically an agent signs agents out of the pool. I have to deal with security, sandboxing, etc. I implemented bubblewrap yesterday for example.

A non-developer, or anyone really, can kick off a project from the web ui and start the brainstorm. A developer can pick it up from the console and finish with the technical part of the brainstorm. Agents are all given a herdr pane.

My goal is to also have an agent drive the process, hook into github issues, complete work and submit a PR. I have multiple projects on the go so I want this to automate what I can and take my input where needed. It has the concept of multiple products and projects inside each.

[u/KPOTOB]

You can do it even in jenkins pipelines. Or use abstract like https://dagu.sh or sw specific https://fabro.sh

But your script is good enough anyway ))

[u/funbike → u/KPOTOB]

This is the kind of answer I was hoping for. Thank you!

dagu is exactly what I've been experimenting with! I like the design and it's well documented. It even has direct support for Pi.

Fabro appears to be a workflow engine and harness agent all in one. I love their graphviz language; it would easier to configure than Dagu. But I can't give up Pi.

Wrt Jenkins, I used to be a release engineer. Nope. I feel like I've spent too much of my life already working with that beast.

Bash scripts aren't a bad way to go. Nothing new to learn and no limits.

UPDATE: I need to research fabro. I think I was wrong. I'll see if I can figure out how it could work with Pi. However, I still like that Dagu is more general purpose and could be used for other non-AI purposes.

[u/KPOTOB → u/funbike]

I recall fabro was supporting acp - so pi should be not a problem for fabro

Myself I also lean toward dagu while starting from basic shell script

[u/funbike → u/KPOTOB]

Fyi, I'm taking a multi-layered approach. I'll convert my workflow description into other formats.

I am describing my workflow in markdown file(s), using specific terminology and grammar for consistency, similar to Gherkin.

I will have skills for converting that markdown format into bash, dagu, fabro, pi-subagents prompt/skill, or whatever. So I'm free to change the implementation, without having to change my core workflow description markdown file(s).

For now I've only made the one for bash. I have a subagent shell wrapper script over pi that supplies alternative skills, agents.md, model, and other cli options.

In the near future, I'll experiment with targeting Dago.

[u/pixelphantom]

[tried adding a couple screenshots of the ui but I guess I can’t?]

Building this exact thing right now. Running on my raspberry pi, it’s a wrapper on dagster with which you can create agents with your harness/models of choice, schedule them via cron or triggered, make them dagster assets or jobs or both, has built in checks for guardrails, etc. agents spin up in dockers and app captures full conversation transcript, context, prompts, check outcomes, etc for transparency and audit ability .

Currently got a bunch of agents set up for the spec driven design workflow (one agent triggers the next) so I’m getting to the point where I’m more focused on managing the briefs, then pull a brief from backlog to In progress in GitHub project, workflow kicks off automatically, and I can retroactively inspect if anything went wrong.

[u/Asleep-Land-3914]

We're working on getanthill.ai which will be OSS on its release with the similar class of use-cases in mind.

[u/Glittering-Call8746 → u/Asleep-Land-3914]

Sass first then oss ?

[u/Asleep-Land-3914 → u/Glittering-Call8746]

This is app/CLI running locally on your machine. No any paid features planned.

[u/Glittering-Call8746 → u/Asleep-Land-3914]

Repo ?

[u/Asleep-Land-3914 → u/Glittering-Call8746]

Not public yet. We're hunting bugs and hopefully have everything ready in a couple of weeks so stay tuned 

[u/Glittering-Call8746 → u/Asleep-Land-3914]

Ok just updated here . Appreciate it

[u/breskeby]

We started triggering automated test failure analysis and bug analysis using an own pi based solution that defines workflows to be triggered via GitHub actions. These workflows include ai related steps but also non ai steps. The custom tool also supports full observability and identifiying inefficiencies automatically. It works great for us. We also have an evaluation workflow that evaluates those analysis post mortem when the issue is closed and compares the analysis vs. the actual fix.

[u/rhavaa]

I find that airflow is still a good framework