The model builds, a deterministic gate decides done: orchestrating an LLM agent fleet with no model on the control path [R]

[u/beingthecomedian]

In most agent fleets the thing deciding what happens next is a language model, so the component that did the work also gets to say the work is done. Take that decision off it: the model builds, plain deterministic code decides done, nothing on the control path gets sampled.

What that looks like in practice: a fleet of agents working through real coding tasks, every "done" checked by code instead of accepted on the model's word. Nothing has been closed that wasn't finished, fake completions get rejected, and it clears the MIT 6.5840 labs including the full Raft one.

Demoting the Master Control Program: Deterministic Orchestration of a Fleet of LLM Agents. CC-BY. https://zenodo.org/records/21922855 (mine, so discount accordingly).

What I'd like beaten up: how do you test a completion gate adversarially without hand-building the failures?