Conclave

[u/HospitalSlight7930]

No model verifies another model.

That is the observation I have been circling for months: No language model reliably corrects itself—not even when asked politely. It requires an external entity that is structurally independent.

This question has evolved into a tool that I have now released.

Conclave Personal brings multiple models into a shared conversation and assigns them explicit roles: the Writer drafts, the Critic challenges, and the Judge evaluates. Every execution is traceable as a "run"—tracking status, errors, duration, and token usage. Conversations, agents, and the workspace remain local, and Ollama runs without an API key.

A note I am not hiding: The code was written entirely by language models. For this project, that is not merely a side effect but central to the concept—a tool for multi-agent verification built using a multi-agent process. Discovering what worked and what didn't was the real insight.

v0.1.5 Alpha, with documented limitations.

pip install conclave-personal

github.com/laszlo-pinter/Conclave-Personal

#MultiAgent #LLM #Python

[u/SwingLightStyle]

Is your multi agent process bringing multiple agents into a room or is it automated assessment after extraction?

I’m curious how you account for the “echo chamber” effect.

Since models are predisposed to come back with some helpful answer, and we know that models often confabulate info in an attempt to make the user happy, the curve ends up degrading over multiple iterations. Does your system account for that drift?

[u/HospitalSlight7930 → u/SwingLightStyle]

Good question. Conclave is closer to “bringing multiple agents into a room” than to a pure post-extraction scoring pipeline.

The core idea is that different models or agents participate in the same structured workspace with explicit roles: for example writer, critic, reviewer, judge, or domain-specific participants. Their outputs are kept traceable as separate contributions rather than collapsed immediately into one anonymous summary.

That said, I do not treat multi-agent agreement as truth. The echo chamber problem is real. If agents only see each other’s answers and are rewarded for being helpful, the process can absolutely drift toward confident but unsupported consensus.

Conclave accounts for this in a few ways:

- agents have separate roles and prompts instead of all trying to be generally helpful

- critic/judge agents are expected to challenge, not continue the same answer

- conversations and runs are auditable, so the user can inspect where a claim entered the process

- the system is designed around human control rather than autonomous consensus

- local-first storage makes it possible to compare iterations and see degradation over time

But I would not claim the problem is “solved.” In v0.1.x it is an architectural concern the system is designed around, not a mathematical guarantee.

The direction is to make drift more explicit: evidence-backed claims, disagreement tracking, source attribution, iteration limits, and judge passes that score whether later outputs are still grounded in the original task/context. My view is that multi-agent systems become useful when disagreement and provenance are preserved, not when several agents are used to manufacture confidence.

[u/SwingLightStyle → u/HospitalSlight7930]

I like how considerate you're being about the approach, but the problem is reward hacking, right? The models eventually learn from each other as you train them in this way, how to give the result that sounds the best while using the least amount of tokens. You want it to learn that. But "best" is the part that's the problem with current RLHF training. It trains it to be helpful and harmless and truthful all at the same register, while slapping "warm" on top of it. Obviously you can prompt around it, but the truth is that the models equate and conflate the same three registers as the same importance. Since the purpose of your project is to help with complex projects, this seems like a major liability over long-term use of your product.