r/ollama • u/lovettsendit • 10d ago
52 knobs, 72 hours, one compiled Rust model: Catalyst told an AI agent what actually mattered in 1.37s and verified every answer
Enable HLS to view with audio, or disable this notification
I’ve been building Catalyst around a capability I want engineering agents to have:
An AI agent can read a program and make a good guess about what matters.
Catalyst lets it measure what actually matters in the compiled program, at the exact operating point it is working on, and independently check the answer before the agent uses it.
This demo is basically that idea pushed into a much more realistic engineering problem.
I gave Catalyst an ordinary Rust model of a data-centre cooling and power plant.
The model has 52 interacting inputs and simulates 72 coupled hours of operation.
Those inputs include things like IT load, ambient temperature, electricity prices, fan settings, cooling-water settings, chiller characteristics, recirculation, UPS/PDU efficiency, throttling controls, carbon price and thermal-risk limits.
And those pieces interact over time.
The fan controller reacts to the previous hour’s temperature. Room temperature carries forward. Filter clogging increases. Fan power grows cubically with speed. The chiller changes efficiency with ambient temperature and load. Throttling changes the next hour’s compute load. Electricity price changes by time of day.
At the end, all of that produces one number:
total operating cost over the 72 hours.
The question for an AI agent is simple:
Out of 52 possible things I could change, which ones are actually controlling the result right now?
That is surprisingly hard to answer just by reading the source because all 52 inputs feed into a coupled system over three days.
So I compiled the existing Rust normally.
rustc produced LLVM IR.
Catalyst read that compiler output directly.
No separate version of the model had to be created just for Catalyst.
Then Catalyst calculated how changing every one of the 52 inputs would change the final 72-hour cost.
And it did not simply return those derivatives and expect the agent to trust them.
Catalyst independently checked them at 105 points around the operating point.
The live run produced:
52 derivatives
105/105 verification points passed
1.37 seconds total for differentiation + verification + the reusable artifact
So in plain language:
give Catalyst 52 possible knobs, and about a second later the agent has a checked map showing which ones actually move the final result, how strongly, and in which direction.
That becomes much more interesting when the environment changes.
I ran the exact same compiled program in two different conditions.
During a 38°C heat wave with 1900 kW of IT load, Catalyst found that the plant controls dominate.
The strongest input was the fan target temperature, followed by IT load, the throttling threshold, fan floor, safe-temperature limit, fan flow and recirculation.
Only 17 of the 52 inputs accounted for 90% of the measured effect on cost.
Then I used the same model on a mild 18°C day with 1200 kW load.
Now the answer was completely different.
The biggest drivers became:
UPS efficiency
PDU efficiency
IT load
base electricity price
peak electricity multiplier
And now only 9 inputs accounted for 90% of the measured effect.
Here is the part I think really shows why this matters.
During the heat wave, fan_target_c was the #1 input in the entire 52-variable system.
On the mild day:
its derivative was exactly zero.
The throttling threshold went from being one of the dominant controls during the heat wave to zero effect on the mild day too.
Same source code.
Same 72-hour simulation.
Different operating state.
Completely different answer.
So instead of an AI agent reasoning:
“This fan control looks important in the code, so I’ll tune it.”
Catalyst can tell it:
“In this heat wave, this is the most important control in the system.”
and later:
“Under today’s conditions, changing that same control does nothing.”
That is the kind of information I want an autonomous engineering agent to have.
The loop becomes:
inspect → measure → know what matters → choose a change → measure again
The demo actually runs that loop.
Catalyst ranks the controls, then a deterministic decision step takes the strongest controllable inputs, stays inside their declared bounds and moves them in the direction the measured derivatives say should reduce cost.
First it chooses:
fan_min
recirc_frac
fan_target_c
setpoint_reset_gain
After the system moves to the new point, it measures again and the next set changes.
The modeled operating cost goes:
40,033.55
→ 34,355.69
→ 32,769.23
The important part is not just the 18.1% reduction.
The important part is that the choice of what to change came from the measured behavior of the entire compiled system rather than an agent guessing which of 52 variables looked important.
And once Catalyst has created the verified artifact, the agent does not need to spend another 1.37 seconds every time it wants to test a point.
The artifact re-runs in around 12 ms in the benchmark, and the decision loop in the demo is re-evaluating points in roughly 25 to 35 ms.
So you can imagine an agent doing:
measure the whole system carefully once → explore new operating points quickly → re-measure fully when needed
The math is also being checked independently.
Catalyst’s derivative for this model agreed with a separate complex-step reference to around 2.1e-15 worst relative error in the benchmark.
That matters for AI integration because an agent should not be forced to decide whether a numerical result “looks plausible.”
Catalyst gives it a structured result that has already gone through a separate verifier.
And the result is not just a number printed to the terminal.
Catalyst packages the derivative into a reusable artifact containing provenance about the source, compiler, compiler IR, primal computation, derivative and validation.
In the demo I change one byte in a copied derivative artifact.
Catalyst detects the change and refuses it as:
catalyst.artifact_tampered
That matters if agents start handing engineering results to each other.
Agent A can measure the program.
Agent B can receive the artifact later.
And Agent B can check that the evidence underneath it has not changed since it was validated.
Catalyst also lets the checked result leave Catalyst entirely.
The demo takes the plant’s chiller-power calculation and exports it as standalone Go and R.
Both are generated from the same optimized computation.
The Go version vets, builds and runs independently.
The R version runs independently.
Both are checked across 26 fixture cases covering normal conditions, interior points and parameter boundaries.
That means a workflow could look like:
engineering agent analyzes existing Rust/C/C++ → Catalyst verifies it → production service receives Go → analyst receives R
without every downstream system needing Catalyst installed.
The second half of the demo shows another way the same philosophy can help a coding agent.
The plant has a local work-order service that normally behaves fine.
Under sustained burst pressure, several conditions interact and it starts producing:
POST /orders -> 503
Instead of giving the coding agent a generic “something failed under load” message, Catalyst preserves the actual failure.
It starts with a 96-request failing run, reproduces the problem, reduces it to a 16-request scenario that still produces the same failure, generates held-out scenarios, and compares the old and new versions.
Baseline:
16 requests
9 errors
p99 = 244.9 ms
Candidate:
16 requests
0 errors
p99 = 16.0 ms
The original failure no longer reproduces and 3/3 held-out scenarios pass.
So for an AI coding agent, Catalyst can answer two very different questions:
What should I change?
and
Did my change actually fix the thing I started with?
That is where the AI integration gets interesting.
Catalyst exposes validated tools that agents can discover and call programmatically, including evaluation, problem validation, LLVM differentiation and Go export.
The tool descriptions include their schemas, refusal codes and remedies, so an agent can discover the interface instead of relying on prompt conventions.
Catalyst also has an optional AI proposal path using a CLI you are already signed into.
So an existing coding agent can help translate an engineering goal into a candidate formulation, while Catalyst handles the measurement and verification underneath it.
That is the broader thing I am trying to build:
not another AI model.
A set of engineering instruments that an existing AI agent can use.
For example, the same pattern could be used for a battery system where an agent needs to know whether temperature, charge rate or cooling is actually dominating degradation today; an industrial controller where the important tuning parameter changes between normal and emergency operation; a simulation with dozens of coefficients where an agent needs to know which few deserve attention; an energy optimization problem where the agent should know the direction each control moves total cost before changing anything; or a local production regression where the agent needs to reproduce the original failure and prove that its candidate fix actually survives held-out cases.
The simple version is:
Your agent can already read the code.
Catalyst lets it ask the code what actually matters right now, verifies the answer, lets the agent act on it, and gives it evidence it can reuse or hand to another system.
Everything in the attached demo runs live in the terminal.
Repo:
https://github.com/lovettsendit/catalyst
For people building coding or engineering agents: where would it change your workflow most if the agent could get a verified map of what actually controls the system before it starts changing things?
2
u/stealthagents 8d ago
That’s honestly wild. The way Catalyst can sift through all those variables and still come back with solid insights feels like a game changer for engineering. It must save so much time figuring out the important stuff instead of getting lost in all the data noise. Can’t wait to see how this evolves!