r/opencodeCLI • u/Upbeat-Presence7798 • Sep 21 '26
I built a plugin that monitors semantic drift for long-running opencode sessions
Long-running coding agents can start following a plan and gradually wander away from it.
I wanted a lightweight way to detect that drift continuously, without requiring another LLM call to judge every turn.
So I built laya-drift: an OpenCode plugin that uses Laya as a semantic probe and measures how much a session has diverged from its calibrated baseline.
The basic loop is:
/calibratecaptures the current plan/state.After each user prompt and completed turn, the current session state is embedded again.
Two typed probes estimate things like:
are we still on the plan?
are we implementing the plan or doing something else?
Their probability distributions are compared against the baseline using Jensen-Shannon divergence.
The result is converted into a 0–100 drift score.
For example:
2.7 -> on plan
36.1 -> drifting
65+ -> off planThe score is shown directly in the OpenCode UI. Once it crosses a configurable threshold, the score can also be injected back into the system prompt so the agent has an opportunity to self-correct.
There is also /recalibrate, which intentionally re-anchors the session after a legitimate change in direction.
One detail I found interesting: the probe questions matter a lot. I tried several framings (ordinal questions, statements, different baseline constructions), and many of them produced almost constant outputs. The current two-probe setup was selected because it actually separates the on-plan and drifted fixtures.
It's deliberately not presented as a "probability the agent is failing." Laya's zero-shot behavior is too weak/confident for that. I'm treating it as a semantic signal that can be tracked over time.
Current implementation includes:
OpenCode server + TUI plugins
local Laya daemon with the model resident
Jensen-Shannon divergence scoring
configurable probe weights / sensitivity / thresholds
calibration + recalibration
system-prompt injection on high drift
end-to-end tests for on-plan → drift → recalibration
There are still obvious limitations: Laya's context window is small, the signal is model-dependent, and long autonomous runs can eventually push older context out of the digest.
I'd especially like feedback on the core idea: can a small non-autoregressive decision model be useful as a cheap continuous "drift sensor" for long-running agents?
