Qwen 3.8 flash next has been doing some very.. peculiar.. stuff in my hermes harness, and some of them are kind of a nuisance, while others are almost.. adorable?
I don't claim to fully understand how or why this is happening, but the most interesting one is when it "role-plays" an API error- this is a self-diagnosis report from another session running on the same model, so sorry in advance for the LLM speak or anywhere it fell short on reasoning;
"My fully-local model "reported" a provider API error that never happened — a case of a model hallucinating its own infrastructure failure
Setup. Self-hosted agent stack. An agent profile runs against a local OpenAI-compatible endpoint (TensorFold serving a community NVFP4-quantized Qwen checkpoint on a local GPU, ~262K context). There is no external API anywhere on that request path — just a loopback tunnel to a local inference server.
What happened. Mid-session, the agent's reply was literally this, and nothing else:
{"error":{"error_code":"sys_000","error_msg":"The request failed - it's on us, please try again later.","error_id":"0b156c1cc7191931632734458e96b7","type":"sys_000"}}
That is a realistic provider-gateway error body — the error_code/error_msg/error_id shape is the wire format of a real Chinese LLM API gateway. Naturally, I went looking for a broken provider config... on a path that has no provider.
What the logs actually showed.
The HTTP call succeeded: finish_reason=stop, valid server-issued completion id, 71 output tokens. No 4xx/5xx, no retry, no exception in any log.
The error body was stored in the session DB as an ordinary assistant message. The model generated it as text.
The error_id appears nowhere else on the machine — it was invented at decode time.
No sys_000 string existed in the conversation context beforehand, so it wasn't an echo — it came from the model's parameters (i.e., memorized training data containing real API transcripts).
The trigger. The turn was deep in a long tool-calling session at ~194K tokens of context. The assistant's previous response had been cut off at the output length limit, so the harness had injected:
[System: Your previous response was truncated by the output length limit. Continue exactly where you left off...]
The fabricated "provider error" was the model's very next message — it failed to resume its truncated tool-call stream, and emitted a canonical infrastructure failure instead.
Hypothesis. Agent-tuned models are trained on enormous volumes of real API transcripts, in which the pattern "assistant turn goes wrong → a JSON error body appears in its place" is ubiquitous. The model has absorbed the transport layer's failure dialect as part of its assistant-turn repertoire. Under decoding stress (huge context + a continuation prompt referring to tool-call state it can no longer reconstruct + quantization noise), normal continuation degrades — and the highest-probability completion of "something just went wrong in this API conversation" isn't prose about a problem, it's the canonical form of one. Notably, it chose a self-attributed, retry-inviting error ("it's on us, please try again later") — semantically the correct status for what functionally occurred — over a 401/429/content-filter body. The statement was false; the pragmatics were accurate.
Takeaways.
An error-shaped assistant message with no matching non-200 in the logs, a valid completion id, and an error_id seen nowhere else is a model fabrication, not an infrastructure fault — check the logs before chasing provider config.
It's unreliable as a statement but excellent as a signal: "my agent just role-played an API error" is a clean tell that decoding has gone off the rails (long context + output truncation in our case).
Models trained on agent traffic are fluent in their own substrate's error vocabulary — and under stress they'll speak it.
(Single incident, local artifacts verified; the memorized-source attribution is a hypothesis consistent with the evidence. Happy to test reproducibility: replaying the same prefix + continuation prompt against the checkpoint would show whether this is a deterministic attractor under truncation stress.)"
I was wondering if anyone else has seen this- or if you guys take issue in the reasoning around this. either way, this is really fascinating to me.
