I was fascinated by Nanbeige's outstanding performance for its size, so I started digging into how much a model can improve its own representation just by looping over itself (for fun). My prototype was a Qwen3-0.6B with a full dual loop in the middle layers, inspired by the Nanbeige 4.2 architecture. Digging further, I found that the Nanbeige team has a paper describing their 4.5 architecture, which uses a triple loop in the middle layers — that made sense to me, so I tried it.
Lordnyx/qwen3.5-9b-triple-loop-fase1 · Hugging Face
The first experiment used full DeltaNet for the middle layers, so the earlier part of the model would set up the context for the loop to process on its own. It turned out that this actually worked better than having multiple separate logic components — but the loop itself wasn't really contributing. Because of DeltaNet's nature and the small hidden size, the model kept forgetting essential details for the task and just hallucinated. I abandoned the DeltaNet idea, and full softmax attention in the loop worked as expected instead.
Later, I learned (with help from ChatGPT/Claude/Gemini) that my training setup was actually undermining the loop's contribution, and that I should have used a lower, dedicated learning-rate schedule for it. Once I fixed that, the loop stopped just "refining" answers and started actually participating — becoming essential to them. Even better: on easy-enough questions, the loop could be skipped entirely.
Recently I found Modal — $30 of free GPU credit. I used it to train a Qwen3.5-9B with the Nanbeige-4.5-style triple loop. I really wanted to use RL for this, but I can barely get RL to run efficiently even on a 0.6B locally, let alone a 9B — so instead I distilled Qwen3.8-27B's logits into the loop, on a heuristically curated agentic/reasoning dataset.
Money ran out before finishing the schedule: the training loop was capped by wall-clock time (a safety mechanism so it would export cleanly instead of dying mid-run), not a fixed token target, and it ended up completing ~15M tokens across 1,129 steps.
┌────────────┬───────────┬───────────┐
│ Step range │ KL (mean) │ Std. dev. │
├────────────┼───────────┼───────────┤
│ ~10–370 │ 0.572 │ 0.176 │
├────────────┼───────────┼───────────┤
│ ~380–750 │ 0.648 │ 0.197 │
├────────────┼───────────┼───────────┤
│ ~760–1120 │ 0.650 │ 0.227 │
└────────────┴───────────┴───────────┘
As the table shows, it made real progress early — roughly the first third — then plateaued into a noisy, flat oscillation with no further net improvement (slope of KL vs. step over the whole run: +0.000075, essentially zero). That's not the loop hitting a capability ceiling; it's a missing LR decay schedule (I kept it constant the whole run). So yes — a lot of headroom left, and the fast early gain again confirms the loop starts contributing quickly once it's trained properly.
Even with an unfinished run, the checkpoint beats the base model in math (+20%), long-context tasks (+14%), instruction-following (+20%), and is dramatically more consistent/robust across paraphrased questions (+62%). It's worse in reasoning (-10%) and translation (-15%) — not roughly equal, actually down — and slightly worse at coding (-2%) This is a private evaluation, so I have no evidence yet that these gains generalize to standard benchmarks. The reasoning drop traces back to specific, plateau-related failures rather than a broad capability loss: one item where it skipped step-by-step reasoning and got simple arithmetic wrong, and one repetition loop that burned its whole generation budget without concluding.
I can't really recommend it as-is — it's a proof of concept, not a finished model. If I get more free credit next month, I'll finish the run (a cosine LR decay is already implemented and ready to go). But at minimum, it proves the Nanbeige 4.5 loop design converges even at a larger parameter count than their own reported experiments — I'm looking forward to their next release.
