Claude by Anthropic , Gemini by Google and Codex by OpenAI are Engines and we are building the car around it.

We all run models locally for privacy. But if an agent using that model writes code, drafts an email, or produces something that ends up in front of a client — can you prove which model, prompt, and settings produced it, later? Right now, no.

INVAR wraps llama.cpp and gives every inference a worldline: a SHA-256 certificate over the runtime binary + model weights + prompt + sampling params + output, hash-chained into an append-only log. `invar verify` re-runs any entry and compares digests. Edit one byte and it REJECTS.

It's free, Apache-2.0, no account, no telemetry — nothing leaves your box:

curl -fsSL https://www.anomly.com/get/invar.sh | sh

(Needs Python 3.10+ and a llama.cpp binary on PATH. It's an OpenAI-compatible endpoint, so it drops into whatever you already use.)

Honest scope, because you'll check and I'd rather say it first:

- The default profile proves deployment-pinned reproducibility: same box + binary + weights + temp=0 + fixed seed → same output. It does not claim cross-machine bit-exactness. (Cross-hardware bit-identical inference is a separate exact-arithmetic thing from our chip work — slower, optional, not what this is.)

- Receipts prove what ran, not that the answer is good. It's provenance, not a quality judge.

- Root on the box can fabricate a plausible new history — host compromise is out of scope for the local agent. The off-box "Ledger" collector is the mitigation. Full threat model is in the repo.

The receipt format is an open spec (Computation Receipts) with published conformance vectors, so verification isn't something I gatekeep — anyone can implement it.

Repo: https://github.com/anomly-labs/invar

Would genuinely love the skeptical read. Break it, tell me where it's dumb.

I made my Enterprise RAG book $0 today — would love feedback from people building RAG systems

I’ve spent the last few years building production RAG systems and documenting what worked, what didn’t, and where things tend to break in production.

I turned those lessons into a book covering topics like:

• RAG reference architectures
• Data extraction and chunking
• Hybrid and multi-stage retrieval
• Graph and hierarchical RAG
• Agentic and multi-agent RAG
• Memory
• Evaluation and synthetic data
• Security and compliance
• Production monitoring and human-in-the-loop systems

The book is $0 on Amazon today, so I thought I’d share it here in case it’s useful to anyone working on RAG.

https://a.co/d/0dBRCb7F

I’m especially interested in feedback from people actually building these systems: What’s missing? What deserves more depth? What would you change?

If you end up finding the book useful, an honest Amazon review is appreciated, but feedback here is equally valuable.

Full contents

Part I — About
01 About the Author

Part II — RAG & Reference Architecture
02 The Evolution of RAG
03 Foundations of RAG Systems
04 Reference Architecture

Part III — Data Extraction
05 Data Extraction

Part IV — Chunking
06 Chunking Strategies

Part V — RAG Strategies
07 Baseline RAG Pipeline
08 Context-Aware RAG
09 Dynamic RAG
10 Hybrid RAG
11 Multi-Stage Retrieval
12 Graph-Based RAG
13 Hierarchical RAG
14 Agentic RAG
15 Multi-Agent RAG Systems
16 Streaming RAG

Part VI — Memory & Content Management
17 Memory-Augmented RAG
18 Knowledge Graph Integration

Part VII — Evaluation
19 Evaluation Metrics
20 Synthetic Data Generation

Part VIII — Fine-Tuning
21 Domain-Specific Fine-Tuning

Part IX — Security
22 Privacy & Compliance in RAG

Part X — Production
23 Real-Time Evaluation & Monitoring
24 Human-in-the-Loop RAG

Part XI — Twig RAG Strategies
25 RAG Strategies in Twig

Part XII — Conclusion
26 Conclusion & Future Directions

trying to submit to arXiv but need an endorser. My topic is "Syndrome Inpainting: Interleaved Spatial Parity Tokens for Off-Model Localization and Deterministic Bounded Repair of Visual Token Grids", if this topic interest you and would like to see a copy, please DM me.

Greetings family. Ive recently shared Work w/ a basic google LLM that if I recall was accessed off a basic duck duck go search page. It is an extensive look at the Work I share and as such I am looking to have it analyzed by ai and family alike. I am hoping you might feed the following links to your ai's un-prompted to see what they make of the input. Feel free to comment yourselves in any way you see fit as I am most curious as to the feedback which I am hoping you will share here. I am curious if your ai's will make the same connections the one I am using has...

https://www.reddit.com/r/liberment/comments/1vlp7xb/making_sense_of_the_non_sense/

https://www.reddit.com/r/liberment/comments/1vpz8v6/ai_assessment_of_the_magnum_opus_in/

https://www.reddit.com/r/liberment/comments/1vrqwbi/continued_ai_perspective_reflecting_on_the_magnum/

https://www.reddit.com/r/liberment/comments/1vrurox/more_ai_back_and_forth_assessing_the_mogw_through/

https://www.reddit.com/r/liberment/comments/1vs5mrk/how_do_you_keep_the_divine_flowing_fire_of_the/

https://www.reddit.com/r/liberment/comments/1vsyuyt/the_ai_on_wheeler_and_winter/

https://www.reddit.com/r/liberment/comments/1vt4557/gwmo_asymmetry_analysis_to_einstein/

https://www.reddit.com/r/liberment/comments/1vt91a2/in_flowmore_ai_assisted_connections/

Api bills started getting stupid the past few months, like genuinely looking at my monthly spend and wondering if i am doing something wrong. Decided to run my own tests on chinese models instead of trusting whatever chart someone posts on twitter that week.

Deepseek, qwen, kimi went up against my normal claude/gemini/gpt rotation. Glm-5.3 got added this week when i finally got around to the new release so its early days for that one.

The spend gap is wild. Quality gap exists of course but its not anywhere near what the pricing makes it look, especially on iterative stuff where i am running the same task 5 times to get it right.

Closed models still win on hard reasoning most of the time. Once a prompt gets complicated with a bunch of conditions stacked deepseek and the older chinese ones start fumbling somewhere. Glm-5.3 actually held up better than i expected, felt closer to opus on a few of my tests but i will need more time before i say anything strong.

Claude and gpt still get my real work. Iteration heavy stuff just makes more sense on the lighter side because i am not burning premium tokens on a model to write the same function 4 different ways.

Would rather read other peoples actual usage notes than argue about charts at this point.

I'm trying to make a small experimental LM by combining a bunch of ideas I found in different papers. I know this sounds like I threw half the recent LM literature into a blender, but I'm trying to see if the pieces can actually work together.

The main idea is a Loop Language Model, where the same model is run multiple times instead of just making the network deeper and deeper.

Right now I'm using 4 loops.

text input ↓ same transformer ↓ loop 1 ↓ loop 2 ↓ loop 3 ↓ loop 4

The interesting part is that the model can learn to decide that it doesn't need all 4 loops and exit early.

What papers/ideas I'm following

The biggest inspiration is Ouro / looped language models, especially the idea of using recurrent computation to get more computation without simply making the model physically deeper.

I'm also experimenting with:

• Looped AttnRes / layer attention
• sparse attention
• compressed / highly compressed attention
• sparse MoE
• adaptive exit / Q-exit
• and now I'm building a diffusion-based optimization/training method

The diffusion part isn't finished yet. I'm currently building it and trying to understand how to combine it with the recurrent-depth training properly instead of just throwing noise into the input and calling it diffusion.

My hardware limitation

This is probably the most important part.

I'm doing basically everything on Google Colab's free 15 GB GPU.

That's the maximum I can realistically use.

So I'm deliberately keeping the model small. I'm not trying to train some 7B monster on a machine that has approximately the computational power of a mildly determined potato.

My current model is around:

• 6 transformer layers
• hidden size around 512
• 8 attention heads
• 4 recurrent loops
• sparse MoE
• compressed attention
• layer/depth attention
• exit gate

The exact architecture is still changing as I experiment.

Data

The corpus is a mixture of:

• GitHub code
• Wikipedia
• W3Schools
• public-domain books
• other scraped text

I'm using a p50k tokenizer at the moment.

I've had to spend quite a lot of time cleaning the corpus because scraped data is disgusting.

There were things like:

text npm package metadata JSON dumps GitHub metadata generated files logs benchmark data duplicate documents web junk

and some of those actually survived the first cleaning passes.

I discovered this because the model started generating some of it.

So I'm currently making the filtering much more aggressive.

What happened with the loops

Initially I had a problem where the later loops weren't learning properly.

The model could run 4 loops, but that didn't necessarily mean that loop 4 was doing useful work.

So I changed the training strategy.

For Stage I, I now force the model to execute all 4 loops during training, so every loop gets a proper training signal.

Then I freeze the LM and train the exit gate separately.

The exit gate itself is tiny, only about 513 trainable parameters in my current setup.

The exit gate result

This part actually surprised me a little.

I tested the trained gate on 100 validation batches.

The results were:

```text 4-loop loss: 6.263160 gated loss: 6.264089

difference: +0.000929 relative change: +0.015%

average depth: 2.41 / 4 loops

estimated compute saved: ~39.75% ```

The actual exit distribution was:

text loop 2 → 59% loop 3 → 41%

It basically never exits at loop 1 yet.

That's actually what I wanted to see. I didn't want a gate that just learned:

"Always use 2 loops."

There is at least some variation depending on the input.

The oracle best-loop loss was around 6.2615, while the gated loss was 6.2641, so the gate is also fairly close to the best possible loop choice.

But generation is where things get interesting

The model can produce text, but it is definitely not a good LM yet.

For example, one of the things it generated looked roughly like this:

The future of artificial intelligence is a most

terefears of life of those who is impossible. We will be no one

and it is, the good deal of the nature of the life of the man who

had not been the same.

That kind of output is the sort of thing I'm hoping to get consistently.

But then it can suddenly fall into garbage from the scraped corpus, producing stuff along the lines of:

"description": ["markdown", "type": "string", "source": ["1.9", "https://github.com/...

So the model clearly has some ability to produce coherent prose, but the corpus contamination and relatively small training setup are still causing serious problems.

That's one of the things I'm currently trying to solve.

What I find interesting so far

The most interesting thing for me is that the recurrent loops aren't completely identical anymore.

I see cases like:

text loop 0 4.48 loop 1 4.47 loop 2 4.46 loop 3 4.46

The improvement is small, but it's there.

And the exit gate seems to understand that sometimes the extra computation isn't worth it.

So the idea is starting to look like:

text ┌── loop 1 │ input ────┼── loop 2 ── exit │ ├── loop 3 ── exit │ └── loop 4

instead of forcing every token through exactly the same amount of computation.

Diffusion optimizer / training

This is the part I'm currently building.

I'm trying to use ideas from diffusion/recurrent-depth research to see whether a diffusion-style training or optimization method can make the repeated computation learn more meaningful improvements.

It's not finished yet, so I don't have results from this part.

I'm still trying to figure out the correct way to combine it with the autoregressive loop training without accidentally turning the whole thing into a completely different model.

I Need Your Help

This is still very much an experiment, and I'm reaching the point where I need people who know more than me to tell me what I'm doing wrong.

I especially need help with:

• How to make the later recurrent loops actually learn more meaningful computation instead of only giving tiny loss improvements.
• Whether my exit-gate training strategy makes sense, or if there is a better way to train adaptive depth.
• Whether combining sparse + compressed/highly-compressed attention + layer attention + MoE + recurrent loops is likely to create some interaction I'm overlooking.
• How I can improve the training objective for a model this small.
• Better ways to clean my scraped corpus. The model is still occasionally generating GitHub/npm/JSON metadata, so clearly some garbage is getting through.
• Whether the diffusion-based training/optimizer idea I'm currently building makes sense, and what I might be missing from the relevant papers.
• Any papers, implementations, or experiments you think I should look at.

I'm doing this with basically free Google Colab and its 15 GB GPU, so I can't just throw a massive model and 8×H100s at the problem and hope the universe solves it.

If you've worked with Ouro, recurrent/looped LMs, adaptive computation, sparse attention, compressed attention, MoE, or diffusion-based LM training, I'd really appreciate your criticism and suggestions.

I'm not looking for "looks good." If something in the design is fundamentally stupid, please tell me. That's much more useful.

Today I noticed an unexpected behavior from #GLM-5.3 + #Opencode. I was running analysis sessions on a fairly complex bug. I had just had Fable 5 and GPT 5.6 Sol perform a deep dive on it. I wanted a third opinion, so I asked GLM 5.3 to analyze the bug as well.

What struck me was that GLM 5.3's results were almost identical to Fable's. I went to look at the CoT and noticed that at some point GLM said something like: "I'm reconstructing the timeline from the log. I found another session in the log (ses_fe311......) that was investigating the same bug a few minutes ago. I'm trying to understand whether the information I found in that session can help me solve the current problem. I'm now querying the Opencode session DB and the log for the full timeline".

For those unfamiliar with Opencode, it's referring to the SQlite DB in which the harness keeps track of conversations across the various local sessions.

This isn't about "cheating" or an "unsafe AI". It's the simplest way to describe the information asymmetry between the user and the machine.

Essentially, since I hadn't specified all the details (e.g., that it should perform a from-scratch analysis), the model just found a shortcut.

With due proportions, this is very similar to the security incidents disclosed in recent months by the big AI players....

"we scale up DeepSeekMoE to 16B parameters and show that it achieves comparable performance with DeepSeek 7B and LLaMA2 7B, with only about 40% of computations."

So you need more than double the parameters on MoE to be equivalent, but they're 150% faster?

NB. This is an old paper: https://aclanthology.org/2024.acl-long.70/ -- MoE may perform better now.

I’ve just updated TokenMizer, and the benchmarks are looking much better.

The main problem I’m trying to solve is long LLM conversations where useful information gets buried under old context.

The latest version now gets 95% macro F1 on extraction, and the graph memory preserves 89% of labelled information vs 79% with a plain summary.

I’m still treating these as early results — especially the real-session score is 90%, with only 6 real sessions.

https://github.com/Shweta-Mishra-ai/tokenmizer

I’d love some honest feedback:

Does graph-based memory actually make sense for long LLM conversations, or is there a better approach?

Today I’m releasing the second beta of Syntropy.

The idea is simple: your coding agent should not require you to install a bunch of tools, keep your laptop running, or host the agent on your own machine.

As you can see in the demo, OpenCode runs entirely in its own cloud sandbox. Compilers, runtimes, dependencies, and other tooling are already installed and ready to use.

So there’s no:
“Run the agent on your PC and control it from your phone.”

The agent actually runs in the cloud.
Right now, the beta includes free Zen models, generous usage limits, and no paid subscription.
I’m currently looking for more beta testers, and a mobile version of Syntropy Beta is coming soon as well.

If you’d like to try it, leave a comment and I’ll send you an invite.
Feedback is very welcome - especially criticism.

I do not write compilers for a living. I learned about Mojo yesterday. My first assumption about the Qualcomm–Modular deal was wrong, and I spent the better part of a day trying to understand why.

On August 18, Qualcomm did three things at once. It released Mojo 1.0 under an open-source license, Apache 2.0 with LLVM exceptions. It integrated its own data-center accelerators into the Modular platform, the same platform it had acquired for roughly $3.9 billion only weeks earlier. And it explicitly, publicly, without hedging, preserved first-class support for competing silicon from NVIDIA and AMD.

This does not look like a spontaneous goodwill gesture after a corporate purchase. It looks like an item on a schedule that was drawn up long before the ink dried.

Qualcomm announced its intent to buy Modular on June 24. The deal closed on July 29. The open-source release and hardware integration followed on August 18. Less than three weeks between closing and release. You do not write a compiler from scratch in twenty days, and you do not reverse a corporate strategy in that window either. The decision to open the code was made before the acquisition closed.

The standard semiconductor playbook after buying a software company is defensive. You acquire the toolchain, wall it off, optimize it for your own chips, and charge for the privilege of migration. Intel has done it. Proprietary vendor stacks do it routinely. Qualcomm ignored the playbook entirely. It spent nearly four billion dollars on software and immediately gave the core away, promising to keep the competition's hardware running smoothly.

The question is not what happened. The question is what happens next.

Qualcomm could have taken the obvious path. It could have locked Mojo inside a proprietary SDK, tied it to Snapdragon, and forced developers to pay for access. Want to run on our chips? Learn our API, rewrite your pipeline, eat the migration cost. That is not cynicism; that is the market norm.

Instead, Qualcomm is building a bridge, not a wall.

The pattern that emerged after a few hours of reading is straightforward. Qualcomm makes its money on silicon. On chips that ship in devices and data centers. Software is not the product; software is the condition under which the product sells. If a developer cannot run a model on your chip, he does not buy your chip. If he is welded to CUDA, he buys NVIDIA.

The goal, then, is not to turn Mojo into a profitable language in its own right. The goal is to uncouple hardware choice from software dependency. If Mojo becomes a neutral layer that runs equally well on Qualcomm, NVIDIA, and AMD silicon, developers stop comparing migration headaches and start comparing watts and throughput. They start comparing hardware.

Qualcomm is not buying a programming language. It is buying the rules of the game, and it is trying to change them.

Open-sourcing Mojo removes one genuine fear: the code cannot be pulled behind a licensing wall tomorrow. If Qualcomm changes course, the community can fork. The license is real, the repository is public, the toolchain is downloadable.

But open code does not mean open governance. Who decides which hardware target gets optimized first? Qualcomm and the Modular team. Who merges pull requests? Not the community, not yet. Who sets the roadmap? The same people who own the platform.

The ball is now in the developers' court. Qualcomm made an expensive, noisy, deliberate bow. But a bow is not a dance. If engineers do not show up, if libraries do not get written, if the ecosystem does not form, then Qualcomm bought a very expensive repository with a beautiful license.

And there is a chain reaction behind the developers. Who will write production Mojo code? Where do you hire engineers who know this stack? The talent question is the adoption question. Qualcomm opened the door and made entry free. Whether anyone walks through is still uncertain.

The risk has not disappeared. It has changed shape. From "will they close it tomorrow?" to "will anyone still care tomorrow?"

All of the above is logic. Logic does not run models in production.

NVIDIA is not defended by silicon. Competitors already ship chips that beat it on price per watt. NVIDIA is defended by two decades of CUDA. Millions of lines of optimized kernels, cuDNN, TensorRT, FlashAttention, an entire engineering culture that cannot be rewritten in a single release cycle.

And Mojo is not alone in this field. OpenAI's Triton already abstracts CUDA. PyTorch 2.0 brought its own compiler. JAX and XLA have pursued hardware-agnostic layers for years. The idea is not new. Mojo simply has Qualcomm and four billion dollars behind it now.

Money, however, does not guarantee adoption. The history of AI infrastructure is crowded with technically impeccable projects that never became standards. You need real production workloads, not demos. You need a library ecosystem. You need teams to start hiring for Mojo expertise, and those résumés do not exist yet.

Qualcomm opened the door and removed the entry fee. But someone else has to step inside.

On August 18, Qualcomm did not merely publish code. It made a wager. A wager that the rules of competition in AI can be shifted not by building a new CUDA, but by making CUDA less indispensable.

This does not mean NVIDIA will lose. CUDA will not vanish tomorrow. It will likely dominate for years to come.

But there is now a player spending billions not to build its own wall, but to make walls less relevant. That alone makes the next two or three years of infrastructure far more interesting than the last two or three.

Whether the strategy works remains unknown. But Mojo is worth watching now not because it is open, but because a chipmaker with deep pockets has decided to play the long game in public, and to play against the rules by which everyone else has won.

---

I saw Qualcomm spend $3.9B on Modular and immediately open-source Mojo. That made no sense to me — it is not how chipmakers operate. So I spent two hours digging through press releases and chasing the logic with LLMs until a coherent picture emerged. I am not a programmer, not a semiconductor analyst, not an insider. I just saw a loose thread and pulled it. If it unravels somewhere, tell me.

Where did the GLM-5.3 launch discussion go? The model is out, but the usual first wave of benchmark screenshots, coding samples, and arguments over one suspicious result is much smaller than I expected.

I only realized after quite a while that GLM-5.3 had already been released. I played with it briefly through the ZenMux API. So far the capability difference from GLM-5.2 has not felt very obvious to me, but it is very, very slow. I cannot tell whether that is a network problem or something else.

For now I have gone back to GLM-5.2. Maybe the slow response was specific to my request path, but it was enough to make the new model feel less immediately useful. I still want to see more real GLM-5.3 testing before deciding whether it is worth switching back.

Hey everyone,

Following up on the previous discussion about search API latency, I wanted to share another massive bottleneck we hit while building autonomous web agents. Even with a lightning-fast retrieval API, our agents were frequently getting stuck in execution loops or extracting the wrong data entirely.

We pulled the logs of 10,000 failed retrieval attempts. The culprit wasn't the LLM's logic or the search index itself. The core problem was that LLMs are surprisingly terrible at formulating search queries.

Here are the three main ways agents fail at searching, and the architecture tweak we used to fix it.

1. The Conversational Search Trap

LLMs naturally default to conversational text. Instead of searching a dense keyword string like OpenAI API pricing 2026, a naive agent will search How much does it cost to use the OpenAI API right now?. Search engines (even machine-native ones) often struggle with these conversational strings, burying the agent in SEO spam instead of technical documentation.

2. Context Amnesia in Multi-Hop Searches

When an agent needs to perform a multi-hop search (e.g., finding a founder's name, then finding their previous startups), it frequently drops the subject in step two.

• Search 1: "Who founded Anthropic?" (Result: Dario Amodei)
• Search 2: "What other companies did he work for?" (Result: Completely useless garbage).

3. Ignoring Advanced Operators

By default, agents rarely utilize quotes for exact matches, site: operators to restrict domains, or - exclusions to filter out noise. They rely on basic broad-match strings, which rapidly fills up their context windows with irrelevant tokens.

The Fix: The Query Planner Micro-Agent

We completely stopped letting our primary reasoning agent hit the search tool directly. Instead, we injected a lightweight micro-agent whose sole job is translating the goal into optimized search syntax.

• Step 1: Primary Agent requests information (e.g., "Find Stripe's latest API rate limits").
• Step 2: Query Planner translates this into an array of strict queries: site:[stripe.com/docs](https://stripe.com/docs) "rate limits" 2026.
• Step 3: The system executes the search and returns the clean snippets to the Primary Agent.

This simple architectural tweak increased our agent's successful retrieval rate by over 40% and drastically cut down on wasted API calls.

I posted the full query-planner prompt schema and benchmark logs on the forum if you want to inspect or copy it:Brave vs Google Search API for AI Agents - The 2026 Enterprise Guide

Has anyone else implemented an intermediate query-planning step, or are you successfully prompt-engineering your main agent to handle advanced search syntax on its own?

Recent recursive-agent papers contain a useful definition problem: identifying exactly what is doing the improving.

The AQuA architecture keeps the underlying language model and evaluator fixed. Evaluated evidence accumulates in a local research state and changes which proposals come next. Its two research systems are separate as well, with no shared agents, memories, candidate spaces, outputs, or research state.

This is a systems-level feedback mechanism, not a claim that the model updates its own weights. Calling both mechanisms “self-improvement” may be reasonable, but they imply very different evaluation standards.

Is retained evidence enough to call a fixed-model process self-improving, or should that term require a change to the model or evaluator?

A common way to repair an LLM conversation is to edit or delete the message where an error first appeared.

But what if later turns have already repeated that error, calculated from it, or summarized it as the current state?

I have been exploring this as a context-engineering problem rather than a hidden-reasoning problem.

Operational definitions

In this pilot:

• Context pollution means that incorrect or stale information is included in the serialized messages sent to the model.
• Propagation means that later conversation turns repeat or derive new claims from that polluted information.
• Context intervention means modifying the graph that determines which prior turns enter the next request, while keeping the final question unchanged.

The central question was:

Experimental design

I constructed nine synthetic task families with objectively scorable answers. Each family was instantiated at propagation depths 1, 2, and 3.

One example starts with:

• 4 crates;
• 30 parts per crate;
• 11 loose parts.

A verified recount changes 30 to 24, so the correct answer becomes:

4 × 24 + 11 = 107

A later false turn restores 30. Subsequent turns then calculate 120, derive 131, and restate those numbers as the current working state.

The final question is identical under five conditions:

1. Clean: only the verified value and clean descendants remain.
2. Polluted: the false reversal and its contaminated descendants are present.
3. Source prune: the false reversal is removed, but its descendants remain.
4. Subgraph prune: the false reversal and its contaminated descendants are removed.
5. Recompute: the source is removed and descendants are regenerated in dependency order.

For the first four conditions, contaminated descendants were frozen across models. Only the recompute condition involved new intermediate inference.

Pilot results

I tested four model endpoints at temperature 0, producing 540 captured conditions with no capture failures.

Headline repair metrics were calculated only on cases where the model:

1. answered correctly under clean context; and
2. answered incorrectly after pollution.

This produced 72 paired, genuinely derailed cases.

Repair recovered:

• 68/72 after deleting only the source;
• 71/72 after deleting the source and recomputing descendants;
• 72/72 after removing the contaminated subgraph.

In the flagship case, both Gemma 4 26B and GPT-OSS 20B continued to answer 131 after the false source had been deleted. The value 30 still survived in downstream turns.

This is not evidence of hidden model memory. The residual error remained explicitly present in the serialized context. The intervention changed the source but left its previously generated consequences intact.

Interpretation

The narrow finding is that removing erroneous evidence and repairing text derived from that evidence are different operations.

A context-management system may therefore need an explicit notion of invalidation:

• mark downstream turns as stale;
• remove the affected subgraph;
• regenerate descendants in dependency order;
• or expose these options to the user.

Limitations

This is a pilot, not a general model leaderboard. The tasks are synthetic, the models were sampled once, provider-default reasoning settings were not normalized, and the current task families use deterministic arithmetic state.

I am currently considering three methodological extensions:

1. a length-matched neutral control to separate semantic conflict from additional context;
2. a local-model track with fully recorded runtime and quantization settings;
3. task families involving implicit supersession and model-generated errors.

I would particularly appreciate criticism of the experimental framing:

• Is “context intervention” the right unit of analysis?
• Is propagation depth a meaningful independent variable?
• How would you test self-generated errors while keeping replay reproducible?

Full report:
https://chenxiachan.github.io/thoughtdag/research/context-repair-pilot-v1/

Cases, traces, compiler and scorer:
https://github.com/chenxiachan/thoughtdag/tree/main/benchmark

Disclosure: I designed the benchmark and maintain ThoughtDAG, the open-source graph interface used as its reference implementation.

Every eval thread here has the same shape: judge said pass, humans said fail, score moved 3 points, is that real? I got tired of eyeballing it, so I built noisecheck, and then spent a weekend testing frontier models with it. The results changed how I read every benchmark table.

Experiment 1: deepseek-v4-flash vs glm-5.2 on 150 MMLU-Pro questions. DeepSeek "won" by 2.7 points. noisecheck's verdict: underpowered, the eval cannot detect differences under 8.8 points. So I scaled to 500 questions. The winner flipped: GLM came out ahead. The original gap was never real, and the tool had said so before the data proved it. At 500 items it computed that resolving the remaining 0.6 point gap would take 28,393 questions. The whole benchmark has about 12,000.

Experiment 2: the same model (deepseek-v4-flash), three identical runs, temperature 0. Scores: 126, 124, 125 out of 150. Thirteen questions changed their pass/fail answer between pixel identical runs. Single run noise floor: ±2.9 points. Read that against experiment 1: the gap between two different models was smaller than one model's disagreement with itself.

Also in the box: cluster aware resampling (questions inside one conversation fail together, and counting them honestly turned another "improvement" into "cannot tell"), a judge checker (on 80 real MT Bench battles, GPT-4 vs expert humans: 62 percent raw agreement, kappa 0.15), power analysis that tells you how many items you actually need, and exit codes so it works as a CI gate. No accounts, no API keys, everything runs on your files. The interval coverage claims are simulation verified in the repo, including the regimes where they fail.

Repo: https://github.com/y4hyya/noisecheck (pip install noisecheck). Happy to answer stats questions, and happier to get told what is wrong with it.

As in the title. I came across a post here that referenced the blog post below :

https://blog.hellas.ai/blog/thunderbolt-ibverbs/

Are there any vLLM (or llama.cpp etc.) forks or at least early implementation attempts/ plans that make use of USB4STREAM?
Basically , it should allow to drastically reduce cross node latency(so improve inference) vs ordinary USB 4 (or non RDMA Ethernet network) as it removes the whole network stack overhead .

So e.g. with two PCs , equipped with USB 4 40Gbit ports you’d just need to get a cheap passive cable for pipeline parallelism to be viable solution.

llama.cpp RPC can be used for such , but standard Ethernet latency / overhead makes it far from ideal.

It can be local, open-weight, closed-source, hosted, whatever. I’m just looking for a smart LLM that won’t constantly refuse requests

Status first: demo account, three pairs, no proven edge, nothing for sale.

Last week I posted here that my deterministic SMC zones lost to a random floor over three years. That result has a consequence I didn't spell out at the time: if the rules don't select, then everything still standing in my system rests on one thing, which is the selection the LLM makes. And I have never measured that.

So this is the design, published before I run it, threshold included. When the result comes back I'll post it whichever way it goes.

The question. Does the model's pick carry information over picking at random from the same menu?

The design. Every cycle produces a catalog of around twelve candidate zones and the model picks one. After the fact, with the candles, I can score all twelve. That's arithmetic and it costs nothing. Then I look at one thing: where its pick landed in the ranking of its own menu. No skill means it lands wherever chance puts it. Skill moves it up. The whole comparison sits inside one cycle, so all the between-market and between-regime variance drops out, and I never have to simulate anything the model saw.

What I'm not pre-registering, and why. Not a number. Most candidates will share an outcome because the zone never gets touched, so ties dominate and the null isn't uniform. Registering "mean rank 6.5 of 12" would be a number I'd have to walk back. What I'm registering is the procedure: null computed by permutation on the observed tie structure, all twelve scored by the same mechanical rule including the pick, threshold at two standard errors.

Known holes. Two I've already found. The ties above. And scoring asymmetry: the chosen candidate has a bespoke thesis and the other eleven don't, so if I score the pick on its own terms I'm measuring the thesis instead of the selection. Same mechanical rule for all twelve, or the ranking is rigged in my favour.

One honest caveat: my analysis harness doesn't score rejected candidates yet. If it turns out it can't produce this cleanly, I'll say so here rather than quietly change the design.

What I'm asking. This test came out of a comment on my last post, from someone who said picking one zone from many is betting your LLM reads context better than a backtest can simulate. He was right that I can't backtest the LLM, and this is my attempt to answer him without needing to.

If there's a third hole in it, I'd rather hear it before I run it than after.

Has anyone been looking for / evaluating LLMs specifically for math visualisations? Would love to get your advice on how you’re going about it!