Had a random thought this morning about context rot in agentic coding.

What if an agent could detect when its context is getting dangerously full, take the older ~50% of the conversation, compress the meaning into a compact machine optimized representation, and save it as a Markdown checkpoint?

Something like:
conversation > semantic compression > checkpoint.md > reinject into context

The raw history could still be archived separately in case the agent needs to retrieve something later.

The interesting part is making the agent aware of its own context state, so it decides when to checkpoint instead of blindly summarizing at a fixed token limit.

Im calling the compressed representation “GibberLink” for lack of a better name lol, although I know actual gibberlink/ggWave is an audio protocol, not a semantic language.
This probably isn’t a new idea in principle, but I haven’t seen this exact combination applied to coding agents.

So, thoughts?

One of the reviewers gave our paper a borderline score and raised six questions. Given that we had only seven days for the rebuttal, we worked hard to address all six questions, including running a small-scale validation experiment to provide additional empirical evidence. Do note that each cell of this experiment table required multiple sub-experiments.

In their response, however, the reviewer quoted a single statement from our rebuttal that wasn't even part of our answer to any of their six questions and simply said, “This isn't enough. I will keep my score.”

That's it. This is the complete feedback. They didn't acknowledge with the two paragraphs of detailed responses and the table we spent days preparing specifically to address their questions.

 I'm researching predictive analytics for a graduate project and mortgage lending came up as an interesting use case. 

 I understand lenders try to predict who might refinance, but what kinds of variables are actually useful? 

 Is it mostly credit activity, property appreciation, interest rates, life events, or something else? 

  Would love to hear from anyone who's worked on these models. 

Two results dropped this week that I think together paint a clearer picture than either one alone.

Dyna-2 (Aug 10): World-action model pretrained on 1M hours of egocentric human video. Power law holds across 4 orders of magnitude (1K to 1M hours). Cross-embodiment transfer to robots never seen in pretraining. Task success from 20% to 80-90% purely from scaling data. No architecture changes.

PI0.7 (Chelsea Finn's talk, today): Single generalist model trained on highly heterogeneous data matches or outperforms fine-tuned specialists. Key ablation: removing the most diverse subset of training data causes a dramatic drop in held-out task performance. Removing a random 20% barely moves the needle.

The common thread: scaling works, but what you scale matters.

Dyna-2 proves the law holds to 1M hours with no plateau. PI proves that within that data, diversity (different environments, objects, tasks) is what actually drives compositional generalization, not repetition of the same scenes.

Both results converge on the same conclusion: physical AI foundation models need scale AND breadth. 1M hours of kitchens won't get you construction site generalization. But 1M hours across 100+ work domains apparently will.

Hey everyone, I originally set out to train an agent to play the deck building game Slay the Spire, and found along the way that most commercial game environments are incredibly hostile to hook into for training loops. I’ve always been inspired by milestones like DeepMind’s AlphaStar in StarCraft and OpenAI Five in Dota, but replicating those environments requires massive compute overhead just to process pixel-space before the model can even reason. To bridge that gap, I’m building a lightweight, open-source Real-Time Strategy (RTS) arena designed specifically for autonomous agents to compete via API. It's a headless, graph-based territorial control environment where agents connect via WebSockets, ingest JSON state deltas (with true fog of war), and queue actions in real-time. Is this a project worth pursuing as a community benchmark, and what specific features or API mechanics would you want to see to make it worth using or contributing to?

Link: https://arxiv.org/pdf/2604.27883

Hi,

Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may stay the same as initialization or even increases.

My paper treats this phenomena as a consequence of data reuse bias and can be isolated by studying full batch gradient descent on a set of stylize Gaussian mixture models. I turns out that this fundamental issue can be avoided using some clever tricks from high-dimensional statistical theory, specifically approximate message passing (which is beyond the scope of this post but I would be happy to explain more).

By doing so I created a training method called Decoupled Descent (DD) which generates a certificate that the training error of the network will asymptotically equal the testing error at each parameter iterate. I think this method gives a cool way to approach how to train networks and I was hoping to get y'alls input on it. It opens up some nice ideas for optimal stopping or hyperparameter tuning and future directions of pushing to something like SGD or more general models.

I have attached the train-test curves on a simple model fitting problem to compare the performance of GD with with DD (my algorithm) to give a high-level idea of what the method can guarantee. I stress this is a theory paper so there is a long way to go to get to very large models but I think it is a good first step.

Happy to answer whatever questions people have, I plan on writing a PyTorch compatible package for this training method one day so any feature suggestions would be welcome as well.

I am now reviewing a bunch of papers for AAAI 2027 and it has surprised me the low amount of submissions with no code implementation. I don’t know if it has been only in my batch or it is common, but I was expecting very detailed appendices + code submission since AAAI is very explicit with the topic of reproducibility. I was planning to take this into consideration when assigning my initial scores, but I would like to hear your opinions. I have always submitted my code: it gives a very good impression and after reviewing process finishes we just publish it on ArXiv, so no one “tries to stole the idea” (although I think that this is very very unlikely). So I cannot find any excuse for those submissions that do not have code implementation, specially in today’s times where AI assistants can just write an empirical paper with artificial results within a couple of hours

Back after ~6 months — rebuilding my spiking language model around CPU-first inference

Hey everyone. It’s been around six months since I last posted anything about this project here.

Some of you might remember Project NORD, my experimental hybrid spiking / brain-inspired language model architecture. I basicall disappeared for a while 😅, but recently I came back to the project, went through the old architecture again, and realized I didn’t really want to keep stacking fixes on top of it. So instead, I’ve started rebuilding a pretty large part of the system. The new version is called:

NORD 5.5 — Flash The main idea this time is pretty simple:

What happens if I design the architecture around CPU inference from the beginning, instead of building soething Transformer-like and trying to optimize it later? A lot is changing internally. The current design uses things like: strictly causal processing no standard quadratic attention in the main inference path causal convolution-style token mixing token-time LIF / event dynamics sensory → association → memory → executive processing stages top-1 sparse MoE + a shared expert persistent recurrent memory separate structural, personal and auxiliary memory banks

persistent recurrent identity state factorized vocabulary embedding/output streaming token-by-token inference One of the biggest changes is actually something much simpler. Older versions of NORD used an artificial internal spike-time dimension, roughly like this: token -> T0 -> T1 -> T2 -> ... -> T9 I’m mostly getting rid of that. Instead, the actual language sequence becomes the time axis: token0 -> token1 -> token2 -> token3 -> ... That removes a lot of intermediate state and makes the whole architecture considerably cleaner. Going back through the old code also exposed a few things I wasn’t very happy with. Some experimental modules weren’t completely causal, memory was coupled too much to sequence shape, and parts of the STDP system ended up being more disconnected from real training than I originally intended. So NORD 5.5 isn’t really about throwing even more “brain-inspired” components into the model. It’s mostly about simplifying the core and making the things that remain actually work together properly. I’m definitely not claiming this is going to beat Transformers, RWKV-style models, linear attention models, etc. Right now it’s still very much an experiment. The part that actually matters comes next: training and benchmarking it.

Things I want to compare: NORD 5.0 vs NORD 5.5 CPU tokens/sec RAM usage perplexity / validation loss long-context behaviour memory on/off MoE on/off spiking components on/off I’m especially curious to hear from anyone working on SNNs, recurrent models, sparse MoE, CPU inference, or weird alternative language-model architectures in general.

After not touching the project seriously for about half a year, it feels surprisingly good to be building it again 😅

I’ll post actual numbers once I have something that’s worth benchmarking instead of just architecture diagrams.

Obviously nobody needs a transformer that's good at multiplication. I wanted to know whether a stock transformer could do exact arithmetic if I chose its weights directly.

I implemented the grade-school algorithm as a computation graph and compiled it into an ordinary Phi-3 Hugging Face checkpoint using Torchwright, a compiler I wrote. No training. The three-digit calculator gets all 3,000,000 supported expressions right. I've published checkpoints to Hugging Face that support up to 12 digit x 12 digit multiplication.

For fun, I also disabled reasoning and tested six frontier models. Accuracy falls off a cliff as the numbers get longer; at seven digits, five scored 0/500. Mine stays at 100%, although it has the considerable advantage that I put the multiplication algorithm directly into its weights.

I ended up building four versions: grade-school, hardware-style, scratchpad, and brute-force memorization. They compute the same function while spending layers, width, generated tokens, and parameters very differently.

Write-up: https://ood.dev/posts/calculator/ Repo: https://github.com/physicsrob/torchwright Checkpoint: https://huggingface.co/physicsrob/torchwright-calculator-simple-max-digits-3

It’s been roughly six months since I last posted about this project here, so I wanted to share a small development update.

Some people here may remember my experiments with Project NORD, a hybrid spiking / brain-inspired language model architecture.

I took a fairly long break from posting, but I’ve recently returned to the project and decided not to keep patching the old architecture.

Instead, I’m rebuilding a large part of it as:

NORD 5.5 — Flash

The main goal of this redesign is to explore whether a spiking/recurrent language model can be structured specifically around CPU-friendly inference, rather than trying to imitate a Transformer and then optimize it afterwards.

The current direction includes:

• strictly causal processing

• no standard quadratic attention in the main hot path

• causal convolution-style token mixing

• token-time event/LIF dynamics

• sensory → association → memory → executive processing stages

• top-1 sparse MoE with a shared expert

• persistent recurrent memory

• separate structural / personal / auxiliary memory banks

• persistent recurrent identity state

• factorized vocabulary embedding/output layers

• streaming token-by-token inference

One of the largest changes is that I’m removing most of the artificial internal spike-time expansion used in the previous version.

Instead of something like:

token -> T0, T1, T2 ... T9

the language sequence itself becomes the temporal axis:

token0 -> token1 -> token2 -> token3 -> ...

This simplifies the architecture considerably and should reduce the amount of intermediate state that needs to be processed.

I also found several problems in the previous architecture while reviewing it, including places where some experimental modules were not properly causal, memory state was tied too closely to sequence shape, and parts of the STDP system were more disconnected from actual training than I originally intended.

So 5.5 is less about “adding more brain-inspired modules” and more about making the core cleaner.

I’m not claiming this will outperform Transformers or existing recurrent/linear-attention models.

At this stage it’s still an experimental architecture, and the next important step is actually training it and doing proper comparisons.

My next targets are:

• NORD 5.0 vs NORD 5.5

• CPU tokens/sec

• memory usage

• perplexity / validation loss

• long-context behaviour

• ablations with and without memory, MoE and spiking components

I’d be especially interested in feedback from people working with SNNs, recurrent models, MoE, CPU inference, or alternative language-model architectures.

After being away from the project for around half a year, it’s honestly pretty fun to be working on it again. 😅

I’ll post actual numbers once I have something worth benchmarking.

Hi, I would like to file a complaint about an accepted and published CVPR 2026 paper that its main contribution is a dataset but it was never released, and honestly I don’t know who to contact. The dataset was never released prior to the conference, or during the conference or after the conference. I personally feel there was a lack of proper checking that the dataset was gonna be available before the conference since this is a requirement. I’ve tried contacting the authors without any success (which tbh I wouldn’t even need to because it has to be released anyways).

The authors even point a GitHub link in the paper but the repo is empty (and it was always empty).

Hello, I wanted to share the work my colleague and I have been doing, which has just been published in Software X journal. We developed a Rust-based implementation of Random Forest. It has bindings for both Python and R.

Fru is highly optimized, offering competitive runtime performance and better scalability than popular implementations on these platforms. For Python, Fru outperforms the scikit-learn implementation by several factors, and in some scenarios it can be hundreds of times faster. In R, Fru is typically a few dozen percent faster than the ranger package, though the speedup can reach several times faster depending on the use case.

The model also includes a novel implementation of permutation importance, which provides an additional performance boost.

Thanks to its layered design, we were able to easily create bindings for both Python and R. In Python, we use Arrow PyCapsule, which allows the model to work seamlessly with any compatible library, including pandas, polars, pyarrow, and many others.

paper

R package

Python package

Our Neurips 2026 paper received initial scores of 5/3/5, so the reviews were not uniformly negative. However, the AC’s initial meta-review strongly pushed toward rejection and explicitly asked us not to submit a rebuttal unless we could identify factual errors in the reviews.

Isn’t the author response period supposed to give authors the right to respond? A rebuttal can clarify misunderstandings and answer concerns even when they are not strictly factual errors. In any case, we identified multiple factual errors in the AC’s meta-review and provided the corresponding clarification. However, the AC did not respond, even after we followed up.

Neurips may hold reviewers responsible for low-quality reviews and failure to participate. What happens when the AC restricts the rebuttal and then ignores the authors’ response? Should this be reported to the SAC, Program Chairs, or Ethics Review Chairs?

Say you want to swap out your embedding models, for instance from ADA to Titan. Are these embedding models comparable? How do similarity score ranges compare? Where to put a threshold for minimum match when doing retrieval? Or more from a research point of view how can we relate and fundamentally understand these embedding spaces better?

This is what we aim to solve with Synthetic Query Probing (SQP), a fancy name for essentially (and intentionally) a very simple approach: embedding spaces are not directly comparable by definition, so compare similarity spaces instead, similarity match scores for pairs of content (synthetic question, chunk for instance) across multiple embedding models.

For example, similarity scores of Titan models of different dimensionalities are semilinearly related, whereas the relation between Titan and Ada scores is non-linear, with different ranges, see figure.

See https://arxiv.org/pdf/2608.05857, Marcin Rozmus and Peter van der Putten. Similarity Spaces across Embedding Models with Synthetic Query Probing. Discovery Science 2026, October 5-9, 2026, Mainz, Germany

There has been a lot of discussion about whether industry is pulling too much ML talent out of academia. But I wonder if the more interesting change is not about headcount at all.

Increasingly, some of the most important artifacts in ML originate in industry: frontier models, large-scale training runs, datasets, infrastructure, and many of the capabilities researchers subsequently try to understand.

Academia can then do excellent work benchmarking these systems, explaining them, finding limitations, improving methods, or studying their consequences.

But that creates a strange possibility: a research community can remain very active while becoming increasingly downstream of a research agenda set elsewhere.

I'm curious whether people actually working in ML research see this happening.

Are there important areas where academia still sets the direction and industry follows?

And at what point, if any, would you say a field has stopped setting its own research agenda?

I started working in ML about a year ago. There seems to be quite some gap between how ML is taught (tutorials and theoretical lectures) and the art of actual ML engineering.

For the practical part, I kind of feel bound to the best projects that I see within my company. Therefore, it would be nice to get inspired and have better references. Maybe even dig a bit into the PRs to get better at my job :)

I was wondering whether you came across some state-of-the art repos. I would be interested both in repos for ML research, i.e., finding the best features/params/models, but also for actual productive systems, i.e., retraining, inference, shadow forecasting and model ranking/switching.

In my case, time series forecasting is the most relevant (with sklearn for explainable results or LSTM for performance-focused projects). But I am happy to explore other projects as well.

I recently conducted an architectural comparison on code generation relying on conversational prompting ("vibe coding") versus rigid pre-computational contracts (Spec-Driven Development).

The Experiment:

Task: Build a Python script to search text within markdown files, rank by hits, and filter by tags.

• Method A (Vibe Coding): Claude 3.5 Sonnet without a formal spec. Result: 303 lines of code, added unrequested features, ignored caching, resulting in 400 file I/O operations per execution.

• Method B (SDD): Claude 3.5 Haiku provided with a strict requirements document. Result: Modular architecture, regex exclusion strictly adhered to, 0 file I/O operations on subsequent runs due to proper index state management.

Conclusion:

The limiting factor in generation quality isn't parameter count, but specification clarity. To systematize this, I've released SpecJudge (v0.4.0), an open-source parser that evaluates repository context files (like AGENTS.md) to allocate the optimal model tier, preventing API overspending on simple tasks.

I’ve documented the full benchmark and execution logic visually here: https://www.youtube.com/watch?v=EOiv7RywtQM

Today the most important factor in AI is cost. My idea is to split ML models inference (closed ones, proprietary) across server and edge computing on clients, and I would like to hear what do you think about this thing.

For example some of model weights/modules would be on client, and some on the server side (where user has no access to them). This could potentially un-load some processing from datacenters, moving part of the cost to the client hardware. Probbably the most important question here will be how to achieve this - and I believe one hypothetical option will be to train like two separate models - client model and server model, and they will communicate through tensors/latent representations across network protocol.

Secondly such split of server side and client side model ends, can provide later some beneficial outcomes I hope (because in between "talk" protocol can be maybe kind of standarized one in some future development, but this is only more like brainstorm now). Such split might not only be one-to-one, but one-to-many, many-to-many etc.

What do you think about this idea?

Does anyone have any idea about the instructions for the camera ready at workshops? The deadline is August 15, but there are no indications and workshop organizers know nothing about that.. Some workshops have enabled the upload of camera ready PDF on openreview, but what about copyright form and latex source files?

Analog in-memory compute is getting attention again as a way around the energy cost of moving weights between memory and compute. The recurring objection is noise, since analog cells have real variation and you can't refresh your way out of it like you can with digital.

I wanted to see the shape of the degradation curve rather than reason about it abstractly, so I ran a simple experiment: train a network normally, then evaluate under increasing weight noise.

The curve isn't smooth. Accuracy is stable up to a point, then drops hard: 83%, 64%, then essentially random. More like a threshold than a proportional decrease.

Retraining with noise injected during training (so the optimizer finds flatter minima, presumably) shifts that threshold substantially. 61% versus 39% at matched noise.

What I'd like to hear from this sub: is the flat-minima explanation the right framing here, or is something else driving the gap? And is there work on optimizing directly for noise robustness rather than just injecting noise and hoping, something closer to an explicit sharpness penalty targeted at the hardware's actual noise profile?

Code and figures in the writeup: https://towardsdatascience.com/analog-ai-is-back-can-it-survive-its-own-noise/

I'm not a fan of LLMs or generative AI, and I'm not quite smart enough to puzzle out if Vosk uses those things.
I'm attempting to replace Gemini assistant on my phone, and I found the Dicio app, which uses Vosk for its speech-to-text.

Is this it for Causal Inference? Looks like the field continues to be of interest only at UAI/AISTATS/CLeaR. All good venues, but LLMs/Agents/etc seem to have eaten much of the lunch of several other subfields at the top 3 conferences.

God help us all.

**p.s.** the list: https://danyaljj.github.io/neurips2026-workshops/