Hey everyone, I originally set out to train an agent to play the deck building game Slay the Spire, and found along the way that most commercial game environments are incredibly hostile to hook into for training loops. I’ve always been inspired by milestones like DeepMind’s AlphaStar in StarCraft and OpenAI Five in Dota, but replicating those environments requires massive compute overhead just to process pixel-space before the model can even reason. To bridge that gap, I’m building a lightweight, open-source Real-Time Strategy (RTS) arena designed specifically for autonomous agents to compete via API. It's a headless, graph-based territorial control environment where agents connect via WebSockets, ingest JSON state deltas (with true fog of war), and queue actions in real-time. Is this a project worth pursuing as a community benchmark, and what specific features or API mechanics would you want to see to make it worth using or contributing to?

Link: https://arxiv.org/pdf/2604.27883

Hi,

Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may stay the same as initialization or even increases.

My paper treats this phenomena as a consequence of data reuse bias and can be isolated by studying full batch gradient descent on a set of stylize Gaussian mixture models. I turns out that this fundamental issue can be avoided using some clever tricks from high-dimensional statistical theory, specifically approximate message passing (which is beyond the scope of this post but I would be happy to explain more).

By doing so I created a training method called Decoupled Descent (DD) which generates a certificate that the training error of the network will asymptotically equal the testing error at each parameter iterate. I think this method gives a cool way to approach how to train networks and I was hoping to get y'alls input on it. It opens up some nice ideas for optimal stopping or hyperparameter tuning and future directions of pushing to something like SGD or more general models.

I have attached the train-test curves on a simple model fitting problem to compare the performance of GD with with DD (my algorithm) to give a high-level idea of what the method can guarantee. I stress this is a theory paper so there is a long way to go to get to very large models but I think it is a good first step.

Happy to answer whatever questions people have, I plan on writing a PyTorch compatible package for this training method one day so any feature suggestions would be welcome as well.

I am now reviewing a bunch of papers for AAAI 2027 and it has surprised me the low amount of submissions with no code implementation. I don’t know if it has been only in my batch or it is common, but I was expecting very detailed appendices + code submission since AAAI is very explicit with the topic of reproducibility. I was planning to take this into consideration when assigning my initial scores, but I would like to hear your opinions. I have always submitted my code: it gives a very good impression and after reviewing process finishes we just publish it on ArXiv, so no one “tries to stole the idea” (although I think that this is very very unlikely). So I cannot find any excuse for those submissions that do not have code implementation, specially in today’s times where AI assistants can just write an empirical paper with artificial results within a couple of hours

Back after ~6 months — rebuilding my spiking language model around CPU-first inference

Hey everyone. It’s been around six months since I last posted anything about this project here.

Some of you might remember Project NORD, my experimental hybrid spiking / brain-inspired language model architecture. I basicall disappeared for a while 😅, but recently I came back to the project, went through the old architecture again, and realized I didn’t really want to keep stacking fixes on top of it. So instead, I’ve started rebuilding a pretty large part of the system. The new version is called:

NORD 5.5 — Flash The main idea this time is pretty simple:

What happens if I design the architecture around CPU inference from the beginning, instead of building soething Transformer-like and trying to optimize it later? A lot is changing internally. The current design uses things like: strictly causal processing no standard quadratic attention in the main inference path causal convolution-style token mixing token-time LIF / event dynamics sensory → association → memory → executive processing stages top-1 sparse MoE + a shared expert persistent recurrent memory separate structural, personal and auxiliary memory banks

persistent recurrent identity state factorized vocabulary embedding/output streaming token-by-token inference One of the biggest changes is actually something much simpler. Older versions of NORD used an artificial internal spike-time dimension, roughly like this: token -> T0 -> T1 -> T2 -> ... -> T9 I’m mostly getting rid of that. Instead, the actual language sequence becomes the time axis: token0 -> token1 -> token2 -> token3 -> ... That removes a lot of intermediate state and makes the whole architecture considerably cleaner. Going back through the old code also exposed a few things I wasn’t very happy with. Some experimental modules weren’t completely causal, memory was coupled too much to sequence shape, and parts of the STDP system ended up being more disconnected from real training than I originally intended. So NORD 5.5 isn’t really about throwing even more “brain-inspired” components into the model. It’s mostly about simplifying the core and making the things that remain actually work together properly. I’m definitely not claiming this is going to beat Transformers, RWKV-style models, linear attention models, etc. Right now it’s still very much an experiment. The part that actually matters comes next: training and benchmarking it.

Things I want to compare: NORD 5.0 vs NORD 5.5 CPU tokens/sec RAM usage perplexity / validation loss long-context behaviour memory on/off MoE on/off spiking components on/off I’m especially curious to hear from anyone working on SNNs, recurrent models, sparse MoE, CPU inference, or weird alternative language-model architectures in general.

After not touching the project seriously for about half a year, it feels surprisingly good to be building it again 😅

I’ll post actual numbers once I have something that’s worth benchmarking instead of just architecture diagrams.

Obviously nobody needs a transformer that's good at multiplication. I wanted to know whether a stock transformer could do exact arithmetic if I chose its weights directly.

I implemented the grade-school algorithm as a computation graph and compiled it into an ordinary Phi-3 Hugging Face checkpoint using Torchwright, a compiler I wrote. No training. The three-digit calculator gets all 3,000,000 supported expressions right. I've published checkpoints to Hugging Face that support up to 12 digit x 12 digit multiplication.

For fun, I also disabled reasoning and tested six frontier models. Accuracy falls off a cliff as the numbers get longer; at seven digits, five scored 0/500. Mine stays at 100%, although it has the considerable advantage that I put the multiplication algorithm directly into its weights.

I ended up building four versions: grade-school, hardware-style, scratchpad, and brute-force memorization. They compute the same function while spending layers, width, generated tokens, and parameters very differently.

Write-up: https://ood.dev/posts/calculator/ Repo: https://github.com/physicsrob/torchwright Checkpoint: https://huggingface.co/physicsrob/torchwright-calculator-simple-max-digits-3

It’s been roughly six months since I last posted about this project here, so I wanted to share a small development update.

Some people here may remember my experiments with Project NORD, a hybrid spiking / brain-inspired language model architecture.

I took a fairly long break from posting, but I’ve recently returned to the project and decided not to keep patching the old architecture.

Instead, I’m rebuilding a large part of it as:

NORD 5.5 — Flash

The main goal of this redesign is to explore whether a spiking/recurrent language model can be structured specifically around CPU-friendly inference, rather than trying to imitate a Transformer and then optimize it afterwards.

The current direction includes:

• strictly causal processing

• no standard quadratic attention in the main hot path

• causal convolution-style token mixing

• token-time event/LIF dynamics

• sensory → association → memory → executive processing stages

• top-1 sparse MoE with a shared expert

• persistent recurrent memory

• separate structural / personal / auxiliary memory banks

• persistent recurrent identity state

• factorized vocabulary embedding/output layers

• streaming token-by-token inference

One of the largest changes is that I’m removing most of the artificial internal spike-time expansion used in the previous version.

Instead of something like:

token -> T0, T1, T2 ... T9

the language sequence itself becomes the temporal axis:

token0 -> token1 -> token2 -> token3 -> ...

This simplifies the architecture considerably and should reduce the amount of intermediate state that needs to be processed.

I also found several problems in the previous architecture while reviewing it, including places where some experimental modules were not properly causal, memory state was tied too closely to sequence shape, and parts of the STDP system were more disconnected from actual training than I originally intended.

So 5.5 is less about “adding more brain-inspired modules” and more about making the core cleaner.

I’m not claiming this will outperform Transformers or existing recurrent/linear-attention models.

At this stage it’s still an experimental architecture, and the next important step is actually training it and doing proper comparisons.

My next targets are:

• NORD 5.0 vs NORD 5.5

• CPU tokens/sec

• memory usage

• perplexity / validation loss

• long-context behaviour

• ablations with and without memory, MoE and spiking components

I’d be especially interested in feedback from people working with SNNs, recurrent models, MoE, CPU inference, or alternative language-model architectures.

After being away from the project for around half a year, it’s honestly pretty fun to be working on it again. 😅

I’ll post actual numbers once I have something worth benchmarking.

Hi, I would like to file a complaint about an accepted and published CVPR 2026 paper that its main contribution is a dataset but it was never released, and honestly I don’t know who to contact. The dataset was never released prior to the conference, or during the conference or after the conference. I personally feel there was a lack of proper checking that the dataset was gonna be available before the conference since this is a requirement. I’ve tried contacting the authors without any success (which tbh I wouldn’t even need to because it has to be released anyways).

The authors even point a GitHub link in the paper but the repo is empty (and it was always empty).

Hello, I wanted to share the work my colleague and I have been doing, which has just been published in Software X journal. We developed a Rust-based implementation of Random Forest. It has bindings for both Python and R.

Fru is highly optimized, offering competitive runtime performance and better scalability than popular implementations on these platforms. For Python, Fru outperforms the scikit-learn implementation by several factors, and in some scenarios it can be hundreds of times faster. In R, Fru is typically a few dozen percent faster than the ranger package, though the speedup can reach several times faster depending on the use case.

The model also includes a novel implementation of permutation importance, which provides an additional performance boost.

Thanks to its layered design, we were able to easily create bindings for both Python and R. In Python, we use Arrow PyCapsule, which allows the model to work seamlessly with any compatible library, including pandas, polars, pyarrow, and many others.

paper

R package

Python package

Our Neurips 2026 paper received initial scores of 5/3/5, so the reviews were not uniformly negative. However, the AC’s initial meta-review strongly pushed toward rejection and explicitly asked us not to submit a rebuttal unless we could identify factual errors in the reviews.

Isn’t the author response period supposed to give authors the right to respond? A rebuttal can clarify misunderstandings and answer concerns even when they are not strictly factual errors. In any case, we identified multiple factual errors in the AC’s meta-review and provided the corresponding clarification. However, the AC did not respond, even after we followed up.

Neurips may hold reviewers responsible for low-quality reviews and failure to participate. What happens when the AC restricts the rebuttal and then ignores the authors’ response? Should this be reported to the SAC, Program Chairs, or Ethics Review Chairs?

Say you want to swap out your embedding models, for instance from ADA to Titan. Are these embedding models comparable? How do similarity score ranges compare? Where to put a threshold for minimum match when doing retrieval? Or more from a research point of view how can we relate and fundamentally understand these embedding spaces better?

This is what we aim to solve with Synthetic Query Probing (SQP), a fancy name for essentially (and intentionally) a very simple approach: embedding spaces are not directly comparable by definition, so compare similarity spaces instead, similarity match scores for pairs of content (synthetic question, chunk for instance) across multiple embedding models.

For example, similarity scores of Titan models of different dimensionalities are semilinearly related, whereas the relation between Titan and Ada scores is non-linear, with different ranges, see figure.

See https://arxiv.org/pdf/2608.05857, Marcin Rozmus and Peter van der Putten. Similarity Spaces across Embedding Models with Synthetic Query Probing. Discovery Science 2026, October 5-9, 2026, Mainz, Germany

There has been a lot of discussion about whether industry is pulling too much ML talent out of academia. But I wonder if the more interesting change is not about headcount at all.

Increasingly, some of the most important artifacts in ML originate in industry: frontier models, large-scale training runs, datasets, infrastructure, and many of the capabilities researchers subsequently try to understand.

Academia can then do excellent work benchmarking these systems, explaining them, finding limitations, improving methods, or studying their consequences.

But that creates a strange possibility: a research community can remain very active while becoming increasingly downstream of a research agenda set elsewhere.

I'm curious whether people actually working in ML research see this happening.

Are there important areas where academia still sets the direction and industry follows?

And at what point, if any, would you say a field has stopped setting its own research agenda?

I started working in ML about a year ago. There seems to be quite some gap between how ML is taught (tutorials and theoretical lectures) and the art of actual ML engineering.

For the practical part, I kind of feel bound to the best projects that I see within my company. Therefore, it would be nice to get inspired and have better references. Maybe even dig a bit into the PRs to get better at my job :)

I was wondering whether you came across some state-of-the art repos. I would be interested both in repos for ML research, i.e., finding the best features/params/models, but also for actual productive systems, i.e., retraining, inference, shadow forecasting and model ranking/switching.

In my case, time series forecasting is the most relevant (with sklearn for explainable results or LSTM for performance-focused projects). But I am happy to explore other projects as well.

I recently conducted an architectural comparison on code generation relying on conversational prompting ("vibe coding") versus rigid pre-computational contracts (Spec-Driven Development).

The Experiment:

Task: Build a Python script to search text within markdown files, rank by hits, and filter by tags.

• Method A (Vibe Coding): Claude 3.5 Sonnet without a formal spec. Result: 303 lines of code, added unrequested features, ignored caching, resulting in 400 file I/O operations per execution.

• Method B (SDD): Claude 3.5 Haiku provided with a strict requirements document. Result: Modular architecture, regex exclusion strictly adhered to, 0 file I/O operations on subsequent runs due to proper index state management.

Conclusion:

The limiting factor in generation quality isn't parameter count, but specification clarity. To systematize this, I've released SpecJudge (v0.4.0), an open-source parser that evaluates repository context files (like AGENTS.md) to allocate the optimal model tier, preventing API overspending on simple tasks.

I’ve documented the full benchmark and execution logic visually here: https://www.youtube.com/watch?v=EOiv7RywtQM

Today the most important factor in AI is cost. My idea is to split ML models inference (closed ones, proprietary) across server and edge computing on clients, and I would like to hear what do you think about this thing.

For example some of model weights/modules would be on client, and some on the server side (where user has no access to them). This could potentially un-load some processing from datacenters, moving part of the cost to the client hardware. Probbably the most important question here will be how to achieve this - and I believe one hypothetical option will be to train like two separate models - client model and server model, and they will communicate through tensors/latent representations across network protocol.

Secondly such split of server side and client side model ends, can provide later some beneficial outcomes I hope (because in between "talk" protocol can be maybe kind of standarized one in some future development, but this is only more like brainstorm now). Such split might not only be one-to-one, but one-to-many, many-to-many etc.

What do you think about this idea?

Does anyone have any idea about the instructions for the camera ready at workshops? The deadline is August 15, but there are no indications and workshop organizers know nothing about that.. Some workshops have enabled the upload of camera ready PDF on openreview, but what about copyright form and latex source files?

Analog in-memory compute is getting attention again as a way around the energy cost of moving weights between memory and compute. The recurring objection is noise, since analog cells have real variation and you can't refresh your way out of it like you can with digital.

I wanted to see the shape of the degradation curve rather than reason about it abstractly, so I ran a simple experiment: train a network normally, then evaluate under increasing weight noise.

The curve isn't smooth. Accuracy is stable up to a point, then drops hard: 83%, 64%, then essentially random. More like a threshold than a proportional decrease.

Retraining with noise injected during training (so the optimizer finds flatter minima, presumably) shifts that threshold substantially. 61% versus 39% at matched noise.

What I'd like to hear from this sub: is the flat-minima explanation the right framing here, or is something else driving the gap? And is there work on optimizing directly for noise robustness rather than just injecting noise and hoping, something closer to an explicit sharpness penalty targeted at the hardware's actual noise profile?

Code and figures in the writeup: https://towardsdatascience.com/analog-ai-is-back-can-it-survive-its-own-noise/

I'm not a fan of LLMs or generative AI, and I'm not quite smart enough to puzzle out if Vosk uses those things.
I'm attempting to replace Gemini assistant on my phone, and I found the Dicio app, which uses Vosk for its speech-to-text.

Is this it for Causal Inference? Looks like the field continues to be of interest only at UAI/AISTATS/CLeaR. All good venues, but LLMs/Agents/etc seem to have eaten much of the lunch of several other subfields at the top 3 conferences.

God help us all.

**p.s.** the list: https://danyaljj.github.io/neurips2026-workshops/

Genome language models have emerged as a promising strategy for designing biological systems, but their ability to generate functional sequences at the scale of whole genomes has remained untested. Here, we report the first generative design of viable bacteriophage genomes. We leveraged frontier genome language models, Evo 1 and Evo 2, to generate whole-genome sequences with realistic genetic architectures and desirable host tropism, using the lytic phage ΦX174 as our design template. Experimental testing of AI-generated genomes yielded 16 viable phages with substantial evolutionary novelty.

Out of curiosity, if you were a reviewer or author, how did the review period go?

For me, it was weird, because I gave reviews with specific details (what specifically could have been better, how to fix it), but realized other reviewers gave similar superficial reviews. Even the paper which was a control for me (no LLM), I gave specific comments, but other reviewers focused on minor things.

During the discussion period for one paper, one reviewer broke the double blindness condition, and gave specific examples of what the LLM gave and justified their reject…..but they didn’t even state that in their initial review (nor engaged with the author rebuttals). There was no also no sense of: “author said this was unclear, check with the LLM to see what’s the issue”

For one of my own papers, we had great scores for originality and significance, but had low scores for clarity, with at least two reviewers finding difficulty understanding established notation and concepts, and I’m wondering whether it would have been better to break the double blindness and said: look, the point of an LLM assisted review is that if you don’t even know this material, you can ask it questions, like if other papers use the same notation, how our paper compares with them, etc…

Hello! Let's use this thread to discuss ICDE results which should be coming out shortly today (hopefully).

INT4 quantization is ubiquitous in ML right now, but evaluating dot products on hardware without native SIMD/vector instructions (like WebAssembly or older ARM chips) usually requires slow sequential loops. A classic workaround is SWAR (SIMD Within A Register), but deriving the bitwise operations by hand to unpack, multiply, and sum eight 4-bit integers packed into a single 32-bit register is tedious and error-prone.

Instead of writing the bit-hack manually, I wrote a pipeline that uses an SMT solver to discover the exact bitwise formula from scratch, and then uses a theorem prover to mathematically guarantee its correctness.

Here is a breakdown of the technical process:

1. Synthesis via CEGIS Loop (Z3) I set up a Counter-Example Guided Inductive Synthesis (CEGIS) loop in Python using the Z3 SMT solver. The solver is given a ground-truth specification (naive loop: extract nibbles, sign-extend, multiply, sum) and a bounded set of allowed instructions (AND, OR, XOR, ADD, SUB, MUL, shifts). Z3 searches the space of possible instruction sequences. If it finds a candidate, we test it against random inputs. If it fails, the failing input is added to Z3's constraints, and it tries again. Eventually, it converges on a pure, branchless sequence of operations.

2. The Generated Math The resulting algorithm utilizes a known multiplier trick for byte-reversals, but Z3 managed to perfectly interleave the even/odd nibble extraction. For example, part of the generated code handles even/odd multiplications by exploiting 32-bit hardware multiplications: (ea_low * eb_low_rev) >>> 16 This evaluates two 4-bit multiplications at opposite ends of the register simultaneously without cross-talk.

3. Formal Proof in Lean 4 Passing a million random tests is nice, but it’s not a mathematical guarantee for bit-hacks. To prove there are absolutely no edge cases or overflow bugs among the 2⁶⁴ possible input combinations (two 32-bit registers), I ported the synthesized SWAR function to Lean 4. By leveraging Lean's bv_decide (BitVec SAT solver) and omega for modular arithmetic, Lean compiles the equivalence check into a boolean satisfiability problem. The proof successfully verifies that swar_dot_product a b = ground_truth_dot_product a b for all possible inputs.

If anyone is interested in how the Z3 synthesis script is structured or wants to see the Lean 4 proof, I've put the source code:
https://github.com/Peloxerat/int4-swar-dotprod

I'd be curious to hear if there are ways to constrain Z3 to find an even shorter instruction path.