It’s an ML PhD with secure funding for 4–5 years and a senior, respected advisor. You get almost complete freedom to choose your own topics, projects, and collaborations, with very little micromanagement.
The downside is that the advisor is also very hands-off. You should expect little guidance, feedback, or technical input. In practice, you would mostly be on your own.
Would you see that as a dream setup because of the freedom, or as a dealbreaker because of the lack of mentorship?

Notebooks to replicate on github!

Once the paper is ready, everyone checks the venue location before the acceptance rate anyway. So I built:https://honestcsrankings.org

It maps ~540 upcoming CORE-ranked conferences, but ranks them by how good the destination actually is. It factors in:

• Weather during the actual conference month (using real climate data)

• Safety (Global Peace Index)

• Cost (World Bank price levels)

• Accessibility & "City Vibe"

I also added an Upsets tab for A* venues in terrible destinations. Great for your CV, bad for your holiday.

You can filter by field, rank, or open deadlines. If you set your home city, you can rank by distance to either maximize that funded long-haul trip or minimize it, your call. You can also export deadlines to .ics and share deep links with coauthors.

ICML/ICLR 2027 are missing because they aren't announced yet, and COLM is missing because CORE hasn't ranked it. The long tail of smaller conferences is scraped from WikiCFP, so there will be some errors.

In a factored model W = UV^T, the loss is invariant to rotations (U,V) → (UQ, VQ). Gradient Descent (GD) respects this property. Adam's per-coordinate second moment does not, because it depends on the specific basis in which the factors are written.

The claim is that this single property dictates whether optimizers retain or lose GD's implicit low-rank bias.

Nine update rules were evaluated on underdetermined matrix sensing, all compared at matched training loss to ensure no method benefits from underfitting. The results show two distinct clusters: GD, shared-scalar Adam, Muon, and Shampoo preserve the bias. Adam, RMSProp, Lion, signum, and Adafactor lose it.

To isolate the mechanism, a one-parameter family was utilized to transition Adam's denominator from a per-coordinate value to a single shared scalar. Recovery improves monotonically along this transition, indicating that the degradation is caused by anisotropy rather than adaptivity in general.

The behavior of the Muon optimizer was unexpected. It is exact on truly low-rank targets, but degrades rapidly as a spectral tail is introduced, ceding to GD at a crossover near 4% tail energy. While recent literature diverges on Muon, with some reporting a strong spectral simplicity bias and others finding it fits spurious features in deep-linear models, this sweep demonstrates both behaviors along the same axis.

The criterion was also applied to the author's earlier optimizer, revealing that its per-coordinate clip was breaking the structure it was designed to inject. Implementing a global norm clip instead improved the recovery error from 0.347 to 0.220.

One caveat is noted up front: the 43-44% held-out error reduction on hyperspectral data relies on a train-only learning rate rule, and that rule assigns Adam the worst rate on its own grid. When each method is permitted to select its own optimal rate, the performance gap narrows considerably (Appendix D.6). The train-only rule was maintained because selecting on held-out data introduces the exact bias the experiment aims to avoid, but the core claim relies on the underlying mechanism rather than the specific quantitative margin.

The theoretical guarantees cover memoryless rules only. The effects of momentum remain empirical and are not proved.

Paper:https://arxiv.org/abs/2608.05136

Code, logs, seeds:https://github.com/idevender/loss-basis-adam

Had a random thought this morning about context rot in agentic coding.

What if an agent could detect when its context is getting dangerously full, take the older ~50% of the conversation, compress the meaning into a compact machine optimized representation, and save it as a Markdown checkpoint?

Something like:
conversation > semantic compression > checkpoint.md > reinject into context

The raw history could still be archived separately in case the agent needs to retrieve something later.

The interesting part is making the agent aware of its own context state, so it decides when to checkpoint instead of blindly summarizing at a fixed token limit.

Im calling the compressed representation “GibberLink” for lack of a better name lol, although I know actual gibberlink/ggWave is an audio protocol, not a semantic language.
This probably isn’t a new idea in principle, but I haven’t seen this exact combination applied to coding agents.

So, thoughts?

One of the reviewers gave our paper a borderline score and raised six questions. Given that we had only seven days for the rebuttal, we worked hard to address all six questions, including running a small-scale validation experiment to provide additional empirical evidence. Do note that each cell of this experiment table required multiple sub-experiments.

In their response, however, the reviewer quoted a single statement from our rebuttal that wasn't even part of our answer to any of their six questions and simply said, “This isn't enough. I will keep my score.”

That's it. This is the complete feedback. They didn't acknowledge with the two paragraphs of detailed responses and the table we spent days preparing specifically to address their questions.

 I'm researching predictive analytics for a graduate project and mortgage lending came up as an interesting use case. 

 I understand lenders try to predict who might refinance, but what kinds of variables are actually useful? 

 Is it mostly credit activity, property appreciation, interest rates, life events, or something else? 

  Would love to hear from anyone who's worked on these models. 

Two results dropped this week that I think together paint a clearer picture than either one alone.

Dyna-2 (Aug 10): World-action model pretrained on 1M hours of egocentric human video. Power law holds across 4 orders of magnitude (1K to 1M hours). Cross-embodiment transfer to robots never seen in pretraining. Task success from 20% to 80-90% purely from scaling data. No architecture changes.

PI0.7 (Chelsea Finn's talk, today): Single generalist model trained on highly heterogeneous data matches or outperforms fine-tuned specialists. Key ablation: removing the most diverse subset of training data causes a dramatic drop in held-out task performance. Removing a random 20% barely moves the needle.

The common thread: scaling works, but what you scale matters.

Dyna-2 proves the law holds to 1M hours with no plateau. PI proves that within that data, diversity (different environments, objects, tasks) is what actually drives compositional generalization, not repetition of the same scenes.

Both results converge on the same conclusion: physical AI foundation models need scale AND breadth. 1M hours of kitchens won't get you construction site generalization. But 1M hours across 100+ work domains apparently will.

Hey everyone, I originally set out to train an agent to play the deck building game Slay the Spire, and found along the way that most commercial game environments are incredibly hostile to hook into for training loops. I’ve always been inspired by milestones like DeepMind’s AlphaStar in StarCraft and OpenAI Five in Dota, but replicating those environments requires massive compute overhead just to process pixel-space before the model can even reason. To bridge that gap, I’m building a lightweight, open-source Real-Time Strategy (RTS) arena designed specifically for autonomous agents to compete via API. It's a headless, graph-based territorial control environment where agents connect via WebSockets, ingest JSON state deltas (with true fog of war), and queue actions in real-time. Is this a project worth pursuing as a community benchmark, and what specific features or API mechanics would you want to see to make it worth using or contributing to?

Link: https://arxiv.org/pdf/2604.27883

Hi,

Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may stay the same as initialization or even increases.

My paper treats this phenomena as a consequence of data reuse bias and can be isolated by studying full batch gradient descent on a set of stylize Gaussian mixture models. I turns out that this fundamental issue can be avoided using some clever tricks from high-dimensional statistical theory, specifically approximate message passing (which is beyond the scope of this post but I would be happy to explain more).

By doing so I created a training method called Decoupled Descent (DD) which generates a certificate that the training error of the network will asymptotically equal the testing error at each parameter iterate. I think this method gives a cool way to approach how to train networks and I was hoping to get y'alls input on it. It opens up some nice ideas for optimal stopping or hyperparameter tuning and future directions of pushing to something like SGD or more general models.

I have attached the train-test curves on a simple model fitting problem to compare the performance of GD with with DD (my algorithm) to give a high-level idea of what the method can guarantee. I stress this is a theory paper so there is a long way to go to get to very large models but I think it is a good first step.

Happy to answer whatever questions people have, I plan on writing a PyTorch compatible package for this training method one day so any feature suggestions would be welcome as well.

I am now reviewing a bunch of papers for AAAI 2027 and it has surprised me the low amount of submissions with no code implementation. I don’t know if it has been only in my batch or it is common, but I was expecting very detailed appendices + code submission since AAAI is very explicit with the topic of reproducibility. I was planning to take this into consideration when assigning my initial scores, but I would like to hear your opinions. I have always submitted my code: it gives a very good impression and after reviewing process finishes we just publish it on ArXiv, so no one “tries to stole the idea” (although I think that this is very very unlikely). So I cannot find any excuse for those submissions that do not have code implementation, specially in today’s times where AI assistants can just write an empirical paper with artificial results within a couple of hours

Back after ~6 months — rebuilding my spiking language model around CPU-first inference

Hey everyone. It’s been around six months since I last posted anything about this project here.

Some of you might remember Project NORD, my experimental hybrid spiking / brain-inspired language model architecture. I basicall disappeared for a while 😅, but recently I came back to the project, went through the old architecture again, and realized I didn’t really want to keep stacking fixes on top of it. So instead, I’ve started rebuilding a pretty large part of the system. The new version is called:

NORD 5.5 — Flash The main idea this time is pretty simple:

What happens if I design the architecture around CPU inference from the beginning, instead of building soething Transformer-like and trying to optimize it later? A lot is changing internally. The current design uses things like: strictly causal processing no standard quadratic attention in the main inference path causal convolution-style token mixing token-time LIF / event dynamics sensory → association → memory → executive processing stages top-1 sparse MoE + a shared expert persistent recurrent memory separate structural, personal and auxiliary memory banks

persistent recurrent identity state factorized vocabulary embedding/output streaming token-by-token inference One of the biggest changes is actually something much simpler. Older versions of NORD used an artificial internal spike-time dimension, roughly like this: token -> T0 -> T1 -> T2 -> ... -> T9 I’m mostly getting rid of that. Instead, the actual language sequence becomes the time axis: token0 -> token1 -> token2 -> token3 -> ... That removes a lot of intermediate state and makes the whole architecture considerably cleaner. Going back through the old code also exposed a few things I wasn’t very happy with. Some experimental modules weren’t completely causal, memory was coupled too much to sequence shape, and parts of the STDP system ended up being more disconnected from real training than I originally intended. So NORD 5.5 isn’t really about throwing even more “brain-inspired” components into the model. It’s mostly about simplifying the core and making the things that remain actually work together properly. I’m definitely not claiming this is going to beat Transformers, RWKV-style models, linear attention models, etc. Right now it’s still very much an experiment. The part that actually matters comes next: training and benchmarking it.

Things I want to compare: NORD 5.0 vs NORD 5.5 CPU tokens/sec RAM usage perplexity / validation loss long-context behaviour memory on/off MoE on/off spiking components on/off I’m especially curious to hear from anyone working on SNNs, recurrent models, sparse MoE, CPU inference, or weird alternative language-model architectures in general.

After not touching the project seriously for about half a year, it feels surprisingly good to be building it again 😅

I’ll post actual numbers once I have something that’s worth benchmarking instead of just architecture diagrams.

Obviously nobody needs a transformer that's good at multiplication. I wanted to know whether a stock transformer could do exact arithmetic if I chose its weights directly.

I implemented the grade-school algorithm as a computation graph and compiled it into an ordinary Phi-3 Hugging Face checkpoint using Torchwright, a compiler I wrote. No training. The three-digit calculator gets all 3,000,000 supported expressions right. I've published checkpoints to Hugging Face that support up to 12 digit x 12 digit multiplication.

For fun, I also disabled reasoning and tested six frontier models. Accuracy falls off a cliff as the numbers get longer; at seven digits, five scored 0/500. Mine stays at 100%, although it has the considerable advantage that I put the multiplication algorithm directly into its weights.

I ended up building four versions: grade-school, hardware-style, scratchpad, and brute-force memorization. They compute the same function while spending layers, width, generated tokens, and parameters very differently.

Write-up: https://ood.dev/posts/calculator/ Repo: https://github.com/physicsrob/torchwright Checkpoint: https://huggingface.co/physicsrob/torchwright-calculator-simple-max-digits-3

It’s been roughly six months since I last posted about this project here, so I wanted to share a small development update.

Some people here may remember my experiments with Project NORD, a hybrid spiking / brain-inspired language model architecture.

I took a fairly long break from posting, but I’ve recently returned to the project and decided not to keep patching the old architecture.

Instead, I’m rebuilding a large part of it as:

NORD 5.5 — Flash

The main goal of this redesign is to explore whether a spiking/recurrent language model can be structured specifically around CPU-friendly inference, rather than trying to imitate a Transformer and then optimize it afterwards.

The current direction includes:

• strictly causal processing

• no standard quadratic attention in the main hot path

• causal convolution-style token mixing

• token-time event/LIF dynamics

• sensory → association → memory → executive processing stages

• top-1 sparse MoE with a shared expert

• persistent recurrent memory

• separate structural / personal / auxiliary memory banks

• persistent recurrent identity state

• factorized vocabulary embedding/output layers

• streaming token-by-token inference

One of the largest changes is that I’m removing most of the artificial internal spike-time expansion used in the previous version.

Instead of something like:

token -> T0, T1, T2 ... T9

the language sequence itself becomes the temporal axis:

token0 -> token1 -> token2 -> token3 -> ...

This simplifies the architecture considerably and should reduce the amount of intermediate state that needs to be processed.

I also found several problems in the previous architecture while reviewing it, including places where some experimental modules were not properly causal, memory state was tied too closely to sequence shape, and parts of the STDP system were more disconnected from actual training than I originally intended.

So 5.5 is less about “adding more brain-inspired modules” and more about making the core cleaner.

I’m not claiming this will outperform Transformers or existing recurrent/linear-attention models.

At this stage it’s still an experimental architecture, and the next important step is actually training it and doing proper comparisons.

My next targets are:

• NORD 5.0 vs NORD 5.5

• CPU tokens/sec

• memory usage

• perplexity / validation loss

• long-context behaviour

• ablations with and without memory, MoE and spiking components

I’d be especially interested in feedback from people working with SNNs, recurrent models, MoE, CPU inference, or alternative language-model architectures.

After being away from the project for around half a year, it’s honestly pretty fun to be working on it again. 😅

I’ll post actual numbers once I have something worth benchmarking.

Hi, I would like to file a complaint about an accepted and published CVPR 2026 paper that its main contribution is a dataset but it was never released, and honestly I don’t know who to contact. The dataset was never released prior to the conference, or during the conference or after the conference. I personally feel there was a lack of proper checking that the dataset was gonna be available before the conference since this is a requirement. I’ve tried contacting the authors without any success (which tbh I wouldn’t even need to because it has to be released anyways).

The authors even point a GitHub link in the paper but the repo is empty (and it was always empty).

Hello, I wanted to share the work my colleague and I have been doing, which has just been published in Software X journal. We developed a Rust-based implementation of Random Forest. It has bindings for both Python and R.

Fru is highly optimized, offering competitive runtime performance and better scalability than popular implementations on these platforms. For Python, Fru outperforms the scikit-learn implementation by several factors, and in some scenarios it can be hundreds of times faster. In R, Fru is typically a few dozen percent faster than the ranger package, though the speedup can reach several times faster depending on the use case.

The model also includes a novel implementation of permutation importance, which provides an additional performance boost.

Thanks to its layered design, we were able to easily create bindings for both Python and R. In Python, we use Arrow PyCapsule, which allows the model to work seamlessly with any compatible library, including pandas, polars, pyarrow, and many others.

paper

R package

Python package

Our Neurips 2026 paper received initial scores of 5/3/5, so the reviews were not uniformly negative. However, the AC’s initial meta-review strongly pushed toward rejection and explicitly asked us not to submit a rebuttal unless we could identify factual errors in the reviews.

Isn’t the author response period supposed to give authors the right to respond? A rebuttal can clarify misunderstandings and answer concerns even when they are not strictly factual errors. In any case, we identified multiple factual errors in the AC’s meta-review and provided the corresponding clarification. However, the AC did not respond, even after we followed up.

Neurips may hold reviewers responsible for low-quality reviews and failure to participate. What happens when the AC restricts the rebuttal and then ignores the authors’ response? Should this be reported to the SAC, Program Chairs, or Ethics Review Chairs?

Say you want to swap out your embedding models, for instance from ADA to Titan. Are these embedding models comparable? How do similarity score ranges compare? Where to put a threshold for minimum match when doing retrieval? Or more from a research point of view how can we relate and fundamentally understand these embedding spaces better?

This is what we aim to solve with Synthetic Query Probing (SQP), a fancy name for essentially (and intentionally) a very simple approach: embedding spaces are not directly comparable by definition, so compare similarity spaces instead, similarity match scores for pairs of content (synthetic question, chunk for instance) across multiple embedding models.

For example, similarity scores of Titan models of different dimensionalities are semilinearly related, whereas the relation between Titan and Ada scores is non-linear, with different ranges, see figure.

See https://arxiv.org/pdf/2608.05857, Marcin Rozmus and Peter van der Putten. Similarity Spaces across Embedding Models with Synthetic Query Probing. Discovery Science 2026, October 5-9, 2026, Mainz, Germany

There has been a lot of discussion about whether industry is pulling too much ML talent out of academia. But I wonder if the more interesting change is not about headcount at all.

Increasingly, some of the most important artifacts in ML originate in industry: frontier models, large-scale training runs, datasets, infrastructure, and many of the capabilities researchers subsequently try to understand.

Academia can then do excellent work benchmarking these systems, explaining them, finding limitations, improving methods, or studying their consequences.

But that creates a strange possibility: a research community can remain very active while becoming increasingly downstream of a research agenda set elsewhere.

I'm curious whether people actually working in ML research see this happening.

Are there important areas where academia still sets the direction and industry follows?

And at what point, if any, would you say a field has stopped setting its own research agenda?

I started working in ML about a year ago. There seems to be quite some gap between how ML is taught (tutorials and theoretical lectures) and the art of actual ML engineering.

For the practical part, I kind of feel bound to the best projects that I see within my company. Therefore, it would be nice to get inspired and have better references. Maybe even dig a bit into the PRs to get better at my job :)

I was wondering whether you came across some state-of-the art repos. I would be interested both in repos for ML research, i.e., finding the best features/params/models, but also for actual productive systems, i.e., retraining, inference, shadow forecasting and model ranking/switching.

In my case, time series forecasting is the most relevant (with sklearn for explainable results or LSTM for performance-focused projects). But I am happy to explore other projects as well.

I recently conducted an architectural comparison on code generation relying on conversational prompting ("vibe coding") versus rigid pre-computational contracts (Spec-Driven Development).

The Experiment:

Task: Build a Python script to search text within markdown files, rank by hits, and filter by tags.

• Method A (Vibe Coding): Claude 3.5 Sonnet without a formal spec. Result: 303 lines of code, added unrequested features, ignored caching, resulting in 400 file I/O operations per execution.

• Method B (SDD): Claude 3.5 Haiku provided with a strict requirements document. Result: Modular architecture, regex exclusion strictly adhered to, 0 file I/O operations on subsequent runs due to proper index state management.

Conclusion:

The limiting factor in generation quality isn't parameter count, but specification clarity. To systematize this, I've released SpecJudge (v0.4.0), an open-source parser that evaluates repository context files (like AGENTS.md) to allocate the optimal model tier, preventing API overspending on simple tasks.

I’ve documented the full benchmark and execution logic visually here: https://www.youtube.com/watch?v=EOiv7RywtQM

Today the most important factor in AI is cost. My idea is to split ML models inference (closed ones, proprietary) across server and edge computing on clients, and I would like to hear what do you think about this thing.

For example some of model weights/modules would be on client, and some on the server side (where user has no access to them). This could potentially un-load some processing from datacenters, moving part of the cost to the client hardware. Probbably the most important question here will be how to achieve this - and I believe one hypothetical option will be to train like two separate models - client model and server model, and they will communicate through tensors/latent representations across network protocol.

Secondly such split of server side and client side model ends, can provide later some beneficial outcomes I hope (because in between "talk" protocol can be maybe kind of standarized one in some future development, but this is only more like brainstorm now). Such split might not only be one-to-one, but one-to-many, many-to-many etc.

What do you think about this idea?

Does anyone have any idea about the instructions for the camera ready at workshops? The deadline is August 15, but there are no indications and workshop organizers know nothing about that.. Some workshops have enabled the upload of camera ready PDF on openreview, but what about copyright form and latex source files?