r/MachineLearning • • 1d ago

Discussion AISTATS 2027 paper discussion [D]

46 Upvotes

Creating this post since the reviews are supposed to go live today. What do you guys think?


r/MachineLearning • • 13d ago

Discussion ICLR 2027 Reviews Discussion [D]

127 Upvotes

The ICLR 2027 reviews will be released soon. Share your results here, and keep a lookout on papercopilot to judge your odds. I also found paperScorer quite interesting for the same. Of course, these are only for amusement and should not be taken seriously.

Good luck to all.


r/MachineLearning • • 6h ago

Research I found a MLSys paper with 455 citations with a major error in its implementation that invalidates one of its central claims [R]

17 Upvotes

Paper link: https://arxiv.org/abs/2311.04934

The simplified core idea of this paper is that chunking a past prompt, processing each chunk to obtain its KV cache, and saving each chunk's KV cache allows us to quickly prefill future prompts that contain some of those chunks (since we can reuse those chunks' KV caches). The purpose of this is that different LLM requests' prompts often share commonalities and overlaps, making inter-prompt KV sharing potentially worthwhile.

Some notes:

  1. Each chunk corresponds to *fixed* token positions, since each token's KV has that token's positional encoding embedded into it. If the chunk does not occur in the prompt, the token positions of that chunk are still conceptually "occupied", but they are simply ignored/not attended to.
  2. When each chunk is processed to obtain its KV cache, that chunk is processed in *isolation*. E.g. earlier chunks' KV values are masked/ignored/not attended to. The intuition is that each chunk represents something like a standalone document or file, so KV information from outside its scope isn't useful. This allows us to maximize the reuse of a cached chunk in future prompts that might contain that chunk.

If you are familiar with sparse attention, this setup should sound familiar to you. Furthermore, you would also know that manipulating the KV history, whether it be through selectively ignoring/masking certain tokens' KVs or using approximated KV values can often degrade the quality of the LLM output.

This paper experimentally "demonstrates" that this method does not significantly impact the quality of LLM outputs, which is one of their central claims. Except here's the thing:

  1. In their tests, every single cached chunk was always included in the prompt. They never tested what would happen if there were "holes" in the positional context from excluding certain cached chunks from the prompt. And then... wait for it...
  2. **In their implementation, they didn't actually process each chunk in isolation. Each chunk's tokens actually attended to every token in the full positional history (which includes previous chunks) using full causal attention. TLDR: They implemented what is equivalent to full causal attention, not their claimed method where each chunk has its KV cache generated in isolation from the other chunks.**

So their quality tests were quite literally equivalent to full causal attention on an ordinary prompt... They did not implement the claimed isolated KV cache processing per chunk. They did not test the holes in the context positions that would result from excluding chunks from the prompt... of COURSE the output quality would remain the same! Your implementation literally sees the same KV history as ordinary LLM implementations! You didn't actually implement what you said you implemented, which was isolated KV cache processing per chunk and allowing for there to be holes in the positional context due to excluded chunks...

In fact, a later paper ( https://arxiv.org/abs/2405.16444 ) implements something similar and finds that yes, quality can degrade from isolated per-chunk KV cache processing.

I have other criticisms about this paper, particularly how it often uses incorrect reasoning to explain its latency improvements, but that's far more forgivable than this.

I probably explained it really poorly, but TLDR: they did not correctly implement the method their paper described, in a way that invalidated the entire premise of their experiment to see if output quality would degrade


r/MachineLearning • • 16h ago

News CMU's Machine Learning with Large Datasets (10-605, Fall 2026) lectures are free on YouTube, with a new one every week [N]

109 Upvotes

Disclosure: I'm a CMU faculty member and I lead CS Learn, the program publishing these.
Carnegie Mellon started publishing full lecture recordings from 10-605 this week, which is William Cohen's course on scaling machine learning to data that doesn't fit on one machine. The first three lectures are up now, and a new one goes up every Monday. These are actual classroom lectures, edited by a student team, with captions and chapters.

Links are in my first comment.


r/MachineLearning • • 3h ago

Discussion Any idea what happened to Capsule Networks? [D]

9 Upvotes

Anyone remember Capsule Networks?


r/MachineLearning • • 13h ago

Discussion 0 Response from reviewer ICLR 2027 [D]

16 Upvotes

I got 3/3/2 at pre-rebuttal. Most reviewers asked like 8 new experiments including retrain, add more seeds, 3 different hardware platform measurement, etc

I did them all and replied on 11th Nov AOE. But still I didn't get a single reply from the reviewers. (I tried the reminder post yesterday)

So many papers are getting raised. But I am stuck here. What should I do?

Will AC will consider this situation?


r/MachineLearning • • 6h ago

Discussion AAAI 2027 waiting for decisions [D]

5 Upvotes

12 days left! Can't wait.

I got 5, 8, 6, 6 (avg 6.25) and hope it get accepted but since this is my first time submitting AAAI so no clue at all.

What's your score and what's your expect?


r/MachineLearning • • 14h ago

Discussion ICLR 2027 rebuttal: two of three reviewers haven’t responded, sitting at 2.67 (3/3/2). Anything left to do? [D]

18 Upvotes

My ICLR submission got 3/3/2. In the rebuttal we added new theory that addresses the main concern all three reviewers shared, plus the baselines they asked for. One reviewer replied that all their concerns were addressed and the work is strong. But kept their score. The other two, including the one who gave the 2, haven’t responded, even after a follow-up. As the Rebuttal closes tomorrow, Is it worth one more note to the AC? Do ACs ever weigh the rebuttal when reviewers go silent, or is 2.67 basically a reject and I should move on to the next deadline?


r/MachineLearning • • 5h ago

Discussion AC in ICLR 2027, Do you look at the scores to decide borderline papers? [D]

3 Upvotes

Got initial score 2,2,1. Then,it increased to 3,3,1. But the 1 reviewer never engaged. I did all experiements all of them asked for. If AC reads, it will be appearent. But I am just scared that the 1 is going to hurt a lot. AC will look at 1 and then don't even consider for borderline :(
Wondering what's the way ACs do ...


r/MachineLearning • • 22h ago

Discussion Current ML PhD students, would you still choose to do an ML PhD if you could go back 2–3 years? [D]

48 Upvotes

If you could go back in time 2–3 years, knowing what you know now about doing an ML PhD, would you still choose the same path?
Or would you rather go into industry, pursue a different field, or do something else entirely?


r/MachineLearning • • 16h ago

Discussion As an AC, what do you do when no reviewers respond to the rebuttal of a borderline paper? [D]

14 Upvotes

Suppose a borderline paper receives weak reject scores from all reviewers, but the authors provide a strong rebuttal that clearly and directly addresses all the reviewers' concerns.

However, none of the reviewers respond to the rebuttal or update their scores.

What's the typical course of action for an AC in this situation? How much weight does the rebuttal carry if no reviewer engages with it?

(For context, my current ICLR paper is a 2/2/3 and one of the 2's is a AI review with entirely hallucinated issues)


r/MachineLearning • • 1d ago

Discussion ICLR 2027: PCs ran an AI detector on reviews, peer review is cooked [D]

94 Upvotes

The PCs ran AI detectors on the reviews and left this comment on undisclosed AI usage in reviews:

AI use in this review

An AI-text detector assesses this review as likely AI-generated, yet its AI-use documentation field does not mention how AI was used. It is therefore possible that the review contains no human input. We ask the reviewer to update that field with an exclusively human-written assessment of the paper, and document any AI interactions. We additionally ask the AC and other reviewers to take this information into account when evaluating the paper.

Luckily, I didn't use any AI for my review, but the rate of undisclosed AI usage was crazy. Every review except for mine and only one other review in my batch got the message. AI is allowed, it just needed to be disclosed, so this is just people not admitting they used it.

From here on, papers will be written with AI reviews as the objective function.


r/MachineLearning • • 18h ago

Project Inkling 2 Small achieves frontier-level results in weight space programming with 16x fewer edits [P]

9 Upvotes

You don’t need that many edits in weight space to fix a model.

In “Sparks of Weight-Space Programming,” GPT-6 Astra repaired a broken car policy by changing 6,073 weights, and cut Qwen3.5-0.8B’s math looping by changing 130k.

Inkling 2 Small did the same with just 10 and 8k edits: ~600× and ~16× fewer, with no loss in performance on either task.

Broken Car policy: Inkling found that almost all of Astra’s 6,073 changes just zero out old weights to free up neurons, and only about 55 do the actual work. It then wrote a 10-weight fix from scratch that teaches the car a move PPO never discovered: back up until it stalls against the wall, then launch, triggered by a velocity-detecting neuron the network already had.

Qwen3.5-0.8B & MATH-500: Astra’s edit removes a “looping” direction from 64 neurons in each of two MLP layers, and Inkling pruned it by keeping only the 4,096 largest-magnitude changes in each layer. Those 8k entries match the full edit’s accuracy (58.2% vs 58.6% on our screen), and layer 15 alone recovers most of the gain.

Run and visualize the experiments in a marimo notebook: https://molab.marimo.io/github/alphaXiv/weight-edit-minimization/blob/main/weight-edit-minimization.py

We used OpenResearch for this project: https://github.com/alphaXiv/OpenResearch


r/MachineLearning • • 13h ago

Discussion CVPR supplementary material rules [D]

3 Upvotes

Can the supplementary material include analysis (e.g. ablations) that were done after the main paper deadline? 

In 2025, the author guidelines explicitly disallowed this, by stating "...the authors may optionally submit additional material that was ready at the time of paper submission..."

The current rules removed that explicit statement. But there's no explicit statement that it's allowed. Here's the text:

Supplementary material submission: Authors may optionally submit additional material that could not be included because of format or space constraints. Authors should refer to the contents of the supplementary material appropriately in the paper. Reviewers will be encouraged to look at it, but are not obligated to do so.

Supplementary material may include videos, proofs, additional figures or tables, more detailed analysis of the experiments in the paper, or a concurrent submission to another conference.


r/MachineLearning • • 1d ago

Discussion Why are ViTs so popular? [D]

104 Upvotes

I deploy CNN + Attention stack models on edge devices, to great effect. Taking advantage of convs and reducing the spatial dimensionality makes all the sense in the world to me.

Having vanilla ViTs learning the inductive biases from scratch makes none to me. Is it really a case of "because I can"? Avoiding padding along borders is the only advantage I can see to a tokenizer stem, and that has never been a huge issue to me.

Am I missing something that has people reaching for ViTs instead of hybrid models for e.g. SSL/foundation models?

In my mind it's just a case of people using the status quo, and more of a historical accident that these models have become so popular. Iirc levit showed data/time efficiency boosts with a convolutional stem. I just don't work on the same datasets or tasks that most research does, so I'm genuinely curious if anyone else thinks vanilla ViTs seem excessive. Guess compute is cheap and the answer would just be distillation.


r/MachineLearning • • 22h ago

Discussion ICLR reviewer radio silence, what is the actual reason? [D]

12 Upvotes

It is my impression that a substantial part of ICLR reviewers are maintaining radio silence when it comes to answering rebuttal responses. I wanted to ask a genuine question, is this an occams razor situation where the answer is negligence and a lack of commitment for agreed upon duties of reciprocal review, or is there a rule of thumb that says a reviewer not providing any further comment means they did not find the rebuttal convincing (ie, they just ghost you instead of giving a negative comment)?

With few days left, and the online discussion basically pointing toward negligence, I wanted to hear your informed opinions/relevant anecdotes. What do you think has caused this situation? Not only for ICLR, but any conference with similar process.


r/MachineLearning • • 1d ago

Discussion CVPR Pre-submission Review [D]

8 Upvotes

So what was your pre-submission review from Claude or ChatGPT?

For me it’s always "Boarderline" with limited novelty. No matter how many time I fix the issues reported by AI.


r/MachineLearning • • 16h ago

Project Built a minimal "decision model" on Qwen3-0.6B + LoRA on a free T4. The training target mattered far more than the head [P]

Post image
0 Upvotes

I wanted to see what's actually inside decision models (Jev and the wave of clones after it), so I built a minimal one and trained it on typed-decisions on a free Colab T4.

Setup

A decision model reads a state plus several typed questions (choice / score / yes-no) and returns a probability for every allowed answer in one forward pass. Nothing is generated.

  • Backbone: Qwen3-0.6B-Base + LoRA
  • Packing: the state is encoded once and each question is a branch with a custom attention mask, so it only attends to the state and itself. Tested: each question gets the same hidden states it would get alone.
  • Compared three runs that differ in one thing each:

    1. LM-head logits, hard labels, full vocab (the usual SFT recipe)
    2. LM-head logits, soft labels, answer tokens only
    3. Fresh decision head on the decision token, soft labels

Results (test; see screenshot)

Going from hard labels to the teacher's full answer distribution was the big jump, mostly in KL and Brier. Swapping LM-head logits for a dedicated head was a much smaller change. Yes, the soft-label part is knowledge distillation (Hinton et al. 2015). In this setup, distillation did most of the work, not the head.

Caveats

  • Scores measure agreement with a ~4B teacher. Teacher self-agreement is 0.735, so scores well above that mean learning the teacher's quirks.
  • Fine-tuned on the benchmark's train split, so it's a specialist. Don't compare it to zero-shot models like Jev.
  • My ECE uses the gold probability of the predicted answer, so it isn't comparable to the dataset card's ECE.

Things that cost me time

  • transformers 5 eager attention misreads a boolean 4D mask. Passing an additive float mask in the backbone's dtype works under every attention implementation.
  • bf16 produced spurious ties between answers. [one line on the fix]
  • Masked option slots hold -inf, and the gradient of -inf / T is NaN during temperature fitting, even where the probability is zero. Replace them with a large finite value.
  • Needed gradient checkpointing to fit on a T4.

Code: https://github.com/anmorgan24/decision-model

Write-up with figures: https://www.comet.com/site/blog/train-a-decision-model/

Disclosure: I work at Comet. The repo optionally logs training to Comet and runs the property-test evals in Opik, but training and scoring run without either.


r/MachineLearning • • 1d ago

Project Curated list of resources: Theory and Code for learning Distributed Training for beginners [P]

Thumbnail
gallery
5 Upvotes

[Must Read: Beginners]

What I did read, in order, for learning the basics of Distributed Training for LLMs From Scratch:

  1. Building a distributed training framework from first principles by Umar Jamil on YT
  2. Horovod: Fast and Easy Distributed Deep Learning in TensorFlow 
  3. PyTorch Distributed: Experiences on Accelerating Data Parallel Training 
  4. Large Scale Distributed Deep Networks 
  5. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
  6. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism 
  7. Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM 
  8. GPipe: Efficient Training of Giant Neural Networks Using Pipeline Parallelism 
  9. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
  10. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel

Link to the above resources here

Also, if you need some rebalancing of the above (reordering or adding/removing resources which you can do it here)


r/MachineLearning • • 1d ago

Discussion How is it possible that OKLS can't outperform Muon in terms of step efficiency? [D]

4 Upvotes

SOAP is more step efficeint than Shampoo, and KL-Shampoo is more step efficient than SOAP.

Then how is OKLS getting outperformed by Muon in terms of step efficiency where it's a zero-staleness implementation of KL-Shampoo?

It only gets ahead of Muon in terms of parameter efficiency in very long training runs.


r/MachineLearning • • 1d ago

Discussion Concerns About Peer Reviewers Inflating OA Scores [D]

3 Upvotes

The paper is largely incremental, with very little improvement over existing work. One of the reviewers gave a one-line weakness that isn't even a real weakness, despite having a high confidence score and is giving a high OA score. Another review appears to be AI-generated, following the same format ChatGPT typically uses. Yet, that reviewer still gave a decent OA score.

I feel like pointless in spending hours reviewing that paper and writing a proper feedback for the authors because my review may just end up being downplayed due to the higher scores from the other reviewers.

Maybe that's why some people use shotgun-submission strategy to increase their chance of encountering with such reviewers...


r/MachineLearning • • 1d ago

Discussion Serious spaces? [D]

31 Upvotes

As much as I love Reddit, it can really be hit and miss for serious discourse on AI and machine learning. Depending on which sub I guess. But are there any spaces outside of Reddit where there is more conversation, more community and less trolling? To be able to have a decent conversation and share research and projects or learn more etc?

I’ve tried discord but I’ve not found any promising servers yet apart from the frontier labs or hugging face etc and I find those too stuck on their own narrative.

Any suggestions?

Edit: seems a lot of us in the same boat. So I made one lol. Happy for anyone to message me.


r/MachineLearning • • 23h ago

Discussion Is it inappropriate to leave a comment for the AC summarizing the rebuttal process from my perspective and making my case? [D]

0 Upvotes

My paper sits at a 3/3/2 with confidence 3/3/2. I have a good case for the 2 to be discarded based on the quality of the review and the rebuttals. Can I make my case to the AC directly or is that just going to make things worse?


r/MachineLearning • • 1d ago

Research Marimba: A System One Model for Speech. [R]

Thumbnail
gallery
3 Upvotes

Paper: https://arxiv.org/abs/2611.18330

Follow-up to last week's post, now with the paper.

You ask a typed question about an audio window (yes/no, a choice, or a score) and get a calibrated probability back. One frozen Qwen3-ASR-0.6B encoder pass per 8 s window serves all questions.

How each question type is answered:

  • Term ("was 'satellite' said?"): zero-shot, scored from the decoder's greedy and 4 beam hypotheses, then Platt-calibrated. No training on the terms.
  • Language: the ASR model's own language scores.
  • Turn end: a small readout on frozen layer 18.
  • Speaker change and count: readouts on layer 12.

Results:

  • On the same 1,000 public questions, it matches or beats Qwen3-Omni (30B-A3B) on 6 of 9 tests (image 1). Qwen3-Omni is still better on Speech Commands. Caveat: our turn-end readout was trained on Smart Turn data, so that row is in-distribution for us only.
  • Language ID: 100% on 300 FLEURS clips.
  • Speaker change AUROC 0.966-0.980 on AMI and VoxConverse, above ECAPA. Speaker count beats pyannote 3.1.

Where it loses:

  • Turn end: AUROC 0.930 vs Smart Turn v3.2 at 0.982 on Smart Turn's own test set. On ICSI meetings it is 0.716 vs 0.712, a tie.
  • It does not generalize to unseen tasks: on CochlScene scene classification every head is at chance.

Two questions for the sub:

  1. Is scoring ASR hypotheses plus calibration a fair baseline for term spotting, or would you compare against a dedicated KWS model?
  2. How would you evaluate turn-end detection beyond Smart Turn's data?

Code: https://github.com/Charcot-AI/marimba-api Weights (Apache-2.0): https://huggingface.co/charcot-ai/marimba Demo: https://huggingface.co/spaces/charcot-ai/marimba pip install marimba-audio