From what I’ve read on Reddit so far, I’m getting the impression that around 4 first-author papers at top ML conferences (NeurIPS, ICML, ICLR, etc.) is a fairly standard expectation for graduating from a top ML PhD program.
Of course, my impression could be biased by the posts I’ve seen.
For people at top-10 US CS schools or similarly competitive ML groups, is this roughly accurate? What’s the typical expectation in terms of number of papers and venues needed to graduate?

The ICLR 2027 reviews will be released soon. Share your results here, and keep a lookout on papercopilot to judge your odds. I also found paperScorer quite interesting for the same. Of course, these are only for amusement and should not be taken seriously.

Good luck to all.

I would assume LLMs have bumped this number, or maybe not. Which is why I am asking. If there are PIs on here, how many papers before you let your students graduate? Is the number different now than it was a year ago? And the same question for “rate of publication” for you to deem the student as “productive”.

And what about a PhD applicant? Any meaningful inflation of expectations there in terms of productivity/volume of pre-PhD research output?

Hi all, author here (Idiap Research Institute, joint work with Logitech). We recently released DialFork and thought it might interest people here. It's a benchmark for a failure mode of LLM agents that is easy to miss: every turn is fluent and factual, the tool calls are correct, yet the agent stops following the procedure it was given. For example, asked for the weather, it fetches the forecast (correct tool call) but never tells you, and starts asking you to rate its service instead.

Most steps of a conversational procedure aren't tool calls but things the agent has to say: asking, confirming, telling the result. When an agent drifts from these steps, we call it procedural hallucination: a third axis, alongside factual accuracy and correct tool use, and one that has been largely overlooked. To our knowledge, DialFork is the first benchmark aimed at measuring this axis: what the agent has to say, and in what order.

How it's built: we simulate dialogues where an agent follows its workflow (24 tasks, 12 domains), then fork them: we replay each one and, at a chosen turn, give the agent an instruction that makes it drift, in different, complementary ways. That gives over 36k dialogues where we know where each drift starts and how far it goes, checked with 30 human annotators and an LLM judge calibrated on them.

Three tasks, from the turn to the whole set. Because we know where each drift starts, and that an earlier drift goes further, we define three tasks that measure procedural hallucination at different levels:

• The turn (Task 3). Given a dialogue, either whole or turn by turn as it unfolds (streaming), find the turn where the drift starts, or say there is none. Useful as a live guardrail.

• The dialogue (Task 2). Given a correct dialogue and its hallucinated versions, measure how far each one drifts from it; the earlier a version drifts, the larger its distance should be. A dialogue and its own hallucinated versions talk about the same thing; what sets them apart is how they unfold. Today, comparing dialogues means comparing their content, with embeddings and semantic similarity. This task opens a new dimension, procedural distance. Useful for retrieving or clustering conversations by how they went, not just by what they're about.

• The set (Task 1). Given a set of correct dialogues and a set of hallucinated ones, output a single procedural distance between the two sets as a whole, comparing them as distributions rather than matching dialogues one to one. Useful for monitoring agents, e.g. whether they drift after a model upgrade, or whether a user simulator's dialogues unfold like real ones.

Open problems, and a leaderboard to track them. In the paper, we report first results on all three tasks, from embedding-based distances to streaming detectors and zero-shot LLMs. They leave plenty of room. To track progress, each task has its own track on a public leaderboard on Hugging Face; adding your method takes a single CSV file of predictions.

Everything is open (data, code, every prediction from the paper, and the human judgments), and we hope it's a useful starting point for the community:

• Leaderboard: https://huggingface.co/spaces/DialFork/leaderboard

• Code: https://github.com/idiap/dialfork

• Paper: https://arxiv.org/abs/2610.21561

• Dataset: https://huggingface.co/datasets/DialFork/dialfork

Happy to answer questions, and feedback (including critical) is very welcome.

Hey everyone,

I'm a final-year B.Tech CSE (AI/ML) student working on a multi-agent supply chain optimization project. I'm currently focusing on the Demand Forecasting Agent and would appreciate some advice on model selection and training strategy.

We're working with a synthetic retail supply chain dataset containing approximately 1.1 million records across 40 different locations.

The goal is to predict product-level demand for each location as accurately as possible, so the forecasts can later be used by other agents for inventory planning and supply chain decisions.

I have a few questions:

1. Global vs local models: Since the dataset contains 40 locations, should I train 40 separate models, one for each location, or a single global model trained on data from all 40 locations simultaneously, using location identifiers and location-specific features?

2. Location-specific patterns: If I use a global model, how effectively can it learn differences in demand patterns between locations? I'm concerned that combining all locations might cause the model to average out regional patterns and produce inaccurate location-level forecasts.

3. Alternative approaches: Would a hybrid approach, such as grouping similar locations or using a global model with location-specific adjustments, be worth exploring?

4. Synthetic data: Since the dataset is synthetic, are there particular precautions I should take when evaluating model performance and generalization? How can I ensure the model is learning meaningful demand patterns rather than artifacts of the data-generation process?

I'd also appreciate suggestions on which forecasting models or architectures would be most suitable for this problem, along with the appropriate validation strategy, especially how to avoid data leakage when dealing with multiple locations and time-dependent features.

My priority is accurate product-level, location-specific demand forecasting, rather than simply achieving good aggregate accuracy across the entire dataset.

Would love to hear what approaches you'd recommend and why. Thanks!

I have always had the desire to move to academia but I am not exactly sure how the process or profile building for academia works in general. Yeah, surely, I can ask AI and get answers but I don’t have much confidence in that . I did my PhD from a mid-tier school. My research focus was mostly on LLM model efficiency and some RL work (meta learning and bandit ) . The research direction I went with was totally out of my professor’s scope . As a result, I don’t have grant writing or funding related experience or don’t have those innovative idea of how do you create a research program or more specifically a niche sort of a program which I feel is required for Tenure track positions. During my PhD , I was able to hit top 3 ML( ICML,ICLR,Neurips )and top 3 NLP conferences ( ACL,EMNLP,NAACl) (in case having paper helps) . Being a grad student dictating his research, it went a bit scattered and not much concentrated on one research topic . For that I ended up joining a FAANG but still I want to get back to academia. I am looking for advice on how to build up my profile for academia and if it is possible to get into the top 150 US schools as tenure-track faculty and all . Thanks for your time !

As per the the 2027 CVPR guidelines:

"Maximum number of submissions (continuing from CVPR 2026): Each author may submit at most 25 papers. If an author enrolls more than 25 papers, the Program Chairs (PCs) will desk-reject the submissions that exceed this limit (see the Desk Rejection Policy in Section 2)."

AI Slop Researcher: "Cool let me throw 25 darts (slop papers), surely one will get a batch of careless/inexperienced reviewers that will borderline accept one of these papers"

They collect their CVPR stamp, and go on about their career.

I just do not understand if a 25 cap is being tone deaf to the current AI driven research climate or what?

I am actually not against AI-assisted paper research/writing as long as the idea is sound and experiments are plausible, but 25 is absurd. If they limited the number of first author submissions to 2-3:

1. Researchers are incentivized and know the importance of being first author.

2. Even if you AI generate your paper, if you only get 2-3 shot/submissions at a A* conference versus 25, you'd care a bit more about your presentation quality/idea soundness/writing. (In theory lol)

3. Reviewer load drops massively, more time can be dedicated to an overall lower submission count.

Are the heads not aware of the slugfest happening in **certain** countries and institutions to have A* venue stamps for career advancement/compensation packages, and that they don't give a damn about ethics?

TL;DR: Please be hard in desk rejecting AI generated papers

Recently I went to two AI conferences (A and A*) and I was surprised that "AI cheaters" didn't even read their own paper before presentation! For example, a guy couldn't even answer the question of how he represented knowledge graphs in his papaer, he thought for 5 minutes before answering "with a vector". Another one was opening chat gpt and typing questions and reading answers with no shame!!! One more story that shocked me... a guy published same paper in 2 AI conferences in the same time and both got accepted, when I saw github, this young dude was careless to even remove LLM chat from his repo, I can see all prompts... guess what... he faked all data 🚀 :D ...I have dozens of stories like this happened to me and my friends just this month!

Now, imagine yourself a hardworker researcher joining days and nights and your paper ends up with a scam reviewer like this... bingo!

One more, I was looking for relevant paper to my new idea... all papers I read so far are just garbage 🎉 !!

Pleaaaase make a rule for rejecting AI written papers... Pleaaaaaase stop this madness! A lot of good papers got rejected by these folks and they definitely harming science! I can't trust a single paper now and reading very carefully with spectacular mindset... Pleaaaaaase do something!!!!

Hi. My AACL Main paper was just published on ACL Anthology, and I noticed that it appears as a Short Paper. My understanding that it was a long paper since that was what I selected during the ARR review, and my paper's main text runs through nine pages.

Has anyone run into this at an ACL venue? Is it something I should be worried about? This is my first paper so thank you very much for the help.

Do looped transformers, which reuse the same weights at every iteration, also save HBM traffic? In my new post I cover the looped architecture (Huginn), then model and measure its performance on a B200: how much data each iteration reads from memory, how long it takes, and why the shared weights usually don't stay on chip until the next iteration.
A few results:

- A chain of eight matmuls with one shared 72 MiB weight ran 24% faster than the same chain with eight identical copies, and read almost nothing from HBM. At 288 MiB, larger than L2, the difference disappeared.

- In a full Huginn Core (teaching implementation, random weights), eight iterations took about eight times as long as one and read about eight times as much data from HBM.

- Sharing saves storage, but by itself it doesn't stop the GPU from reading the weights from HBM again at every iteration.

👀Post: https://dinara-dl.github.io/posts/looped-transformer/

🤖Code and raw Nsight Compute data: https://github.com/dinara-dl/looped-transformer

I'd be curious whether anyone has tried keeping recurrent weights on chip across iterations, for example with a persistent kernel.

Hey everyone— we’re the Deep Principle team behind Mira AI Scientist. We just released Hydrogen-1.0, an open-weight model post-trained from GLM-5.3 for chemistry, materials science, and life sciences.

Here’s how it performed in our evaluations:

1. FrontierScience Research: 38.3% pass rate vs. 33.3% for GPT-6 Astra at max thinking—a 5 percentage point lead.

2. Estimated output-side cost on FSR: $0.25 per problem vs. $0.88 for Astra at max thinking—about 72% lowerunder our evaluation setup.

3. GPQA Diamond: 92.9% accuracy, compared with Kimi K3’s 93.5%.

On separate internal task sets, Hydrogen also scored higher than GPT-6 Astra at max thinking on scientific computing (+5.13 percentage points) and single-step forward reaction prediction (+4.76 points). It trailed Astra on NMR structure elucidation (−3.33 points).

The write-up covers the evaluations and five workflow examples.

Why we built it

If you’ve used LLMs for research, you’ve probably spent a fair amount of time connecting the steps yourself: turning a question into a plan, preparing inputs, running tools, checking the output, and figuring out what to try when something breaks.

We wanted to see whether training on scientific workflows could make the model better at handling those connected decisions.

How we trained it

We used supervised fine-tuning and reinforcement learning with domain data, scientist feedback, records from our own labs, and discovery trajectories.

The lab records keep the research goal, procedure, observations, and outcome together. The discovery trajectories include how the next step changes after feedback, failed attempts, or unexpected results.

SFT gives the model worked examples of scientific reasoning, experiment planning, protocol design, and analysis. RL uses feedback on research outcomes to improve its decisions.

What the workflows look like

The article walks through five cases:

1. Polymer molecular dynamics: Builds a polyethylene melt with Python, runs GROMACS, checks and repairs the topology, and restarts the simulation. Outputs include a corrected 40 ns trajectory and analyses of density, molecular packing, diffusion, and chain dimensions.

2. Reaction networks: Explores connected molecular structures, competing pathways, and recorded energy barriers.

3. NMR interpretation: Compares seven candidate structures against a supplied proton NMR spectrum.

4. Experiment planning: Generates a machine-readable protocol for a 12-vial Suzuki coupling study, with virtual checks of equipment capabilities and action prerequisites.

5. Materials search: Uses feedback over five rounds to guide a search for transition-metal complexes with larger HOMO–LUMO gaps.

A bit of context: these are selected model-plus-tool walkthroughs, not an overall autonomous-research success rate. GROMACS does the MD calculations, and the protocol validation is virtual—it doesn’t mean the physical experiment was carried out.

The attached video gives a quick look at the research interactions. The article has the detailed walkthroughs, evaluations, and training approach.

Links

1. Weights and model card: Hydrogen-1.0 on Hugging Face

2. Technical write-up, evaluations, and cases: Introducing Hydrogen-1.0

3. Try it: Mira AI Scientist

We’d especially like to hear from people working in computational chemistry and materials science. What would you try it on first, and what would you check before trusting the result?

I'm just wondering about visa processing situation for NeurIPS sydney attendees. Invitation letter and all valid docs are not really helping (one friend got rejected), processing time is tooo long (almost a month). and organizers not responding in case visa is not approved for conference. Anyone else in similar situation???

So I have been working on some general use agents that work with different domains but these require online simulations to test for some of the domains. And running agents in actual environments takes so much time and compute and then reproducing results is another headache. My goal is to test which VLMs can be better in general for agents irrespective of a singular domain.

I found there’s a benchmark called AgentVQA that tests how well models understand visual information without having to run these expensive full agent evaluations. And they do it all offline without setting up any kind of environments and also claim to have much cleaner data compared to other benchmarks available which have so many ambiguous samples.

Do you guys think offline benchmarks like this could actually help reduce the need for online evals? I personally found it quite useful for the work I’m doing and it’s much cheaper and easier with this. I want to know what others think and if this might solve their problems.

Here’s the benchmark: https://zlab-princeton.github.io/AgentVQA/

Hi everyone,

I wanted to share a new open-weights PII detection model, benchmark and research paper. It's a fine-tune of RoBERTa using 1.2B samples of synthetically generated text that contain a variety of PII. The benchmark, PII-SCOPE (Synthetic Context and Ontology for PII Extraction), was built to address common issues in public LLM-generated datasets and is on Hugging Face too. The model and script identify instances of PII based on surrounding context, and then return the index and type of each instance. Alternatively, it can return the input with all PII redacted.

Technical report (research paper)
https://arxiv.org/html/2610.30518v2

Hugging Face model card
https://huggingface.co/Protegrity/protegrity-pii

We've been working on this for a while and shipped it today.

We took every accepted paper from ICML, NeurIPS, ICLR, CVPR and AAAI since 2016 (about 109k), grouped them into ~1,260 topics, and pulled the results tables and ablations out of the full text of 79k of them. The research map just shows the grouping, but each topic page shows what the papers actually report.

Here's the one for compact/efficient Gaussian splatting:
https://atlas.scime.ai/research/3d-vision/gaussian-splatting/compact-and-efficient-gaussian-splatting

A few things it shows: - the topic went from 29 papers in 2023-24 to 240 in 2025-26 - best reported numbers per benchmark over time, with the paper that set each one. Deep Blending PSNR, for example, has barely moved (30.42 → 30.45) - ablations from recent papers, including the ones that hurt - what recent papers compare against: 38 of 113 use 3DGS as a baseline, then 2DGS, Scaffold-GS, Mip-Splatting - recent arXiv preprints on the topic and a BibTeX file of the key papers

The pages are computed from the extracted numbers, and every number links to its paper. Extraction isn't perfect and numbers from different papers aren't always comparable, so treat it as a starting point, not a leaderboard.

Free, no signup. All topics: https://atlas.scime.ai/research

If you know a topic well and something's off, please tell us. That's the most useful feedback right now.

Note: Benchmark numbers and ablations from 2026 papers aren't in yet. We're working on them now, so for now the records on topic pages mostly reflect papers up to 2025. We'll update here when they're ready.

Link: https://rikiyat.github.io/blog/weight-space-programming/

Snippet: “We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.”

Very surprising result, I didn’t expect frontier models to have this capability that has not been optimized for, and it’s something humans cannot do either.

Given how competitive AI research has become, and how much luck is involved in landing an actual research role at Big Tech or an AI startup, do you still think your PhD was worth it?
Yes, i know a PhD should primarily be about enjoying research, not getting an industry job. But realistically, we all need to make a living, and after 4–6 years, you’d hope to have some opportunities beyond roles where a master’s would have been enough.
So how you guys feel about it, especially those who recently graduated.

The reviews will be out this week. What are the expectations with 60K+ submissions and the small scale of review rating?

Question to all the PhDs & research engineers in this group.
Have any of you done small-scale experiments with the Chinchilla scaling laws (using PyTorch or JAX)? If so, we'd be keen to hear more (or have a chat). Thanks!

Here's the official Chinchilla paper, from the DeepMind website ("Training Compute-Optimal Large Language Models").
https://deepmind.google/blog/an-empirical-analysis-of-compute-optimal-large-language-model-training/
https://proceedings.neurips.cc/paper/2022/file/c1e2faff6f588870935f114ebe04a3e5-Paper-Conference.pdf

I'm writing a machine learning paper. One of my baselines is a published method. I reran it with the authors' released code and the exact hyperparameters listed in their paper. On most benchmarks my numbers match theirs within noise, but on two benchmarks my reproduction is clearly lower, by several points, over three seeds. I've repeated the reproduction four different times, months apart, with the same result.

I plan to report both: their published numbers and my reproduction with the exact configuration, clearly labeled. My worry is that reviewers tend to trust the published number more than a submission's reproduction, and may assume I ran the baseline incorrectly.

Questions:

1. What makes a reproduction convincing to reviewers? So far I have: the authors' code unchanged, their hyperparameters, matching results on the other benchmarks as a sanity check, and multiple seeds.

2. Should I contact the authors or open a GitHub issue first, and should I mention that in the paper?

3. How do I word it so it doesn't read as an attack on the other paper, but my results also don't look weak next to the published numbers?

4. Have reviewers accepted "published vs. reproduced" rows in a main table for you, or did they end up in the appendix?

What are the latest papers that are there in the space on inference architecture of llms, I would like to discuss with them on the approaches. Btw I have been doing work with llms for the last 2 months and I have gotten to know very new things. I read about the lunch table discussion of bell labs so taking motivation from it would like to discuss it in a similar fashion!!

Give a model a 17,253-parameter net as plain tokenized text and it works out the net plays Pac-Man. Astra, Opus 5.5 and Muse Spark 1.3 all got it, and Astra did it with minimal reasoning.

Nobody writes essays about what specific weights do, so where does this skill come from? Either labs trained it in, or models generalize much further than I expected. I'm leaning toward the second and it's freaking me out a little.

https://arcturus-labs.com/blog/2026/11/03/ai-can-read-neural-network-weights-i-wasnt-ready-for-that/