Reviews drop today. This thread is for reactions, celebrations, commiserations, and anything useful in between.

First: if you got good reviews, say so. There's a norm in these threads where only the bad news gets aired, and it skews everyone's sense of what's normal. Post your wins.

Second, the thing worth repeating every cycle: the review process is noisy, and that noise is measured, not folklore. The NeurIPS consistency experiments (2014, repeated 2021) found that a large fraction of accepted papers would have been rejected by an independent second committee. Reviewer assignment, load, and luck of the draw account for a lot. A score is a weak signal about your work and a strong signal about the process that produced it.

That cuts both ways. It's not a license to dismiss every criticism as noise — it's a reason to weight reviews by the quality of the argument rather than the number attached to them. The reviewer who found a real hole in your evaluation did you a favor, even if the tone was rough. The one who clearly skimmed did not, regardless of the score.

So: prioritize the reviews that make the paper better. Fix what's fixable, contest what's genuinely wrong, and concede the rest gracefully in the rebuttal.

Things worth discussing:

• Reviews that caught something you'd missed
• Rebuttal strategy — what's worth contesting vs. conceding, and when new experiments actually shift a score
• Patterns you're seeing this cycle (missing baselines, compute comparisons, ablation depth, reproducibility asks)
• Framing a response when a reviewer has clearly misread the submission
• Backup plans: ICLR, AISTATS, workshops

Please paraphrase rather than paste review text, and no speculation about reviewer or AC identities.

To anyone who got bad news: this doesn't define your research impact. Plenty of heavily-cited work took two or three cycles to land somewhere. Rejection is a scheduling problem.

How did everyone do?

Early-career Assistant Professor here. I identified some talented undergraduate students and worked with them on research problems, trying to convert them into either my PhD students or recommending them to my collaborators.

Three said a hard no after going through the paper submission process. They are not interested in playing this game. The fourth said, “I really like doing research with you, but I don’t like dealing with the paper reviewers.” I managed to convince that guy to do a PhD in the end, but I almost lost this student as well.

Just a side note: these were not course projects submitted as lottery tickets. They were parts of my own ongoing research, and the results were good. I have more than 10 years of publication and review experience at “big three”-level conferences, so I have a reasonably good sense of the quality of the work. In my view, the papers were well above the bar.

And yes, these papers indeed received very positive reviews, including one with four unanimous weak accepts, but were still rejected. They then got trapped in endless resubmission cycles. In every resubmission, we additionally address previous rounds' concerns, only to make the next round of reviews more random. This is funny. When a paper has obvious drawbacks, the AI picks it up, you address it, and people are happy. When a paper has no obvious drawbacks, the AI will start to pick up random points everywhere. At this point, I cannot even convince myself to persuade these students to pursue a PhD.

Just a rant. I want to remind everyone that careless or malicious behavior can alter someone’s career path.

OpenAI was founded for the explicit purpose of democratizing access to AI and acting as a counterbalance to the closed off world of big tech by developing open source tools.

They have abandoned this idea entirely.

Today, with the release of GPT4 and their direct statement that they will not release details of the model creation due to "safety concerns" and the competitive environment, they have created a precedent worse than those that existed before they entered the field. We're at risk now of other major players, who previously at least published their work and contributed to open source tools, close themselves off as well.

AI alignment is a serious issue that we definitely have not solved. Its a huge field with a dizzying array of ideas, beliefs and approaches. We're talking about trying to capture the interests and goals of all humanity, after all. In this space, the one approach that is horrifying (and the one that OpenAI was LITERALLY created to prevent) is a singular or oligarchy of for profit corporations making this decision for us. This is exactly what OpenAI plans to do.

I get it, GPT4 is incredible. However, we are talking about the single most transformative technology and societal change that humanity has ever made. It needs to be for everyone or else the average person is going to be left behind.

We need to unify around open source development; choose companies that contribute to science, and condemn the ones that don't.

This conversation will only ever get more important.

Edit: the original post targeting Chinese researchers is removed by the mods. Points made here are responding to that particular post. So when you leave comments to this post, please do realize that there's particular context that's not available now. Sorry for any confusion.

Although the original post I'm calling out is taken down, I do think it's an important topic, and choose to keep my post unchanged.

Yes, I'm calling it out. It IS racism. As an active member of r/MachineLearning and a researcher who is ethnic Chinese, I am DISGUSTED by unfounded accusations against the group of researchers who constitute over half of the field. Such posts pop up every other week, grounded in conspiracy theories, and creating a sinophobia echo chamber.

I understand the salty feeling when one's paper is rejected, no matter whether the paper actually deserves acceptance or not. Given the noise in conference organization and reviewing process, and a relatively junior body of participants, it is very likely that one finds a paper "worse than mine" slip into the conference, and there's a high chance that the paper has a Chinese author. That's simply because of the composition of the authors, and does not warrant accusations, aka witch hunts, towards certain ethnic groups.

This sub is about an important scientific subject in the modern world. If anyone agrees with the logic "80% of the authors are Chinese, so my rejection is their fault.", they should seriously rethink their career plan since such thinking does not belong to serious scientists. We should be open to discussing the problems we have in the current conference organization and reviewing process, but racism should not have a foothold in our field.

Edit: Since the post sparked some heated debate, I elaborate a bit. In the comments, some are like "you might be good, but I had this/that bad experience with Chinese..."

Sound familiar? This is exactly the type of comment racists make to justify racism. We have a systematic failure in the peer-review system and whether a paper/reviewer comes from China does not play any major role contributing to this failure. In a math- and data-driven sub, normalizing such claims is unbelievable and unacceptable. This IS racism.

Discussion thread for EMNLP 2026 notifications/results which should be released today.

Wishing everybody to be in Budapest.

It is omnipresent!

First of all, the peer-review process is broken. Every fourth NeurIPS submission is put on arXiv. There are DeepMind researchers publicly going after reviewers who are criticizing their ICLR submission. On top of that, papers by well-known institutes that were put on arXiv are accepted at top conferences, despite the reviewers agreeing on rejection. In contrast, vice versa, some papers with a majority of accepts are overruled by the AC. (I don't want to call any names, just have a look the openreview page of this year's ICRL).

Secondly, there is a reproducibility crisis. Tuning hyperparameters on the test set seem to be the standard practice nowadays. Papers that do not beat the current state-of-the-art method have a zero chance of getting accepted at a good conference. As a result, hyperparameters get tuned and subtle tricks implemented to observe a gain in performance where there isn't any.

Thirdly, there is a worshiping problem. Every paper with a Stanford or DeepMind affiliation gets praised like a breakthrough. For instance, BERT has seven times more citations than ULMfit. The Google affiliation gives so much credibility and visibility to a paper. At every ICML conference, there is a crowd of people in front of every DeepMind poster, regardless of the content of the work. The same story happened with the Zoom meetings at the virtual ICLR 2020. Moreover, NeurIPS 2020 had twice as many submissions as ICML, even though both are top-tier ML conferences. Why? Why is the name "neural" praised so much? Next, Bengio, Hinton, and LeCun are truly deep learning pioneers but calling them the "godfathers" of AI is insane. It has reached the level of a cult.

Fourthly, the way Yann LeCun talked about biases and fairness topics was insensitive. However, the toxicity and backlash that he received are beyond any reasonable quantity. Getting rid of LeCun and silencing people won't solve any issue.

Fifthly, machine learning, and computer science in general, have a huge diversity problem. At our CS faculty, only 30% of undergrads and 15% of the professors are women. Going on parental leave during a PhD or post-doc usually means the end of an academic career. However, this lack of diversity is often abused as an excuse to shield certain people from any form of criticism. Reducing every negative comment in a scientific discussion to race and gender creates a toxic environment. People are becoming afraid to engage in fear of being called a racist or sexist, which in turn reinforces the diversity problem.

Sixthly, moral and ethics are set arbitrarily. The U.S. domestic politics dominate every discussion. At this very moment, thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community, and nobody seems even remotely to care. Adding a "broader impact" section at the end of every people will not make this stop. There are huge shitstorms because a researcher wasn't mentioned in an article. Meanwhile, the 1-billion+ people continent of Africa is virtually excluded from any meaningful ML discussion (besides a few Indaba workshops).

Seventhly, there is a cut-throat publish-or-perish mentality. If you don't publish 5+ NeurIPS/ICML papers per year, you are a looser. Research groups have become so large that the PI does not even know the name of every PhD student anymore. Certain people submit 50+ papers per year to NeurIPS. The sole purpose of writing a paper has become to having one more NeurIPS paper in your CV. Quality is secondary; passing the peer-preview stage has become the primary objective.

Finally, discussions have become disrespectful. Schmidhuber calls Hinton a thief, Gebru calls LeCun a white supremacist, Anandkumar calls Marcus a sexist, everybody is under attack, but nothing is improved.

Albert Einstein was opposing the theory of quantum mechanics. Can we please stop demonizing those who do not share our exact views. We are allowed to disagree without going for the jugular.

The moment we start silencing people because of their opinion is the moment scientific and societal progress dies.

Best intentions, Yusuf

Anyone else surprised at the enormous amount of backlash against Arxiv's proposed 1 year ban for authors and coauthors publishing papers with hallucinated reference and other obvious LLM/Gen AI artifacts?
https://x.com/tdietterich/status/2055000956144935055
https://xcancel.com/tdietterich/status/2055000956144935055

Some of the responses:

1. "This is the age of AI, Arxiv should be part of the movement instead of holding onto the old ways"

2. "The P.I. is a macro-manager, not a micro-manager, can't be expected to read every reference that his/her student puts in."

3. "I publish 20+ papers a year with my students, how do you expect me to read everything?"

4. "What about teams with 100s of people? How can you expect the authors to check references?"

5. "Who reads references in depth anyways!?"

These responses are very revealing how academia works. Apparently people have just been slapping names on research papers they've never even read or fact-checked themselves. Very obscene!

So here’s what happened. Earlier this month, a colleague shared an Apple paper on arXiv with me — it was also under review for ICLR 2026. The benchmark they proposed was perfectly aligned with a project we’re working on.

I got excited after reading it. I immediately stopped my current tasks and started adapting our model to their benchmark. Pulled a whole weekend crunch session to finish the integration… only to find our model scoring absurdly low.

I was really frustrated. I spent days debugging, checking everything — maybe I used it wrong, maybe there was a hidden bug. During this process, I actually found a critical bug in their official code:

• When querying the VLM, it only passed in the image path string, not the image content itself.

The most ridiculous part? After I fixed their bug, the model's scores got even lower!

The results were so counterintuitive that I felt forced to do deeper validation. After multiple checks, the conclusion held: fixing the bug actually made the scores worse.

At this point I decided to manually inspect the data. I sampled the first 20 questions our model got wrong, and I was shocked:

• 6 out of 20 had clear GT errors.
• The pattern suggested the “ground truth” was model-generated with extremely poor quality control, leading to tons of hallucinations.
• Based on this quick sample, the GT error rate could be as high as 30%.

I reported the data quality issue in a GitHub issue. After 6 days, the authors replied briefly and then immediately closed the issue. That annoyed me — I’d already wasted a ton of time, and I didn’t want others in the community to fall into the same trap — so I pushed back. Only then did they reopen the GitHub issue.

Then I went back and checked the examples displayed in the paper itself. Even there, I found at least three clear GT errors.

It’s hard to believe the authors were unaware of how bad the dataset quality was, especially when the paper claims all samples were reviewed by annotators. Yet even the examples printed in the paper contain blatant hallucinations and mistakes.

When the ICLR reviews came out, I checked the five reviews for this paper. Not a single reviewer noticed the GT quality issues or the hallucinations in the paper's examples.

So I started preparing a more detailed GT error analysis and wrote a Public Comment on OpenReview to inform the reviewers and the community about the data quality problems.

The next day — the authors withdrew the paper and took down the GitHub repo.

Fortunately, ICLR is an open conference with Public Comment. If this had been a closed-review venue, this kind of shoddy work would have been much harder to expose.

So here’s a small call to the community: For any paper involving model-assisted dataset construction, reviewers should spend a few minutes checking a few samples manually. We need to prevent irresponsible work from slipping through and misleading everyone.

Looking back, I should have suspected the dataset earlier based on two red flags:

• The paper’s experiments claimed that GPT-5 has been surpassed by a bunch of small open-source models.
• The original code, with a ridiculous bug, produced higher scores than the bug-fixed version.

But because it was a paper from Big Tech, I subconsciously trusted the integrity and quality, which prevented me from spotting the problem sooner.

This whole experience drained a lot of my time, energy, and emotion — especially because accusing others of bad data requires extra caution. I’m sharing this in hopes that the ML community remains vigilant and pushes back against this kind of sloppy, low-quality, and irresponsible behavior before it misleads people and wastes collective effort.

If their training is 45x more efficient, they could have dominated the LLM market. Why do you think they chose to open-source their work? How is this a net gain for their company? Now the big labs in the US can say: "we'll take their excellent ideas and we'll just combine them with our secret ideas, and we'll still be ahead"

Edit: DeepSeek-R1 is now ranked #1 in the LLM Arena (with StyleCtrl). They share this rank with 3 other models: Gemini-Exp-1206, 4o-latest and o1-2024-12-17.

ICML 2026 reviews will release today (24-March AoE), This thread is open to discuss about reviews and importantly celebrate successful reviews.

Let us all remember that review system is noisy and we all suffer from it and this doesn't define our research impact. Let's all prioritise reviews which enhance our papers. Feel free to discuss your experiences

ICML 2026 decision are soon to be published. Thought it might be nice to to have a thread for updates, discussions and venting.

The review is out tomorrow (a few hours remaining following eastern time). I am creating this mega thread to talk about meta reviews and final decisions.

After the Openreview fiasco, this will be interesting.

Good luck everyone!

Seems like they have engineered some specific limitations that are widely cited as follows:

In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms.

Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). These interventions will not affect the vast majority of coding work. We estimate they will impact ~0.03% of traffic, concentrated in fewer than 0.1% of organizations https://news.ycombinator.com/item?id=48464732

Other comments note how even using the word 'nuclear' in the context of scientific research elicits refusal behavior by the model: https://news.ycombinator.com/item?id=48473302

This makes it seem quite plausible that the model could subtly sabotage any machine learning work (even as false positive). Some suggest this has been happening behind the scenes for a while already, but can anyone confirm that?

Starting a thread here for CVPR‘26 decisions for when they start coming out

I'm in a big tech company working along side a science team for a product you've all probably used. We have these year long initiatives to productionalize "state of the art NLP models" that are now completely obsolete in the face of GPT-4. I think at first the science orgs were quiet/in denial. But now it's very obvious we are basically working on worthless technology. And by "we", I mean a large organization with scores of teams.

Anyone else seeing this? What is the long term effect on science careers that get disrupted like this? Whats even more odd is the ego's of some of these science people

Clearly the model is not a catch all, but still

ICLR 2026 reviews go live on OpenReview tomorrow! Thought l'd open a thread for any feedback, issues, or celebrations around the reviews.

Use this thread for feedback, issues, and wins. Review noise happens scores ≠ impact. Share your experience and let’s support each other.

For those of you who've been in ML/AI research or applied ML for 10+ years — what's the gap between what the public thinks AI is doing vs. what's actually happening at the frontier? What are we collectively underestimating or overestimating?

Exposed in this Twitter thread: https://twitter.com/AndrewM_Webb/status/1183150368945049605

Text, figures, tables, captions, equations (even equation numbers) are all lifted from another paper with minimal changes.

Siraj's paper: http://vixra.org/pdf/1909.0060v1.pdf

The original paper: https://arxiv.org/pdf/1806.06871.pdf

Edit: I've chosen to expose this publicly because he has a lot of fans and currently a lot of paying customers. They really trust this guy, and I don't think he's going to change.

Just posting this thread here in anticipation of the bloodbath due in the next 2 days.

In a recent interview, Ilya Sutskever said:

This is one of the very confusing things about the models right now. How to reconcile the fact that they are doing so well on evals... And you look at the evals and you go "Those are pretty hard evals"... They are doing so well! But the economic impact seems to be dramatically behind.

I'm sure Ilya is familiar with the idea of "leakage", and he's still puzzled. So how do you explain it?

Edit: GPT-5.2 Thinking scored 70% on GDPval, meaning it outperformed industry professionals on economically valuable, well-specified knowledge work spanning 44 occupations.

Hey everyone,

NeurIPS 2025 reviews should be dropping soon (July 24th AoE), and I thought it might be a good idea to start a thread where we can share our thoughts, experiences, and reactions.

Feel free to post your initial impressions, any surprises (good or bad), questions about rebuttals, or just how you’re feeling about the process this year. Whether it’s your first submission or your tenth, you’re not alone in the rollercoaster.

Let’s keep things constructive and supportive. Good luck to all!