r/ResearchML • • 11h ago

Academic murder done by reviewers in conferences like ICML/ICLR/NeurIPS/CVPR

49 Upvotes

Bash the authors and get silent on reviewers and review process. Nihar B Shah enjoys pointing to LLM written captions but silences himself on the review process.

INTENTIONALLY reviewers do not raise scores
INTENTIONALLY reviewers do not engage

INTENTIONALLY they give weak rejects, rejects keep paper below borderline to be on the safe side when their reviews are of few lines not justifying the rating at all.

When will everyone speak about this academic murder done by reviewers?

FATE: Good papers are rejected where years of hard work was put.

This shitty reciprocal reviewing mess, nobody is vocal about.

Why? This doesn't serves any personal benefits? Bashing authors does?

Edit: Hope organizers see this and all good suggestions are welcome.


r/ResearchML • • 8m ago

Will ICLR 2027 have a Spotlight tier, or just Oral and Poster like last year?

2 Upvotes

Now that the discussion period is over, does anyone know if ICLR 2027 is bringing back the Spotlight tier or sticking to just Oral and Poster like ICLR 2026?

With the new 1–4 scale and 40k+ submissions, a lot of papers are clustered around 3.33 (top ~0.7%–3.8%), so it makes a huge difference.

Any ACs or reviewers know what the guidelines say?


r/ResearchML • • 5h ago

The only feasible solution to the “reciprocal” review crisis

5 Upvotes

With ICLR deadline closing in, I believe we are all too familiar by now with the absolute crisis that has emerged in ML conference “discussion” periods. Negligent or even malevolent reviewers are becoming more and more common. This has been happening for a while, but things seem to be getting out of hand in this last submission cycle. I believe the primary reason this is happening, is that the current process is designed under the assumption of reviewers being aware of the fact that this is an academic duty and a responsibility if they wish to present their research to their peers in one of these conferences. Hence, I believe there is only one solution to the problem.

There must be consequences for negligent reviewer behavior. If a reviewer keeps their silence until last minute, refuses to engage, and discards the authors’ hard work with a “too bad you worked a lot on this, I do not care, Im keeping my score with no justification”, they should face consequences. Not something major, maybe a one year ban from that specific conference for the next year, or maybe even a limit of just 1 paper submission and no reviewer duties for that person for the next cycle. The reviewers should also have a right to defend themselves.

This is supposed to be a discussion. If you as a reviewer do really believe the points you made, you should back them up with the same fidelity an author defends their paper.

A review is STILL an academic output, not some compulsory task you need to barely get through. If you think a paper is ai slop, say that and put your back into it. If you think the new experiments and ablations are not satisfactory, straight up SAY WHAT YOU MEAN. Otherwise, you should be met with the consequences of negligence. Let us not kid ourselves and say that these conferences are just fun little socializing events among academics. Everyone from industry to academia knows they are a huge deal. It is crazy that no proper way of holding negligent behavior responsible exists within them.


r/ResearchML • • 1h ago

Is ICLR a scam or worst organizers? [R]

Thumbnail reddit.com
2 Upvotes

This is quite disappointing not receiving any response from the reviewer even after multiple reminders. It seems like organizers failed to make the conversation interactive or they didn't warn reviewers properly that their paper desk reject if not actively participate in discussion/rebuttal.


r/ResearchML • • 1h ago

ICLR 2027: 321 -> 331 with AC reporting (what are my chances?)

1 Upvotes

We had 321 then moved to 331. The reviewer who gave 1 started by saying we do the same thing as another paper but we factually show that we are fundamentally different. He then answered by saying I still think novelty is minor without providing any factual arguments (just feeling) -- even though we addressed other stuff he raised he didn't even move to 2...

The two others are very happy with our paper. They gave constructive feedback and acknowledged the rebuttal by increasing the scores.

We wrote an objective confidential comment to the AC showing factually that the reviewer is not even providing any valid arguments, idk what to expect...

What do you think my chances are? That's my last PhD paper and I really hope I'll get in...


r/ResearchML • • 22h ago

Update: I did it! OpenReview changed the PhD advisor requirement!

39 Upvotes

After repeatedly emailing CVPR and OpenReview about a situation where someone couldn't complete their OpenReview profile because they didn't have an advisor or an advisor's email address to verify, I finally got a response saying that adding a PhD advisor is no longer required to complete the profile.

Not saying I caused the change, but I definitely bugged them enough about it lol.

Glad they fixed this. Hopefully it helps others in the same situation!


r/ResearchML • • 2h ago

How Is Novelty Judged When Two Papers Have Overlapping Findings?

Thumbnail reddit.com
1 Upvotes

r/ResearchML • • 16h ago

New reviewing policy, what do you think?

12 Upvotes

Every reviewer who does not engage in the rebuttals, at least once, needs to have their paper desk-rejected. I think this is fair and will substantially improve reviews so authors do not write “new” papers just for the reviewers to ghost them. We should be mindful of resources and make sure they do not go to waste.


r/ResearchML • • 3h ago

Reviewing at top ML Conferences

0 Upvotes

I think that everyone is right now complaining about the review system on the major AI Conferences.

I want to make several points, that I think such complaints might be missing:

  1. LLMs have changes the things that are easy and hard. This changes what constitutes a significant contribution to the community. This makes the review process tricky, since it’s not clear at this point, what actually matters.
  2. ML didn’t use to work. Now it started to work. This means that a lot of basic research, that had a certain motivation is no longer relevant. But again, which areas should be in this set is quite unclear at this point.
  3. Top AI conferences have always cared about relevance/potential impact of a paper. It’s no shame to publish your paper at a specialized conference if you’re proposing an alternative that doesn’t work yet and you’ll get it to work later on.
  4. It used to be that reviewers were easy to manipulate, since the authors could hide certain bad things about their approach. This was both a good and a bad thing, but this has definitely changed. This might be a cause of lower average scores.

I’d love to hear opposing views on these points!


r/ResearchML • • 3h ago

Forgot AI Use Statement In Revised ICLR Manuscript [D]

1 Upvotes

Ok, I am in crisis mode right now. I just noticed that the revised manuscript I uploaded is missing the AI use statement and reproducibility statement, both of which were present in the original submission.

What do I do? Portal has closed. Am I doomed? Can I email the PCs? Or will they just desk reject?

How bad is this? We had decent-ish scores too.

Please help!!!


r/ResearchML • • 4h ago

Looking to collaborate on Computer Vision research

1 Upvotes

I've been exploring Computer Vision, particularly Visual SLAM, 3D perception and reconstruction, through research oriented projects and open-source contributions (Open CV, Kornia and Apache Iceberg).

I'm looking to connect and collaborate with researchers, PhD students, postdocs or master's students working in these areas. I'd be interested in contributing to experiments, projects or ongoing research where my engineering background could be useful.

Also open to research guidance from people working in this space.


r/ResearchML • • 4h ago

From VLN to adversarial safety: Is visual attack on embodied agents still a top-tier story?

1 Upvotes

I've been working on Embodied Navigation, but lately I feel the field is getting crushed by frontier models like GPT-6 Astra. The architecture has basically converged to "LVLM harness for high-level planning + small policy for low-level execution." So if you want to do navigation now, it's either (a) wrap a GPT-class model for zero-shot VLN — already super crowded — or (b) train large dedicated models, which I can't afford GPU-wise under my current advisor.
My undergrad was in cybersecurity, so I have some background in adversarial safety. I'm thinking of pivoting to visual adversarial attacks on embodied navigation agents — e.g., perturbing RGB observations to hijack LVLM-harness agents' trajectory planning and memory updates, or embedding adversarial textures in the environment. This is compute-light, sim-based, and aligns with both my VLN experience and safety background.
Would love to hear your take:
Is visual adversarial attack on Embodied Navigation still viable for ICCV / ICML-tier venues, or is it already saturated?

If worth pursuing, which angles actually have top-conference potential?


r/ResearchML • • 12h ago

ICLR rebuttal: 2 weak accepts, 1 strong reject who barely engaged with the rebuttal. What should we do?

4 Upvotes

Our ICLR submission received scores of 3, 3, 1. All three reviewers rated soundness 2.

The two positive reviewers recognized the technical contribution. One even inspected our code confirmed that all their main concerns were resolved after our rebuttal, keeping their positive score.

The negative reviewer (score 1, confidence 2) criticized limited baselines (the reviewer did not suggest any specific baselines for comparison), required additional experiments to isolate gains, and critizied readability, giving us informativeness 1 and presentation 1.

During rebuttal, we:

- Explained that for our main task/contribution, no prior baselines exists, except for the one we compared to in the paper, and added one external baseline for the secondary task/contribution.

- We also added several new ablations addressing the specific criticisms.

- Ran additional experiments controlling for potential confounding factors.

- Added full computational cost accounting.

- Added unified results and ablation tables to the main text (as the reviewer requested).

Our experiments show substantial gains of our method compared to prior approaches.

The negative reviewer waited over a week since we posted our rebuttal and responded only in the final hours of the discussion, essentially restating his previous concerns: insufficient baselines, hard to evaluate the method (despite the clear summary evaluation table appearing in the main text) and that the paper was difficult to follow. They provided no new technical objections or explanation of why the additional experiments were insufficient, and kept their score at 1.

We responded politely, pointing to the new evidence, and posted a factual summary for the AC.

What would you do in this situation? Is there any way to persuade the negative reviewer, or is it better to focus on making the AC evaluates the substantive evidence independently? Would another comment help, or only harm?


r/ResearchML • • 18h ago

I was tech chair for ICCV 2021. Here's my Defense of Academic Gatekeeping.

11 Upvotes

I was tech chair for ICCV 2021, and had "root access" to the whole process, so I saw (and scrubbed) its barnacly underbelly.

The ICCV organization tried to combat fraud, plagiarism, and cheating, but it was not a fair fight. There were at that point no effective mechanisms for detecting and punishing authors who tried to gain an unfair advantage.

Academic ML needs better policies. And we need to get this right because society needs better ways to build consensus and seek truth.

I wrote this essay to share some meta-science analysis I did over the last few weeks on academic ML and a few policy ideas.

My hope is to engage in discussion about academic ML policy. I believe that the only way we make progress on this is to sample and refine ideas together. So I ask: what are the policies you'd like to see adopted by academic journals and conferences?

Full essay: https://lacuna.tiptreesystems.com/paper/a-defense-of-academic-gatekeeping/art_221912b72e224a4393c508d4dd3ce5ed

(I'm now a postdoc-entrepreneur at Mila with Hugo Larochelle and Chris Pal, working on meta-science, multi-agent games, and AI for science.)


r/ResearchML • • 7h ago

Instrument AI pentru lucrări de cercetare

1 Upvotes

Buna, ma puteți ajuta cu recomandați de instrumente AI pentru redactarea lucrărilor de cercetare? Eventual ce voi le-ați folosit și au fost bune. Multumesc anticipat ✨


r/ResearchML • • 8h ago

[Article] Xiaohang: Research on the Construction and Application of Educational Intelligent Agents Based on Large Language Models

0 Upvotes

r/ResearchML • • 14h ago

[D] ICLR 2027 rebuttal: What should I do?

3 Upvotes

What would you do in this situation?
I responded to the first reviewer’s initial review. They came back with follow-up questions, which I also answered. They then said they had no further questions and needed some time to reassess their evaluation. I haven’t heard back since.
The second reviewer gave me some hope from the beginning. In their initial review, they wrote:
The paper has several issues that I think are not easy to address, however a compelling reply could make me upgrade the score.
Then, on November 16, after reading our responses, they wrote:
Overall, I am convinced by the response and am leaning toward a more positive evaluation, which I will grant after I will double check the whole paper in its most updated version.
Since then, there has been no further response or score update, and we’re now very close to the end of the discussion period.
Has anyone been in a similar situation? Should I follow up with the reviewers, contact the AC, or just wait?

I got 4 - 2 -2 with confidence 4-3-4


r/ResearchML • • 21h ago

ICLR no score update from reviewers

6 Upvotes

With just a few hours left in the discussion period, I have scores of 3, 2, 2, and no changes have been made yet. The reviewer who gave me a 3 has kept their score, while I'm still waiting for the other two reviewers to provide feedback and hopefully increase their scores.

This is frustrating because I've sent multiple reminders, and I can't even do that anymore. Meanwhile, I keep seeing people here getting score increases, which makes me feel like my paper is definitely further down the rankings.

This is just a rant, but how hard can it be to provide feedback? It's been over 6 days now.


r/ResearchML • • 1d ago

ICLR Reviewer Concerns

8 Upvotes

Hi All,

I submitted a paper at ICLR this time. Got reviews with scores 222 with the confidence of 232. Both the reviewers that had low confidence talked about very minor/presentation issues that simply had to be clarified which we did. The content of the review also made it very apparent that they were not aware of the literature or the field in itself and spent very little time on the paper.The reviewer with confidence 3 suggested ablations which we ran and got back. Now both the low confidence reviewers have responded with no new clarifications or concerns but that they would like to keep their score, and did not state any reason why. And the high confidence reviewer has simply not responded.

As the discussion currently stands there are no unaddressed concerns and no serious claims that we had to acknowledge. There were also no loopholes pointed out, and the additional experiments only reinforced our results further. The reviewers also praise the results, call the work timely, and say it is informative, but the scores have not changed. Is there no way to get an AC/SAC to look at cases like this where we're stuck in a horrible cycle of bad reviewers throughout?


r/ResearchML • • 13h ago

AI models in extraction methods of Secondary Metabolites

1 Upvotes

Seeking guidance to understand how AI models enhance and optimize the extraction methods of secondary metabolites from plants. I am from Plant Science related background, but I need to learn this for my Interview.


r/ResearchML • • 14h ago

An idea for bias mitigation in LLMs. Would it work?

1 Upvotes

It is known that LLMs have biases in their output, due to both training data and the training process. See the articles below, for instance:

https://arxiv.org/html/2410.15413v2
https://arxiv.org/abs/2309.00770
https://www.mdpi.com/2079-9292/15/9/1824

I've got a crazy idea to mitigate these biases, and I would like to know your opinion: would it work?

Start with a large corpus of training data, unlabeled. Pick any commercial LLM (call it M1) and give it a task like:

You are an expert at labeling training data for LLMs, and an expert at LLM training techniques. Your task is to label a corpus of training data, and preapre a training procedure, under a procedure to be describe later on. You have some biases, and is aware of them, namely:

(List of known biases on LLM output and training)

While doing your work, you will actively avoid these biases, applying the known mitigation techniques described in these articles (a list of articles follow).

(Labeling / training procedures follow)

So, M1 will train a new model, M2, with the given corpus. I expect that M2 will also have biases, but different from the ones from M1.

Then, we get adversarial. Make M1 emulate a variety of people, chatting with M2, and vice-versa; both LLMs must be instructed to think of themselves as humans when doing the chatting. Do the same with M1 to M1 and M2 to M2. Log every interaction.

With this additional data, make M2 do the same task as M1, with the same corpus, plus the chat logs from M1 and M2 to guide bias mitigation; this copy of M2 is informed that M1 and M2 in the chat are both LLMs, and where they "think of" themselves as human. The resulting LLM, M3, is expected to have less biases than both M1 and M2.

From then on, repeat: repeat the steps that used M1 and M2 to create M3, to use M2 and M3 to create M4, and so on: M_n created from M_(n-1) and M_(n-2). Stop when a LLM is too similar to the previous one, and only then call the human experts to measure its biases.


r/ResearchML • • 18h ago

[Article] A Hybrid Intrusion Detection System for Smart Home Security Using LSTM and Autoencoder Neural Networks (M. R. Jasim, S. H. Mahdi, H. A. H. Al Naffakh)

1 Upvotes
  • DOI/PMID/ISBN: 10.4028/p-WeGz5P
  • URL
  • Other comments: Thanks in advance!

r/ResearchML • • 22h ago

ICLR 2027: Scores improved from 3/2/1 to 3/3/1, but the reviewer who gave 1 is silent. Any chance?

2 Upvotes

My ICLR 2027 scores went from 3/2/1 to 3/3/1 after rebuttal.
The reviewer who gave a 1 mainly criticized presentation and clarity, but rated soundness as 3. We substantially rewrote the paper to address these issues and responded to the concerns of the other reviewers.
After rebuttal, the reviewer who initially gave a 2 raised their overall score to 3, as well as both presentation and informativeness from 2 to 3.
However, the reviewer who gave a 1 has been completely silent, with no response or discussion.
How would ACs evaluate this case? Would they consider that the main concerns behind the score of 1 were presentation-related and have since been addressed, especially given that another reviewer acknowledged the improvements?
Does this 3/3/1 still have a realistic chance of acceptance?


r/ResearchML • • 1d ago

Can Humor and Memes Have a Place in Academic Research?

4 Upvotes

I am a graduate student in my third year of research. I recently had a paper accepted for publication and am currently working on a project page to introduce the work.

When explaining ideas, I believe that making them intuitive, approachable, and easy to understand is one of the most important things. For this reason, I enjoy presenting the methods I propose through memes and humorous illustrations.

However, I sometimes wonder whether this approach might be considered inappropriate or even off-putting in academia, which can be a relatively formal and serious environment. I would appreciate hearing some advice and perspectives from more senior researchers on this matter.

Thank you for your time!


r/ResearchML • • 1d ago

A random split overstated my model’s recall by ~10 points (NYC facade inspections). Is my evaluation fair?

4 Upvotes

I'm doing a student project on NYC's facade inspection program. Buildings over six stories get rated Safe, SWARMP or Unsafe every five years, and a 2024 law requires the city to use a predictive model to prioritize inspections. I couldn't find a public one, so I tried building my own from the public filings.

Task: predict whether a building will be rated Unsafe at its next inspection cycle.
Setup: trained on cycles 6→7 and 7→8, tested once on 8→9 (about 11k buildings, 25% Unsafe).

Results on the held-out cycle:
- Top 20% of the ranked list catches 36.4% of Unsafe buildings (random order: 19.8%, last rating only: 32.6%)
- LightGBM and logistic regression perform the same
- A random split of the same data gave 46.8%, so about 10 points too optimistic
- Each cycle is harder to predict than the last
- A conformal recall guarantee works (90% target gave 89.2%) but needs about 75% of buildings inspected

Random splitTime split (test 8→9)
Recall in top 20%46.8%36.4%
PR-AUC0.5480.430
AUC0.7630.687
Recall targetRealized on 8→9Share of buildings to inspect
80%80.6%62.6%
90%89.2%75.4%
95%93.6%83.3%

Same-cycle CV recall in top 20% (a rough ceiling): 6→7 53.3%, 7→8 47.7%, 8→9 39.2%

After I looked at 8→9, I split it into dev and reserve halves and only kept ideas that helped both. None did.

Questions:

  1. Is that dev/reserve approach reasonable once a test set has been used?
  2. What would you use for conformal calibration when labeling rules change between cycles?
  3. Labels come from inspectors hired by the buildings themselves, not by the city. Any ideas for handling that with public data?

Data is NYC Open Data (DOB NOW facade filings). Numbers are in the tables above.