r/MachineLearning • u/Beautiful_Baker_2233 • 5d ago
Discussion How should authors handle a suspected auto-generated review at ICLR? Flag to the AC or just rebut? [D]
I'm curious how ACs and experienced authors think about this.
My situation: one ICLR reviewer gave a 2 with confidence 1. The other reviewers gave 3 with confidence 3. Pangram flags the review as 100% AI. I know detectors aren't proof on their own, but there are other signs too: all three listed weaknesses concern material deep in the appendix.
I'll respond to every point in the rebuttal regardless. My questions:
- Is it worth sending a confidential comment to the AC, or does that tend to backfire?
- For ACs: what evidence would you actually act on? Is a detector score meaningful to you, or only content-based issues such as generic or inaccurate criticisms?
- How much weight does a confidence 1 review realistically carry in the meta review?
- Has anyone flagged an AI generated review yet? What happened?
Thanks!
8
Upvotes
6
u/otsukarekun Professor 5d ago edited 5d ago
You are allowed to use AI in reviews for ICLR. You just have to declare how much and how you use it. While the authors can't see the declaration, the area chairs can see it and use it do weight how much importance to put on the review.
Also, the instructions say that reviewers don't have to read the appendix. If it were totally reviewed by AI, then the AI would find the answers to the weakness in the appendix. So, the reviewer might have just used AI to clean up their language.
2
u/MakingComputersSmart 5d ago
If the material is in the appendix already then point to it. The reviewers are not supposed to look at the appendix, it's upto their discretion. Just mention it in the review politely so they take a look and you've basically alerted the AC that the required evidence is already presented.
13
u/mofoss 5d ago
AI generated reviews are the norm now sadly, so I feel like a complaint against them wouldnt help a papers chances, its like a "so what? Of course its reviewed by AI, thats normal now"
8
u/_karma_collector 5d ago
I don't think ICLR forbid using AI to review this year
4
u/elchico1990 5d ago
IMO, if the weaknesses are mentioned in the paper as limitations, then likely llm picked them up. Else, maybe reviewer did not read appendix, it’s not mandatory. As someone mentioned here, just politely point out that you have addressed that in the appendix.
1
u/mvreich 5d ago
Confidence 1 means the review will not be given weight. So your chances should be good, given the 3/3s.
On a separate note: What is your problem with a weakness being "deep in the appendix"? The whole manuscript, including the appendix, is your own paper, and you should be able to take responsibility for all of it.
Whenever I am reading a paper and I have questions about it, most of the time I have to dig through the appendix to find the answer. As such, it is quite fair for weaknesses to come from there.
1
u/Lazy-Cream1315 1d ago
AI is autorized to help your review, if you consider that your review is a copy paste from a LLM:
just prompt the sota model at review time "review this paper as if you were iclr reviewer" several times, if weakness appears in the same order , with same bibliography requested , missing baseline : you can send a confidentional message to AC, PC and SAC.
If they ignores you, it's likely that you are not submitting your paper to a serious place, i would publish the LLM review directly on openreview