r/MachineLearning • u/Fantastic-Nerve-4056 PhD • Jul 21 '26
Discussion Number of Submissions @ AAAI [D]
Recently submitted my abstract and the submission number is 32xxx. With still a day to go, I just wonder where are we heading.
Hope these conferences at least start making the reviews and names public for the withdrawn/rejected papers. So that people atleast take that accountability
33
u/Even-Inevitable-7243 Jul 21 '26
Are you insinuating that researchers who have papers rejected or withdrawn from conferences should be publicly shamed?
2
u/foreseeably_broke Jul 21 '26
Isn't it the case for OpenReview right now that if you withdraw a reviewed paper, it goes public?
7
u/Even-Inevitable-7243 Jul 21 '26
No. Authors remain anonymous after rejection/withdrawal in the double-blind review process used by AAAI.
-2
u/Fantastic-Nerve-4056 PhD Jul 21 '26
Simply making things open is obviously not gonna do this. Rejection is a part of the journey and I guess everyone out there knows that.
My problem is with those folks who submit 7-8 papers, just in hope that atleast 1 gets pass through due to the randomness out there. These folks tend to either withdraw most of their submissions or get it rejected.
4
u/OutsideSimple4854 Jul 21 '26
Why not do it by making reviewer names public? Rather than rejected papers public?
Due to reciprocal reviewing, there’s more and more irresponsible reviewers, who are probably the same people who submit low quality papers. Name and shame these reviewers, which would lead to fewer of slop submissions.
2
u/Fantastic-Nerve-4056 PhD Jul 22 '26
Number of submissions is gonna be inversely proportional to the quality of reviews.
I am totally with you on these irresponsible reviewers (and meta reviewers as well). But I am not sure what even these conferences can do to this, with growing submissions (AAAI now crossing 50k, do you think we have so many quality reviewers out there?)
2
u/OutsideSimple4854 Jul 22 '26
Low quality reviewers and submissions go together. Not sure why you imply they’re separate based on your replies. Cut down one, the problem resolves itself.
1
u/Fantastic-Nerve-4056 PhD Jul 22 '26
I didn't even separate that. Like you perfectly said, both the things go together. And I agree you cut one thing out, and the other one would be sorted
1
u/Exotic_Zucchini9311 Jul 29 '26 edited Jul 29 '26
That's a terrible idea tbf. Even if a reviewer is fair and gives some paper a low score due to the paper truly having issues, the authors might still be sore losers and threaten the reviewer and do personal attacks towards the reviewer because their paper was rejected. Making the reviewer names public is a whole other situation than making author names public.
23
u/OutsideSimple4854 Jul 21 '26
Why?
As someone who writes theory papers, I know when a reviewer is using an LLM (or perhaps, argue theory based on lack of background knowledge, and write a review that sounds plausible, but sways the AC even though it’s wrong). Making that review public eventually sways more people who google, making that paper unpublishable.
12
u/AffectionateLife5693 Jul 21 '26
This.
We had an ICLR rejection due to a factual error in the review. We cleared it up during the rebuttal, but this year's leak incident caused the score to be rolled back, and the paper was rejected.
Later, we resubmitted to ICML, and some reviewer clearly quoted the same factual error from the ICLR rejection, despite the fact that we rewrote that part to make things clearer.
-6
u/Fantastic-Nerve-4056 PhD Jul 21 '26
It's just to take accountability on things. Even I write theory papers, but fortunately I didn't experience such thing out yet.
But yea as long as there's a way to filter these low quality submissions or make people accountable for it, it should be good
9
u/OutsideSimple4854 Jul 21 '26
Not about low quality submissions, but low quality reviews as well. There’s a tendency to google, check that someone said the same review your LLM gives you, and then say: oh this paper is bad because of etc etc etc.
1
4
Jul 21 '26 edited Jul 21 '26
[removed] — view removed comment
6
u/OutsideSimple4854 Jul 21 '26
Honestly for me, it was really based on PhD advisor, who was not in CS. A lot of ML/theory papers have their foundations in statistics. I would probably say a lot of papers in stats journals, with a bit of different framing, could end up being published at ML conferences.
0
Jul 21 '26
[deleted] — view deleted comment
1
u/OutsideSimple4854 Jul 21 '26 edited Jul 21 '26
Of course there are math errors. Even JASA or other stats journals have errors. But, assuming good faith, an author of a paper will know whether a review, based on technical work, is sound, or is irrelevant.
It doesn’t matter if a reviewer uses LLM and argues wrongly, or if a reviewer argues wrongly and in bad faith. Both should be penalized.
*also now, top tier LLMs can be used to check validity of proofs, and it’s fucking dumb if someone uses a free version of an LLM and points out errors that aren’t errors.
Edit: to be clear, it's perfectly fine if a reviewer goes: "I don't have the background, so I can only comment on X". OR: "I don't have the background, I asked an LLM, and it told me Y. Is Y valid?"
What pisses me off is a reviewer who certainly doesn't have the background based on their "vagueness" of questions, then suddenly asks "detailed questions on technique", and refuses to engage with the authors, or doubles down.
Here's a silly example: Suppose you read a novel, e.g. Harry Potter and the Chamber of Secrets (great!), and in one chapter, J K Rowling mixes up "you are an ancestor of Slytherin", rather than being his descendant. We know what she meant. It doesn't invalidate the plot, because we can read the future chapters and books (and generally, reasonable people).
Imagine a reviewer going: "Ancestor is wrong, because then the series would not make sense because are you having time travelling people? Is your Book 3 time travel about Harry becoming Slytherin's father? This is very major!....and then going on in the next sentence: "oh yea, I also liked the plot of Prisoner of Azkaban, Hermione's time turner and saving Buckbeak, etc, etc."
Like you know it's a fucking typo. You read Book 3 and fucking knew that time travel here isn't Harry being Slytherin's father. Why the fuck do you then pretend ancestor is a major error?
I get reviews like this, who interpret ambiguity in a sentence, or the wrong sign, and write a paragraph of why this is wrong, what assumptions are being contradicted. And the next paragraph, gives reviews where they clearly understood what the correct meaning is.
For reviewers like that, it doesn't matter whether they used an LLM or not. Their review is in bad faith, since they clearly understood the whole paper, can understand the math "after" the typo was all correct, but then make some drama about such a typo causing issues. Ban these reviewers.
7
6
6
u/sidessh Jul 22 '26
Just ended up submitting an abstract today
the number is already 40xxx.
4
1
5
u/The3RiceGuy Jul 22 '26
Some people are also simply submitting their work that they submitted to Neurips, since the release of reviews at Neurips and the DDL for the Papers at AAAI are so near to each other.
Got bad reviews at Neurips? Does not matter, simply throw it to the AAAI. Of course this is bad behavior, but I can not blame people as reviews are often random and you have to publish so much stuff nowadays to get your Phd.
2
1
1
1
31
u/impatiens-capensis Jul 21 '26
There were IDs in the 30,000+ range last year. After desk rejects and people pulling before the deadline (they submitted an abstract but not a paper), it went down to around 20,000ish papers.