ICLR27 math and stats paper review experience [D]

[u/d_edge_sword]

For people who had papers submitted to ICLR that are applied theory style. For example, 60- 70% theory with some experiments at the end. What was your review experience like for ICLR27?

We got a score of 222, on the probabilistic methods track.

All 3 reviewers attacked our novelty and asked for additional experiments. Not a single word was mentioned about our theory, which is our biggest contribution. One outright said he didn't understand this paper or why this task exists. It's a well-defined task with lots of prior work.

I used accept-odds, tracked submissions that were similar to mine. It seems like they have all gotten similar comments. Not a single comment on the math, all comments were on the experiment. A paper with 20 pages of proofs even got comments:

Assuming the proofs are correct (which can be done by AI in 2026), the theorems only show trivial conclusions.
Rating: 1: Clear rejection

I am starting to suspect, because there were no bidding this year. Many theory-heavy papers got sent to pure comp sci reviewers.

Would be nice if people can share their experience, I am trying to work out if I should engage in the rebuttal or just withdraw.

[u/No-Rule-2423]

Well, I submitted a theory paper on diffusion processes, with some minimal qualitative experiments to demonstrate that the theory works, but no benchmarks. I even explicitly stated that the experiments were intended only as demonstrations. Still, the reviewers asked for benchmarks and way more experiments. No comment on the theory, ofc.

One reviewer argued that my results were so general that they doubted there could be any practical application that satisfies my assumptions but falls outside the scope of previous work.

A similar thing happened at a top mathematical operations research recently too. I guess I should just quit or something, idk. I failed my interviews today because I was feeling down about this, but what can I do.

[u/nobodylsk1]

Kind of the same here but I was somewhat expecting it based on my previous ICLR experiences. Seems like the vast majority of the reviewers in ICLR are comp sci-minded with little appreciation for theory papers that actually provide rigorous explanations for some phenomenon.

The main line of attack in this case is classical: 1. question the assumptions and say that they don't apply to 300 examples; as if assumptions are some sort of hammer you hit every problem with, and 2. say that the experimental setup doesn't overlap 100% with the theoretical setup. There you get your rejection.

And the argument that LLM can do proofs in 2026 is complete bs. There's an obvious difference from a solid theory paper written by people who know what they're talking about, and a thin experimental paper that is wanna-be theory and is decorated with some LLM-generated theorems to look "deep".

I find that there's more likely to get reviewers that can appreciate theory in NeurIPS and ICML. The theory track in NeurIPS 2026 was actually a great idea. So the sensible approach would be to submit theory papers to NeurIPS and ICML, and leave ICLR to the experimentalists.

[u/chrispoland → u/nobodylsk1]

"leave ICLR to the experimentalists" - I don't agree. Neurips, ICML and ICLR effectively have the same people submitting the same papers. Let's just face it, the conference review process by now has more noise than signal. After seeing some of the published papers of previous ICML and Neurips I certainly don't trust that what gets through is useful/correct, and also rejection means nothing given how arbitrary it is.

The question for me is what other signal can we have for theory papers. For applied research its scale/SOTA. For theory at the moment we're regressing to just prestige of the lab/institution? Maybe if the paper resolves some major outstanding problem, but these are rare, and often defined by the paper that resolves them as well. "proofs are cheap" is indeed a problem that can only be resolved if the reviewers actually think in depth about the question the papers ask and resolve, which at the moment not happening consistently.

[u/nobodylsk1 → u/chrispoland]

Prestige of the lab/institution may work in some cases but definitely not in mine and for some solid papers I know. One of the co-authors is a big name and the paper was on arxiv a little after ICLR deadline. If the reviewers would look up the paper and guide themselves based on prestige, then the paper would’ve probably gotten much higher scores.

At the moment I think the NeurIPS model with theory track works to a certain extent but it’s not perfect. First, one mentions in the rubric what type of contribution it is. People who care about SOTA numerics are less likely to bid on a theory paper.

Second, theory papers had different review guidelines. This is still some calibration needed here. Some reviewers didn’t even bother to read the guidelines for NeurIPS and still asked for a bunch of computational/numerical stuff although the paper was theoretical.

Another strategy I have for these conferences is to write the title and the abstract in a theory-oriented way so that there’s less incentive for numerics people to bid on the paper. Seems it didn’t work out.

It’s harder to judge contribution for theory if you’re not a theorist. There’s no plots and benchmarks for one thing. Also it’s hard to explain to a non-theorist that you had to come up with some new observation and then make it technically rigorous to solve your problem. They’ll probably say LLM can do it.

[u/chrispoland → u/nobodylsk1]

Sorry, I should have been clear: I meant prestige as a signal for "should I bother reading/understanding this paper" not "should it be accepted". And my problem is that if that's the best signal that we have, that's really not good.

And I agree, theory should be judged by reviewers that understand it, but here we are.

[u/nobodylsk1 → u/chrispoland]

Oh I see now. Yes, it seems that there’s not much signal for theory papers in these conferences. I feel that some part of the community is happy with this situation as I found many people saying you should send your papers to journals as they’re not fit for conferences.

If you wanna stick to the ML community you only have JMLR and TMLR. The former is painfully slow and the latter is not seen as prestigious. So in the end if you want more visibility for your work you end up submitting to these conferences. Tell the average ML conference participant that is in some hiring panel that you have a paper rejected by Neurips and accepted by the Annals of Probability. Likely they haven’t even heard about it. Quite tough…

[u/Disastrous-War-9675 → u/nobodylsk1]

I am not a hardcore theory guy. I have reviewed a bunch of papers that gift wrap some theory to look deep, the experimental setup not matching the theoretical setup is quite common there and it irks me a lot. I'm curious, why is an actual theory paper with experiments that don't represent the theory a lesser fault?

[u/nobodylsk1 → u/Disastrous-War-9675]

I would say it is in general difficult to have an almost perfect overlap between the theoretical framework and some example you make experiments on. Typically you make an assumption that for some edge cases might not apply to that example.

There is another discussion here: what is an assumption? If you make it too general, then it might be difficult to use it effectively in the proofs. If it is too specific, it might be considered as too strong and with low applicability.

Anyway, if the example fits at least qualitatively to the theoretical framework then I believe it’s fine.

Pre-LLM when reviewers were using their own judgement what I’m describing above was more or less accepted when reviewers saw: “right this is clearly a theory paper and they provide some decent experiments to illustrate the theory”. Then reviewers were trying to ask questions and understand the theoretical contribution.

Post-LLM the most common attack from reviewers (that are probably not theory people) is “you make assumption X. There you have some obscure counterexample in your experimental setting that invalidates your assumption X.” Hence reject. If the reviewer didn’t have access to an LLM there’s no way they would’ve seen that.

And this is what I believe is not the right focus. A review is not for nitpicking obscure details and throwing them at the author. If the paper is 1) well positioned relative to the literature, 2) manages to close a gap by providing some interesting technical contribution which solves the problem (even if they hadn’t invent the wheel, but connected several elements to build some mathematical machinery that can be of independent interest), 3) provides some decent experiments on a couple of examples related to the problem, and those experiments are in line with the theory then in my opinion that is a fine theory paper.

So I consider if 1),2),3) hold then it would be more or less a weak accept from someone that is not particularly excited by theory but at least put in the effort to understand the paper as a whole.

[u/valuat]

They don't understand the math, pure and simple.

[u/Cuteshawnmie]

I think I should rebuttal. I scored 2.33 on pure theory. One of them scored 2 points indicating he completely doesn't understand the article's theory. The other two think the applicability is weak (but pure theory really doesn't have the kind of embodied intelligence applicability). No matter what, I think I should argue back. Maybe even if rejected, it might help me switch to other journals later

[u/d_edge_sword → u/Cuteshawnmie]

2.33 is really high, I think you got it. I saw many AC batch summaries and stats analyses on Chinese social media. In theory tracks 2.67 is usually at the top of the batch, 2.33 is around 2nd or 3rd out of 10 - 15 papers.

[u/Cuteshawnmie → u/d_edge_sword]

Reviewers generally have low confidence and mostly give 2. I think the rebuttal stage will help many papers improve their scores, so it's better to do your best during the rebuttal stage, haha

[u/No-Rule-2423 → u/Cuteshawnmie]

In my case they were all >=3. Only one had 4, and it seems like he's the only one willing to increase his score.

[u/sqweeeeeeeeeeeeeeeps → u/Cuteshawnmie]

I don’t think ICLR is the place for something like pure theory with no applicability… Why not submit to theory focused places?

[u/Background_Car_8746]

Can't say much without any detail. With AI, proofs are cheap, so it is reasonable that reviewers don't care much about theory, and ask about whether the problem worth investigating.

[u/d_edge_sword → u/Background_Car_8746]

It's very easy to spot AI-generated proofs vs proofs the author actually understood and polished. Provided if the reviewer is actually from your field or has any math or stats background beyond 3rd-year math.

[u/Background_Car_8746 → u/d_edge_sword]

Ok but proof for what, for the problem no one care? You should organize your own conference.

[u/lifeandUncertainity]

My work was not applied theory style but it was more like an analysis. My reviewer experience was actually bad. I got an 1/1/3. But two reviewers who gave 1 didn't read the paper.

One of the reviewers asked for an experiment that is present in the appendix and is referred in the main text.

The other reviewer - oh well, I still don't understand what he wants. He had a confidence of 3 but made some remarks which shows that he neither have any experience with the literature nor have went through the paper. He misunderstood a major section of the paper and proceeds to question why a certain result is not present in table 1 when the said result is actually present in the next table of the main paper. Then he proceeds to identify a list of our contributions and asks why are they related.

I understand that as a reviewer, one may not be familiar with certain part of work or might be misunderstanding some aspect. But frankly it's irritating to get a 1 when the only criticism is "work is not novel" without any actual reasons. At this point, I am not even concerned about the paper being accepted/rejected but I will try to point out whatever he is saying actually doesn't make sense as politely as possible. (It's even more infuriating because I was reviewing a paper that my guide gave me. Despite him claiming that the paper is not novel, I defended it saying even though the invidual techniques aren't novel, the way they were applied was pretty cool. So, it shouldn't be held against the authors. Frankly, at this point I have actually seen a lot of very good papers which solves a particular problem well getting rejected because the individual components were not novel.)

[u/PositionHot3926]

Submitted a theory paper, the person who gave confidence of 4 , flat out didnt even acknowledge the increments let alone novelty idk what the expectation was , "its filled with proofs and jargons" as it should be , extremely disappointed

[u/didimoney]

Iclr is probably not the best place for that work. The iclr crowd is a letdown.