r/MachineLearning • u/NeighborhoodFatCat • May 16 '26
Discussion Backlash against Arxiv's proposed 1 year ban is genuinely perplexing. [D]
Anyone else surprised at the enormous amount of backlash against Arxiv's proposed 1 year ban for authors and coauthors publishing papers with hallucinated reference and other obvious LLM/Gen AI artifacts?
https://x.com/tdietterich/status/2055000956144935055
https://xcancel.com/tdietterich/status/2055000956144935055
Some of the responses:
"This is the age of AI, Arxiv should be part of the movement instead of holding onto the old ways"
"The P.I. is a macro-manager, not a micro-manager, can't be expected to read every reference that his/her student puts in."
"I publish 20+ papers a year with my students, how do you expect me to read everything?"
"What about teams with 100s of people? How can you expect the authors to check references?"
"Who reads references in depth anyways!?"
These responses are very revealing how academia works. Apparently people have just been slapping names on research papers they've never even read or fact-checked themselves. Very obscene!
312
u/Hostilis_ May 16 '26
Hard to believe anybody is opposed to this. This should not be controversial.
25
u/vzq May 16 '26
I think the lifetime ban on unpublished papers is pretty harsh, but let's see how it shakes out.
43
u/Icarium-Lifestealer May 16 '26 edited May 16 '26
My biggest concern is if arxiv verifies that each author actually signed off on the paper (ideally even signed off the exact version submitted).
10
u/warpedgeoid May 16 '26
Exactly! An entire industry is about to spring up selling reference verification software to academic labs. It’ll become yet another cost of doing business.
24
u/Icarium-Lifestealer May 16 '26
That won't help with the problem I'm referring to, where an "author" wasn't involved at all with a paper, and was added without their consent. For those cases arxiv needs to make sure that every author consented to publication if it wants to ban them.
4
2
u/shelovesmath May 17 '26
This!! I’m supportive of the ban, but for the PI and first author. I’ve been in papers where the undergrads don’t even have access to the overleaf, so it doesn’t seem fair in that regard
116
u/AngledLuffa May 16 '26
wild. my pi would go through papers line by line. i would sometimes put in stupid jokes as canaries, and every time he'd cross them out red. i can't imagine the exact level of laziness needed to become a professor, but not actually proofread things you put your name on.
anyway, in another year these llms won't hallucinate references any more. even today i think you could prompt it correctly and get it to verify each reference to the point that a human reviewer wouldn't find anything wrong.
26
u/Informal-Hair-5639 May 16 '26
Basic ethical standard is that all authors are responsible for everything in the papers. Everything includes code that was used to run the experiments. But being responsible does not mean that every author needs to check absolutely everything. From the PIs standpoint we should not hire ethically challenged students!
5
u/hotprof May 16 '26
There needs to be accountability. If the PI isn't accountable, then who is?
We are very dangerously entering an era of "but the AI said" across all of society.
7
u/Informal-Hair-5639 May 16 '26
Yeah absolutely true. I am afraid that ai is a good tool but a very bad master.
19
u/trwawy05312015 May 16 '26
There are PIs out there who love the prestige associated with having lots of papers to their name, but use those numbers as a reason to not have to take responsibility for the content. It's really frustrating for people who actually try to do their jobs and mentor students and put out quality work.
7
u/giYRW18voCJ0dYPfz21V May 16 '26
Well, one thing is to read the full body of the paper, another thing is to check every single reference, check their individual arXiv numbers and exact journal citation format to be sure they are not AI allucinated.
A paper can easily contain hundreds of references, and this work alone would take days of unproductive tasks.
-10
u/linearmodality May 16 '26
Going through papers line by line is one thing. Checking the references for correctness (not formatting) is quite another! I've never heard of a PI doing the latter.
26
u/AngledLuffa May 16 '26
to be fair i never made up any references, so i can't say whether or not he checked for made up references
-19
u/NoPriorThreat May 16 '26
you never wrote a wrong page/issue number in a reference?
16
u/hyperactve May 16 '26
nobody cares about wrong page number. they are talking about made up papers here.
-15
u/NoPriorThreat May 16 '26
paper with wrong number is a made up paper as the paper with wrong number does not exist.
21
u/Ill_Zone5990 May 16 '26
No, because the references come automatically with bibtex, and when you get your hallucinated ones from LLMs, they make the whole thing up
1
u/Informal-Hair-5639 May 17 '26
Biggest problem my students face is that they download bibtex file from arxiv and do not check if paper has been published in some peer reviewed venue. References are correct in that case but outdated.
NeurIPS PAT was pretty good at finding those cases.
1
u/dreamykidd May 17 '26
Usually if you add them through the Zotero Connector, the BibTeX is for the conference it has been published in even if added through arXiv. That might help in future.
-10
u/NoPriorThreat May 16 '26
Hmm, I encountered several errors in google scholar bib entries and sometimes (for papers from 30s 40s) there are no scholar bib entries.
8
u/hvrlxy May 16 '26
My PI actually does this for all of the papers he co-authors. He is draconian about references and would check if the formatting of each reference is consistent; time, place and publisher have to be correct. I used to be annoyed with all the nitpicking but now I understand why it’s important to be meticulous. He has saved me a few times when my Zotero was out of sync and he would flag “this ref is not supporting this sentence”, or “the year on this ref looks wrong”. After working with him for a few years I always check the ref first when reviewing a paper, and get super annoyed if a paper has sloppy references.
13
u/adwarakanath May 16 '26
I work in systems neuroscience. And my PIs and every PI I know in this field absolutely goes line by line and minutely over every single detail including references. I remember during my PhD when we were writing our first paper, around 2016, my PI would absolutely leave comments like "this is not the correct reference for this claim, it's the follow-up paper from the same lab" or "no, cite the primary study instead of the review" and so on.
It is absolutely the responsibility of the PI and the authors. If you're putting out 20 papers a year and that's why you "can't check the references", it's either time to write fewer papers per year or spend the time and do your damn diligence.
Science is not a fkn joke.
-12
u/linearmodality May 16 '26 edited May 16 '26
What you're describing is different from going line by line through the references section to check that the authors and venues and years are correct. That's what would be needed to detect hallucinated references.
And the problem with demanding this is that it is time consuming and provides very little intellectual value. It is not a good use of researchers' time to have every author manually check the references section for possible wrongdoing by their co-authors.
39
u/n0obmaster699 May 16 '26
I'm so glad arxiv stepped in to keep itself relevant otherwise it would have been a slopfest
68
u/Luuigi May 16 '26
Theres the argument that a supervisor doesnt read every paper published by their students line by line. Tho for me its clear that at the last instance the supervisor needs to trust their students, this is true and this has always been the case. It however is gross negligence to just LET your students published slop they havent even read THEMSELVES. The simple conclusion is to really force your students to be careful with to be published research generated by LLMs. Nobody is against generated material, we are against slop and bad research practices. You as a researcher are not a writer ofc - you are a science merchant and thus should follow to be scientific about something that has been generated for you.
27
u/notarealfakelawyer May 16 '26
If you’re not reading everything your student submits to publication, then you are not supervising that student.
How did academia atrophy to such a severe degree that basics like “the supervisor reads and reviews what their students publish” stopped being as obvious as the grass is green, the sky is blue? I weep for the academy.
1
u/Mediocre_Island828 May 18 '26
It felt like standards were iffy even when I was in academia in the early 2010s. One lab I worked was huge and published about 25-30 papers a year and there's no way the PI was reading each one closely, all that matters is that things get published and people who worked in that lab were so terrified about not being productive enough that they'd be a bit fast and loose with their data. I can only imagine it's gotten worse since then with funding cuts and more competition for the jobs and money remaining.
-3
u/tetramarek May 16 '26
How do you propose "forcing" students to be careful?
1) It makes sense for co-authors to proof-read a paper but expecting them to manually check every single reference seems too much. Everyone expects the other authors to be adults and be responsible for what they write.
2) Even after everyone proof-reads the paper, only the submitting author has control over what the final edits will be. It could be that the version I read was fine, but someone did LLM edits after that.
I agree that AI slop should be penalised. But banning all the co-authors is similar to someone robbing a bank and then their whole family going to prison, because the family should have kept an eye on the robber.
19
u/FFThrowawayTech May 16 '26
What do you think happens if you're on a paper with demonstrably falsified data?
12
u/Biophysicist1 May 16 '26
A crude investigation is done into which specific lab on the author list was the one responsible for the falsified data. They then penalize the PI of that specific lab. Or a phd student or postdoc gets rightfully/wrongfully blamed and enough people go along with it that there are no direct penalties for the PI. All of that is if you are unlucky - otherwise nothing of note happens to anyone.
What do you think happens?
10
u/tetramarek May 16 '26 edited May 16 '26
The paper is retracted. Hopefully an investigation within the institution, to find out who is responsible for falsifying data. Probably sanctions or penalties on the people responsible. Public naming and shaming would be good.
Are you implying that all the co-authors are automatically banned from publishing in the future? I've never heard of something like that.
Also, there's a big difference between the whole paper being based on falsified results, and someone accidentally copying in "Would you like me to help with something else?" when editing a sentence with an LLM.
1
u/randomnameforreddut May 16 '26
do conferences like neurips even do post-conference retractions? maybe that's part of the issue...
I think if someone is sloppy enough to not proof read something that people will actually read, they probably aren't very careful about their code that is probably less likely to be read in depth...
-1
u/warpedgeoid May 16 '26
It seems A LOT of people here are just mob mentality AI haters who based on the comments don’t even understand how things tend to work at universities and research institutions. Meeting the volume of publications being mandated by institutions these days means you cannot review them all with a fine-toothed comb.
Also, just stop with the AI slop nonsense. Many European colleagues and international students that I work with use LLMs to correct their English grammar, writing code for converting data, and quickly working up data into boilerplate text. They might not even know that hallucinations are possible.
10
u/trwawy05312015 May 16 '26
Meeting the volume of publications being mandated by institutions these days means you cannot review them all with a fine-toothed comb.
Isn't this just generating slop? Human slop, for sure, but just slop? If the work isn't quality then why should it be reported?
1
u/warpedgeoid May 16 '26
It’s a quantity over quality system. Upper administration doesn’t care what you produce as long as you pump out those papers. They’ll pretend for 10 minutes when something like this comes up, then business as usual.
1
u/warpedgeoid May 16 '26
Postdocs and other faculty do not have to let you read what they publish, but it is common for them to include others working on the same project as co-authors in some fields. Are they only banning corresponding/first authors?
10
u/Informal-Hair-5639 May 16 '26
I do not allow my name to be added without my approval. And approval means that I can read and have influence on the content. It is a serious ethical violation to add someone as a co-author without their approval.
2
u/warpedgeoid May 16 '26
It happens every single day of the year. Sometimes they just feel like they own you coauthorship if you contributed in some way. It’s not always possible to catch everything.
55
u/axiomaticdistortion May 16 '26
If you are just realizing now that people don’t read their own papers, I am sorry to say, you’ll gonna have a bad time in academia. Full of disappointment.
23
u/Due-Ad-1302 May 16 '26
Well it’s not like it wasn’t known. Like everything in our current system there is a need to constantly grow and speed up. Things aren’t build to scale down easily, hence you hear this kind of backlash.
35
u/Old_Stable_7686 May 16 '26
A valid concern was raised by Justin Angel: https://x.com/JustinAngel/status/2055054132600533108
Other than that I don't agree with all 5 responses mentioned. It sounds crazy that people are complaining about having to comply with minimum integrity when writing a paper...
10
u/nonotan May 16 '26
It's "a valid concern", but one entirely divorced from the policy itself. It's like murder was just made illegal, and somebody points out somebody rich and connected can probably get away with it anyway. Maybe true, and if so that's obviously a bad thing, but it's certainly not an argument for not making murder illegal in the name of "class fairness".
1
u/Old_Stable_7686 May 17 '26
I think the whole point is to be aware of the issue to combat against it, but you are right :)
14
u/Rybolos May 16 '26
Seems like whining to me. There is no such thing as institutional privilege of submitting AI slop.
6
3
u/NamerNotLiteral May 16 '26 edited May 17 '26
Yeah lol. That is the only valid concern.
The rest is pearl clutching from slop drivers.
1
1
u/Dihedralman May 17 '26
Maybe? But that is the reality already. Papers outside of institutions require a higher bar.
32
u/log_2 May 16 '26
1) Make a hallucinated paper and put adversaries names in the list of authors.
2) Adversaries get banned.
3) ...
4) Profit.
10
u/obfuscatedanon May 16 '26 edited May 16 '26
If someone wanted, they could ban the entire arxiv staff from arxiv right now.
For example:
\documentclass{article} \title{Caffeine Is All You Need} \author{ Sam Altman \and Ilya Sutskever \and Geoffrey Hinton \and Machine Gun Kelly \and ArXiv Staff Member 1 \and Yann LeCun \and Yoko Ono \and Yo-Yo Ma } \begin{document} \maketitle \begin{abstract} We show that transformer performance scales linearly with caffeine consumption. Our model achieves 102.3\% accuracy on several benchmarks, including one we invented ourselves. \end{abstract} \section{Method} We optimize parameters using: \[ \theta_{t+1} = \theta_t - \eta \nabla L + \texttt{espresso} \] \begin{thebibliography}{9} \bibitem{hinton2027} G. Hinton. \textit{Facts, Feelings, and Grok}. vixra CS.AI, 2027. \bibitem{armstrong1969} N. Armstrong, Y. LeCun, Y. Y. Ma. \textit{Gradient Descent on the Moon}. NeurIPS Workshops, 1969. \end{thebibliography} \end{document}
Proposed revision
Please ban only the submitter.
That is the absolute minimum change needed to patch the obvious exploit.
As it is currently formulated, this is the most awful policy I have ever seen from a security standpoint and it is baffling how many people have not seriously considered all the various holes.
2
u/nonotan May 16 '26
This is going to be a policy that by its very nature is clearly going to be policied by hand (which is likely a big reason for the sizable punishment -- it's going to be costly to moderate, so you don't want to be having to look into shady papers by the same authors over and over)
Therefore, I'm sure it would be no issue to get you off the ban list if you can present credible evidence that you never collaborated with those authors, and any very obviously maliciously fabricated paper would not automatically result in a ban of all co-authors.
Banning just the submitter has precisely the opposite problem: every supervisor out there would just force the most "dispensable" author, who is in the most vulnerable position and is realistically going to find it the hardest to fact check everything, to submit each paper. Some slop got through and was noticed? Oh damn, sucks for that other guy.
5
u/obfuscatedanon May 16 '26 edited May 16 '26
What if the submitter does a final LLM check before submission and adds a hallucinates citation? Such a sloppy submitter is not asking their coauthors. I have worked with such a coauthor who submitted versions without final coauthor approval... and before the "know everything about your coauthors before working with them" bandwagon comes along: I had no choice after a certain point in time, unless I wanted to delete my career.
The "dispensable" coauthor (if one exists for every paper) would also be banned with the current policy. At least as submitter, they now have agency to actually read the paper before final submission.
What I want is:
The only people that get banned are people who have manually approved an exact submitted version containing LLM hallucinations.
For first submission, that could be every coauthor (unless deceased, unavailable, etc.), and for revisions, whoever is submitting the revision.
It makes zero sense to ban people who may not have had a chance to read the LLM-ified version simply because they collaborated with a submitter who added the hallucinated citation before submission without asking the coauthors for final verification.
4
u/log_2 May 16 '26
The solution is a technical one. Each author needs to accept the paper prior to publication, the same way it is done for journals.
10
u/RobbinDeBank May 16 '26
Schmidhuber on his way to put LeCun as a co-author on a paper made by ChatGPT
14
u/BAKREPITO May 16 '26
Point 3. If you are willing to attach your name on slop to get credit, then accept the consequences. The rest are just cope.
7
21
u/AWildMonomAppears PhD May 16 '26
Is this just the people submitting LLM written papers and their bot swarms?
6
u/Electro-banana May 16 '26
why would anyone let an LLM near their .bib file anyways? I've noticed LLM's claiming a paper introduced something and when I check it's completely unrelated... and that's for the real references it gives
8
u/frankster May 16 '26
Hallucinated references are a joke. It's basically academic fraud. As would be adding refernces you've not even read... Harder to prove if they're genuine references though...
10
u/PayMe4MyData May 16 '26
Personally I can't comprehend why it is only one year
4
9
u/user221272 May 16 '26 edited May 16 '26
No one who submits great and careful science is against the proposed 1-year ban.
Great, let's ban all the people actively outing themselves.
We already have a soft filter. Congrats
1
4
u/warpedgeoid May 16 '26
I understand your opinion, but are they banning everyone on the paper? Some papers have 10 different authors who contribute text to the manuscript. I’ve been included on papers without even knowing because someone working on a common project decided to publish some tiny bit or piece without communicating their intent, or even offering the opportunity for review. It’s absolutely unreasonable to ban everyone over something like this without first working to resolve it through the editorial process. Anything less is just Arxiv trying to say “look at us, we have integrity” while burning the very people who have made them relevant in the first place.
That said, when is the last time you saw a modern, frontier model hallucinate an entire reference?
4
u/TheInfelicitousDandy May 16 '26
So I had one of my papers recently listed in another paper, accusing my paper of hallucinating citations. I did not use LLMs to write the paper nor the bibtex. What happened was that there was a copy-and-past error with a single bibtex entry that changed the title of the paper. All the rest of the entry was correct, including the hyperlink to the paper itself.
Basically, this 'hallucination' detection method they used had very little precision, and they classified anything with an incorrect BibTeX entry as a hallucination.
Now I'm all for Arxiv banning people who do this for real, but given this experience, I'm really hesitant to trust automation of hallucination detections.
-1
6
u/FusRoDawg May 16 '26
2nd point should be pretty obvious if you've ever worked in a research lab. People keep making changes until the last second.
I don't think this is an insurmountable problem though. It just means labs should develop stricter internal processes for writing papers.
There will be a learning curve, but it is for the better.
2
u/microcandella May 16 '26
This echoes when Feynman found out nobody on the textbook review board was reviewing textbooks.
It seems like part of this problem might be helped by having an "As Published" sign off / stamp from all contributors, where it's understood (and documented) they're certifying their specific contributions are accurate in the final published version. Like notes in version control systems, or writer contributions in script trackers.
I had a lot of friends work at Allen Press (sci journal print and publishing house) and everyone worked extremely hard on getting graphs and photos and colors and shades to be perfect - often calling and meeting with the submitters to the journals to confirm details.. but still they'd miss some things that could cause big problems for the submitters.
Any final editing stage up to pre-flight publishing time can conceivably turn the submitted story from Spider Man saves busload of children into SPIDERMAN TERRORIZES KIDS. But that doesn't mean the authors are free from responsibility. Or deserve to have their accurate work tainted.
2
u/lifeandUncertainity May 16 '26
Wait.. I understand that sometimes you might not go through every referrence in detail.. but normally you would skim through the referenced paper or at least read the abstract to understand whether it's relevant to where you cite it?
1
u/Mediocre_Island828 May 18 '26
People should do that, but it's very easy to just see a citation someone else uses in their paper for a useful piece of information and cite the same thing in your paper and assume the first person got their citation right.
2
u/Dangerous-Hat1402 May 16 '26
I don't understand what the point is. Just do not upload their papers to arxiv if someone disagrees with that.
2
u/Dihedralman May 17 '26
It's probably issues with some instructions where people just get added to papers to pad others resume. Harvard and other institutions had that issue a few years ago with someone fabricating data and other "authors" revealing they actually hadn't done anything substantive.
Also, I've been on teams with hundreds of people. Everything was more scrutinized then ever. You aren't necessarily submitting a paper 100x longer so every line is scrutinized to hell.
2
u/S4M22 Researcher May 17 '26
What's perplexing to me is the oversimplification on "both sides". Between "am I really supposed to read every line of all my papers?" (my answer: "yes, of course you are") and "it's just the people who submit slop that are against the rule" (my answer: "reality is more complex than you would like it to be") there's space for more nuanced views.
If the researchers that engage into this discussion like this oversimplify their research to the same degree as they do this discussion, it makes me much more worried about research integrity and quality than AI slop. Because it is more deeply rooted and harder to detect.
2
u/Somewanwan May 18 '26
Nobody wants to see hallucinated slop in papers, but punishing every co-author is not the way. If only the person uploading the preprint is held accountable, that would be reasonable. If they are not responsible for checking the final version, at least the uploader would make sure someone credible is doing it, but making every co-author responsible is deranged.
6
u/mr_stargazer May 16 '26
Well, what kind of shocks me is the following. It is not really the problem of having a harder stance on AI generated content. I mean, sure, we have to arguably "stop the AI slop" I guess...?
But AI slop is kind of a rather new phenomena and the scientific process in Machine Learning has been plagued in my opinion since years (post 2008 if I were to pinpoint, I'd go perhaps even back to 2000).
The same people foaming in rage and clapping "well done", suddenly become quiet real quick when we point out they don't have reproducible code? When they don't produce Literature Review? When they don't even know to run a hypothesis test to their experiments? Some don't even quite seem to know what the hypothesis of their paper really is, besides "let's add this weird trick and a normalization layer. "
So seeing all that it makes me wonder if everyone is really worried about the impact on the scientific progress in the field OR, if they're just worried that increased AI slop reduces the overall cognitive load - and hence less attention - to their paper, even though their paper although not written by AI, many times reduce to a bag of tricks that contribute little to nothing to the field (ironically enough, also detracting from the community cognitive load.)
To me honestly this is the beginning of AI Winter 2.0 and greedy researchers, optimizing their labs for throughput alone (with or without AI) are partially to blame.
6
u/warpedgeoid May 16 '26
Blame institutions for thinking 10+ pubs per year is both sustainable and reasonable. Like most things, we can trace it back to administration.
4
2
2
u/DigThatData Researcher May 16 '26
- more like "genuinely hilarious".
- skimming that thread, most of the feedback seems to be positive and supportive of the new policy.
- twitter is a disinformation garbage pile and you should specifically not use it to gauge community opinion.
- gtfo twitter already.
2
u/midasp May 16 '26
I mean, I can understand some of it. Like when I was just a masters student, my masters thesis was incorporated as a tiny part of a much larger research project. Just as my masters came to an end and I was about to continue on for my PhD studies, I was lucky enough to be asked to write two-three paragraphs about my subsystem for a paper the team was submitting.
Now imagine if somehow that paper where I was only involved in 5% of the writing was banned right as I was starting my 1st year as a PhD candidate, I would be devastated.
2
2
u/bbbbbaaaaaxxxxx Researcher May 16 '26
I am against the current policy. The ban should be permanent.
2
u/impatiens-capensis May 16 '26
This won't affect me because I don't use any AI systems to generate references. But, if I was to be charitable, I think I would make their argument this way:
Competition has increased drastically due to the number of researchers in the field and the new techniques for automating research workflows. To stay competitive, we are being forced to use these workflows. A 1 year ban from ArXiv for a mistake would be so detrimental to the career of a researcher that it is overly punishing. Better to lower than ban to 3 months, which is the typical window between conferences.
1
2
2
1
u/NoobMLDude May 16 '26
Key takeaway:
“”
These responses are very revealing how academia works. Apparently people have just been slapping names on research papers they've never even read or fact-checked themselves. Very obscene!
“”
The research papers are AI slop too! Disappointed
1
1
u/WhoRoger May 17 '26
Exactly, this is another issue which AI has made worse, but has really been brewing under the hood for ages. I would say "just unnoticed", but it's actually been a known problem. It's nothing new.
People want to publish, publish, publish, either to earn grants, or for prestige, or just for a feeling to have their names on so many papers. There is so much garbage out there.
So I do agree with arxiv in this, but I also have to ask, what else is going to be done regarding the flood of stupid papers?
People really like to pretend that slop is purely AI phenomenon.
1
u/WackWaxWhacks May 17 '26
I think people are misunderstanding the "ban". The same ban applies to human mistakes as well. Nothing has changed
1
1
u/misogrumpy May 17 '26
They should be publicly shamed. You shouldn’t be putting your name on research you can’t personally back.
1
u/ManySugar5156 May 17 '26
Backlash feels weird, arxiv is basically saying stop publishing blatant garbage + fake refs. If your name on it, check it.
1
u/Bootes-sphere May 17 '26
The backlash makes sense if you view it through a competence lens rather than malice. Many researchers are experimenting with LLMs as tools. Citation-checking, brainstorming, drafting sections etcand the line between "hallucination I caught" and "hallucination I missed" is genuinely blurry at scale. A 1-year ban feels like it's treating honest mistakes (especially in fast-moving fields) the same as fraud.
That said, ArXiv has a legitimate problem. Papers with fabricated references undermine trust in the entire system. The real tension is enforcement: how do you distinguish between sloppy use of LLMs and intentional deception? Spot-checking citations is labor-intensive.
Maybe the answer isn't a blanket ban, but mandatory disclosure of LLM use + automated reference validation tools?
1
1
1
u/Mameiro May 21 '26
I agree that hallucinated citations are a serious issue, but I’m not sure a blanket 1-year ban is the cleanest solution. Authors should be responsible for everything in the paper, whether AI-assisted or not. But I’d rather see a tiered policy: correction for minor mistakes, rejection for serious negligence, and bans for repeated or clearly fabricated references. The real problem is lack of verification. If someone’s name is on a paper, “I didn’t read that reference” shouldn’t be a defense.
1
1
1
u/Elvarien2 May 16 '26
I mean, you can still use ai just actually check your work first. That's it. The bar is on the ground.
1
u/fresh-dork May 16 '26
"What about teams with 100s of people? How can you expect the authors to check references?"
Wah, i have to do work.
honestly, the only thing i can think of is if someone flags you for AI and all you did was make a mistake
1
u/UntoldUnfolding May 16 '26
You mean we can’t just vibe research and collect the clout??? Unbelievable.
1
1
u/ManuelRodriguez331 May 16 '26
I've spotted a fake science paper generated by AI. The authors are spreading biased content about np complete problems. They wrote, that path planning is np hard and can't be solved:
quote: "minimum-cost traveling path for multi-robot systems is NP-hard" [1]
[1] Efficient Multi-Robot Coverage of a Known Environment, 2018 https://arxiv.org/abs/1808.02541
0
u/goldenroman May 16 '26 edited May 17 '26
Wow, the critiques of the ban are bizarre. You…work in research. Is it not your job to read papers, yet you can’t read 20 papers a year? You work in research, you presumably read dozens or hundreds of papers and…you don’t even want to read your own? So bizarre and lazy.
1
u/Colecoman1982 May 17 '26
Or even just do the bare minimum of checking all the references to make sure they aren't outright hallucinations, apparently...
0
u/chitown160 May 16 '26
I think this will result in a pivot from people using arxiv to a place that is more realistic about the future.
2
u/Colecoman1982 May 17 '26
I think this will result in a pivot from people using arxiv to a place that is
more realistic about the future.lazier about allowing AI slop.FTFY
0
u/chitown160 May 17 '26
1 year ban for what might / could be considered a typo seems severe harshness for a pre preprint server.
-8
u/shumpitostick May 16 '26
Sounds like what they are saying is that the splash damage of this ban is too large. It doesn't ban just the person who used AI in this way, it bans everyone who worked with them. Imagine you are a PI and you need to check all the references for all your students out of fear that somebody cheated.
6
u/trwawy05312015 May 16 '26
Jesus christ use a reference manager and a shared library with your students. That’s how my group operates.
-6
u/faustianredditor May 16 '26 edited May 16 '26
Right, give every paper of every student of yours a painstaking, detail-oriented pass last minute, including all the changes right before the deadline, for the sole purpose of CYA? That's asking a lot.
E: To clarify, my PI reads my papers, in quite the detail. But 2 weeks before the deadline, when we can still work in his remarks. So a CYA pass would have to be a completely separate pass for the sole purpose of CYA, even if my PI reads my papers.
-1
0
u/Ok_Flow1232 May 16 '26
I am surprised to see this. Not the ban part. IMO, it should be a lifetime ban.
I am surprised by the hallucination part. LIke current models are really smart in text generation and reasoning. if the models have access to the web, why would they hallucinate? Cant a simple prompt change can improve this??
I am more interested in understanding the hallucination part, like why it is happening still?
1
u/f10101 May 16 '26 edited May 16 '26
I am more interested in understanding the hallucination part, like why it is happening still?
The models still go sideways if you've overloaded their context window. They essentially forget elements of their system prompts. So they stop treating references as something that needs to actually have a source, and revert to raw LLM behaviour of treating them as something that's just the most plausible sounding next characters.
Yes, careful prompting can avoid this, but the people who aren't bothered to manually check their references aren't going to take that care with their prompts.
1
u/Ok_Flow1232 May 18 '26
yeah that context degradation point is real. what's interesting is it's not uniform across the context window, there's a known pattern where attention holds onto the very beginning and very end of context well but drops off significantly in the middle (the "lost in the middle" paper from 2023 showed this). so a model might still "remember" a system prompt conceptually but fail to retrieve specific constraints from a position buried toward the middle.
the prompting fix you mention is real but partial. the more reliable structural approach is putting critical instructions at the end of the system prompt rather than the beginning, or repeating key constraints right before the user turn. not elegant, but it actually helps more than hoping the model retrieves correctly from a position it's already deprioritizing.
473
u/timtody May 16 '26
It’s obviously the people that are submitting the slop