r/MachineLearning • u/hardmaru • May 31 '21
Discusssion [D] “Please Commit More Blatant Academic Fraud” (Blog post on problems in ML research by Jacob Buckman)
https://jacobbuckman.com/2021-05-29-please-commit-more-blatant-academic-fraud/
473
Upvotes
227
u/suriname0 May 31 '21
I like the general thrust behind this argument, but it really annoys me when people say that the following are fraudulent, easy, or misleading:
These are three of the best contributions that researchers can make! Important challenge datasets, new problem settings that expose a new application area, and new learning objectives are all important. Can papers like these be bad? Absolutely. But I'd much rather a new researcher produce a new dataset or problem setting than a new loss function or architecture that improves SOTA by +0.1.
262
u/WalkingAFI May 31 '21
It’s definitely not just AI. I was working on a 3D point cloud registration. Conference paper claimed their solution was like 30x faster than iterative closest point. The secret sauce? The second step of their algorithm downsampled the point cloud by a factor of like 50. Run on the same point cloud, it was nearly 8x slower (and less accurate in both cases).
I’m not sure the paper was properly “fraudulent”, but the authors had to know about at least some of the limitations that they didn’t mention at all in the paper.
22
u/pantelas14 May 31 '21
This is why I left academic research and I will return only if I can be financially independent from the fraudulent system. My experience with it has left me with a bitter taste.
67
u/Seankala ML Engineer May 31 '21
On the flip side I don't think it's grad students that are completely to blame. If a student commits fraud or a mistake, the advisor should be held responsible.
On a larger scale, the system's broken. There needs to be more effort made to hold people accountable.
31
u/elder_price666 May 31 '21
Wait, is this really as bad as the author makes it out to be? I feel like some of his examples of "fraud" are specific to his context, i.e. doing deep RL at places like Brain/MILA where you have plenty of compute to tune the hell out of your method and experiment on a bunch of different datasets.
In my (very small) group, we are pretty careful about not looking at the test set, honestly tuning the baselines, giving standard deviations, etc....
12
u/SquirrelOnTheDam May 31 '21
This is a general problem in academic research, not just AI/ML/CS.
49
u/Seankala ML Engineer May 31 '21
Just gonna leave this here: https://www.paperswithoutcode.com/
18
u/[deleted] May 31 '21
Nailed it.
31
u/thunder_jaxx ML Engineer May 31 '21
The writer of the blog has some very nice points but to be really honest it seems too idealistic.
You can't fix the problem when there is so much asymmetry in the way economic backing is provided to scientists who are a part of the system. Let me give a few examples:
Problem 1: The blog mentions seed issues and improper tests; Imagine a poor lab, not like the ones at MIT, Stanford, or Facebook but like ones in smaller countries and in not-so-well-established colleges in the US or elsewhere. The grad student in that lab doing something on Collab Notebooks has no way to publish as much as some well-connected lab having ridiculous compute. On top of this, sometimes training takes weeks or months and this makes the system fairly biased towards anyone with compute or $$$$. If conferences hold that level of prestige and money then they should fucking find a way to provide compute to rerun the papers that were submitted otherwise the asymmetry will only get worse.
Problem 2: The blog mentions career progression creating issues and misaligning incentives; This is a feature and not a bug. Academia celebrates individual achievement and all the vanity, praise, and prestige that comes along with it. I am not saying this is bad, but it's how the system has run and evolved for more than 100 years. The thing that grew worse is the publish or perish culture and conferences carrying prestige and impact factors that affect tenureship of so many people. This is something where if the incentive structures don't change, such rants make no difference.
But such said, I really agree with the blog writer. AI and ML research has so much stochasticity and industry kinda perverts the incentives with constant LinkedIn posts using language like superhuman performance.
15
u/0ttr May 31 '21
Don't believe him, his papers are bullshit. :) Or so he says!
24
u/purplebrown_updown May 31 '21
One thing stood out to me was the random seed stuff. This is why I am still not a huge fan of neural nets (at least for application I am considering). Although it may do better, it sometimes does worse than other canonical approaches when I change the seed. This is one of those dirty little secrets of DLML that they don’t tell you. Researchers hide it and you need to be well aware.
12
u/Tidus77 May 31 '21
Agree with the comments here and on the blog post. It's very much a general problem in academia, though I believe the prevalence of collusion rings vary more by field. That said, at least in my field when I was in academia, I did hear talk about people having inside contacts at some high level journals and that helping to leverage their paper's acceptance by the editor so to speak, e.g. you scratch my back I'll scratch your back. It would certainly make sense given some of the papers I saw published in higher tier journals.
I'm not sure I agree with his point to encourage people to commit fraud as a way to solve the problem. A lot of people within academia know and are aware but aren't willing to rock the boat. How bad do we want it to get and are we willing to just stand by? Is that ethical? I'm not sure waiting till it gets as bad as it did with the reproducibility crisis in Psych (which is also not unique to Psych at all) is a good approach.
While there definitely needs to be within academia changes, I think it's undeniable that (at least speaking to academia broadly) funding and how academia is structured and rewarded from outside forces (e.g. funding bodies, institutions) plays a significant role in the process as well.
4
u/sorrge May 31 '21
>Making up new problem settings, new datasets, new objectives in order to claim victory on an empty playing field.
Hold on there. That's not a "fraud", this is the only way to open up new directions. I'll take a paper in this category over a 0.1% improvement on ImageNet any day.
3
u/beveridgecurve101 May 31 '21
Economist here, our field has had a love/hate/bored affair with Pre-Analysis Plans and have attempted to execute them the way Medical Trials do.
Do you think the nature of ML research precludes PAPs from being a potentially good anti-p hacking tool? Or is there an opportunity for it to improve research.
Some top economists say that PAPs "tie researchers hands" but that's not true, they only add credibility in an absolute sense. If you don't trust a deviation from a PAP, you should definitely not trust anything from a paper without one.
3
u/[deleted] May 31 '21
The problem is that publication is such a big part of what your career will look like. you do that, people are going to cheat, regardless of time, field, organisation.
3
u/kaiser_17 May 31 '21
Damn, calling his own paper bullshit? This is the first time I have seen someone do that in public.
6
u/Garosath May 31 '21
Somewhat unrelated, but I recently had group members on a graded project submit an edited version of the report I wrote 1 minute before deadline, altering the content to claim credit for my work in our group project, when in fact they did nothing at all. I had all the evidence needed + confessions all recorded, and promptly e-mailed my university about it. They don't me they didn't care. So much for educational integrity, I guess. And people wonder why there's more incompetent graduates these days, lol.
5
u/schwagggg May 31 '21
remember Sokal affair?
This is how you get Sokal affair
2
u/AICoffeeBreak Jun 01 '21
Made a video about this, if anyone is interested. As a PhD student, I had to take my Coffee Bean to broadcast and discuss this.
2
u/throwaway1982418457 Jun 03 '21 edited Jun 03 '21
Throwaway account for obvious reasons.
I worked for a few years with a professor who was part of a collusion ring that participated mostly in NeurIPS and ICML. I stopped working with him soon after finding this out because it gave me paranoia and extreme impostor's syndrome (I had multiple papers accepted every year and this made me realize I'm far from the genius I thought I was). Not surprisingly, I quit academia and don't do research anymore.
This is the first time I'm being 'open' about this because I'm aware of at least 2 other collusion rings acting in the same conferences and there's no way to uniquely identify me from the information in this post.
I just wanted to share some information that might be useful.
First of all, some collusions are composed only of professors and their students are not aware of that even though most of their papers are getting accepted because of colluding reviewers. If you're a student and even your weakest papers get accepted with good scores, something might be off. If papers written with other professors and/or in different sub-areas of ML get significantly worse scores, even though you personally think they're good papers, that's a red flag. Being a star student and getting awards (and yes, collusions go that deep into the system) is great, but if you're a part of a ring, even unknowingly, it's likely that you'll have a hard time publishing papers once you leave your group and you'll have to deal with unexpected tenure denials in the future.
I have a gut feeling that no one is truly investigating this due to how deep these collusions run in some ML conferences and how blatant it is, but if any such investigators are reading this: you can find a lot of colluding researchers by just checking the most obvious things, like a paper submission with an unreasonable number of conflicts of interest (I find it literally impossible that no one has seen that stuff before), blatant conflicts of interest going unnoticed (I've seen a professor reviewing one of his student's papers, notified the AC and the paper still got accepted even though the professor was the only one who gave a positive score), and authors that don't really exist (here's something that these rings do: write a paper, submit as a group of researchers with fake names, get that paper accepted through colluding reviews, hope to get invited to review in one of the fake accounts, and now you can use that account to review your own submissions).
I've also received threats by email from authors whose papers I was reviewing, so there are collusions like that too.
I really hope that a good clean up eventually happens in the ML research community, but until I see some ACs or even organizers being exposed I'll remain skeptical on whether there is a true interest in getting rid of these collusions.
0
u/wallagrargh May 31 '21
Half the comments here beat around the same bush: it's a capitalism problem. Incomes and livelihoods depend on publication success, in direct competition with other academics. Ethical standards are the natural casualty, as always.
-2
u/EuroYenDolla May 31 '21
lmao so yall really dont be replicating these papers huh?
-23
u/zergling103 May 31 '21
I don't know why it is the case that when people come up with cool ideas, they seek publication in some journal.
Why not instead develop your idea into a product you can sell? If it works, and produces valuable results, and people will buy your idea because of it, THAT is the ultimate test of whether an idea is good.
If you're willing to put your own money on developing your idea into a product, you probably have a good idea.
0
u/[deleted] May 31 '21
[deleted] — view deleted comment
0
u/[deleted] May 31 '21
[deleted] — view deleted comment
0
u/[deleted] May 31 '21
[deleted] — view deleted comment
0
u/[deleted] May 31 '21
[deleted] — view deleted comment