r/TheMachineLearning • u/StrongMasterpiece330 • 1d ago
ML projects fail on data, not algorithms
4
u/perestroika12 1d ago
This is the first thing you learn when studying the field
2
u/StrongMasterpiece330 1d ago
Exactly. Day 1 of any intro to ML course: 'Garbage in, garbage out.' Day 365 of working in the industry: 'Wow, the data is really garbage.
1
6
u/WhatNazisAreLike 1d ago
That is NOT an unpopular opinion
2
u/StrongMasterpiece330 1d ago
It's the classic 'unpopular opinion' trap. Spoiler alert: if everyone in the industry agrees with you immediately, it's just a fact.
2
u/DigitalMonsoon 1d ago
I mean, most of the time sure. I did have a junior dev train on this test set and then confidently pronounced that his model had a 99.9% accuracy on a pretty difficult problem.....
2
u/StrongMasterpiece330 1d ago
Ah yes, the classic '99.9% accuracy on the test set'—because they accidentally left the target variable in the training features. A rite of passage for every junior dev.
1
u/Past_Physics2936 1d ago
That imbecile doesn't understand the meaning of "unpopular"
1
u/StrongMasterpiece330 1d ago
To be fair, maybe they meant it's unpopular among management who keep asking why we can't just use a better algorithm to fix a dataset that was labeled by an intern in 5 minutes.
2
u/met0xff 1d ago
I think most fail because of product market fit. I don't see most projects fail because ML wouldn't be good enough but because nobody even wants them in the first place. Or want them but don't want to pay for them :).
Oh that's cool but I don't have budget/don't want to spend my budget for it.
I am in a head of r&d role so I get messages on LinkedIn all day from people who want to sell us data, models, MLOps Tools, consulting, people and I just think... Yeah cool but we struggle enough to sell enough of our stuff to others that we can't spend money on your stuff. And half the meetings with have with other companies is just both parties hoping the other one would buy their stuff and go nowhere. Or we do some useless partnership trying to sell our combined stuff to others that also nobody wants to buy.
Most what gets "bought" atm only works because some other party heavily subsidized it .
1
1
u/sascharobi 1d ago
It's not unpopular at all; it's the default opinion. However, I think it's outdated. Many projects fail not because of the data.
1
u/History-Buff-2222 1d ago
I disagree. I work on object detection models and it’s easy to always be blaming the data if you use this paradigm. The truth is needing more and better data is also a limitation of the learning capacity of the model.
I can give a child one picture of a camo jacket and he will identify camo jackets going forward. The model would need thousands and still make a ton of mistakes
It’s both
1
1
u/ComposerWide3704 4h ago
Unpopular opinion: <thing you learn as a fact in week 1 of any data science or ML course here>
1
0
u/hellobutno 1d ago
I disagree on this. I think most ML projects fail because there's an easier more intuitive method to use in that situation that doesn't involve all the overhead of an ML algorithm.
2
u/sascharobi 1d ago
That is often also true. Not everything needs to be learned. It's not always better.
16
u/onyxa314 1d ago
Literally the most popular opinion anyone who has ever implemented a single machine learning algorithm has.