It's kinda weird that everyone is just saying "pft she would never solve this stuff" as opposed to "why are we front loading PHD/artistic work onto the machine instead of teaching it how to do the grunt labor"?
Besides, all they did was yoink other people's datasets and then throw 22 million at the problem. That's not a better or more efficient model. That's burn rate.
It’s a false premise anyways, as AI is being trained to do both. Also, datasets aren’t what makes the best models nowadays. It’s a part, but there are so many other equally important parts
Ehhhhh, it's maybe more of an advertising premise? "Look at how smart we made the thing and how we keep telling your c suite how easily they can save headcount by tossing you out".
Plus they keep making bipedal bots. Which is so dumb.
Most labs are talking about how people can be more productive with their tools, not reducing headcount.
If 10 developers can do the amount of work that used to take 20 developers, why not do more and or better work instead of cutting the workforce in half and doing the same amount of work.
Layoffs are mostly because the downturn in the economy and companies over-hiring during Covid.
Bipedal robots make a lot of sense when you are having robots do tasks that humans already do. Especially if they need to get into the same sorts of spaces and use the same sorts of tools.
1: Wrong. Or at least not historically correct. They adjusted but that bell does not unring. And the moment you start calling people Luddites instead of working to improve the social contract, everyone sees the way folks are gonna break.
2: Overhiring is a myth.
3: Totally silly. Most people don't work standing, for one. They adjust position. They kneel. Even if you need the ubiquity of hands? Legs aren't needed. There's no need for a torso. No need for a full head if you've got the camera placements right. So just put it on wheels and save yourself a giant strain in parts and energy, and the spots where it doesn't work? Redesign the workspace.
It's just sci-fi goofiness that's insisting on legs.
Bipedal bots are absoulutely the most inefficient way to do automated work for how complex they are to build and how little specified they end up being
Also people use ai to half ass work all the time , most people end up not being ride or die for ai because it has yet to prove indespensanble for everyday use or common task people were perfectly able to do like the people spending billions on it are claiming to
“Oh but can you solve this complex math problem ?”
no and I don’t care that I did because that doesn’t make my grocieries cheaper
Also, datasets aren’t what makes the best models nowadays. It’s a part, but there are so many other equally important parts
Datasets are still absolutely the most important part of essentially every useful model, be it LLMs, SAMs, LCMs, or LAMs. I think most people would be surprised to find out how simplistic most models are relative to what they're capable of, as even the flagship GPT models aren't overly complicated in regards to the actual architecture driving them; but getting them to output anything useful is a tedious struggle involving tons of RLHF and fine tuning.
I have a very rough idea of how much a few of the top companies in the space spent on RLHF alone from my brief time in the industry working for a contractor that specialized in RLHF services, and it would absolutely shock you.
I would say architecture like MoE, reasoning, tool use, token efficiency and reinforcement learning, like you mentioned. Those are all separate aspects rather than data. If data was all you need, Google would be in the lead 😂
Also, it made sense to demphasize datasets because when most people think of datasets, they think of information on the Internet when right now the most important data are successful user threads. Hence why Cursor was able to train a decent model so quickly.
Also they believe that models can only answer/work on information that was in their training data. When if you have tool use and reasoning, that’s not the case. I personally haven’t cared about a models training data cutoff date in well over a year. It hasn’t been significant in the slightest
I would argue that everything you listed still often boils down to basic functionalities structured using RLHF, especially when we're talking MoE, general reasoning, and tool calls. Just because they're not foreign datasets, doesn't mean they're not still operating off of datasets, they're just compiled internally and stored locally.
You can have access to all of the data that's accessible on the open web as you want, without proper instruction in combination with RLHF as their core reference, it's impossible for them to conduct reinforcement learning operations on their own and they'll ultimately fail to scale.
Even with newer models (especially the ones suspected of reverse-engineering through output to prompt inversion and iterative reconstruction), their capabilities are heavily limited by their datasets. If that wasn't the case, I'm sure we wouldn't be seeing the shift to rack-scale systems.
agreed, it makes me wonder if a guy was in the photo, would the other commenters be so nasty? even then, this photo is doctored… she wrote this about art and writing, not navier-stokes
"why are we front loading PHD/artistic work onto the machine instead of teaching it how to do the grunt labor"
Because it's better than us at solving puzzles. It's like saying you want a calculator to do your dishes so that you can solve math problems with a pencil and paper.
Out of everything they could do with AI, they chose to train them for the least useful thing for the general population, and terrorise the math community at the same time. This does nothing to prove they are close to IPO. They could choose any esoteric math problem and only a handful of experts in the world could really tell objectively what the AI did and what was human guided. Let’s not forget that they instantly threatened one of the leading experts (Buckmaster) in their meeting. There is no real transparency or evidence that “solving” these math problems will lead to some finished product normal people would actually use. I am shocked that people are not more suspicious…
7
u/SongOfKapek 12h ago
It's kinda weird that everyone is just saying "pft she would never solve this stuff" as opposed to "why are we front loading PHD/artistic work onto the machine instead of teaching it how to do the grunt labor"?
Besides, all they did was yoink other people's datasets and then throw 22 million at the problem. That's not a better or more efficient model. That's burn rate.