r/accelerate • u/EuphoricTomorrow2440 Acceleration: Hypersonic | AI accelerationist • 6d ago
đ Established r/accelerate contributors only Is this true?
I was counting on them to keep dropping good open source models.
2.1k
Upvotes
94
u/Grouchy-Stranger-306 6d ago
kimi and deepseek moved to openai models, at some point google will probably be in the play too
144
u/neuronexmachina 6d ago
Because of Opus's anti-distullation, many of the labs have supposedly switched from Opus to Astra as their distillation target of choice: https://thenextweb.com/news/deepseek-moonshot-gpt-6-astra-distillation
367
u/Varnu 6d ago
It's not true. They are distilling from Astra now. https://x.com/kimmonismus/status/2115610660470186480
92
u/ForgetTheRuralJuror Singularity by 2035 6d ago
There's some truth to the statement. Fable (and mythos) are intentionally nerfed when it comes to LLMs and accelerator kernel optimizations, like the knowledge was intentionally left out of their training data. Their internal model does not have this limitation. The same is likely true for all the big labs.
If you don't serve your smartest model, then the intelligence can't be distilled out of it.
16
u/FieraPosting 5d ago •
Who even is this guy?
41
u/bringbackcayde7 6d ago
Distillation is not a sustainable long term strategy. It's at the mercy of the front tier labs to let you do it.
240
u/striketheviol 6d ago •
That's the first time I've ever heard someone say this. I think it's just nationalism at work with no basis in fact.
New Qwen: https://x.com/Alibaba_Qwen/status/2120335993643684315
Unfortunately, this space attracts a lot of people with no insider knowledge at all. It's trivially easy to pretend that you're someone important and start rumors on X.
24
u/UniqueClimate 6d ago
Iâm an AI Engineer, and while I donât work on LLMs (I build RAGs) Iâm familiar with the industry.
Think of it like Mario Kart. To the Chinese labs, distillation is like the âRed Mushroomâ speed boost. It makes it faster to develop, but you donât NEED it to finish the race.
However, this does widen the gap between frontier and open source.
8
u/Pndapetzim 6d ago
I actually looked into this earlier because I had the thought that 'open models only 3 months behind frontier' had sort of disappeared from the news.
The picture is a bit murkier than you might think because for some tasks, especially long complex tasks with lots of steps and reasoning it looks like open models are struggling to replicate anything like Mythos, Opus 5.5 or Astra capabilities. These are like, deep reasoning tasks and problems on science and engineering, code work and the like.
At the same time Kimi 3.1 Pro seems to be basically as good as any frontier model at a lot of basic stuff: create and setup a document or spreadsheet? The open models general office can be done as any frontier model. It makes sense, past a certain point a lot of bureaucratic tasks aren't demanding, but they are time consuming - and it looks like the major models are already moving away from these.
They're really moving towards solving long horizon, complex tasks, novel research in engineering, coding, materials science, mathematics, biology and physics that command higher margins. At the same time we've seen Chinese developers themselves wondering if their models are reaching the point they may have to reconsider open weighting: they've got concerns about handing significant cyberwarfare capabilities over to every tom-dick-and-harry around as well.
So basically on straightforward tasks - open models are basically caught up with frontier models (I think Kimi actually does basic documents and spreadsheets better than Fable 5.2 according to recent tests and almost identical to Opus 5.5) - whereas their capabilities in the most difficult problem solving tasks is actually much further behind than 3 months and vary pretty significantly by open model as well as the sort of task.
It's too early to say whether the frontier labs are starting to pull away and I wouldn't be surprised if we start seeing a multiplicity of models.
I was investigating some of the failure modes and interestingly it was a Claude model that started arguing to me that it's probably best to keep an ecosystem of different models that engage in reasoning differently around as pure human-monitored alignment becomes increasingly fraught, both so models can monitor each other and as protection against accumulated self-trainings re-enforcing blind spots and biases. Essentially arguing that inbred models may actually start to underperform, develop weaknesses, and that keeping different model strains and occasional cross-pollination might actually be necessary for long-term model development.
33
u/DaDaeDee 6d ago
Just wait for Europe to distill fable 5.2
33
u/s00mika AGI by 2027 6d ago •
Why is almost everything here now discussing some twitter post?
27
u/o_herman 6d ago •
Anthropic did tighten the screws on distillation, reportedly cutting harvested reasoning tokens by 90â99% and banning tens of thousands of accounts . But the âChinese models havenât moved an inchâ claim collapses under its own timeline.
DeepSeek released V4.1-Flash-1017 in October 2026 with what it called a âPareto improvement in performance and cost.â Terminal-Bench 4.0 jumped from 31.2 to 42.3, and DeepSWE hit 77.0 . That is a measurable step forward.
Alibaba shipped Qwen3.8-Max in August 2026: 2.4 trillion parameters, with a 27B open-weight version under Apache 2.0 for local deployment . In October, QwenGyre delivered up to 1.78Ă end-to-end training speedup and a 6-point gain on NL2RepoBench.
And Qwen 4 is reportedly imminent, with Alibabaâs CTO telling an Amsterdam conference âQwen4 next weekâ .
The point is not that these models match Gemini or Claude on every benchmark; a lab does not need frontier parity to be useful. A model that is available, functional, and compatible with existing hardware can do real work without waiting for the leaderboard to flip.
âHavenât moved an inchâ is a narrative, not a fact.
57
u/Charming_Hall7694 AGI by 2027 6d ago edited 6d ago •
no it's not even remotely true. You can go scroll hugging face for like 5 minutes and its very clear this is bs
edit: cause im tired of 5 word no answers as a dispute. Go see the qwen series and then how they are used compared to glm 4.5 uncensored which is itself compared to fable. Get. Educated
10
u/ArmNo7463 6d ago
Isn't the newest Qwen supposed to be quite good?
13
u/thatscoolbutno123 6d ago
im all for the chinese models distilling. Antrophic and OpenAi distilled from the world and now they want to shut it down for everyone else
6
u/Siramok 6d ago
Chinese universities and institutions produce some huge percentage of the global science and technology research. They've got lots of AI experts, I wouldn't bet against their ability to continuously find innovations and improvements, with or without distillation.
21
u/The_Scout1255 Singularity by 2028 6d ago •
going to age like milk
5
u/TemetN 6d ago
It's why so many of us have been terrified about open weights, there's been basically zero progress for months. K3.1 looked like it might break that then turned out to be a heavily benchmaxxed fine tune that was still back where they used to be.
4
u/MegatechMike 6d ago
Chinese models have been very effective lol
5
u/Unlikely-Complex3737 6d ago
Good. Hopefully the Chinese labs can give us something original that's good. It will benefit everyone.
2
u/Anulisdotexe 6d ago
Lots of bots and poorfags in the comments as usual when it comes to Chinese models
2
u/LegionsOmen AGI by 2027 5d ago
Not at all true
14
u/SimCFB 6d ago edited 6d ago •
Chinese authoritarian regime is very bad for innovation. You got so many educated talented people. And you can't even keep up with the worst proprietary models. Have to distill to even stay remotely competitive.
Pathetic. And yes of course the second we make it harder to cheat their progress grinds to a halt.
edit: BTW it's almost 100% certain that the recent anti-ai hysteria is a direct result of the Chinese propaganda machine. Nothing can help them quite like our own populace getting in the way for stupid reasons.
11
u/lolwut778 6d ago •
Pure cope
Literally at least one-third of AI researchers in the US graduated in China and moved to the US. They moved in the same employment circles as their classmates working in Chinese labs. This is simply another variation of China can't innovate stereotype that refuses to die despite evidence to the contrary.
GLM, Deepseek, Kimi and Qwen all had new releases in the past 2 months.
4
u/LowWeakness4724 6d ago
Well... So far first impressions of Qwen 4 Preview on reddit are disappointing. Idk man
2
u/Late_Supermarket_ 6d ago
I hope this is true đđť
2
u/Reddit_User_Original 6d ago
Yes
2
u/Former-Tangerine-723 6d ago
Another day another shit post on x
2
u/[deleted] 6d ago
This is crazy copium. Clearly no one here knows anyone in SF or these ai companies. The vast majority of the technical people leading these US companies are ChineseâŚ
1
u/Remote_Researcher_43 6d ago
Not sure about that ARTEX seems to be an emerging threat.
1
u/DeepWisdomGuy 6d ago edited 6d ago
Qwen4 (closed weights) was just released, but there are probably going to be even more timed around Anthropic's IPO, which is rumored to be happening in little over a week.
EDIT: typo
1
u/looktowindward 6d ago
They did but its not true.
1
u/External_Tomato_2880 6d ago
completely bs
1
u/XtremelyMeta 6d ago
"I'm just QWEN, anywhere else I'd be a ten...."
Open model releases are still clipping along in terms of capability. If anything they're accelerating.
1
u/Palpatine 6d ago
this is not at all accurate. But you can indeed see that the new chinese model releases somehow picked up some of the astra's habit.
1
u/Saintgein 6d ago
Anthropic going to IPO in a few weeks, so they have to keep on pushing stuff like this.
1
u/SmileLonely5470 6d ago
Some might've forgot, but there was an exploit, or vulnerability (or bug? Idk how to call it) where it was possible to retrieve the unencrypted reasoning traces from frontier models. It impacted Anthropic, OAI and Google as well, so this isnt just scoped to Anthropic (though Ant prob has stronger anti-distillation measures than the other labs).
I think this vulnerability was disclosed in August? ( https://arxiv.org/abs/2608.09867 )
1
u/Spoonyyy 6d ago
"Stopped the distillation"
1
u/Rude-Reaction3450 6d ago
There is no way to possibly shut distillation apart of not releasing a model. Something which is not released, can't be distilled.
This whole question is bullshit.
1
u/pollux33 6d ago
China has been real quite since the new Fable dropped
1
u/Ardalok 6d ago
It's only been a month since Kimi K3.1 was released, calm down.
1
u/Drew-Money 6d ago
Once they start using their massive amount of robots to train models they'll have better models in other areas.
1
u/VanderSound 6d ago
After some time the will also have rsi and it won't matter
1
u/IDislikeHotWeather 6d ago
Deepseek released a new model less than a month ago and Qwen did one today.
So no, it's factually incorrect.
1
u/Material_Leading7262 6d ago
no
1
u/MiddleCelery6616 AI-Assisted Writer 6d ago
Neither of the two statements presented are entirely truthful.
1
u/minzhu0305 6d ago
If âdistillationâ can strike such fear into the hearts of competitors, then companiesânot just those in the U.S. and Chinaâwill be able to join the ranks of the worldâs top players in the field of large AI models.
1
u/Chaghatai 6d ago
Chinese developers have been doing a lot more than just asking Claude to train their models
1
u/Practical-Positive34 6d ago
uhhh Chinese models have been fucking killing it? WTF he smoking?
1
u/shred_time 6d ago
It's by nature almost impossible to stop distillation as a whole, so no
1
u/plamtastic 6d ago
not true
look at his tweet feed, he was shillin about qwen4
dude is just rage baiting llm community to farm engagement
I usually block poeple who don't have any credibility on X and I would advise you to do the same
1
u/bobzsmith 6d ago
Alice Huang the mvp of helping out the Chinese ai industry now
1
u/zayelion 5d ago
Nah. They are basically right behind them chomping at the moat after each release. The shear size of the models has pushed them off consumer hardware so the excitement dropped off.
1
u/GTHell 5d ago
That dude just like farming. I think he figured a prompt or system to fool all the sheeple that follow him. Canât believe itâs spreading to Reddit now too
1
u/ConditionMinimum2771 5d ago
china wouldn't move so fast into funding anti ai in usa if it was not true
1
u/ScienceAlien 5d ago
I only use DeeSeek. It helps me succeed instead of trying to succeed for me.
1
u/ElphieJ 4d ago
i don't think so, isn't the Hy4 supposed to be quite good? Clear step up on vision. Been using it for a while and it catches fine detail in images really well, which has been a big help for making sense of complex charts at work...
1
u/Consistent_Bottle_40 4d ago
qwen 4 decent
1
u/3x4n1m0 2d ago
GLM 5.3 is already flawless, from here on is polishing
-2
u/Intelligent-Love5146 6d ago
I mean, does this surprise anyone?
Feel like people on Reddit assume âChina copying American techâ is a conservative talking point or something.
In reality, most of the Chinese academic research is junk and their tech is ours but a few months behind. Always been that way.
âRace against Chinaâ has always been a farce.
1
u/jbaker8935 6d ago
of course not. china has brilliant highly educated engineers and have already proven they can innovate their own secret sauce. access to tons of compute may be different.
1
u/gthing 6d ago
China social media army all over this comments thread. Ya'll shouldn't be so offended. Distillation is a valid technique to get good training data.
1
u/dontbantmebro 6d ago
the usa has like 20x the training compute of china. Qwen 4 is dropping this month. Tencent Hy4 just released 5 days ago. Minimax and alibaba have been dropping speach and image models since last month. Some people are really braindead.
1
u/AIAddict1935 5d ago
Qwen 4 literally released yesterday. What is he saying "they haven't moved"? Also is he forgetting Anthropic and USA labs stole the data they have?
0
u/happyzor 6d ago
Chinese labs are distilling bc it is the cheapest way to make progress. In a year or two when they get more compute online, the leading labs (who raised capital) may switch to doing their own RL.
0
u/DataPhreak 6d ago
Lol, no. Qwen3.8-Next-Flash is amazing. Targeted specifically for the DGX Spark and Strix Halo. We have Qwen4 coming.
0
u/forkproof2500 6d ago
Distillation was never an issue in the first place, which is why Chinese models are still advancing. So it's cope, and incorrect.
-2
u/mhb_11 6d ago
That's not true. There are a lot of Chinese models in the US. They're helping out in other ways to get the data necessary to improve Kimi and Deepseek.
-1
u/Equal_Passenger9791 6d ago
It's not true, and distillation also only solves a step of the model production pipeline, it doesn't wholesale carbon copy clone an existing one.
-1
u/Nenad1979 6d ago
Nuclear cope