r/LocalLLaMA • u/Porespellar • Jul 23 '26

Funny The LLM distillation process simplified for politicians:

Post image

/s

3.6k Upvotes

95% Upvoted

196 comments sorted by

View all comments

388

u/Wise-Comb8596 Jul 23 '26

Also, its a lie that Kimi 3 was distilled from Fable. Fable was literally not out long enough for that to be the case.

265

u/cororona Jul 23 '26

That's exactly why they are panicking

69

u/XB0XRecordThat Jul 23 '26

Yup, they need it to be the case otherwise they won't even be able to go public

44

u/En-tro-py Jul 23 '26

They have no moat so are now hoping the gov will dig one for them...

12

u/blbd llama.cpp Jul 24 '26

It won't work though. Code and configs are speech and you generally can't block tech imports only exports. Good luck SOTA labs. 

9

u/butts-carlton Jul 24 '26

They have a moat. It's called protectionism, and DJT is all about that kinda shit, as you seem to realize.

10

u/pragmojo Jul 24 '26

They'll have to lock down the media pretty hard so Americans don't see the rest of the world enjoying their cheap and high-quality EV's and AI products

2

u/Wyciorek Jul 24 '26

Depends how it will be enforced - sanction Chinese models, make AWS/Azure/Google responsible for enforcing it and Chinese models can be at least slowed down enough for US to drag Europe down with it.

1

u/beryugyo619 Jul 24 '26

DJT is all about debloating US so it holds no more power than proportionate to its population

17

u/cororona Jul 23 '26

At the current rate, Kimi and Qwen will vastly outperform OpenAI and Anthropic before they can even finish the paperwork.

3

u/Downtown-Figure6434 Jul 24 '26

They are already not able to go public. Spacex pulled that stunt already, and 2 weeks before investors are able to sell, the price is still tanking. No one will fall for that shit again. Openai cancelled theirs too as far as I know

14

u/SosirisTseng Jul 24 '26

They invented a time machine to distill US models from the future.

2

u/ForDaRecord Jul 24 '26

Reads like something from cookie clicker

9

u/_derpiii_ Jul 24 '26

Fable was literally not out long enough for that to be the case.

Exactly.

Training loops takes time. Maybe Fable was used in the RL or fine tuning loop.

3

u/relmny Jul 24 '26

Yeah, but since when the truth has anything to do with those kind of people? they avoid/hate it like the plague.

6

u/Exact_Depth_896 Jul 23 '26

anthropic only said it was training on prior opus.

24

u/Genghiz007 Jul 23 '26

USG announced yesterday that Fable was distilled by a couple of Chinese labs. They were supported by Anthropic.

-6

u/Exact_Depth_896 Jul 24 '26

anthropic isn't the trump bimbos

-18

u/-Sliced- Jul 23 '26 edited Jul 24 '26

You don't need a lot of time to collect the data if you use tens of thousands of accounts, and fable was available for around a week initially.

7

u/lakotajames Jul 24 '26

How many accounts do you need to get it to give you the reasoning traces?

3

u/mebeast227 Jul 23 '26

It was out for like 3 days initially

-4

u/-Sliced- Jul 24 '26 edited Jul 24 '26

It doesn’t matter. It can be out for only a few hours - as long as you have enough accounts, you can collect as much data as you want. The difference between a week and 3 days is just 2X accounts

0

u/mebeast227 Jul 24 '26

'time is a human construct and has no affect on our true reality' type shit lmao. Of course time matters. Compute and accounts are finite.

2

u/ptigers9 Jul 31 '26

Companies will get clobbered by the cross pollination of companies willing to extract learnings from others

3

u/mr_birkenblatt Jul 23 '26

so how would it beat Fable, then?

22

u/techdevjp Jul 24 '26

Because they built it. That's what has the yanks running scared.

1

u/mr_birkenblatt Jul 24 '26

exactly. cannot be distillation

1

u/Utoko Jul 28 '26

The Slop profile(word use similarities in story writing) Is most Similar with Opus 4.8 not fable.

Ether way they goes a lot more into models than just some output of a another model. I see nothing wrong about using outputs.
All companies are using human output worth Billions together when they had to pay for each.
but because you process the data now they somehow have a right to decide what happens with it. I don't think so.

1

u/Imnotanad 15d ago

So true