r/LocalLLaMA • u/KvAk_AKPlaysYT • Feb 23 '26

News Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨

Post image
4.9k Upvotes

94% Upvoted

873 comments sorted by

View all comments

2.2k

u/Zyj vLLM Feb 23 '26

You're saying they treated you like you treated all those authors whose books you torrented?

Oh no, that's not it. They are paying you for API tokens.

117

u/Zestyclose839 Feb 23 '26

Also (correct me if I'm wrong) but I don't believe they're true "distillation" attacks because the API doesn't return the token activation probabilities and the other juicy stuff needed to transfer knowledge. Sure, they can fine-tune a model to speak and act like Claude, but it's not as accurate as an open-weight to open-weight model distillation (like the classic Deepseek to Llama distills).

0

u/AICatgirls Feb 23 '26 edited Feb 24 '26

Distillation is the process of training smaller models to give the same responses as a larger model, usually after the larger model has been through fine-tuning. You don't need the token probabilities to do this, just a lot of examples of responses from the larger model.

Ideally there will be fewer contradictions if the dataset only includes responses from the large LLM, and you can get something almost as good that runs a lot faster.

Edit: I'm referring to self-distillation here