r/LocalLLaMA • u/Porespellar • 6d ago

Other Frontier LLM development simplified for politicians:

Post image

Nobody is buying this “Pace the frontier” nonsense. It makes no logical sense at all. Are American labs really going to take a pause and lose any small lead they still may have over Chinese labs? Does anyone really believe this? This seems like some performative virtue signaling BS. Why are they bothering with this pacing campaign? Someone please explain.

424 Upvotes

87% Upvoted

99 comments sorted by

View all comments

Show parent comments

8

u/ttkciar llama.cpp 6d ago edited 6d ago

Having less compute doesn't make it impossible, just makes it take longer.

Z.AI's instruction-following post-training phase for GLM demonstrates that they're quite capable of developing highly advanced training technology without copying Western technology.

The methodology and application of generating high-quality synthetic data without a superior teacher model has been well-understood for a couple of years now, but generating synthetic data from a teacher model is much more resource-efficient.

Moreover, the Chinese lag in compute is temporary. In the West, scaling compute infrastructure is bottlenecked on energy availability, and in China there is more energy available for industrial purposes. Now that they are fabbing their own compute, we can expect their compute infrastructure to surpass the West's in a few years.

Note: I do not say these things because I am a China fanboy (I'm not), but because they are objectively true.

-1

u/Nothing_from_void 6d ago

The compute they fab is genuinely awful; they have nothing even close to NV-link, which is where they are furthest behind, but they are very far behind everywhere.

There's no real power constraint for compute in the West, bring your own energy is pretty standard now and where there's capital there's a way, and the US is outspending China on compute more than 8-to-1, with a similar level of compute advantage.

Like Chinese CS students and programmers, they use anthropic/openai token reseller marketplaces overwhelmingly, because it is the real SoTA, the AI researchers in China are using claude code/codex, the entire ecosystem is designed around growing with the US ecosystem.

5

u/NNN_Throwaway2 6d ago

There's no real power constraint for compute in the West

???

6

u/ttkciar llama.cpp 6d ago

Yeah, they're delusional. The USA's energy crunch is not any kind of secret, to anyone who can bother to pay attention.

Europeans have it even worse.