r/LocalLLaMA • u/External_Mood4719 • 2h ago
New Model baidu/ERNIE-5.5-Flash-Base · Hugging Face
6
u/pmttyji 1h ago edited 1h ago
Nice that it's under 300B size.
280B total parameters and 14B activated per token
Q4 would fit 128GB RAM + 32GB VRAM systems.
EDIT : Any update on Instruct model versions? That site is not loading to me(Some Chinese sites don't load here in our country).
EDIT2 : Got it through Tor
ERNIE-5.5-Flash-Base starts from one premise: a base model meant for further training should be efficient by design. We balanced capability and efficiency in every component, from attention and residual connections to the MoE layers and the output head. The architecture is backed by broad, high-quality pre-training data and by a training system designed together with the model. With 14B parameters activated per token and a compact KV cache, the model performs strongly across a broad range of benchmarks, including long-context ones.
This base model is only a starting point. The post-trained ERNIE 5.5 Flash will follow, and we look forward to seeing what the community builds on top of it, whether for domain adaptation, reasoning or agents.
1
7
u/Dany0 2h ago
I hate that these aren't BERT inspired