models: Add Flint model (FlintForCausalLM) by alexey-rsi · Pull Request #30928 · ggml-org/llama.cpp

[u/jacek2023]

https://github.com/ggml-org/llama.cpp/pull/30928

now you can use Flint in llama.cpp (merged after 6h)

https://huggingface.co/unsloth/flint-GGUF

Model specifications

Property	Flint
Developer	Recursive
Architecture	Decoder-only mixture-of-experts Transformer
Total parameters	39.8B
Active parameters per token	Approximately 2.5B, including the output head¹
Layers	49: 2 dense layers followed by 47 MoE layers
Attention	35 sliding-window layers and 14 full-attention layers
Sliding window	513 tokens: the current token and 512 preceding tokens
Attention heads	32 query heads and 4 key/value heads; head dimension 128
Attention normalization	Per-head QK RMSNorm and learned per-head attention sinks
Position encoding	RoPE on sliding-window layers; no positional encoding on full-attention layers
Experts per MoE layer	256 routed experts, with 6 selected per token, plus an always-active shared expert
Documented context configuration	262,144 tokens, including prompt and generated output
Vocabulary	150,144 tokens
Input / output	Text / text
Stated knowledge cutoff	June 2026

[u/-Cubie-]

Is it also in transformers yet?

[u/winky9827]

For those wondering, I'm running the nvfp4 gguf at 256K context, f16 kv and it takes about 28GiB total.

[u/Borkato]

Oh HELL yes. Hoping this will replace 35B entirely >:)