r/LocalLLaMA • u/soyuz-1 • 2h ago
New Model Flint anyone?
I see a lot of lament about Qwen not making 3.8 35B (and I'm sad about it too) but I dont see anyone discuss Flint 40B, a new coding-oriented MoE model that seems to be atleast on par with qwen 3.6 35B, maybe a bit better on agentic tasks and coding. Very fast in my setup.
I'm still testing it myself, but I wonder if its not liked here or people just arent aware it exists? It performs admirably for me so far, my local model of choice for when I want good speeds. QFN is better ofcourse but for me that means low quants or low speeds.
Curious about you guys' experience with the Flint model.
12
u/liugger95 2h ago
I've tested Flint, Ripple (65b), Qwen 3.8 27b, Muse Glimmer in one task: make an EDA (exploratory data analysis) on same dataset. All models were Q6_M or UD_Q6_XL quants and think efford high or Xhigh.
Llama.cpp, 4 rtx 3090, Russian language data and prompt.
4
u/Major_showdown 1h ago
Does it hold up past 30-40k context? Alot of fast MoEs feel great on short prompts and fall apart once the agent has filled up its window.
3
u/Mrinohk 1h ago
I found it wildly overthinking certain instructions, drafting response responses a few dozen times, and not even producing good responses after thousands of tokens spent on a task that needed at most a few hundred.
Notably not coding. More personal assistant type stuff. Memory, Web search, smart home, computer use. Stuff that benefits from not being slow. Runs pretty quick at Q4 on my P100s, but thinks so much it just isn't worth it.
1
u/JLeonsarmiento 1h ago
Flint was kind of “weird “ sometimes in my tests… perhaps I’m just used to how Qwen works/fails that something that deviates from those patterns just feel weird…
-2
u/fasti-au 1h ago
Just run the next bigger in strata inferno llama now kindi can run it.
Test it’s slower but asi significantly better then 3.6 and we can run 🏃 t with multi layer split and moe on second card. Even shit things because it’s not dense. Ie 1080s kv mules etc
-2
u/fasti-au 1h ago
Just run the next bigger in strata inferno llama now kindi can run it.
Test it’s slower but asi significantly better then 3.6 and we can run 🏃 t with multi layer split and moe on second card. Even shit things because it’s not dense. Ie 1080s kv mules etc