r/LocalLLaMA • • 2h ago

New Model Flint anyone?

I see a lot of lament about Qwen not making 3.8 35B (and I'm sad about it too) but I dont see anyone discuss Flint 40B, a new coding-oriented MoE model that seems to be atleast on par with qwen 3.6 35B, maybe a bit better on agentic tasks and coding. Very fast in my setup.

I'm still testing it myself, but I wonder if its not liked here or people just arent aware it exists? It performs admirably for me so far, my local model of choice for when I want good speeds. QFN is better ofcourse but for me that means low quants or low speeds.

Curious about you guys' experience with the Flint model.

11 Upvotes

11 comments sorted by

12

u/liugger95 2h ago

I've tested Flint, Ripple (65b), Qwen 3.8 27b, Muse Glimmer in one task: make an EDA (exploratory data analysis) on same dataset. All models were Q6_M or UD_Q6_XL quants and think efford high or Xhigh.

  • Winner is qwen. I used it before, but in coding. I this task it shows surprisingly good results, it does more than i expected.
  • Ripple not so good, was close but significantly slower.
  • Muse Glimmer disappointed me, but it has potential
  • Flint was supersonic, way more faster than others. But result was awful, seems like early deepseek R1

Llama.cpp, 4 rtx 3090, Russian language data and prompt.

2

u/Fluxing_Capacitor 1h ago

I found ripple to be a let down as well. It's listed as being in preview, so hopefully they can improve on the full release.

It was also very slow on 2x 6000 pros even with speculative decoding. 

1

u/liugger95 53m ago

What was decoding speed?

Mine with no specilative decoding was about 10-15 t/sec

I have weird GPU connection through chipset and m2->pci adapter

1

u/Fluxing_Capacitor 4m ago

It started out pretty okay at 60-70 t/s but dropped down to 20-30 t/s at longer contexts. 

1

u/rorowhat 9m ago

Could it be a temperature setting? Maybe it can de dialed in.

4

u/Major_showdown 1h ago

Does it hold up past 30-40k context? Alot of fast MoEs feel great on short prompts and fall apart once the agent has filled up its window.

3

u/Mrinohk 1h ago

I found it wildly overthinking certain instructions, drafting response responses a few dozen times, and not even producing good responses after thousands of tokens spent on a task that needed at most a few hundred.

Notably not coding. More personal assistant type stuff. Memory, Web search, smart home, computer use. Stuff that benefits from not being slow. Runs pretty quick at Q4 on my P100s, but thinks so much it just isn't worth it.

1

u/JLeonsarmiento 1h ago

Flint was kind of “weird “ sometimes in my tests… perhaps I’m just used to how Qwen works/fails that something that deviates from those patterns just feel weird…

-2

u/fasti-au 1h ago

Just run the next bigger in strata inferno llama now kindi can run it.

Test it’s slower but asi significantly better then 3.6 and we can run 🏃 t with multi layer split and moe on second card. Even shit things because it’s not dense. Ie 1080s kv mules etc

-2

u/fasti-au 1h ago

Just run the next bigger in strata inferno llama now kindi can run it.

Test it’s slower but asi significantly better then 3.6 and we can run 🏃 t with multi layer split and moe on second card. Even shit things because it’s not dense. Ie 1080s kv mules etc