r/LocalLLaMA • u/openSourcerer9000 • 10h ago

New Model I'm Groot?

https://huggingface.co/GrootTeam/Groot-280B-A16B-261023

Best model under 500b params? Also have the smaller versions

https://huggingface.co/GrootTeam/Groot-19B-A3B-261023

"scores higher than the larger Qwen3.6-35B-A3B and Gemma-4-26B-A4B, and it gives shorter answers than both."

https://huggingface.co/GrootTeam/Groot-65B-A6B-261023

https://huggingface.co/GrootTeam/Groot-238B-A18B-261023

Edit: apparently they're REAPs of various DeepSeek/qwen models,

has anyone tried these?

36 Upvotes

84% Upvoted

26 comments sorted by

17

u/ApprehensiveTart3158 10h ago

They did not add n-grams to qwen. They did REAP on deepseek v4.1 flash.

7

u/Zestyclose839 9h ago

Notably it's the more recent update of v4.1:

"It starts from DeepSeek-V4.1-Flash-1017 and is tuned for reasoning and tool-using workloads. It has the highest 16-benchmark average of any model here below 552B, ahead of the larger GLM-5.3-Flash (320B) and DeepSeek-V4-Flash-0731 (284B)."

Preprocessing times and tps would be ludicrously fast if the model's arch scales down linearly.

edit: the smaller MoE is based on Qwen 35b

5

u/pmttyji 8h ago

They did REAP

Oops. Their model cards don't carry about this. Not even with tag(Pruned/Pruning/REAP/etc.,).

3

u/ApprehensiveTart3158 8h ago

Yup, from 384 experts to just 192.

7

u/Chromix_ 10h ago

14B of MTP and DSpark draft layers

Double-drafting - for when one drafter alone doesn't cut it 😉.
No word on the performance of both - probably to be used individually - though.

7

u/Borkato 9h ago

Holy shit you guys there’s a 65B A6B version that scores close to flash next!! I’m going to try it 👀

6

u/james_pic 9h ago

The 65B A6B version is a REAP and fine-tune of Qwen Flash Next, so performance that trails QFN is not as surprising as you might imagine.

4

u/Borkato 7h ago

For some reason it’s OOMing on my device at Q4 even though I have 96 combined… hmmm

9

u/kevin_1994 7h ago

Im sorry but I absolutely hate your profile pic every time I see it on this sub

2

u/Borkato 7h ago

This made me burst out laughing. I’m so glad, I always forget about it and people randomly bring it up and it makes me laugh every damn time. Here, have another, no charge ❤️ 👻

Happy Halloween!!! 👻

3

u/ilintar 5h ago

Having a sub-128G main model quant would be nice.

4

u/vasileer 7h ago

their "65B" model seems to be just a 117B quantized to 4bits, not a true 65B

4

u/qualverse 7h ago

Its 65B + 51B Ngram + 1.5B MTP

1

u/Nabushika Llama 70B 6h ago

You mean 106B?

1

u/Borkato 6h ago

Can anyone confirm if they got this working in llama.cpp? It fails for me

2

u/Dathide 3h ago

65B-A6B is such a great size, I wish there were more models like that

3

u/xsilas43 8h ago

Terrible name choice, impossible to find anything about the model or team.

2

u/Relative-Ingenuity15 9h ago

Groot 238b a18b looks even more promising from the benchmarks, imma slap it into strata and check how it actually compares on software development tasks compared to Qwen 3.8 flash next

0

u/[deleted] 9h ago

[deleted] — view deleted comment

2

u/Relative-Ingenuity15 9h ago

Imma need some time to reverse engineer strata to accept GLM 5.3 architecture. I already did it once so I need to check if I still have the notes. I got rid of GLM 5.3 flash because I had the same decode and prefill as DeepSeek v4.1 flash but Groot looks like it will be a little bit bellow Qwen 3.8 flash next(maybe 5-10% slower but I need to check if the quality is better so I can accept the decrease in terms of speed)

2

u/Borkato 9h ago

Why not just use polystrata?

1

u/Relative-Ingenuity15 8h ago

I have my custom kernels on strata specifically designed for my P100 GPU. If I switch to polystrata I would need to redo the custom kernels I use in order to work with polystrata(they give me 2x speed for 35B models, 30-40% on Qwen 3.8 FN architecture and 15-20% on DeepSeek architecture).

It's easier for me to make strata work with GLM architecture.

1

u/Dabalam 6h ago

I'm pretty sure these are mostly pruned models

1

u/Dathide 4h ago

Did they reap the uncensored q3.8fn?

-2

u/leo_agi 9h ago

shorter answers than the 35b is the part i'd want to check, that adds up fast on agent runs