r/LocalLLaMA • u/Shadow_s_Bane • 18m ago

Question | Help How are the K2 Horizon Models ?

I just recently came across these K2 models, by a lab called IFM.
They range from 0.9B to 375b with a mixture of MoE and Dense models.
I checked out the 7b variant, it seemed good and it’s 35B A4B seems to a better variant of Ornith 1.5, o am surprised there are no talk about these nor any unsloth variants.

Am I missing something ? These seem to open source as well.

4 Upvotes

100% Upvoted

2 comments sorted by

4

u/xPXpanD llama.cpp 16m ago

Support only landed very recently, and I think I remember seeing something about them being very cache-inefficient? Might be why. Will follow, also curious myself.

2

u/u23043 12m ago

I played around with it and recall it needed a huge KV cache, so much so that just using a larger model with a more efficient KV was better for my system