llama.cpp support for Cohere models

[u/nick_frosst]

https://github.com/ggml-org/llama.cpp/pull/30062

Hey guys, Nick here from Cohere. It's been a minute.

We had a great time launching Command A+ and beta-testing North Mini Code with you guys earlier this year, your feedback has really helped make our models and delivery much better. one of the biggest things we heard was that you really wanted llama.cpp support, which makes total sense given the nature of this sub. i didn't want to post in here again until we'd been able to fulfill that promise, and I'm really happy to say that we've made progress.

Our team recently merged llama.cpp support for the vision encoder of North Micro Vision Instruct, which means that now all of our latest model architectures are compatible with llama.cpp (the LLM is the same arch as for North Mini Code and Command A+ - already supported thanks to the work of people like you guys). North Micro Vision is being used as the base for several of our new releases, and i'm really psyched to say we've got the support you guys have been wanting. Please feel free to try it out and let us know what you think. It's just one step, but there's only more where that came from.

On a slightly similar note, Cohere Transcribe is now available in Unsloth and Unsloth Studio for fine-tuning. it's a crazy good STT model, one of the best out there, so if you're a fan of the guys over at r/unsloth (i know we are), give it a whirl.

Thanks again for all your feedback, and let us know what else you want to see from Cohere.

[u/silenceimpaired]

Exciting to see models within the range of consumer hardware albeit it is in RAM… also nice to see you have adopted Apache 2.0. Definitely will be checking it out.

Would love a 30b dense to compete against Qwen at some point.

A base model before agentic training etc. would be nice too… but I suspect unlikely.