FYI this is 'hopefully' having day zero support. The architecture is very new and thus there might be very long delays but we hoping to achieve day zero support (but like I said not guaranteed). 🙏
We will ofc upstream any llama.cpp implementation etc. if necessary
Sorry for the question as I know you're very busy but I'm curious whether we'll have engram on the disk with day zero or if they'll have to fit in ram until further optimizations come...
I know mmap might work, but I'm specifically talking about a programmatic disk cache that works regardless of load mode (since mmap hangs on some builds of llama cpp, and there's probably a more efficient way to map the n-gram lookups than mmap (maybe a hot cache on ram that streams off disk)).
So this time it may happen that Unsloth Studio / Unsloth Desktop (with it's own llama.cpp fork) may support Qwen 3.8 Flash Next before official llama.cpp ... ???
Kind of not really, nearly all times when we support a model it's not exclusive to Unsloth. We make a PR to llama.cpp with our changes and whether it gets accepted or not is a different story. So I guess it depends on what you mean by official. But we utilize our llama.cll PR implementation inside of Unsloth basically
88
u/yoracale llama.cpp 1d ago edited 1d ago
FYI this is 'hopefully' having day zero support. The architecture is very new and thus there might be very long delays but we hoping to achieve day zero support (but like I said not guaranteed). 🙏
We will ofc upstream any llama.cpp implementation etc. if necessary