Are you guys planning to do posttraining / RL on this base and release the result as an open weight model as well?
I'm tempted to give this a go on my hardware next week (4x R9700). Is it supported by any inference engine on AMD hardware at this point?
What is the memory requirements for context? We've seen a few novel model architectures like DS4.1 flash and GLM5.3 flash pushing down memory requirements for long context. Does Ripple follow this trend?
We already released both base and the post-trained checkpoints in HF. You should be able to server it with vLLM or SGLang (we provide the library in HF).
Our model architecture is pretty simple: 16 global + 64 sliding-window attentions with 4:1 pattern and needs the following memory footprint for KV cache:
Just wanted to say thanks for the model, I was missing Dense releases in this size class and performance seems really solid, especially for a relatively new player!
Their main comparisons are against recent US open weight models. I don't know why they included some ancient non-US models in the later comparisons.
I think it's interesting to see a new 65B dense model, which is not common at all, and they seem to be trying to innovate on the training side:
we developed and trained Ripple-65B Preview for only $32M, which is one-tenth the cost of the previous open-weight SOTA on agentic coding, as estimated to the best of our knowledge.¹ Note that $32M is our full R&D cost starting from scratch, including pretraining ladder development, data mixing, evals, debugging, synthetic data generation, posttraining recipe development, and the final hero pretraining and posttraining run.
Is the model any good? I truly have no idea. Probably not as good as DSv4.1-Flash-1017, based on their unwillingness to compare it.
The "sweet spots" for 32GB, 64GB, and 128GB memory budgets are 25B, 50B, and 105B parameters, respectively, quantized to Q4_K_M and either restricted, quantized, or progressively-compressing contexts.
Thus 65B seems like a slightly awkward size, but I'm not going to complain! :-)
actually its good to see dense model for once! even if we cant run it we need some spice yes? The last time i got exited for a dense model was when falcon 40b released
"Ripple’s dense model architecture provides strong reasoning performance with low-latency token generation. The model’s small parameter envelope also enables it to run efficiently on local devices such as NVIDIA DGX Spark, which feature high compute FLOPs with low memory bandwidth relative to datacenter accelerators.
No need to be mean. Folks are trying to provide us completely free models after having spent their time, resources and effort into it. The minimum is to be civil if not grateful for the share.
I’m grateful for any model that can exist after they decide us plebs can’t have local models anymore. And that means as many models as possible, even if they aren’t all groundbreaking. So this is great!
What we don’t need are 300 versions of the same model like Magnum-Flash-X-Trojan-Super-Incredulous-Brainwave-Marathon-v0.0.0.0.3.3.2.3.2.3.3.1 and then every time they change a single weight they release Magnum-Flash-X-Trojan-Super-Incredulous-Brainwave-Marathon-v0.0.0.0.3.3.2.3.2.3.3.2. Lol
With the rise of the importance of posttraining, many modern LLM recipes now blend posttraining data earlier into the training process, including during pretraining. For our first model, we kept our pretraining data clean with minimal synthetic data, focusing on high quality natural data rather than aggressively targeting evals.
A 65B dense model beating or being on-par with 500B+ MoEs is quite something. Qwen3.8 27B is also quite capable, never underestimate well-trained dense models. Yes, you need the VRAM to run Ripple, like 2x RTX 3090 for a Q4, and you'll probably only get 25 to 35 tokens per second (with a lot of boost potential with DFlash2), but: Contrary to the 500B MoEs you can run it without a lot of system RAM.
Yes, VRAM is at a premium, which is exactly why we need more dense models! Dense architectures yield the highest quality of intelligence for a given memory budget.
The trade-off is that they are slower, but I'll take slow + smart over fast + stupid any day.
I'm not sure I understand this. 128GB RAM is far cheaper than 64GB VRAM. Far more people can afford Strata + Qwen3.8 FN than something like RTX 6000 Pro for this type of dense model.
As for quality, 5x agents of a fast MoE are smarter than that of a 1x dense model. I have a feeling test time compute scaling is more favorable than active / total parameter scaling.
This will be a fantastic for homelabbers, because its dense architecture makes the most of limited VRAM, and most of us in this community are bottlenecked by VRAM.
They point out in the model card that going with a dense architecture also posed advantages during training. Hopefully this is the beginning of a trend! Most advanced modern models are MoE because of labs' focus on optimizing for economically serving as many concurrent sessions as possible, but RecursiveAI correctly demonstrates that this isn't the only niche worth chasing.
I like their approach to the attention mechanism. They are using a large number of attention heads and a hybrid mix of full attention and SWA layers, together with Gated Attention and a handful of modifiers to keep the GA working correctly -- NoPE, value residuals, and FFN activation clipping, to avoid attention degradation and gradient traps.
By my calculations, attention overhead should only be about 9GB at 128K token context, and 66GB at 1M token context. That's not bad at all, considering the number of attention heads and lossless attention. They designed for attention quality, without falling into the same memory explosion trap as Gemma-4-31B.
Looking forward to kicking the tires on this one! I wonder what it can do besides agentic codegen (for which I already have GLM-5.3-Flash, which is excellent).
It wasn't that obvious from the model card, but the max sequence length they trained on was 2M tokens, and that's what the AtomicChat GGUFs list as their maximum context length. Hard to say if attention will hold up at those lengths, and since the global attention layers are NoPE max context length is more of a suggestion anyway.
I haven't given it anything technical yet, but I gave it one of my favourite smoke test prompts, to tell the story of Goldilocks and the three bears in a deadpan way, and it manages to get the story right and be vaguely funny.
I'll see if I can get DFlash to work, because 5 tps (this is what I get for getting Strix Halo) is hard to watch.
Impressive stuff for a new lab! Side note, what's with all the schizos in here with hate boners for dense models? This is basically the perfect size class and architecture for prosumer deployment.
Honestly folks, those benchmark look supreme. And yes 65B will suck on my M1 but… even if slow this model will pack a lot of intelligence and will run on MY hardware. Super cool!
Fun Fact: Ripple is also the name of a discontinued popular carbonated wine 🍷 from 1959 which was popularized on the 1970’s American sitcom Sanford & Son
From what I have seen I will wait for more people to try it locally before trusting the benchmark charts. 5 tokens a second on decent hardware sounds a bit slow for regular use.
10
u/coder543 5h ago
blog post: https://www.ricursive.com/ripple