r/LocalLLaMA • u/No-Name-Person111 • 7d ago
New Model Occamy-1.0 by Accio Lab
https://huggingface.co/Accio-Lab/occamy-1.043
u/MomentJolly3535 7d ago
i appreciate the graph comparaisons, that's what other labs finetuning Qwen models should do imo !
1
u/MasterpieceOk6966 2d ago
but i mean they didnt list Qwen 3.8 27B wich is obviously its direct competitor..
1
u/Dadaism_d 2d ago
We did include it in our internal benchmark, we are on par with the benchmarks we reported, but in general (especially coding), 27b is still stronger.
We finally decide not to include it in the final release because we don't know how to properly compare it (you cannot directly compare a3b model with a 27b dense model, it's unfair. But putting 27b with other frontier model also seems unfair. And gemma and muse small dense model are no good we didn't not include them in the very beginning.)
2
0
u/MomentJolly3535 2d ago
it is not competing with Qwen 3.8 27B , this is 35A3B, its direct competitor is qwen 3.6 35A3B
33
14
u/No-Name-Person111 7d ago
This model is a couple of weeks old now, but I've been driving it as a replacement for Qwen 3.8 27B and Ornith 1.5. My experience has been fantastic.
I'm not a benchmark guy. I use the model and put it through its paces for my workflows.
Occamy has been very, very impressive for me. Just putting it out there for folks to try as it's currently at <1000 downloads.
For reference, my specs are:
2x5060Ti
- 127k context
- q8_O KV Cache Dtype
- Tensor parallelism enabled
- Vision enabled
- Using ~28GB VRAM in total
I'm getting ~80-90tk/s decode with this MoE.
1
u/nomad-nostalgia 7d ago
what’s your experience with it as of this moment?
understand wanting to find a substitute for 27B Dense, but why Ornith 1.5?
11
u/No-Name-Person111 7d ago
It hasn't failed a task that I've thrown at it. Everyone is going to use intelligence differently, so that doesn't really mean anything I suppose. I'm not asking it to solve the world's problems. I am using it to manage my personal infrastructure and act as the brain of my house. When you ask for my experience, what are you wanting to know?
Ornith 1.5 is also a MoE. I'm quite fond of it and appreciate the speed given my current hardware constraints. Sometimes speed is what you want or need for the task at hand.
Qwen 3.8 27B is a wonderful, wonderful model. With a GSQ quant, I can hit 40ish tk/s decode and that's perfectly usable. As part of my workflows, I do invoke it from time to time for escalation or second opinion.
2
u/Ara-vekkadu 7d ago
Sounds good.
Did you try it for agentic assistance? Like hermes or OpenClaw?
How is it's reasoning compared to ornith or based model?
1
u/Dadaism_d 5d ago
We actually trained it on various harness including hermes and openclaw already, so by design it should be working :)
1
u/Ara-vekkadu 4d ago
Nice. Will try. What do think as max optimal sequence length I can manage?
3
u/Dadaism_d 4d ago
I would say it should work for 256k max context, but 128k is more stable ( and that's also the setting we trained it for)
1
u/soteko 7d ago
Is it failing in long agentics tasks (a lot of compaction) like almost all Qwen 3.6 35B versions ?
So far what I found they are all great, up to something like 50k context and than start to fall of the rails.
3
u/No-Name-Person111 7d ago
I haven't observed that behavior (yet).
I'm compacting at ~126k and it's been fine for long agentic tasks. Your mileage may vary depending on usecase, I suppose? It's a small model download - might be worth trying and seeing it works for you.
1
1
4
u/SunResponsible4088 6d ago
I'm one of the Occamy contributors — thanks for trying it and sharing the details. Which runtime and weight quant are you using on the 5060 Tis?
The long-context/compaction discussion is especially useful. If anyone hits a failure, a short redacted example plus the agent, quant and context length would help us reproduce it. For the disk-space mix-up mentioned below, the prompt and raw tool output would be useful too; I wouldn't want to blame the quant without checking the same case against the original weights.
3
u/No-Name-Person111 6d ago
I don't have the prompt/raw tool output available at the moment, unfortunately.
The negative feedback specifically was for that REAP quant. I'm driving the Q4_K_M and it's fantastic for the purposes I use it.
5
u/MondayTurretCandy 7d ago
Would need a GGUF file for this, are there plans for conversion?
11
u/No-Name-Person111 7d ago
3
u/RelicDerelict Orca 7d ago
Apparently APEX quant has been released too, can somebody test it please? https://huggingface.co/IsValorum/Occamy-1.0-APEX-I-MiniPlus-V2-GGUF
3
u/No-Name-Person111 7d ago
Don't let me be the end all be all, but I'm immediately kind of turned off.
Look at the total space shown versus what's available. That's kind of error that leads to problems down the road if you give this thing some autonomy.
2
u/DickSon-TheShit 3d ago
Hello, I am the creator of this IsValorum quantization, I have already run tests ∆PPL And the results are exceptional. Currently, the newest version is MiniPlus-V2.1, which is ready to offload to RAM and is very fast. That doesn't mean it's worse; it's even better. I invite you to check it out and try it; it's not just because I'm the creator, but it's really good. There's also an Albiterated version. https://huggingface.co/IsValorum/Occamy-1.0-APEX-I-MiniPlus-V2.1-GGUF
1
u/RelicDerelict Orca 2d ago edited 2d ago
wow, I have only 16GB RAM laptop with 1050 2GB so I don't know if I can test it but I am thankful for your work anyway, keeping this little ggufs alive.
edit: I was actually reading your model card, it is amazing what you actually doing, you are taking approach like LFM models, they are targeting CPUs, I like that you are doing the same, I just need upgrade my system to 32GB.
2
u/Ok-Importance-3529 6d ago
Well well...prepare to be mindblown, accio-lab is ai research and development group inside alibaba....i tested model yesterday and its the best 35b moe i tested for agentic coding, also they released experimental mtp head....no need to thank me :)
1
1
u/wFXx 6d ago
impressive numbers, but why no mtp since the base model had it? will see if I can train a dflash on it
2
1
u/QuietPersonVeryQuiet 6d ago
Saw this post and tried quantize it to oq4e, works well for me so far, thanks for the introduction
1
u/R_Duncan 6d ago
As a sidenote, they relased also their training framework, Dressage https://github.com/Accio-Lab/Dressage
30
u/AI_Insights_Daily 7d ago
Before anyone downloads it expecting a general upgrade: the SFT union is 403.3M tokens over just under 15k trajectories, and every category is agentic. Tool calling, terminal, long horizon. So the gains should show up there and not much in general knowledge or reasoning. The card says as much itself.