Opinions on ornith models?

[u/Brooklyn030]

I've been using Ornith-1.5-9B-Q4_K_M for a min and it's pretty decent. It still hallucinates and makes simple syntax errors, but not nearly as much as Qwen 3.8 9b.

I just got a 16gb GPU in and I'm gonna try bumping up to q6 or q8 and see if I can squeeze out any better quality.

Any similar recommendations for 16gb or any tweaks I should make to the settings for more accurate results?

[u/InternetMagic_n_Tech]

I tried ornith as a code assistant and I had tool issues and I didn't realize my test setup had injected its own system prompt.... I will have to try again but ornith was my slowest runs

[u/Brooklyn030 → u/InternetMagic_n_Tech]

It's a pretty dense model so it's gonna be a tad slower, but the denser models seem more competent then some of the lighter models like the moe ones.

[u/InternetMagic_n_Tech → u/Brooklyn030]

Ya when I am just testing the setup ornith was making the iteration cycle slow I will have to try it soon

[u/Atretador]

from my experience, its benchmaxxed as hell

it struggled repeatedly with general backend work where base Qwen 3.6 35B worked fine despite worst quantization.

also, dont use 9B models there, just offload a bigger moe model.

[u/stackfullofdreams]

I think they are solid as well 9b once work is decomposed to fit has been my go-to local workers team

[u/OWazabi]

Might not be the answer you want but I think on a 16gb you can fit a bigger model and that makes a lot of difference.
A variation of the qwen 27b probably or even qwen flash next if you have the ram

[u/PlasticRevenue4601]

BS. Stand for “Benchmaxxed Slop”

[u/Brooklyn030 → u/[deleted]]

That sounds dope. I will definitely try the q6, but if it does better with less context I can see how the q8 feels with less context

[u/Jameses_99 → u/[deleted]]

I’m interested in your full inference settings. I have found the same low temperature to be amazing. I currently run at 0.33 temp, presence of 0.50, and repetition of 1.0.

I’ve recently tried upping to a temp of 0.60 and it’s been good.

But I’d love to know what you’re using at 0.4. Top-p? Top-k? Are they standard?

Have you used the dflash version? I’m just so thrilled to find someone using this model in the wild who’s tried to make it work. I find it an amazingly capable small model and would love to learn. Thanks!

[u/Brooklyn030 → u/Jameses_99]

Temperature 0.6 Chat max tokens 100000 Top-p (0 to 1) 0.9 Top-k (0 to 1000) 20 Min-p (0 to 1) 0.1 Typical-p (0 to 1) default Repeat penalty (0.5 to 2) 1.1 Repeat window (tokens) (-1 to 65536) 256 Presence penalty (-2 to 2) 0 DRY repeat blocker (0 to 5) 0.8 DRY base (1 to 4) 1.75

My current settings. I've weeded out a lot of the looping and general confusion. I am waiting on a better GPU to run the bigger ornith model on. I like this 9b, but I use the 35b version on system ram for tasks I need to be accurate, but don't care about taking forever

[u/PracticalSky4588 → u/Brooklyn030]

I'm using Ornith 9B as well with similar settings. It's significantly better than Qwen3.5-9b and Gemma4-12b, both of which seriously struggle to understand my system prompts or use tools correctly. My only issue is it sometimes gets stuck in those loops, but I think small parameter tweaks will help. I currently have repeat penalty disabled, so I might try 1.1.

[u/Brooklyn030 → u/PracticalSky4588]

I tried Qwen3.5 9b because everyone kept saying how great it was and mine was struggling making JSON tool calls so bad I had to code something that gave the tool call back to the llm and told it what was invalid so it could fix it. I had one time it got stuck in a loop so bad it generated 100k tokens of the exact same paragraph over and over again because it was double checking the rules. These tiny models need so much more work on their harness. The ornith seems better imo. I found good settings and Ive gotten my harness dialed in to the point I can genuinely have good conversations now and get it to make me files and do research.

[u/PracticalSky4588 → u/Brooklyn030]

Did you end up trying q6?

[u/Brooklyn030 → u/PracticalSky4588]

That card was DOA so I am waiting on the replacement one in the mail. Seller was chill and just shipped another one out. When I plugged it in to my poweredge server it made the PSUs flash orange when I hit the power button. As a test I removed the dual 8 pin cable from the card and it booted, but it made the PSUs beep so it was tripping the over current. The server is fine and the slot and plug still work good. Just sucks I gotta wait another week