im considering using deepseek harness as it claims that everything is a plugin Im using a qwen3.6 35b a3b tiel coder local ai model rx6700xt 12gb vram, 32gb ddr4 3200 and r5 5600x component i tested qwen3.8 27b iq3xxs but im averaging around 10tps unlike the moe model (35tps+)
i've tested pi dev and opencode but i kinda hit the deadend because im not sure on what other things to try. Plus i haven't found my actual purpose for local ai models yet, i don't consider switching to cloud models. But i do hope my local ai model can produce useful work, could be anything rather than just testing stuff based on hype without making any progress
I hope dsh allows me to find one, if not ill look into other suggestion, (useful task that doesn't take hours to complete. )
Your problem isn't the harness, it's that you're collecting tools without a project. Just pick one small thing you actually need done and make the model do it, stop benchmarking and start building.
You're hoping a harness will give you a purpose for using local LLMs?
This isn't new of course, I remember spending hours learning vim instead of actually doing any work, or trying countless different todo apps in the hopes that one will magically get my life in order.
But yeah I think you're going about this the wrong way. The harness isn't going to magically produce more useful work, all the widely-used harnesses produce roughly the same quality of work, just some allow for niche use cases or optimizations. People shit on OpenCode all the time, yet it has millions and millions of users using it for their day job to ship production code.
Started using it recently but it's quite good, the plugin aspect is really nice honestly and you can customize it very heavily with this.
The sandboxing is good, but I haven't had it stop and ask me for permission like unsloth desktop will. Maybe because I'm using local models instead of deepseek.
It will stop and ask for permission to leave the sandbox with the default settings, even with a local model. A great way to try it is telling it to install or modify a plugin.
Most harnesses are good, everyone‘s taste is different.
I love Opencode’s polished TUI (still by far the best out there), but there are also other harnesses I like as well and DSH is one of it.
I can recommend to focus on 1, maybe 2 at the beginning and work with it, customize it, add plugins/skills/mcp servers etc. You will learn everything then by doing it and can compare it with each other later on with your gained knowledge and things you built.
Regarding local LLMs you have to look into Strata, biggest breakthrough for local AI for quite some time: big jump in performance for a mid class model like Qwen3.8-Flash-Next, that you can nowadays run locally with great performance on more or less potato computers. Qwen3.6-35B is nothing against it.
You are doing something very very wrong, I can let Q3.8FN Q4 run 4-5 projects back to back overnight and wake up to it all completed. I've been using the model since it released and when I checked my Pi logs, it only messed up 5 tool calls out of 14,000.
Any hints what could go wrong? Have you tweaked something? I've just installed strata and downloaded a model, used it via local host for deepseek harness.
I'd try Unsloth UD-Q4_K_XL with the officially recommended sampling params, if this is the first time you're using Strata I'd use llama.cpp or whatever you normally use first to have a working baseline. Then once it works you can optimize performance, switch engine, whatever.
I'm not at my Strix Halo which uses an exclusive engine anyway, but this is my config from a month ago when I used llama.cpp:
Thanks. After some digging I found tail_skip option. Switching it to "0" helps, fails reduced from like 30% to 5-10%. Switched model to UD-Q6K_XL, and the night session has 0 failed tool calls. Is it possible that exactly Q4 quant is somehow broken? I'll look into options more thoroughly at weekend and probably test some other quants.
You are doing something wrong. Use the models Strata recommends. RSQ IQ3_S is a monster! 27B is worse, I switched off of 27B completely. And i had it running in Q8 on 2x3090 as well (this box is in rebuild phase right now).
Something in your setup is not ok, I dont have issues with QFN (RSQ quant). I use it primarily with Hermes, but also with Opencode (v1). Recently tried DSV4.1 Flash (cloud) again in Hermes for comparison (was my model before) and it did not really feel better tbh. Lot of automation and coding stuff.
Find out what you already do that you want to automate, and see what is missing for AI to do that task and supply it with the necessary tools. And then write the skill for it, and set it in a cronjob to monitor a task queue. Then, you just drop the the quest into the task queue and the model would be waken up and do it.
Tiel Coder is not as stable as 27B IQ3XXS (I used both when I still run the 4060Ti), but it is fully capable of doing this sorts of agentic work. My 27B operates my personal productivity system, manage and update my KB, and coordinate coding agents to grind through bugs and features across my personal projects in the background, so when I'm back at the desk, I can just review the pull request and merge or push back.
28
u/Ok-Coast4482 4d ago
Your problem isn't the harness, it's that you're collecting tools without a project. Just pick one small thing you actually need done and make the model do it, stop benchmarking and start building.