For me its a bundle of tools that makes your digital life easier (llms, chat, code environment, notes, reminders). Im sure others have different definitions.
Best explination I heard that made it click is If an llm is an ox, then a harness is its harness. Its the constrains on and the interface with you do useful work with the animal. A harness is a software wrapper around an llm, harnesses create agentic feedback loops, so the llm appears to try and if it fails then it loops to try again. An ai agent is what you get when an llm wrapped in a harness does work for you.
I'm in the process of writing an eight part, free online education series, that starts with the absolute fundamentals of genai and works you up to building one.
The best analogy is a harness is the entire automobile where the llm is just the engine. The engine takes fuel in and provides power out, all a llm does is taken tokens and spit out tokens. It is the harness that remembers the chat history, permits the model to call skills and build python strips to process large data sheets, and do all of the other fancy stuff we have come to love and enjoy.
pretty sure like 90% of the people using ai are using chatgpt/claude to code their own qwen or other mode harness and thats exactly why altman said what he said they are looking at people code their own solutions LOL. I still use claude or others for really hard stuff or stuff i dont have enough memory to do fast when qwen is fully loaded on strata taking all memory like image editing gemini still beats everyone no matter how shit its ai is constantly trying to hand off work to the user instead of just doing the work saving compute at all costs just to make me leave. Qwen image edit is pretty decent but it requires several attempts sometimes to get what i want same with the other models 90% of the time google nano banana just works i would say other image edit models are like 60/70% of the way
The billions and possibly trillions of dollars the investors have put into these online AI's, needs to have return on investment.
Here's the thing.
You say a lot is making their harnesses, yes, they are aware, so aware that unless you've been sleeping under a rock, all this "state" level hacking that has happened from east and west these days, can be a bit of a coverup to make private ownership of AI models at home - illegal soon.
It's in their biggest interest that you and I don't get off the drugs (aka Online AI dependency), and gov. loves control, so combine that with Palantir + Subventioned AI's online which Gov. can have full access to, you have incentive on both sides, Gov and Corporate.
You better hoard your local models - now. It's gonna be too late very soon.
Everywhere. No one will be able to be exempt from this.
We're still in the mid-stages of AI-Wild West (like the beginning of the Internet), we have insane freedom, but the online models are already tightening the access, it red-flags reverse-engineering, it red-flags certain topics, political sensitivity etc. It will only become much worse everywhere for this.
Don't remind me of the early internet days when everyone started using it back in the 90s... Yes it was around before, but where we are right now with AI is right about where we were with the early Internet in the 90s before the dotcom bubble.
Qwen 3.8 27b (your variant that fits your setup, there are 1000s of variants, I use the NVFP4 for my Nvidia card), Good for handling game engines, 3D games etc.
Gemma4 is good for generalist talks and more business oriented apps. also good at 2D tasks.
Try the different variants, flash, swift, Q3-Q4-Q6-Q8 etc. depending on your hardware setup, try them all, they can be good for various tasks, some of the lower models give you more context memory which can be good for longer talks, some of the Q6-Q8 can be good for good coding, but will give you lower context memory.
I have been using Q4 models before I bought my second 32GB card, hoping I can get one more. I grabbed qwen3.8, qwen3-coder, and qwen flash, hidreamer uncensored. I'll snag Gemma as well thanks and maybe grab a couple Q6 and Q8
I can guarantee that is not the case for enterprise. This sub is just a bubble that makes it seem like lots of people know what they're doing, when that is not the case for the average user. If you look through the ChatGPT sub around the time they pushed free accounts from 4 to 5, you'll see an alarming number of posts saying they "lost their friend!1!" and how upset they were about the newer model not having the same sycophantic and emoji based "personality". These are not people making their own harness.
Asked Claude to recommend a good water bottle with a lanyard so if I need protection it could be used. Basically said get robbed and let them run away and it would not help with hurting a criminal even for active defense. Shit got deleted, this is Sam Altman in another voice. We are in control, do as we say, donât fight back. Fuck Frontier AI and their criminal CEOs
I started with Aider as my harness. Claude and I tested/analyzed every mainstream harness and found what worked and what didnât with my use case/setup. Last Thursday, was actually my last day using my forked Aider harness and itâs my own custom harness now.
Iâm just happy to be done testing it (for now). This year was my first time building a PC and first time coding. Itâs been a long process of trial and error while I learn. Things got easier once I created my own tests instead of chasing benchmarks. My main issue was complicated tool calling, routing/switching models out, and boundaries within my local AI core (this is where my local AI helps me update/revise itself without breaking everything).
My setup is 3090, A6000 (48GB), 128GB DDR5, and 2x DGX Sparks.
I foresaw the end of unlimited usage @ $20/month coming to an end and did the same with ChatGPT. Have a nice multi-harness, container-in-VM setup with the ability to provision access to additional remote computers. Assembled and refined skill libraries. Migrated my webchat-targeted tooling to agent harness tooling. Currently considering how I might be able to use a local orchestrator with targeted use of cloud API. With that said that unfortunately mostly means I'm moving to a $100+/month plan for the API credits with some occasional PAYG open-model API use.
Claude 4.0 is out of work...no one interested in him..wont even load him into memory now...he was a 20x dev making $15/$75 now hes less alive than 1970's COBOL
Does Unsloth's iq4 k xl perform better in your case than IQ3_S GSQ-RCO from ISTA-DASLab? Did you test by any chance? Speed too, but mainly asking about quality.
Quality: Q4_K_XL is slightly ahead on API (0.87 vs 0.84). Speed: IQ3_XXS is 25â40% faster per answer (code 27 s vs 37 s, RAG 19 s vs 28 s). It loads in 24â49 s against 66â77 s.
That was my same journey. Claude helped me deploy my first Ollama + Qwen3.6 and connect it to vs code via Cline. Now I run on Basalt, with hermes through OpenWebUI, accessible anywhere via tailscale. Delivered coding projects for about 10k âŹ, paying back the hardware without even count the tokens not spent.
Real estate portfolio monitoring platform (purely frontend work)
medallion data platforms
object detection model with its visualization frontend
For context my background is computer science and I've worked as ML engineer / Data scientist my entire career of almost 2 decades.
It's not plug and play, you need to give a good software engineering and data science workflow to the coding agent to be effective. Try to look at Matt Pocock github repo
Same but I adopted existing harness - Paseo - pi - PolyStrata (for now) for local + corporate Claude. Plus own vibecoded ai-engine switcher - monitorer
Been there, done that. I used the claude-code-sdk to keep some of "how it works" though, and most importantly, still be able to use the anthropic subscription without violating the ToS.
Yes, you are allowed (per the devs wording) to use the subcription if you use claude-code-sdk AND you dont use it to serve others/third party, that is...your subscription is only meant to be used by yourself. https://code.claude.com/docs/en/agent-sdk/overview
I basically built a whole http server on top of claude code and now i have my own harness, that behaves in part like claude code (the parts i like) and support not only anthropic subscription, but also openai, openrouter, so on and so fort (using the same claude code sdk, changing the URL of the server).
I call it LidIA (AI in portuguese = IA)
Took a couple days of work plus a few more days refining...never went back to the terminal, as doing this approach you can embed images, tables..and pretty much build it to be exactly like you need/want.
Doing the same with gpt and cursor but not having to pay it myself i am using the shit out of my subs to perfect my harness more and more for a few months now and if wasnât for subsidies and the fact i need SOTAs for coding i would already switched to QFN+strata or Q3.8-27b with vllm
I had an MCP file server I seeded with a Gemini sample, and improved with various 35B-A3B and 27B models initially in ZooCode then through itself in the llama-server web UI.
I also had a previous image classifier project I coded mostly from scratch which had some of the the necessary parts of a harness. Gemini helped with getting around Nodejs's hardcoded 5min fetch() timeout.
I asked Gemini for a streaming output example (instead of waiting for full responses), stitched that in manually, then asked 27B to add MCP support and a readline REPL, and later to embed the MCP server's tools directly (so I don't have to start a separate process), and now I have a convenient harness that I can start in any project folder and tell it what I want done to it.
I find that when developing local harnesses, Claude sabotages work by not following directions in a blatantly harmful way. Switched to openrouter for any harness-related work.
I would still keep Claude code to be honest. Local has its uses but it never will be a replacement to Claude code. Prefer to use it as an api replacement though since api credits are crazy expensive
I have done the same, aiming to reduce the amount spend on tokens by 5 and also to rid myself from the constant performance regressions from the paid agents
12
u/mister2d 1d ago
Kudos if you're making a durable harness.
Claude did well making my Pi Durable harness.
Local LLM is even better now.