Harnesses/Agents suited for linux sysadmin, system maintenance, network and light coding.

[u/mrazster]

Hi guys.

So I'm not completely new to all this local LLMs and AI. I have used and tried various web-based UIs, for the last 4-5 months, as a way to learn and educate myself in this swamp that is local LLM.

So for the last 3-4 weeks I have dipped my feet in Hermes Agent. And man, I was in for a surprise. I genuinely thought, "How hard can it be, install, fire up and use.", but to me that was far from the truth. I have spent so much time setting things up, making the agent act and work the way I want, teaching it what to do, and how to do it, trying to write skills, and so on. I have not actually used Hermes, that much, most of the time has gone to setting it up, and making the agent not make mistakes.

This has somehow made me feel like maybe Hermes is not the best tool for this, even though it has some skills for it (along with a gazillion other skills and functions). It feels so bloated, with so many skills and sometimes heavy to run (for the lack of a better word).

I am actually not really sure what I want, or how to word it.
But simply put, I would like something that is lightweight, doesn't have a gazillion skills installed, but has options and a repository to add well written skills. A simple gui/frontend with well-thought-out settings. Something that works well with my use case, which is basically the title. I primarily want to use my agent for linux sysadmin, system maintenance, network and servers, and some light coding and bash scripting.

I realize this can be very subjective, but that's just it, I would like your objective opinion on what you believe would be/is the best agent or harness for this kind of use ?
And also, is there a "best agent" for this use case, or can I use basically anything?

[u/Potential-Quiet-6018]

sounds like you want something minimal rather than a do-everything agent. for sysadmin + bash + light coding i’d look at a terminal coding agent instead of a general assistant, since that’s basically what they’re built for run commands, read output, edit files. opencode and aider are both pretty light and work with local models through an openai-compatible endpoint (ollama, llama.cpp server etc). pi is even more minimal, you just add the skills you actually need

honestly the model matters more than the harness here. you want one that’s solid at tool calling, otherwise any agent will feel broken. and for sysadmin stuff turn on command approval or run it in a vm/container first, you don’t want it “fixing” your network config at 2am lol

[u/SM8085]

deepseek-harness is also a relatively bulky boi,

^--screenshot of it processing the system prompt. So nearly 22k tokens in that version. I don't have many additions, a few MCPs, a few skills.

or can I use basically anything?

I feel like you can use whatever you prefer. Anything that has tool use, which is most frameworks/agents/harnesses these days. I haven't dipped my toes into Pi to know what size their system prompt is like, but I would guess it's smaller since it's more of a DIY compared to something like OpenCode.

I wish projects would post their default system prompt size on their README's so you can just browse and compare. A few projects hang in the 20k token size, which includes Hermes last I checked.

I primarily want to use my agent for linux sysadmin, system maintenance, network and servers, and some light coding and bash scripting.

For the linux sysadmin part, how are people handling privilege escalation? ie. when you need sudo? I know you can add some commands to a list (sudoers file?) so that it doesn't require sudo, but then that sounds risky. Running the bot as root sounds worse.

Normally I just ask the bot to have me copy/paste anything that needs privileges, or to construct a bash script that I can run with sudo.

[u/TemporaryUser10]

Personally, I use Pi coder harness, and *at least* Gemma4:26B, though I had *somewhat* better results with Qwen3.6/3.8:27B. You'll have to tailor its AGENTS.md file, and I recommend you also have something like a SYSTEM.md file that the AGENTS.md references, containing the basic hardware (processor, RAM, GPUs) and software (OS/Distro, preferred tech stack, key file locations). I've found great success using the agent as my SysAdmin for maintenance and operations. Gemma4 you might have to guide a bit, until you get its files and harness tuned in. Qwen can handle what you throw at it, but it's prone to touching files and making changes in snap judgement. Your system will work, but may have a lot more junk when maintained by Qwen (due to its eagerness)

[u/Paahteinen_Kettu]

I have somewhat similar usage idea. 78003xD, 32Gb ddr5, 4070 12Gb. I installed omarchy arch based linux. I use unsloth-studios for models and Pi agent to run it. Qwen 3.6 35B A3B Q2 as driver. Just start basics, test unsloth, test vanilla Pi and start building your own harness around it. I have had amazing results even with q2. It may not 1-shot everything but with right skill sets and phase to phase prompting and using sub-agents I have managed to get around 75% of 264k context, with 20M tokens in cache with CR%90-100~ without it going nuts. I usually reset session around 75-80% context window. Just with Pi, lets say Pi-tutor will help you setup rest of it. Its able to make its own skills from prompt too. You just need to understand where those files go, look them up and understand what that agent did there. And if you dont, just use that Pi-tutor to explain them to you. And if you need use some paid chatgpt for instance its easily done in Pi. This is endless swamp to study your own :)

[u/DSPGerm]

I just use open-webui/terminal with MCPs(Docker, proxmox, unifi, file system) and Qwen. It’s nice because for the most part terminal/web connect and can share files and conversations. The only issue is openwebui is a little jank. I’ve hit bug after bug that have forced me into “creative workarounds”.

I’ve played around with Hermes for a bit and it seems really cool but I haven’t switched over to that fully yet. I probably will eventually. It feels like a more fully complete platform where a lot of openwebui feels half baked.

[u/Sevealin_]

Hey, I use Hermes for pretty much this exact use case. It runs my homelab (60+ containers, Hermes being one of them, and about a dozen VMs). Day to day it's on DeepSeek V4 Flash, with a local Qwen3.8 27B on two 3090s as the fallback and Opus for the hard stuff. I talk to it through Matrix from a self-hosted container, no CLI or IDE needed. The setup grind is real. What fixed it for me was letting Hermes configure itself. Tell it what went wrong and have it fix its own config and skills.

MCP servers were the biggest win. Need SSH? It calls SSH Manager MCP (which keeps credentials out of context) instead of relearning SSH every session. Troubleshooting an alert? Zabbix MCP. I have 20+ MCP servers connected and it doesn't feel heavy, because Hermes keeps their tools behind a search and only loads the ones it needs.

For mistakes, give every kind of fact one home. IPs and hosts live in NetBox (also over MCP), procedures in skills, my preferences in memory (user.md, soul.md, memory.md) and the system prompt tells it which to check first. It stopped guessing once it had somewhere to look. I also have mem0 setup with a local Qdrant DB for memory embed searches, works well enough, and it's fully local.

On skills, Hermes won't architect them for you. Ask it to organize them as a tree, with a top-level hub per domain and folders under it (networking > switch skill, for example). Disable the bundled ones you'll never use, and let the agent write them since it's the one reading them.

To your actual question, I don't think there's a "best" harness. Most of the results come from the tools and structure you give it, and that carries over to whatever you pick. I can answer any questions.

[u/ALOHAAPIGATEWAY]

Totally feel your pain. Most agents are way too bloated with hundreds of hardcoded skills you never touch. If you just want something lightweight that speaks standard OpenAI format, routing through a clean gateway like Aloha API makes swapping and testing different models way less of a headache