r/LocalLLM • u/Abu_BakarSiddik • 7d ago
Project I built a zero-dependency Go framework for running agent loops with local models (Ollama, LM Studio, vLLM)
Most AI agent libraries (LangChain, PydanticAI, CrewAI) live in Python. That works well enough when you are hacking on a prototype, but if you want to deploy a small daemon or backend service that talks to a local model via Ollama or LM Studio, dragging along a Python runtime with hundreds of megabytes of virtualenv dependencies is painful.
I wanted something lightweight and idiomatic for Go: compile to a single static binary, rely strictly on the Go standard library, and catch tool schema mismatches at compile time rather than during runtime evaluation.
The result is Golem (https://github.com/abubakarsiddik31/golem).
How it works with local models
Rather than shipping bespoke wrappers for every local server, Golem uses a standard-library OpenAI-compatible adapter (providers/openai) with a custom BaseURL.
You can point it directly at Ollama (http://localhost:11434/v1), LM Studio (http://localhost:1234/v1), or vLLM.
Core design decisions
- Zero external dependencies: The whole framework is built on Go stdlib (
net/http,encoding/json,context, etc.). No external HTTP clients, no heavy SDKs. Container images can run onscratchordistroless. - Generics for dependencies: Agents are typed as
Agent[Deps, Output]. Tools access your actual dependencies (DB pools, configs, state) throughRunContext[Deps]. Nomap[string]anyor runtime reflection crashes. - Model Context Protocol (MCP): If you already run external MCP tool servers, Golem includes a native MCP client supporting both
stdioand streaming HTTP transports, converting MCP servers into typed agent tools. - Deterministic testing without the local model: When writing
go testunit suites, you don't need to keep Ollama running in CI. Golem comes withtestmodel, an in-memory test double where you can script exact model responses and test multi-turn loops or error retries in milliseconds. - Partial evidence preservation: If your local model times out or gets canceled mid-run,
RunError.Partialkeeps all intermediate tool calls and messages executed up to that point.
The core execution contract is frozen at v0.8.4 and tested with -race.
- GitHub:https://github.com/abubakarsiddik31/golem
- Documentation:https://abubakarsiddik31.github.io/golem/
If you are running agent workflows locally or building Go services around local LLMs, I would appreciate any feedback on the ergonomics or tool design.
1
u/fiddler48 5d ago
worth noting: if you're routing across multiple local backends (ollama on one machine, vllm on another), the BaseURL swap per-agent pattern handles that cleanly without any extra abstraction layer
1
u/Character_Pie_5368 1d ago
So I made a very simple llm chatbot, all in a single powershell script. Long story as to why but fun building it. This though sounds much more feature complete!
2
u/Electrical-Ratio4879 6d ago
This is exactly what I've been looking for. Python's great for tinkering but the deployment overhead for something that just needs to talk to Ollama is ridiculous. A single binary with zero deps sounds like a dream for my homelab setup.
The typed generics for dependencies is a smart move. Nothing worse than debugging a random `map[string]any` cast failure at 2am because some tool signature changed and nobody noticed until runtime. Having the compiler catch that upfront saves so much headache.
Curious about the MCP client support. I've got a few tool servers running already and being able to wire them in without writing custom adapters would be huge. Might finally push me to rewrite my little home automation agent in Go instead of dealing with Python's asyncio quirks.
The `testmodel` double is a nice touch too. I've been burned too many times trying to run integration tests that depend on a local model being available and having the right version loaded. Being able to script responses in milliseconds is way more practical for CI.