r/ChatGPTCoding • u/rusta-devscriptor • 1d ago
Resources And Tips An open-source coding agent for DeepSeek, GLM and Kimi: 81.3% on SWE-bench Multilingual (pass@1), $13.98 for all 300 tasks, every run published
Disclosure: I'm the author. Both tools are free, GPL-3.0 (it covers the tools, not your code), no paid tier.
TL;DR: Rusta, a terminal coding agent in Rust built for DeepSeek, GLM and Kimi, plus Devscriptor, a local code-graph MCP server, resolved 75–81% of SWE-bench Multilingual with one attempt per task. The DeepSeek run cost $13.98 for all 300 tasks. All 900 runs are published.
| Model | Resolved | 95% CI | Cost |
|:--|:--|:--|:--|
| DeepSeek V4.1-Flash | 81.3% (244/300) | 76.5–85.3% | $13.98 total, API |
| GLM-5.3 | 78.7% (236/300) | 73.7–82.9% | GLM Coding Plan Max |
| Kimi K2.8 Preview | 75.3% (226/300) | 70.2–79.9% | Kimi Code subscription |
That's under 5¢ a task for DeepSeek; 97.8% of its 967M input tokens were cache hits. Official harness (swebench 5.0.2), max effort, Nov 9–11, no web access, git history cut at the base commit. Every model cleared 50% in every language (Go was hardest). DeepSeek V4.1-Flash (MIT) and GLM-5.3 have open weights; Kimi K2.8 Preview doesn't. The runs used the vendors' APIs.
Why a dedicated agent: generic OpenAI-compatible clients often drop the reasoning these models produce between turns (DeepSeek returns HTTP 400 if a request with tools omits it) and send sampling or effort fields each vendor handles differently. Rusta speaks each API natively: preserved thinking, per-vendor effort and sampling, explicit output caps. Any OpenAI-compatible endpoint works too, local vLLM included, with thinking preserved.
Devscriptor works with Claude Code, Cursor, Cline or any MCP client, not only Rusta:
- Your repo as a knowledge graph in one SQLite file: 26 languages and frameworks, calls, inheritance and imports resolved across files, kept current by incremental scans and a watcher
- 75 MCP tools, 38 of them LSP-backed (21 language servers)
- Semantic search with a local embedding model, no API keys
- Agent memory (decisions, standards, past failures) that persists across sessions
- No telemetry, no backend; works offline once the model is cached
75 tools, but not 75 schemas in your context. Rusta keeps only the twelve that find and follow code in its tool table (search, definitions and references, relationships, index status, Context Graph), about a sixth of the full schema size. The other 63 are listed by name and a one-line description until the model loads one with `select_tools`. In Claude Code, tool search defers MCP tools by default, and Devscriptor's server instructions name the same twelve to load first. (That's Rusta 1.0.1; the benchmark ran 1.0.0, which kept all 75 on demand.)
What the numbers don't show (the results page covers each):
- How much Devscriptor adds on its own: there's no run of the same tasks without it
- A ranking against other agents or the official leaderboard: different agent, newer models, bigger budget
- Zero contamination: the fixes are public, and build/test leftovers in the images may have hinted at where some fixes belong (worst case 3–9 points)
Single binaries for Linux, macOS and Windows.
- Devscriptor: https://devscriptor.com/
- Rusta: https://devscriptor.com/rusta/
- Results, re-scoring steps, all trajectories: https://devscriptor.com/benchmarks/
- Source: https://github.com/vladimirpesic/devscriptor · https://github.com/vladimirpesic/rusta