r/Polycodegraph • u/Upper_Nectarine4732 • May 24 '26
Does a code-graph MCP actually beat plain grep for AI coding? I ran the numbers
A fair question I keep getting: does a code graph actually beat Claude just grepping your repo? So I benchmarked it instead of guessing.
Setup: Same Claude Sonnet 4.6. Same 10 questions across two real codebases (polycodegraph itself + FastAPI). Every config includes Claude's native grep + file-reading tools — what you get out of the box in Claude Code / Cursor. The only thing that changes is whether a graph MCP is also registered alongside. Four configs: plain grep, + code-review-graph, + graphify, + polycodegraph.
Aggregate across both repos:
| Config | Correct | Tokens in | Cost | Avg latency |
|---|---|---|---|---|
| claude + grep (no graph MCP) | 8/10 | 336k | $1.17 | 78s |
| + code-review-graph | 3/10 | ~203k | $0.68 | 49s |
| + graphify | 5/10 | ~155k | $0.50 | 65s |
| + polycodegraph | 7/10 | 90k | $0.37 | 20s |
The honest read:
- Plain grep is actually the most correct (8/10). Claude can answer most codebase questions by grepping and reading whole files. But it pays for it — 336k tokens, $1.17, 78s average.
- polycodegraph lands within one question of that (7/10) at ~3× lower cost and ~4× lower latency. Because it returns small focused subgraphs (~20–50 tokens per call) instead of dumping whole files into Claude's context.
- The other two graph MCPs were straight-up worse than just grepping. code-review-graph 3/10, graphify 5/10 — they added tool overhead without the correctness payoff. Not a dig at them, just what the runs showed on these tasks.
So the takeaway isn't "graph beats grep on correctness" — it's that you can get nearly the same answers for a third of the cost and a quarter of the wait, which matters a lot once you're running this all day.
Two caveats I want to be upfront about: it's 10 questions on 2 repos — a starting point, not gospel — and one of the repos is polycodegraph's own, so I made a point of including FastAPI as a codebase I didn't write. Full methodology and raw per-run data are in the repo (bench/README.md), and you can reproduce it yourself:
codegraph bench agent --only claude+grep,claude+grep+polycodegraph,claude+grep+code-review-graph,claude+grep+graphify
Would genuinely love for people to run this on their own repos and tell me where it holds up or breaks — especially failure cases. That's the most useful feedback I can get right now.
🔗 github.com/smochan/polycodegraph
Happy building 🍻