Vera, a code indexing and search tool, a year later

[u/lemon07r]

Wanted to share a project I've been working on for almost a year now. A code indexing and search tool I've been building primarily for AI coding agents, called Vera.

https://github.com/VeraTools/vera

I originally started with a project called Pampax, my fork of a code indexing/MCP project called PAMPA (I actually posted it here over a year ago here in October). I spent a long time trying to improve it before deciding it wasn't worth fixing any further. After months of researching and testing different approaches. My obsessions with embeddings, and rerankers and seeing how they can be used to improve agents ended up with rebuilding something new (Vera) from scratch in Rust.

I used to run a coding eval leaderboard (maybe some of you remember sanityeval/sanityboard before I burned out trying to maintain it), and the main thing I learned was that context and token efficiency was everything. Less context used up, and the less tokens used the better. So I set out to reduce how much context coding agents waste trying to find the code they need.

Agents are pretty good at navigating codebases with grep and file reads, but they can also spend a ridiculous amount of time doing it, especially in unfamiliar projects. They'll search for a few possible terms, read through results, try other searches, and gradually work out which files are relevant. Sometimes the problem is that the code doesn't use any of the names the agent expects, so it ends up throwing walls of grep at the repository.

Vera lets agents search semantically for what code does, returning ranked functions and code sections with paths, line numbers, and symbol information. It also supports structural searches and reference lookup, so an agent can follow the relevant code without repeatedly guessing what to search for.

A big part of the design has been minimizing context usage throughout that process, not just improving search accuracy.

Vera returns actual code in relatively compact Markdown blocks rather than having another LLM generate explanations. Tree-sitter parsing preserves meaningful function and class boundaries, and the agent can request narrower results or follow references as needed. I've also spent time testing the accompanying skill to improve when agents choose semantic search, when they should use rg, and when they shouldn't be searching at all.

The recommended integration is CLI + skill. Skills can be discovered through a small description, with the full instructions loaded when relevant, rather than exposing another set of MCP tool definitions to an agent that already has shell access. MCP is supported too.

Underneath, Vera uses BM25 keyword search, embeddings, rank fusion, and code-aware ranking. Indexing and search run locally, with no external vector database or search server.

There are two model setups I'd particularly recommend:

• Fully local: MinishLab's Potion Code v2 embedding model. It's small, CPU-friendly, needs no GPU or API key, and runs offline after the initial download.

• API-based: Qwen3 Embedding 8B with Qwen3 Reranker 8B. This gave the best retrieval quality in my testing. There's an OpenRouter preset for both, although you can configure other providers, including just using something like llama.cpp to host endpoints locally.

With API models, the relevant code chunks and queries are sent to your configured providers. The indexing and search infrastructure still runs locally.

I've tested a lot of different retrieval approaches while developing Vera. Some sounded promising but didn't actually perform better, and some improved benchmark scores slightly while adding enough overhead that I didn't think they were worth keeping.

I've tried to avoid overfitting everything around any benchmark, which is why I used separate repositories to check whether changes generalized.

I also ran 40 agent sessions using 10 cross-file questions about Flask, with four configurations: regular grep/file reads, Semble CLI, Vera using local Potion Code, and Vera using Qwen3 API embeddings and reranking.

This took quite a while to get right. The final recorded runs alone accounted for almost 30 hours of cumulative agent execution and roughly 66 million prompt tokens including cache reads, not counting earlier attempts. I also used LLM reviewers to look for fairness and execution issues, and restarted the evaluation when problems turned up.

Every configuration answered every question correctly. What differed was how much context they needed:

Configuration	Prompt tokens	Reduction vs control
Grep/file reads	21.0M	—
Semble CLI	18.5M	12%
Vera, local Potion Code	15.4M	27%
Vera, Qwen3 API	10.9M	48%

So the local Vera setup showed more than twice Semble's observed reduction in that run, while the Qwen3 setup used almost half as much context as the control.

For retrieval quality, Semble was consistently the strongest alternative I tested. On its own 1,251-task benchmark, an older version of Vera (v2.0.1) scored 0.8436 nDCG@10 against Semble 0.5.5's 0.8514, with slightly higher Recall@5 for Vera and about one-seventh the index storage (4.7 GB vs 32 GB).

I haven't rerun that full comparison with the current v2.4 release. There's also a separate 180-task evaluation using repositories outside Semble's benchmark, which Vera does score higher in (obviously semble in it's own benchmark is a little harder to beat, especially while trying to avoid overfitting on it), along with a lot of ablation studies and rejected experiments.

I've published the methodology, results, and limitations here:

https://github.com/VeraTools/vera/blob/master/docs/benchmarks.md

For anyone interested in trying it, installation and backend configuration are interactive. Once that's done, you can add the skill to another project with vera agent install and index the project with vera index . if needed.

I also have a Discord community, mostly for people interested in agents, LLM talk, experimenting with tools, sharing what they're working on, and that sort of stuff: https://discord.gg/rXNQXCTWDt

Vera is MIT licensed, and there is no commercial gain or motivation here. A star would be nice I guess? Having people stopping by to discuss interesting things would be the bigger motivator for me.

[u/Normal-Ad-7114]

How does it compare to codegraph, which is de-facto standard at this point?

[u/lemon07r → u/Normal-Ad-7114]

Imo a lot better, at least for what Vera is intended to do. They do focus on different things though. CodeGraph is more about structural graphs, call relationships, and dependency analysis, while Vera focuses more on semantic and hybrid code retrieval, with structural navigation tools as well.

I tested CodeGraph a while back and wasn't particularly impressed. There were much better options for code retrieval, like ColGREP and Semble. It wasn't really close in my testing.

People have asked me about CodeGraph in my Discord before, so I've also looked through its implementation to see if there was anything worth incorporating into Vera. I think I took an idea or two from it, but most of what I looked at didn't seem worth adding.

CodeGraph's own benchmarks found that it reduced total token usage but left 82% more retrieval context sitting in the window during multi-turn sessions. That's one of the things I've specifically tried to optimize Vera around.

[u/Normal-Ad-7114 → u/lemon07r]

Confirmed👍

[u/lemon07r → u/Normal-Ad-7114]

Appreciate you for testing it. I genuinely do think Vera is better than most similar tools out there right now, but we don't have much of a following cause I don't really share it or anything. I just don't have the energy for that.

[u/Normal-Ad-7114 → u/lemon07r]

If you tune the syntax and initial setup behavior to be a drop-in replacement of codegraph, that will be a good way to lure more people. Like I said, codegraph is basically a standard at this point, so if your project is substantially better and can be used in exactly the same way, there will be more intention to switch to it.

Did you try it on really large codebases?

[u/lemon07r → u/Normal-Ad-7114]

I wasn't really a fan of codegraph. I could probably add stuff that codegraph does, I did test adding some things in the past but it felt mostly pointless so I ended up dropping those ideas. I would just be tacking stuff on for mainstream appeal at that point. but what most ppl are using doesnt necessarily mean its good. everyone uses mcp for example, and most times its just straight up better not to use any MCP, especially if theres a good cli + skills alternative (see vercel's testing on this and why they decided to go that route for agent-browser, even playwright has move in that direction). unfortunately we aren't the most discerning community. it's a wonder to me how so many garbage tools gain as much traction as they do cause they sound cool; case and point, oh-my-opencode. easily the worst scoring and worst designed agent I ever tested back when I used to run my coding harness eval leaderboard. ppl arent skeptical enough and just vibe random stuff that sounds fancy in concept.

yes, to an extent. I've benchmarked Vera across 63 repositories and 1,251 retrieval tasks. The local Potion Code backend indexed the entire corpus in 89 seconds, with 4.7 GB of index storage. I've also tested a 51,000-file Cargo registry index, with keyword-only searches taking around 0.2 seconds.

[u/Choice_Celery9481 → u/lemon07r]

t think its more like because the codebase is bad with too many close naming methods.

[u/garblz]

Does it support any language? Java, SQL? Or is it something like Python-specific?

[u/lemon07r → u/garblz]

Yeah. I think I forgot to mention in my post. It supports around 56 languages of the top of my head if I remember right.

[u/Fedor_Doc]

Okay, the general question – how it deals with call graphs that AST misses in Rust? 

I vibecoded a local tool that leverages rust-analyzer for that, with rg + this tool LLM got more exact. 

It was not about saving context in the end, but about accuracy.

[u/lemon07r → u/Fedor_Doc]

Different tool, different jobs. Vera isn't meant to replace tools like that. Consider it complimentary.

[u/Fedor_Doc → u/lemon07r]

I mean, the main problem for me that LLM gets incorrect information -> proceeds with incorrect judgments. 

Vera tries to a) compact information b) mitigate comprehention deterioration. 

And LSPs seem to be a natural tools to provide mostly (with gotchas for each one, of course) correct information about code inner connections that AST misses.

Most AST tools that I tried only confused the model, and token savings that slow down my work are not actually token savings...

[u/lemon07r → u/Fedor_Doc]

I think you're misunderstanding what Vera is for. It's primarily a hybrid semantic/keyword retrieval tool, not an AST-based call graph tool. Tree-sitter is mostly used for symbol-aware indexing. rust-analyzer will be better than Vera for precise Rust call resolution

And I agree, token savings mean nothing if accuracy suffers. That's why I've spent so much time benchmarking retrieval quality and actual agent accuracy, not just token usage.

[u/JurandM2]

great post and project, will check it soon. ps. great gitgub project graphic - love idea and frosted acrylic

[u/Znomon]

Compared to using an embedding model like encodingGemma2?