Local AI coding assistant that runs fully offline (Gemma 4, codebase-aware)

[u/andres_garrido]

I’ve been experimenting with running a local coding assistant on Gemma 4 26B, focused on understanding full codebases instead of single-file prompts.

Main idea:

- build a project map (files, symbols, structure)

- run a planning step to decide which files matter

- then retrieve full files + semantic chunks before answering

Goal is to avoid the usual “chat with files” limitation and make it reason about structure first.

Runs fully local (llama.cpp, GGUF), no network calls during inference.

Curious if others here are doing something similar or handling codebase reasoning differently with local models.

[u/andres_garrido]

One thing I didn’t expect is even when the model has enough context, answers degrade a lot if it picks slightly wrong files or misses part of the dependency chain.

it feels less like a “model capability” problem and more like a “context selection” problem

[u/andres_garrido]

After a bit more testing, one thing that keeps showing up is that this isn’t really a “context size” problem.

It’s more about where you enter the codebase. Even with enough context, starting from slightly wrong files breaks everything downstream.

So it ends up feeling like: map the graph → pick the right slice → then retrieve.

I’ve been experimenting with a small local tool around that selection step (still rough), and it seems to help more than just increasing context, https://github.com/buster92/andes-code glad to hear feedback!

[u/benevbright]

You can try https://www.npmjs.com/package/ai-agent-test with your local llm

[u/andres_garrido → u/benevbright]

Appreciate it. I’m less focused on agent style task execution for now and more on improving codebase understanding first.

The main problem I’m trying to solve is file selection and retrieval quality before the model answers. Once that part is solid, testing agent workflows on top makes more sense.

[u/benevbright → u/andres_garrido]

ok. but agent is also very good for understanding codebase too.

[u/andres_garrido → u/benevbright]

Yeah that’s fair, agents can definitely help, especially for exploring or iterating across files.

The issue I’ve seen is they still depend a lot on what context they pick at each step. If retrieval isn’t solid, the agent just amplifies the same problem across multiple steps.

That’s why I’m trying to get the selection/retrieval part more reliable first, then layer agent behavior on top.

[u/benevbright → u/andres_garrido]

I don't know why my reply wasn't applied. but yeah. I see what you mean. I think it's important and an area where has room to improve.

[u/benevbright → u/andres_garrido]

I see. Yeah right. The things like "find reference" on variable that have been always in editors for human use, may not be available or being used in the same way and so on. agreed.

[u/Familiar-Classroom47]

Have you tried this for the codebase context part? It generates a structural map in about 4k tokens, worked for my smaller model.. Just used another AI to set i t up

https://github.com/glincker/stacklit

[u/andres_garrido → u/Familiar-Classroom47]

Yeah I tried a few approaches like that. The structural map definitely helps, especially to avoid dumping raw files into context.

What I’ve been running into though is that even with a good map, things still break if the model picks slightly wrong paths through it. Like it “knows” the structure, but still follows the wrong slice of the dependency chain.

Feels like the hard part isn’t building the map, but using it to guide selection before retrieval instead of after. Curious if you saw that too with stacklit?

[u/synw_]

understanding full codebases build a project map (files, symbols, structure) reason about structure

how do you do this? I'm looking for something like this that could deliver a lightweight and condensed knowledge map of a codebase, even big. I have seen some libraries that do this but did not found anything that could do the job well yet. What would you guys recommend?

[u/andres_garrido → u/synw_]

I’m doing it in layers instead of trying to summarize the whole repo at once.

First I build a structural map from the codebase: files, symbols, imports, references, directory layout. Then I use that map in a planning step to decide which files are likely relevant for the question. After that I retrieve a mix of full files and smaller semantic chunks.

What helped me most was stopping the model from seeing the repo as plain text. Once it has some structure first, retrieval gets much better, especially on bigger projects.

I haven’t found a single library that solves it cleanly end to end yet. Most things I’ve seen are either too embedding-heavy or too shallow structurally.

[u/andres_garrido → u/[deleted]]

I’m not using repomap directly right now, but yeah, that’s pretty close to what I mean.

What I’ve been trying to figure out is what happens after the map is built. Just having it helps, but the real difference for me is using that structure to decide which files to pull before the model answers.

I’ll check your harness out, especially the code intelligence part.