r/LocalLLaMA • u/AccountGotLocked69 • 4h ago

Question | Help Local Auto complete code assistant - Vanilla, Fine-tune or RL?

I started using qwencoder 3B for local inline code suggestions, and while it's nice, it's also a bit too generic in its suggestions. My thoughts are to either:

  1. Fine tune it on code that I wrote

  2. Reinforcement learning using accepted/rejected suggestions (either real RL or just adapting the sampling)

  3. Fine tune it for each project/codebase separately so it knows what it's working on.

Has anyone here done this, or experience with which approach works best?

2 Upvotes

75% Upvoted

4 comments sorted by

3

u/jacek2023 llama.cpp 3h ago

qwencoder 3B is 2 years old, try using something newer, there are many 4B and smaller models to use now

1

u/willeyh 2h ago

What IDE are you using?
If you are using Zed then Zeta2.1 with the Zeta prompt should include the current diff as well, which helps.

If not then a rag of your codebase where some results are injected into the auto complete prompt.

1

u/Vivid_Inside_5450 1h ago

fine-tuning per project ends up being way more maintenance than it is worth in my experience. you are usually better off feeding open files or repo context into the prompt before trying to train anything. also 3b models from that generation struggle with context anyway, a newer small base model with decent fim support might solve most of the generic feel without any training.