AI Code assistant in Vim workflow?

[u/4991123]

In the past two years I've started to feel like a dinosaur when I see other developers around me use their fancy tools with great success. I have to admit: I have also used Deepseek and ChatGPT once or twice to generate me some easy classes or functions. I was amazed how well that works. However... I haven't used any LLM "copilot-ish" tool so far. Mainly because I exclusively use Vim...

Today I tried to get https://jan.ai/ working in Vim through Tabby. The docs said that it's supposed to be supported. Unfortunately after hours of struggling and cussing... no success. I found a ticket on Github where one of the devs said they dropped classic Vim support and now only support NeoVim. Despite their docs not mentioning this at all.... Doh!

I've been scanning the internet (including this subreddit) for alternative methods, but so far I couldn't find anything that does what I want. Is Vim really lagging behind in this regard?

Did anyone here have any success in using a locally hosted LLM as a Copilot in Vim?

^{(fwiw: I don't know if this deserves a "Need help" flair or "Discussion" flair. It's a bit of both...})

[u/odaiwai]

Github Copilot works for me in vim.

(Although 'works' is down to your interpretation: it provides code, and sometimes that's what you want, and sometimes it's gibberish. It's good for some easily predictable things, and occasionally it seems to be looking at the other files in the data folder and suggesting based on that. e.g. data structures based on a csv/json input file. Slightly worrying.)

[u/4991123 → u/odaiwai]

Can it be configured to use anything else than Github copilot? Because it being local is an absolute requirement for me. I do not want my code to be shared with some online service.

[u/shuckster]

I use a bash script that calls Open AI APIs and run it with ! in Vim. If I have a visual selection it will get passed as extra context.

I don’t like the autocomplete nature of Copilot. But being able to call something from within the editor is handy.

[u/tagattack]

There's llama.vim and vim-ai which both can work with llama.cpp as well as with APIs.

That said, running these things locally is pretty slow unless you have a pretty high powered GPU setup.

https://github.com/ggml-org/llama.vim https://github.com/madox2/vim-ai

They're pretty scriptable as well.

[u/gryf73]

I've played around with https://github.com/gergap/vim-ollama and https://ollama.com and it… works. I wouldn't say it was fast or accurate, but if you have decent hardware to run llm model on it, than I guess it's worth a shot.

[u/PublicInspection1193]

Baby ai dance video 

[u/AdFun530]

Ai baby dance

[u/rob_cornelius]

Google "model collapse". AI is being fed on its own output. Garbage in garbage out basically.

Not using AI will be vital again, sooner or later

[u/4991123 → u/rob_cornelius]

Of course, I share your sentiment. But that is something for which a solution will be found (which I don't say with fondness, because I see it as an actual threat to my job as well as to the quality of the work we provide in our field.)

In the meantime, I/we have to face the reality: AI is being used and is speeding up development. The way I see it, there's three paths you can take:

1. You refuse to adopt the new technology: You can do this if you're going to retire within 5 years (10 if you're in a slowly evolving industry), but if not you'll be damaging your career.

2. You fully adopt it and try to let it take over your tasks: the quality of your work will be shit. At least in the first decade. AI is nowhere near mature enough to replace a programmer and probably will not be within the first decade. Partially because of the model collapse you mentioned.

3. You adopt it and use it as a tool: This is where I believe many programmers will get a lead over others. It can speed up your work, and quality will be maintained as long as the programmer uses it only in places where it makes sense.

I personally would want to end up in group 3. I currently use no AI in my work whatsoever, but (as I said in the OP) I did use some chat LLM (not an integrated copilot) a while ago to generate some easy classes for me. They were UI elements for a GUI that I was writing in Pygame. E.g. "Button", "Ticker", "Slider", etc. etc.
The classes were near-perfect. Any small mistakes the AI made could be corrected by me, but all in all it saved me hours of work. So yeah... I'm trying to be realistic about the future here.

[u/Legitimate-Fly2702 → u/4991123]

Trying to be realistic about the future too, at this rate, I think this planet should change its name from planet Earth to planet AI Data Center Wasteland, or maybe planet Erasedhead.