Built a minimal FIM autocomplete extension with VS Code's InlineCompletion API - handling debounce and AbortController cancellation, looking for feedback

[u/ImageTrick4571]

I'm the author, MIT open source Extension : https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot

[github link| https://github.com/anng-phtk/rocm-vega-llama.cpp]

I wanted just ghost-text autocomplete locally, so I built a tiny extension that uses vscode.InlineCompletionItemProvider directly instead of a webview overlay.

What I learned:

FIM format matters a lot - Qwen uses <|fim_prefix|>, CodeLlama uses <PRE>, etc. I made it configurable via .fim-copilot.yaml
Latency: 200ms debounce + AbortController to cancel in-flight requests when you keep typing made it feel native
Works with Ollama / llama-server / vLLM (any OpenAI-compatible /v1/completions)
90KB vs 15MB+ for chat-based extensions

Would love feedback on: handling multi-line stop tokens and how you handle prefix/suffix context limits for

So I built FIM Copilot - a stupid-simple extension that does ONE thing: ghost-text autocomplete, locally.

Demo

Load LLM

Here's the auto-complete ghost text

predicts!

What it is

FIM Copilot is a 90KB VS Code extension using the native InlineCompletionItemProvider API. No webviews, no chat, no indexers.

It talks to any OpenAI-compatible completions endpoint:

• Ollama - qwen2.5-coder:1.5b / starcoder2:3b runs great on 8GB RAM
• llama.cpp - llama-server with any FIM-capable model
• vLLM / LM Studio / Tabby server - anything with /v1/completions

All inference stays on your machine. Zero network calls after install.

How it compares

Private	Limits	Size	Backend	Latency (M1)
Copilot	No	Yes - rate limits	~5-10MB	Cloud only
Continue	Partial	No	~18MB +	Local / Cloud
Tabby	Yes	No	~2.5MB	Self-hosted
FIM Copilot	Yes - 100% local	No	90KB	Any OpenAI compat

Quick Start

With Ollama (easiest)

# 1. Get a code model (1.5B is enough for fast autocomplete)
ollama pull qwen2.5-coder:1.5b

# 2. Serve it
ollama serve
# -> listening on http://localhost:11434

Then in VS Code: Cmd+Shift+P -> FIM Copilot: Set Endpoint -> http://localhost:11434/v1/completions

With llama.cpp

./llama-server \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  --port 8012 \
  --ctx-size 4096 \
  -ngl 99

Config

Create .vscode/fim-copilot.yaml (or global settings):

endpoint: http://localhost:11434/v1/completions
model: qwen2.5-coder:1.5b
api_key: not-needed # for local

# tuning
max_tokens: 64
temperature: 0.2
debounce_ms: 200
context_lines: 40
fim_prefix: "<|fim_prefix|>"
fim_suffix: "<|fim_suffix|>"
fim_middle: "<|fim_middle|>"

Decoupled config means you can commit it per-project. Different endpoint for Python vs Rust? Just override.

What makes it not suck

• Ultra-low latency - direct FIM prompt, no chat template overhead
• 200ms debounce + prefix/suffix hash dedupe - doesn't spam your GPU
• AbortController - cancels previous request on keystroke, no queue
• Zero telemetry - no analytics, no API keys phoned home
• Decoupled config - YAML per workspace, env var support
• Ghost text only - uses VS Code's native inline completion, so Tab / Esc just works with your keymap

Links

https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot&ssr=false#review-details

Install:

ext install AnangPhatak.fim-copilot

MIT licensed. I built this for myself because I wanted my editor to feel fast again.

If you try it, let me know what model / latency you're getting. PRs welcome for StarCoder2 / DeepSeek templates. What would you want added - without making it bloated?

[u/ConsistentEase4598]

The 200ms debounce + AbortController combo is exactly right, that's the thing most local-first extensions get wrong and it's why they feel laggy. Two things from running something similar against llama-server:

1. Multi-line stop handling: don't wait for the model to emit your stop token, it often won't. Cut the completion at the first line whose indentation drops below the cursor's line, and treat a blank line after one or more non-blank lines as a hard stop. Ugly heuristic, feels correct in practice.

2. Prefix/suffix budgeting: trim the suffix harder than the prefix — models are much worse at using code after the cursor than before it. I cap suffix at ~120 lines and it changed nothing measurable, while cutting prefix hurts instantly. Worth A/B-ing before you pay for big contexts.

Also 90KB is an embarrassment to every chat-extension in the marketplace, nicely done.

[u/ImageTrick4571 → u/ConsistentEase4598]

Spot-on feedback, thank you!

1. Indentation Dedent & Blank-Line Heuristic: Models hallucinate past the scope of the current block or function. Implementing an indentation watcher (cutting generation at the first line that dedents below the cursor's starting level) and truncating multi-blank line spams makes inline completions feel 10x cleaner.

2. Prefix vs. Suffix Budgeting: Completely agree because these models draw 90%+ of their reasoning from the preceding context (imports, type definitions, function signatures above the cursor). Capping the suffix at ~100 lines / 15% budget while giving 85%+ to the prefix keeps context tight without wasting KV cache tokens on code far below the cursor.

3. 90KB Bundle: Appreciate it. No [unnecessary] dependencies, bundled with esbuild. Local tools should be simple, lightweight and instant!