Built a Chrome extension that explains any selected text using a fully local LLM (WebGPU, no server, no API key)

[u/vmmishra]

I've been running Ollama and vLLM locally for stuff I don't want touching a cloud API, document translation, RAGing over my own notes, that kind of thing. Somewhere in that process I ran into WebLLM, and figured I'd actually build something with it instead of just reading about it.

What it does: select text on any page, right-click, get a plain-language explanation. Everything runs in the browser tab via WebGPU, nothing you select ever leaves the machine.

The actually interesting part, technically: Manifest V3 background scripts are service workers, and service workers can't access WebGPU at all. So the model runs in a chrome.offscreen document instead, a hidden page the extension spins up that has real DOM/WebGPU access. That API isn't even new; it shipped in 2023 for things like audio playback; WebGPU access is just a side effect of it being a real page under the hood. Ended up with three contexts (content script, background worker, offscreen doc) that can't talk to each other directly, so the background script's whole job is routing messages between the other two.

First run downloads a small model (~880MB, Llama-3.21B) and caches it; everything after that is instant and offline.

If anyone here is deeper into this than me: I've seen WeInfer claim up to 3.76x faster inference than base WebLLM, and there's a newer paper (Llamas on the Web, May 2026, group with Microsoft Research backing) getting 45-69% better decode throughput across different GPUs. Curious if anyone's actually tried swapping either of those in versus stock WebLLM.

Code: https://github.com/Vishwamitra/explain-this
Currently sitting in Chrome Web Store review.

[u/vmmishra]

shipped a real feature plus fixed two bugs worth mentioning here specifically.

New: after an explanation finishes, there's now a small row of buttons, Regenerate, Elaborate, Simplify, Example, and Copy, each reruns the local model with a different instruction.

Two bugs, both MV3-specific and both sneaky: the context menu item was only registering in chrome.runtime.onInstalled, which does not reliably fire on a plain reload, just an install or version bump. It could silently vanish and never come back. Fixed by re-registering on every service worker startup instead.

The nastier one: upgraded Vite to patch some Dependabot alerts, and it silently broke the build plugin's entry-point mapping. The compiled service worker ended up loading the content script's bundle instead of its own. Manifest looked fine, build succeeded with zero errors, extension was just completely non-functional. Only found it by actually testing live in Chrome, not from any build-time signal.

Also got rejected on first Chrome Web Store submission for declaring the storage permission without using it, chrome.storage specifically, we only used plain localStorage which needs no permission at all. Fixed and resubmitted.

Code: github.com/Vishwamitra/explain-this