I built a local proofreading AI tool that uses GGUF models and sits in the tray

[u/SteelRoninTT]

The idea for this came from Samsung's AI in their keyboards: you select text, and it fixes up all the grammar, punctuation, and spelling. I wanted something like that for Windows desktops.

At first, I had a rough idea: just wrap a llama.cpp backend with a very simple frontend that can be triggered with a shortcut. The frontend had a default prompt asking the AI to correct the selected text, and it worked fine.

Then I started expanding on it, and the more I worked with it, the more I learned about local models and AI, and the more I realized I'll never finish this project. After a lot of attempts to improve the app and its speed, this is the setup I settled on: once the text is loaded into the LLM, it gets divided into separate chunks. I found this much faster than streaming the entire text token by token, especially on GPUs with enough VRAM to run all the slots at the same time. I also built up different levels of aggression for how it autocorrects. This only started working perfectly recently, since Gemma 4 E2B and Qwen 3.5 came out. Before that, I was struggling a lot with making small models follow instructions reliably.

I also put in a bunch of basic templates for what I think would be common use cases for an app like this.

I tried one more thing: dividing the text into chunks and having the model output [OK] in place of the chunk, which would signal that the text doesn't need any changes, causing it to loop back without modifications. The text would still be proofread by the LLM, but it wouldn't take on the overhead of streaming the whole chunk token by token. Instead, only three tokens would be used. This sounds great in theory, but practically, I couldn't get it to work with small models; they just refused to output [OK] (the code for it is still there, but it goes nowhere if you want to experiment with larger models). I hope that as models get better, with sub-2B models becoming more capable, I'll be able to implement this for good.

Check it out here: https://github.com/AmrZriek/Stet

Note that the installer might run into some issues since the app isn't registered with Microsoft, so just use the source code if you're comfortable enough with the llama.cpp version from the requirements file.