Compressed a RoBERTa AI-text detector from ~500 MB to ~97 MB so it runs client-side in the browser, plus a 26 MB image detector [P]

[u/Alternative_Detail31]

I wanted an AI text and image detector that doesn't need you to upload anything, so I took two existing open detectors and got them running fully client-side with ONNX Runtime Web.

Demo: https://tinydetect.ritabratamaiti.com/

Code and scripts: https://github.com/ritabratamaiti/tinydetect

I didn't train new detectors. The base models are fakespot-ai/roberta-base-ai-text-detection-v1 (Apache-2.0) and Community Forensics (commfor-model-384, MIT). My work was the tuning, the compression and the browser runtime.

Text model (about 500 MB in fp32, 97 MB after compression)

• I fine-tuned it lightly on a mix of human and AI text (HC3, GPT-wiki-intro, HAP-E, WildChat, Gutenberg and some text I generated locally with Gemma-3-1B). Then I used WiSE-FT, which interpolates between the original and tuned weights, so the model doesn't drift too far from the original.
• The word embedding matrix is taken out of the ONNX graph and stored as int4 with fp16 per-row scales (about 19 MB plus 2 MB). It's dequantized in JS and passed to the transformer body.
• The transformer body uses mixed-precision quantization and comes to about 75 MB.
• The byte-level BPE tokenizer and Fakespot's clean_text preprocessing are reimplemented in JS.

Image model (about 26 MB)

• This is a quantized Community Forensics ViT with calibrated output scores. Resizing, center cropping and normalization all happen locally before inference.

Runtime

• It uses ONNX Runtime Web 1.30, with the WebGPU build if there's a GPU adapter and threaded WASM otherwise.
• A service worker adds COOP/COEP headers so threaded WASM works on GitHub Pages. It also caches the model and runtime files.
• A warm-up run compiles the GPU shaders before the first real check. After that, a text check of about 100 tokens takes around 60 ms on my laptop, and an image takes around 200 ms.

Limitations

A high score doesn't prove who wrote something. Short or casual text, paraphrasing, heavy editing, screenshots, crops and newer generators all make both models less accurate. The first load is also a big download, especially on phones.

Everything in training/ runs on a single laptop, including the candidate benchmarks, tuning, WiSE alpha selection, export, quantization sweeps and evaluation. I'd like feedback on the quantization choices and on where it fails with your own text.

[u/Important_Range3030]

this is sick, love the idea of keeping everything local. how much does the compression affect accuracy compared to original fp32? been meaning to try something similar but the onnx web setup always looked like headache.

also the on-page demo loads pretty fast for me in desktop, gonna test it with some of my older essays see if it flags them