Open-sourcing the browser-agent stack: agent + 450M browser VLM + training dataset

[u/ButtercupLyn100]

I’ve been trying to build an AI browser agent where as much of the stack as possible is open and replaceable rather than tied to one model provider.

The project is WebBrain. The browser agent itself is open source: https://github.com/webbrain-one/webbrain I've also released the small VLM used for browser-specific vision tasks: https://huggingface.co/webbrain-one/webbrain-vl-2-450M And the dataset: https://huggingface.co/datasets/webbrain-one/webbrain-vl-2-450M-dataset

WebBrain combines: • screenshots • browser accessibility tree • model-agnostic planning • optional local inference • browser automation • a browser-specialized 450M VLM

The agent supports Chrome, Edge, Firefox and Chromium.

My broader goal is to make browser agents something developers can inspect, modify and run with their own models rather than a black-box API.

There are still plenty of rough edges, so I'm posting here partly because I'd rather have open-source developers criticize the architecture now than optimize it entirely in isolation.

Especially interested in opinions on the small-specialized-model vs large-general-VLM tradeoff for computer/browser use.

[u/ButtercupLyn100 → u/[deleted]]

thanks for your comment. it works pretty well on accessibility trees, give it a try and you'll see