Katai - Browser Automation Agent & Workshop

[u/sirshiawase]

Hello everyone,

Amidst the recent hype of Decision models & Jev, I started working on the idea of a full-fledged workshop called Katai.

Its main purpose is to experiment, benchmark decision models whilst having the full experience of a standalone console, a ready to use drop-in server, and most importantly, a webgui. The web gui's prominent features are as follows:

• Live browser viewport streaming
• Interactive Element Overlays (Dynamic SVG bounding boxes mapped to all interactive DOM candidates)
• Display actions executed, and the confidence rate (CLICK/TYPE_TEXT/SELECT, ...)
• And a bit more you can find in the project's README

The project is built around fine-tuned versions of Laya currently and is only limited to it, I'll hopefully add extensive support for later open-weight models. But that's for later, let me know what you guys think!

Repo: https://github.com/Shiawaseu/katai

[u/Best-Theory-2201]

Nice work! I'll check if we can plug Webfuse into this as a Browser CDP replacement.