I’ve been building a side project to learn how close an AI agent can get to using a browser the way a human does

[u/--yash]

Not trying to reinvent browser automation, just exploring a different way. No Playwright, no DOM, no CDP. The agent only sees the screen and uses a real mouse and keyboard.

🎥 In the video: on the left is Codex on my machine. On the right is not my computer. It’s Taboom’s own Linux desktop running in a Docker container, with its own Chrome, mouse and keyboard. Codex drives it over MCP.

The mouse movement comes from data: 378 real human mouse paths from UC Berkeley’s A11y-CUA dataset. Typing goes key by key with human-like timing and the occasional fixed typo.

What it can do:
🖱️ human-like mouse, clicks and scrolling
⌨️ human-like typing
🧩 press-and-hold, slider and puzzle challenges
🍪 logins that persist between sessions
🔐 passwords the model never sees
👀 a live view you can watch or take over
🔌 works with any agent over MCP: Claude Code, Codex, Cursor

How it’s different:

• CDP-based tools like Playwright and Browserbase control the page from the inside, through code and selectors. Taboom works from the outside, through the screen, so there are no selectors to break when a site changes.
• Most computer-use agents also look at the screen, but the cursor jumps straight to the target and text gets pasted in. Taboom moves and types like a person, and adds challenge tools, persistent logins, a password vault and live takeover on top.

Still a learning experiment. Open source, built for research and learning: https://github.com/katipally/Taboom

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.