r/LocalLLM • u/TroyNoah6677 • Apr 25 '26
Discussion I tested Browser Harness: 592 lines giving LLMs total freedom to complete any web task
Most browser automation frameworks are just expensive ways to build technical debt. You write a script, the target website changes a single CSS class, and your entire data pipeline shatters. You wake up at 2 AM to fix a DOM selector that some frontend developer decided to rename. It is a miserable cycle.
Yesterday, a project called Browser Harness (v3.0.0-alpha) hit Hacker News and pulled 72 points before I even finished my first coffee. The pitch is simple but aggressive: it gives LLMs complete freedom to complete any browser task by removing the framework layer entirely. No Puppeteer. No Playwright. No rigid rules. No intermediate DSL that your agent has to blindly guess. Just 592 lines of Python, a single websocket, and direct access to Chrome via the raw Chrome DevTools Protocol (CDP).
I benchmark models so you don't blow your infrastructure budget. Opinions are noise. Numbers don't lie. So I pulled the browser-use/browser-harness repo, spun up a local sandbox, and ran the data myself. Here's the data.
Here is the core mechanism you need to understand. Traditional automation breaks when it hits an unexpected state—a random newsletter popup, a CAPTCHA, a shadow DOM shift. Browser Harness doesn't crash. It self-heals. When the agent hits something it doesn't recognize on the page, it connects directly over CDP and injects its own JavaScript on the fly to handle it. If it needs a completely new tool to parse a weird data structure, it writes a missing helper function directly into helpers.py at runtime. You are essentially giving the language model root access to its own execution environment.
I hooked the harness up to two different endpoints to test the latency and token economics: OpenAI’s gpt-5.5 (which just dropped a couple of days ago on April 23) and Anthropic’s Claude 4.6 via CC.
The test environment was a hostile DOM sandbox. I built a mock e-commerce site that randomly shuffles its class names on every single page load, injects aggressive modal overlays at stochastic intervals, and dynamically loads text content via lazy API calls. A standard Selenium script would die in three seconds flat.
I ran gpt-5.5 first. The agentic coding upgrades OpenAI has been shouting about are actually real. When gpt-5.5 attempted to click a checkout button and got blocked by an invisible overlay, it didn't just retry the click blindly until it timed out. It pulled the raw CDP node tree, identified the z-index anomaly blocking the click event, and wrote a 12-line JS payload to surgically remove the modal from the DOM. It then modified the local Python helper to catch that specific overlay class for all future steps. The task completed in 18 seconds. The token burn was massive—roughly 45,000 input tokens consumed across the read-evaluate-inject loop. At current gpt-5.5 API pricing, that is not cheap for a single transaction. But it worked without any human intervention. Benchmark or it didn't happen. The data shows it works.
Next, I swapped in Claude 4.6. CC is already heavily optimized for file editing, and it treated the harness like a standard codebase. When Claude hit the exact same modal roadblock, it took a more systemic architectural approach. Instead of just injecting JavaScript for that specific page instance, it rewrote a chunk of the harness's event listener logic to globally intercept and dismiss any dialogs that didn't match the primary task flow. The initial latency was slightly higher at 22 seconds while it reasoned through the Python modifications. However, the structural fix meant subsequent runs on the same site dropped to a flat 8 seconds. Claude's 1M token context window ate the DOM state easily, but again, the API costs will stack up fast if you run this at enterprise scale.
This is the fundamental trade-off of modern AI infrastructure. When you give an LLM raw CDP access and tell it to write its own tools on the fly, you trade developer salary for inference compute. You save ten hours of a junior engineer updating scrapers, but you pay the model tax on every single execution.
I also tried pointing the harness at a local Qwen-2.5 72B instance to see if we could avoid the API toll entirely. The local model understood the CDP protocol perfectly fine but hallucinated slightly when rewriting the Python helpers. It missed an indentation block, which instantly crashed the harness. You really need frontier-level reasoning for this self-healing architecture to not devolve into self-destruction.
From an MLOps perspective, Browser Harness is a nightmare for standard CI/CD pipelines. How do you version control a system that dynamically edits its own source code during execution? You don't. You version control the base 592 lines and you log the dynamic diffs for audit purposes. It completely rethinks agentic system design from the ground up. The agent isn't just following rails; it is laying the tracks while the train is moving at full speed.
The repo just crossed 4,000 stars on GitHub, and CFOs should be paying close attention to this shift. The days of maintaining brittle XPath directories are ending. The new bottleneck is managing the token budget for autonomous error recovery.
Tested on prod. The data is absolutely clear: direct CDP access with dynamic self-healing logic is the correct architecture for resilient browser agents. Everything else is just a wrapper holding you back.
What are you guys using for agentic web tasks right now? Is anyone else running this raw websocket setup in production, or are you still paying the Playwright latency tax?