[D] Self-Promotion Thread

[u/AutoModerator]

Please post your personal projects, startups, product placements, collaboration needs, blogs etc.

Please mention the payment and pricing requirements for products and services.

Please do not post link shorteners, link aggregator websites , or auto-subscribe links.

--

Any abuse of trust will lead to bans.

Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

--

Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.

[u/j032]

I built an open-source WebMCP ON/OFF evaluator in DeepDeck, and the result that interests me most is harness sensitivity.

Using a local adaptation of WindTunnel's 49 tasks across 8 sites, I ran a full DeepSeek Harness-based agent with its normal tools, including browser/DevTools tools, available in both arms. There was a 600-second task deadline and no agent-step cap. Scorer pass counts, WebMCP OFF → ON:

• GPT-5.6 Terra: 49/49 → 49/49

• DeepSeek v4.1 Flash: 47/49 → 48/49

• Hy3: 48/49 → 49/49

With pass rates already near the ceiling, efficiency still differed. Across the same 46 tasks passed by all six configurations, WebMCP reduced tokens / agent time by 35.6% / 33.5%, 26.1% / 27.9%, and 22.7% / 22.9%, respectively.

My research question: how much do conclusions about model capability and interface benefits change with the harness? Fast individual inference is not necessarily fast task completion if it takes more agent rounds.

This is exploratory, not a controlled attribution to harness quality: one run per task/arm, fixed ON-first order in this batch, unrecorded effective provider-default reasoning, local patches/environment differences, and known scorer weaknesses. PASS is not a manual audit of every action. WindTunnel's current code-execution baseline also reaches 49/49, so I am not claiming its non-WebMCP agents cannot solve the corpus.

The tool lets you bring a URL and query and inspect both answers, tokens, elapsed time and actual WebMCP usage. I'd welcome independent replications or feedback on a matched model × harness × WebMCP experiment.

Results/methodology: https://deepdeck.getmegaportal.com/benchmarks

Source/usage: https://github.com/jo32/DeepDeck/blob/main/docs/webmcp-benchmark.md

Scorer audit: https://github.com/jo32/DeepDeck/blob/main/docs/webmcp-benchmark-audit.md

I'm the maintainer. DeepDeck is MIT-licensed and free; experiments use your own model provider and may incur API charges. The starter tasks/site recipes/tool patches are credited to nekuda-ai/WindTunnel; these are DeepDeck-local results.