Linknekuda-aiResource
For engineers developing web automation agents, the benchmark demonstrates that dedicated protocol-level tool calling like WebMCP offers vast advantages in speed, cost, and reliability over visual coordinate clicking or DOM traversal.
The benchmark numbers highlight how token consumption explodes when agents rely on vision or page markup instead of tool calling. For example, GPT-5.6 Luna uses a median of 2,596 tokens per task with WebMCP, but requires 20,914 tokens with computer use and 29,561 tokens with DOM and vision. Sonnet 5 climbs to 64,424 tokens on DOM and vision runs. This underscores that exposing explicit application actions directly to LLMs eliminates substantial overhead inherent to rendering and parsing visual layouts.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Repository:nekuda-ai/WindTunnel
WindTunnel is an open-source benchmark suite designed to compare how browser agents interact with websites. It evaluates WebMCP (direct site-exposed actions) against coordinate-based computer use via screenshots and page structure navigation using DOM and accessibility trees.
34starsHTML
X post:Idan Levin
Idan Levin introduced WindTunnel, an open benchmark comparing WebMCP tool calling against computer-use and DOM-based approaches across 49 tasks on eight real websites.
581bookmarks138.3kviews
Link:Rogerio Bonatti, Dan Zhao, Francesco Bonacci, et al.
Researchers from Microsoft, Carnegie Mellon University, and Columbia University have launched Windows Agent Arena (WAA), an open-source evaluation framework for multi-modal AI agents operating in native Windows 11 environments. Featuring over 150 real-world tasks, WAA supports parallel execution in Azure to shorten evaluation cycles from days to around 20 minutes. A newly introduced reference agent, Navi, attained a 19.5% success rate against human performance of 74.5%.
Link:webmcp.com
The WebMCP Ecosystem Adoption Tracker monitors the implementation status of the proposed WebMCP standard across browsers and agent runtimes. While OpenAI has enabled WebMCP by default in ChatGPT and Chrome targets release 157, Mozilla remains neutral and WebKit formally opposes it.
Article:Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Felix Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, Ruoming Pang
ToolSandbox is an evaluation benchmark that assesses LLM tool-use capabilities using stateful environments and multi-turn interactive dialogues.
Repository:ChromeDevTools/chrome-devtools-mcp
chrome-devtools-mcp connects AI coding agents to Google Chrome via the Model Context Protocol (MCP). It enables agents to automate browser actions using Puppeteer, inspect network traffic and console errors with source maps, and collect detailed performance traces.
50.8kstarsTypeScript
Parts of this content were created with the assistance of AI.
