ArticleJiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Felix Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, Ruoming PangResearch
Practical AI agent workflows involve interacting with systems where prior operations alter future state. ToolSandbox provides an evaluation setup that mirrors these real-world state and communication dependencies far better than static benchmarks.
The authors identify critical limitations in standard tool-use evaluations, such as the neglect of state transitions and interactive ambiguity. By demonstrating that current top-tier models struggle with implicit dependencies and missing context, the paper highlights clear targets for improving agentic reasoning and execution.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Article:Jiaqi Liu et al.
A research team led by Jiaqi Liu introduces AutoResearchClaw, an autonomous multi-agent research pipeline. Moving away from rigid linear workflows, the system implements iterative error recovery, verifiable reporting, and adaptable human intervention modes. On the ARC-Bench benchmark, it outperforms AI Scientist v2 by 54.7%.
Link:Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, Joseph E. Gonzalez
The Berkeley Function Calling Leaderboard (BFCL) V4 evaluates the ability of large language models to call functions and tools accurately. Version 4 expands previous iterations by introducing holistic agentic evaluation.
Link:Rogerio Bonatti, Dan Zhao, Francesco Bonacci, et al.
Researchers from Microsoft, Carnegie Mellon University, and Columbia University have launched Windows Agent Arena (WAA), an open-source evaluation framework for multi-modal AI agents operating in native Windows 11 environments. Featuring over 150 real-world tasks, WAA supports parallel execution in Azure to shorten evaluation cycles from days to around 20 minutes. A newly introduced reference agent, Navi, attained a 19.5% success rate against human performance of 74.5%.
Link:arcprize.org
ARC-AGI-3 is an interactive reasoning benchmark designed to evaluate human-like intelligence in AI agents. Moving beyond static puzzles, agents must explore novel environments, derive goals on the fly, and adapt through multi-step interaction.
Article:ARC Prize Foundation
The ARC Prize Foundation has introduced ARC-AGI-3, an interactive benchmark designed to evaluate agentic intelligence within novel, turn-based environments. While human testers achieve a 100 percent completion rate, frontier AI systems score below one percent as of March 2026.
Article:Siru Ouyang, Jun Yan, I-Hung Hsu, et al.
ReasoningBank is a memory framework for LLM agents that distils generalisable reasoning strategies from self-judged successful and failed experiences, enabling continuous self-evolution over time.
Parts of this content were created with the assistance of AI.
