Berkeley Function Calling Leaderboard (BFCL) V4

LinkShishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, Joseph E. GonzalezTool

The Berkeley Function Calling Leaderboard (BFCL) V4 evaluates the ability of large language models to call functions and tools accurately. Version 4 expands previous iterations by introducing holistic agentic evaluation.

Key points

  1. BFCL evaluates tool use using real-world data; V1 introduced AST metrics, V2 added enterprise and open-source functions, V3 multi-turn interactions, and V4 holistic agentic evaluation.
  2. The benchmark distinguishes between native function calling support (FC) and prompt-based workarounds relying on standard text generation.
  3. Core metrics include overall accuracy (unweighted average of subcategories), latency in seconds, and benchmark cost estimates in USD.
  4. Evaluations are reproducible using the PyPI package bfcl-eval==2025.12.17 and publicly shared evaluation responses.
  5. The platform includes an interactive function calling demo producing OpenAI-compatible outputs, alongside a 'Wagon Wheel' comparative chart.

Why it matters

Accurate tool invocation is fundamental for building reliable AI agents and API-driven automation. BFCL provides an objective, reproducible metric for comparing LLM capabilities across accuracy, response time, and operating cost.

Analysis

BFCL V4 marks a progression from single-turn API execution tests to full agentic evaluations. By differentiating between native function-calling interfaces and prompt-based alternatives, the benchmark highlights trade-offs in model reliability and formatting sensitivity. Providing both the PyPI evaluation package and complete raw model outputs ensures transparency and independent verification.

Mentioned links

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Article:Ishan Patel, Sahil Sen, Elias Lumer, Vamse Kumar Subbiah

    Research Compares Programmatic Tool Calling to JSON Tool Calling

    A research paper evaluates Programmatic Tool Calling (PTC) against native JSON tool calling across 14 language models using the BFCL v4 benchmark. In PTC, tools are represented as typed Python stubs invoked via code in a single agent turn. The approach matches or exceeds native JSON tool calling in 11 out of 14 models.

    AI & LLMs· Research

  • Article:Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Felix Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, Ruoming Pang

    ToolSandbox: Stateful and Conversational Benchmark for LLM Tool Use

    ToolSandbox is an evaluation benchmark that assesses LLM tool-use capabilities using stateful environments and multi-turn interactive dialogues.

    AI & LLMs· Research

  • Link:clickrank.ai

    LLM Leaderboards in 2026: Benchmarks, Methodologies, and Model Rankings

    A guide by ClickRank AI examines the state of LLM evaluations in 2026. With legacy benchmarks like MMLU and HumanEval largely saturated, evaluation platforms now focus on harder reasoning tests, human pairwise blind comparisons, and composite indices covering cost and latency.

    AI & LLMs· Resource

  • Link:The Cognition Team

    FrontierCode: Benchmarking AI Code by Maintainer Mergeability

    Cognition has published FrontierCode, a benchmark and leaderboard evaluating whether pull requests generated by AI models would realistically be merged by open-source maintainers. Scoring relies on unit tests, verifiers, and maintainer-defined rubrics.

    AI & LLMs· Resource

  • Link:Masato Okuwaki

    Programmatic Tool Calling vs Native JSON: When Should an Agent Write Code?

    Masato Okuwaki reviews a PwC study comparing native JSON tool calling against programmatic tool calling (PTC) using Python scripts across 14 LLMs on BFCL v4. Letting models emit executable code matched or beat JSON baselines in eleven of the fourteen tested models.

    AI & LLMs· Research

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.