Vellum LLM Leaderboard: Benchmarks, Throughput, and Pricing Overview

LinkVellumResource

Vellum's LLM Leaderboard evaluates leading language models across common industry benchmarks, tracking throughput, latency, and token pricing.

Key points

  1. Claude Fable 5.1 and Claude Mythos 5.1 top the Humanity's Last Exam benchmark with 65% each.
  2. Claude Sonnet 5 leads GPQA Diamond reasoning at 96.2%, followed closely by GPT-6 Astra at 96%.
  3. In agentic coding (SWE Bench), GPT-5.6 Sol ranks first at 96.2%, followed by Claude Mythos 5 at 95.5%.
  4. Llama 4 Scout delivers the highest throughput at 2,600 tokens/sec, while GPT-5.3 Codex records the lowest TTFT at 0.003s.
  5. Nova Micro ($0.04 input / $0.14 output per 1M tokens) and Gemini 1.5 Flash remain among the cheapest options.

Why it matters

The leaderboard offers technical decision-makers and developers clear empirical guidance to balance benchmark accuracy, operational latency, and per-token API costs.

Analysis

The benchmark data highlights distinct model specialisations: Anthropic holds top spots in complex reasoning and computer use (OSWorld), OpenAI excels in agentic coding and browser workflows, and specialised models like GLM-5.3-Flash challenge the frontrunners in automation-focused tasks at lower price points.

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Similar in content, found by the AI search.

  • Link:clickrank.ai

    LLM Leaderboards in 2026: Benchmarks, Methodologies, and Model Rankings

    A guide by ClickRank AI examines the state of LLM evaluations in 2026. With legacy benchmarks like MMLU and HumanEval largely saturated, evaluation platforms now focus on harder reasoning tests, human pairwise blind comparisons, and composite indices covering cost and latency.

    AI & LLMs· Resource

  • Link:StackAI

    StackAI LLM Leaderboard: Comparing Models by Benchmarks and Cost

    StackAI provides an interactive dashboard comparing large language models across multiple benchmarks, throughput, context window sizes, and pricing. It compiles evaluation metrics such as MMLU, HumanEval+, GPQA, SWE benchmarks, and GSM8K for side-by-side model comparison.

    AI & LLMs· Tool

  • Link:Vercel

    DeepsecBench: Vercel Publishes AI Leaderboard for Vulnerability Detection

    Vercel's AI Gateway features the DeepsecBench leaderboard, evaluating AI models on their ability to detect security vulnerabilities in application code using the Deepsec cyber harness. The benchmark calculates an overall score balancing recall and precision.

    AI & LLMs· Resource

  • X post:Guillermo Rauch

    DeepsecBench: GPT-6 Astra Tops Vercel's Cybersecurity Leaderboard

    According to Vercel CEO Guillermo Rauch, OpenAI's GPT-6 Astra has taken the top spot on Vercel's DeepsecBench. The model completes benchmark evaluations in under 50 minutes, compared to nearly four hours for its predecessor.

    180bookmarks81.1kviews

    AI & LLMs· News

Let's talk about your project

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.