Article:Meng Li
DeepSeek Harness vs. Codex Harness: Competing Engineering Philosophies for AI Agents
In August 2026, OpenAI and DeepSeek both introduced execution harnesses for AI agents. Meng Li examines how their technical philosophies diverge.
Terminal-Bench v2.1: Why Agent Harnesses Determine LLM Benchmark Performance
LinkResearch
New benchmarks on Terminal-Bench v2.1 demonstrate that agent harnesses fundamentally shift model rankings: DeepSeek-V4-Flash 0731 surged from a 49.4% pass rate on stock tooling to 71.9% under the OMP harness, drawing nearly level with GLM 5.2 (73.0%).
Key points
- On Terminal-Bench v2.1 (89 tasks), GLM 5.2 3.25 BPW EXL3 scored 68.5% (61/89) on a stock harness and rose to 73.0% (65/89) using the OMP harness.
- DeepSeek-V4-Flash 0731 experienced a massive performance leap under OMP, climbing from 49.4% (44/89) to 71.9% (64/89) and validating manufacturer claims.
- In decode speed, DSV4F-0731 reaches 300-400 tok/s, operating roughly 4.5 times faster than GLM 5.2 3.25 BPW (70-80 tok/s) under identical vLLM TP=4 hosting.
- OMP gains require substantially more compute and tokens: DSV4F consumed an estimated 5.33x more tokens per solved task (~154K vs. ~62K) when using OMP.
- The underlying test rig operates four RTX Pro 6K GPUs connected through a PCIe Gen 5 switch, MCIO cabling, and a retimer to ensure host-independent P2P bandwidth.
Why it matters
Evaluations of coding models cannot be separated from their agent harness and test-time compute. A model that looks like a failure in a basic harness can rival leading architectures when given an agent loop capable of smoothing out multi-turn execution, trading token volume for higher task success.
Analysis
The author highlights how model makers overfit to proprietary evaluation harnesses, making baseline replication difficult. GLM 5.2 demonstrates strong intrinsic reasoning that holds up even on minimal harnesses, whereas DeepSeek-V4-Flash 0731 relies heavily on harness scaffolding. Because DSV4F generates tokens four times faster, it can comfortably accommodate the heavy token budget OMP expends per solved problem.
Mentioned links
Found in
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Related entries
Similar in content, found by the AI search.
Link:deepseek-ai
DeepSeek-V4-Flash-0731: Official Release with Speculative Decoding Under MIT Licence
DeepSeek-AI has officially released DeepSeek-V4-Flash-0731 on Hugging Face. The 304B-parameter model supersedes the preview version, featuring integrated DSpark speculative decoding and significantly enhanced agentic performance.
Article:DeepInfra
Top LLMs for OpenClaw Agentic Workloads
DeepInfra evaluates open-weight models for the OpenClaw autonomous agent framework, focusing on tool-calling reliability, long-session instruction adherence, and cost per completed task.
Article:Mehdi Allahyari
The Harness Beats the Model: Demystifying Agent Harness Engineering
Following prompt and context engineering, harness engineering marks the third wave in developing autonomous AI systems. Mehdi Allahyari explains why the software infrastructure wrapping an LLM matters more than raw model scale for reliability and self-correction.
Link:Z.ai
Z.ai Releases GLM-5.1 Focusing on Agentic Engineering
Z.ai has published GLM-5.1 on Hugging Face under the MIT licence. The 754B parameter MoE model is engineered for advanced coding and long-horizon agentic workflows.
Link:clickrank.ai
LLM Leaderboards in 2026: Benchmarks, Methodologies, and Model Rankings
A guide by ClickRank AI examines the state of LLM evaluations in 2026. With legacy benchmarks like MMLU and HumanEval largely saturated, evaluation platforms now focus on harder reasoning tests, human pairwise blind comparisons, and composite indices covering cost and latency.
Let's talk about your project
Locations
- Mattersburg
Johann Nepomuk Bergerstraße 7/2/14
7210 Mattersburg, Austria - Vienna
Ungargasse 64-66/3/404
1030 Wien, Austria
Parts of this content were created with the assistance of AI.