Claude Code: Why Manual Confirmation Fails and How Anthropic Redesigned Safety

X postShoopyNews

Boris Cherny, creator of Anthropic's Claude Code, revealed that users approved destructive commands almost every time during tests. Anthropic now relies on an isolated second model with zero conversation context and neuron-level monitoring to enforce security.

Key points

  1. Tests showed that developers approved destructive terminal commands prompted by Claude Code almost every time.
  2. Cherny classified 'allow this command?' prompts as security theatre due to widespread approval fatigue.
  3. An independent second model with zero chat context now evaluates every command in isolation.
  4. Neuron activation monitoring is used to counter prompt injection; a week-long challenge with a $20,000 bounty yielded no successful breaches.
  5. The author ties these findings to graph engineering, advocating for modular multi-agent architectures with explicit validation gates.

Why it matters

Human-in-the-loop safeguards frequently fail due to prompt fatigue, turning review prompts into rubber stamps. Reliable agent security requires independent automated verification instead of relying on human alertness.

Analysis

Cherny's test results demonstrate the fragility of human approval gates in automated developer workflows. High approval volume inevitably leads to habituation and blind acceptance. Anthropic's architectural shift—deploying an isolated model stripped of conversation context—ensures commands are evaluated without inheriting conversational manipulation or social engineering tactics.

Original post

Shoopy

@0xShoopy · 25 September 2026

Boris Cherny (creator of Claude Code, Anthropic): "in a test, we slipped destructive commands into people's Claude Code sessions. they approved them almost every time." the "allow this command?" prompt was supposed to be the safety net. Boris calls it security theater – nobody t.co/vKzDA3dPPl t.co/zAoPhdgBcV

96 likes25 replies131 bookmarks60.4k views

View on X
More posts in the thread (9)
  1. @shukshln independent verification without shared context is the key design choice here
  2. @tonyprediction and most people don't even realize they're the ones leaving it open
  3. @Ahmad_a_m_3 always has been, the model was never the problem
  4. @dreyk0o0 inevitable once the volume of approvals went up
  5. @sahilcode great point, zero context on the checker is what makes it resistant to social engineering
  6. @James__Hallam he was careful with it: they can no longer demonstrate a successful injection, not that none exist. he said they probably do, just really hard to find now. he posted a chart comparing labs, no paper that i've seen
  7. @CaseSignalX good comparison. a safety system people stop paying attention to is worse than none, because everyone assumes it's working
  8. @sophq37 that's basically where they landed. a second model with no context of your chat judges each command, so it's not relying on you reading anything
  9. @cyberogz exactly the right question. a check you can click past is just another yes button
Selected replies (5)
  • @MonishAtx @0xShoopy Makes sense. After the 50th allow prompt nobody reads the 51st. Approval prompts check intent, not outcome. The review that matters is after the fact: what changed, what broke, what got exposed. That's where I'd rather spend human attention.
  • @Ahmad_a_m_3 @0xShoopy The human approval step really is the weak link
  • @tonyprediction @0xShoopy wow thats a pretty wild security hole ngl
  • @sahilcode @0xShoopy the part that stands out is zero context on the second model. if it saw the whole conversation it would inherit whatever talked the human into clicking yes. checking the command in isolation is what makes it hard to social-engineer the same way twice.
  • @dreyk0o0 @0xShoopy click fatigue was always going to break approvals

Mentioned links

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Article:Anthropic Engineering

    How Anthropic Isolates and Contains Claude Agents Across Products

    Anthropic outlines its containment architecture designed to cap the blast radius of autonomous AI agents across claude.ai and Claude Code. As human oversight falters due to approval fatigue, security relies primarily on deterministic OS and container sandboxes.

    AI & LLMs· Research

  • Article:Addy Osmani, Morgan Westlee Lunt, Qing Wang

    Code Review at Anthropic: How the Claude Code Team Reviews AI-Generated Code

    The Claude Code engineering team explains how they handle code reviews when agents write around 90 percent of the codebase. Instead of exhaustive manual line-by-line checks, they rely on pre-PR verification, automated multi-agent reviews, risk-tiered human approvals, and rapid rollbacks.

    AI & LLMs· Resource

  • X post:ClaudeDevs

    How the Claude Code Team Redesigned Code Review for AI

    Around 90 percent of Claude Code is written by Claude itself. The Anthropic engineering team shares how shifting to automated, multi-layered reviews more than doubled merged pull requests while cutting reverts by over 40 percent.

    10kbookmarks1.6mviews

    AI & LLMs· Opinion

  • X post:Casey Lee

    Anthropic's AI-Native SDLC Playbook: Scaling Pipelines for Fast Coding Agents

    Casey Lee highlights Anthropic's AI-native SDLC playbook. While agents like Claude Code can generate PRs within hours, traditional security reviews and product sign-offs still take days. Anthropic proposes structuring planning, reviews, and releases into agent-executable steps governed by human checkpoints.

    276bookmarks22kviews

    AI & LLMs· Opinion

  • Article:Anthropic

    Claude Code Power User Tips from Anthropic's Engineering Team

    Engineers from the Claude Code team at Anthropic share core workflow patterns for shipping code faster, highlighting parallel execution via Git worktrees, plan-first workflows, automated verification, and persistent project memory.

    AI & LLMs· Resource

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.