Steering AI agents with ambitious metrics for test suite pruning

X postPeter SteinbergerOpinion

Peter Steinberger advises against giving AI agents generic cleanup instructions, as they terminate tasks prematurely. Instead, developers should assign ambitious, measurable targets, such as removing 20 percent of the least useful tests while keeping code coverage within two percent.

Key points

  1. Generic cleanup prompts cause AI coding agents to halt work far too early.
  2. Ambitious, quantified targets compel the agent to perform deeper iterations.
  3. An example metric: prune 20% of the least useful tests while keeping coverage within 2%.
  4. Commands like /goal can run autonomously over extended periods to execute these goals.

Why it matters

Bloated test suites hinder CI/CD performance and increase maintenance friction. Using AI agents for test pruning requires strict, quantifiable guardrails rather than open-ended directives.

Analysis

Steinberger highlights an important operational pattern in agentic workflows: autonomous agents require quantitative constraints to avoid early stopping. While open-ended refactoring requests quickly satisfy loose stopping criteria, explicit mathematical trade-offs force continuous evaluation cycles. However, commenters rightly noted that relying purely on code coverage as a boundary condition carries risks, as removing redundant-looking tests may inadvertently sacrifice edge-case coverage without impacting overall percentage metrics.

Original post

Peter Steinberger 🦞

@steipete · 24 September 2026

If you just tell the agent to clean up, it will stop far too early. Give it an ambitious goal. Try "remove 20% of the least useful tests while maintaining code coverage within 2%"

593 likes16 replies515 bookmarks102.7k views

View on X
More posts in the thread (3)
  1. @PavelVanchugov Google rejected the update and it’s stalling. Working on it.
  2. @bvirlet It’s not meant for humans.
  3. That /goal is still running.
Selected replies (5)
  • @ZainanZhou @steipete Bill Gates has a great analogy: "Measuring programming progress by lines of code is like measuring aircraft building progress by weight." Reducing the lines of code in testing is like reducing the weight of ... the hangar. Yeah maybe it's helpful but also doesn't makes sense to
  • @akshaymarch7 @steipete A useful follow-up would be to reintroduce a few previously fixed bugs and check whether the smaller suite still catches them. That would give me more confidence in the cleanup than the coverage number alone.
  • @gauravdhiman_ai @steipete How can you trust them now, that what they did is right .. when you earlier trusted them assuming what they did was right 😄 You are just driving the car blindly without knowing when it's going to get into an accident.
  • @ChristianLempa @steipete @NousResearch did it first 😜
  • @CycleOrpington @steipete All tests are useful t.co/wNjKWr7Eci

Mentioned links

Summary generated by AI (Gemini 3.8 Flash, 30 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.