Phone Harness: AI Agents Controlling Real Smartphones Without Companion Apps

RepositoryShawnPana/phone-harnessTool

Phone Harness is an open-source tool that allows AI agents such as Claude Code or Codex to control physical smartphones. It bypasses jailbreaks, Xcode, and WebDriverAgent by relying on macOS Sequoia iPhone Mirroring for iOS and ADB for Android.

Key points

  1. iPhone control runs through macOS Sequoia iPhone Mirroring using screencapture, Apple Vision OCR, and CGEvents.
  2. Android control runs via adb over USB or Wi-Fi using the device accessibility tree, screencap, and input commands.
  3. No companion app installation on the phone and no jailbreak required.
  4. Integrates as an agent skill, allowing LLMs to generate missing helper functions in agent_helpers.py on the fly.
  5. Setup relies on a CLI doctor command (phone-harness --doctor) and macOS Accessibility and Screen Recording permissions.

Why it matters

Automating physical mobile devices usually requires cumbersome test infrastructures like WebDriverAgent or instrumented apps. Phone Harness simplifies this by using existing desktop mirroring and debugging transports.

Analysis

Targeted at developers automating mobile UI workflows via LLM agents, the tool employs Apple Vision framework OCR and HID events for iOS alongside native accessibility trees and ADB for Android. With over 2,300 GitHub stars and built-in diagnostic tooling, it presents an accessible harness, though it inherently depends on Mac host permissions and Android developer settings.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:droidrun/mobile-harness

    Mobile Harness: Control Skills for AI Agents on Android and iOS

    Mobile Harness provides portable operating instructions and interfaces enabling AI agents to control Android and iOS devices locally or in the cloud. Rather than acting as a standalone agent runtime, it functions as a compact Markdown harness powered by the mobilerun_core Python library.

    340stars

    AI & LLMs· Tool

  • X post:Niels Schmidt

    Mobile Harness: Open-Source iOS and Android Control for AI Agents

    Niels Schmidt has introduced Mobile Harness, an open-source harness enabling coding agents such as Codex or Claude Code to drive real iOS and Android devices locally and in the cloud through a unified API.

    1.3kbookmarks124.6kviews

    AI & LLMs· Announcement

  • Repository:mobile-next/mobile-mcp

    Mobile Next: MCP Server for Cross-Platform Mobile Automation

    Mobile Next is an open-source Model Context Protocol (MCP) server that connects AI agents and LLMs to iOS and Android environments. It abstracts platform-specific quirks across simulators, emulators, and real physical devices.

    6.3kstarsTypeScript

    AI & LLMs· Tool

  • Link:mobilenext.ai

    Mobile Next: Programmable Mobile Runtime for AI Agents

    Mobile Next provides an infrastructure that allows AI agents to control real iOS and Android devices, simulators, and emulators via API and the Model Context Protocol (MCP). The stack comprises an MCP server, a CLI utility, the Playwright-inspired Mobilewright testing framework, and an on-demand cloud device farm.

    AI & LLMs· Tool

  • Video

    Video:Greg Isenberg

    Five Open-Source GitHub Repos for the Agentic Era

    Greg Isenberg highlights five trending open-source GitHub repositories designed for AI agent workflows: removing generic AI writing patterns, agent-driven CRM management, automated video editing via coding agents, security auditing for skills, and smartphone control.

    AI & LLMs· Resource

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.