agents-on-modal: Run Coding Agents Against Open-Weight Models on Modal

Repositorysshh12/agents-on-modalTool

This Python utility connects coding agent CLIs like Claude Code and Codex to self-hosted open-weight LLMs on Modal serverless GPUs. It enables running isolated developer agents locally, in Docker, or via Modal sandboxes.

Key points

  1. Bridges agent harnesses (Claude Code, Codex CLI) with open-weight models such as Qwen3, GPT-OSS, DeepSeek-V4, and GLM-5.3.
  2. An in-container gateway maps Anthropic Messages and OpenAI Responses protocols to chat completions APIs.
  3. LLMs are served serverless via SGLang or llama.cpp on Modal GPUs with scale-to-zero capability.
  4. Offers three execution sandboxes: host machine, isolated local Docker containers, or parallel Modal sandboxes.
  5. Model parameters, GPU requirements, and reasoning-effort mappings are declaratively configured in catalog.json.

Why it matters

It lets development and security teams run agentic coding workflows using proprietary-style harnesses without relying on closed APIs, maintaining privacy and controlling GPU compute costs.

Analysis

Overview and Architecture

Developer agent CLIs typically demand proprietary cloud APIs. agents-on-modal removes this vendor tie-in by introducing three orthogonal layers:

  • Model layer: Deploys open-weight or abliterated models (e.g. Qwen, DeepSeek, GLM) via SGLang or llama.cpp on Modal infrastructure, supporting dynamic scale-to-zero.
  • Protocol gateway: A lightweight bridge translates agent requests (Anthropic Messages API or OpenAI Responses API) into engine-compatible completions endpoints, mapping model context and reasoning efforts transparently.
  • Execution sandboxes: Workflows can be executed on bare metal, inside local Docker containers without host/LAN network access, or dispatched across parallel Modal sandboxes.

Target Audience and Project Maturity

The project is designed for software engineers, offensive security researchers, and red teams running automated code tasks or uncensored evaluation pipelines. As of November 2026, the project remains in an early stage (5 stars, 0 forks, and no explicitly listed software license). While not yet hardened for strict enterprise governance, its architectural decoupling provides a viable pattern for serverless agent testing.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 8 November 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:FeiZhuLulu/Agent-Bridge

    Agent Bridge: Orchestrating Local Coding Agents via MCP

    Agent Bridge is an open-source Python tool connecting local coding agents using the Model Context Protocol (MCP). It allows users to interact with a single coordinator agent that delegates tasks to multiple specialized CLI worker agents.

    31starsPython

    AI & LLMs· Tool

  • X post:Shrivu Shankar

    agents-on-modal: Running Abliterated LLMs for Red Teaming

    Shrivu Shankar highlights abliterated open-weight models for autonomous security assessments and shares 'agents-on-modal'. The framework connects agent CLIs like Claude Code and Codex to self-hosted models running on Modal GPUs.

    157bookmarks2.7kviews

    AI & LLMs· Resource

  • Link:Anomaly

    OpenCode: Open-source AI coding agent

    OpenCode is an open-source AI coding agent running across terminals, IDEs, and desktop environments, offering integration with over 75 model providers.

    AI & LLMs· Tool

  • Repository:Gitlawb/openclaude

    OpenClaude: Provider-Agnostic Open-Source Coding Agent CLI

    OpenClaude is an open-source TypeScript CLI tool functioning as an AI coding agent. It standardises terminal-based development workflows across varied local and cloud model providers.

    32.3kstarsTypeScript

    AI & LLMs· Tool

  • Link:modal.com

    Modal: Serverless GPU Infrastructure for Production AI and Agents

    Modal is a cloud infrastructure platform designed to develop, train, and run AI workloads at scale. It offers serverless compute, fast model inference, training clusters, and lightweight sandboxes with per-second billing. It enables AI engineering teams to deploy compute-heavy systems without managing hardware.

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.