Neuron Explorer: Interactive Steering and Interpretability Tool for Qwen3.5-4B

Repositorylyramakesmusic/neuron-explorerTool

Neuron Explorer is a mechanistic interpretability tool for Qwen3.5-4B that tests and visualises model steering. By manipulating individual MLP neurons across prompts and doses, it maps activations and rollouts in an interactive 3D interface. It helps AI researchers and engineers examine internal representations.

Key points

  1. Executes systematic neuron sweeps by shifting MLP neurons up and down across prompts.
  2. Features an interactive 3D map of model layers alongside logit and steer lens projections.
  3. Includes a 3-column multi-turn chat comparing negative, baseline, and positive steering doses.
  4. Integrates with the Claude CLI via MCP, enabling agentic ablations, queries, and UI navigation.
  5. Requires a vLLM steering fork on CUDA 13 for batch sweeps, with local runtime support via MPS, CPU, or CUDA.

Why it matters

Mechanistic interpretability and activation steering provide practical pathways for AI alignment and behaviour control. This project simplifies exploring how individual MLP neurons shape model outputs.

Analysis

Purpose and Architecture

Neuron Explorer tackles the challenge of understanding internal model mechanics through a steering-first methodology on Qwen3.5-4B. Rather than relying solely on passive feature observation, it actively perturbs MLP neurons across various dose levels and records complete output rollouts.

Core Capabilities

The local web explorer (http://localhost:8791) combines multiple analytical views:

  • Spatial Navigation: A 3D layer perspective arranges neurons based on down-projection matrix similarities (writes). Hovering surfaces steer and logit lenses.
  • Evaluation & Clustering: Provides list rankings against random controls, live activation filtering on user text, and UMAP embeddings of rollout clusters.
  • Agent Integration: An MCP server (explorer/agent_mcp.py) connects the claude CLI, allowing an AI assistant to inspect lenses, trigger ablations, and control UI highlights autonomously.

Technology and Maturity

Built with Python and managed via uv, the pipeline uses a custom vLLM fork (RhizoNymph/vllm) for high-throughput GPU sweeps. While popular among researchers with roughly 1,500 GitHub stars, it remains an experimental research instrument with an unspecified software licence, tailored specifically for mechanistic interpretability practitioners.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 28 October 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:ahujasid/mcp-for-blender

    mcp-for-blender: Controlling Blender via the Model Context Protocol

    The open-source tool mcp-for-blender connects Blender 3D to any LLM using the Model Context Protocol (MCP). It enables prompt-assisted 3D modelling and scene manipulation for 3D artists, technical directors, and developers.

    32.9kstarsPython

    AI & LLMs· Tool

  • Repository:ahujasid/blender-mcp

    blender-mcp: Controlling Blender via the Model Context Protocol

    blender-mcp is an open-source community integration that connects Blender 3D to LLMs using the Model Context Protocol (MCP). It enables prompt-assisted 3D modeling, scene generation, and manipulation directly through AI clients.

    26.7kstarsPython

    AI & LLMs· Tool

  • Repository:MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install

    One-Click Installer for Qwen3.8-27B on 16 GB Nvidia GPUs

    A serving kit for Windows and Linux enables running the Qwen3.8-27B language model locally on consumer Nvidia GPUs with 12 to 16 GB of VRAM using EXL3 quantisation.

    529starsPython

    AI & LLMs· Tool

  • Repository:img2threejs/img2threejs

    img2threejs: Procedural Code-Only Image-to-3D for Three.js

    img2threejs turns reference images into procedural, animation-ready Three.js models generated purely as code, avoiding traditional mesh extraction or photogrammetry.

    15.1kstarsPython

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.