MCDMA Enables Bonded Thunderbolt 5 Connections in TensorFold on Apple M5 Ultra

X postAsh HartAnnouncement

Developer Ash Hart has announced bonded pair Thunderbolt 5 connectivity within the MCDMA RDMA framework. Paired with the TensorFold inference engine across two Apple Mac Studios running M5 Ultra chips, the dual-cable link delivers an 83 percent throughput uplift for tensor and pipeline parallelism.

Key points

  1. MCDMA adds dual-pipe RDMA across two Thunderbolt 5 connections between Macs running macOS 27.0.
  2. Streaming transfer throughput increases from around 6.4–6.67 GB/s on a single cable to 11.72–12.06 GB/s over two bonded cables (~1.81x to 1.83x).
  3. In TensorFold, the bonded link reduces time to first token by an extra 1.5–2 percent on 8k and 32k token contexts when serving models like Qwen3.8 Flash Next.
  4. Decode phase throughput sees negligible improvement (0.5–1 percent) as small token exchanges depend on network latency rather than aggregate bandwidth.
  5. The functionality was benchmarked on two M5 Ultra Mac Studios and is pushed to GitHub.

Why it matters

Clustering consumer and prosumer workstations for local LLM inference has historically been choked by interconnect bottlenecks. Unlocking bonded high-speed Thunderbolt 5 RDMA provides an accessible pathway for distributed pipeline and tensor parallelism on Apple Silicon clusters without needing enterprise InfiniBand infrastructure.

Analysis

Data striping across two Thunderbolt 5 ports delivers near-linear bandwidth scaling for raw memory transfers. However, inference workloads reveal nuanced bottlenecks: bandwidth gains visibly assist the memory-heavy prefill stage on long prompts, while the iterative single-token decode stage remains largely latency-constrained. Ash Hart also noted that broader scaling is capped by memory bandwidth boundaries.

Original post

Ash Hart

@ashxhart · 6 October 2026

MCDMA now unlocks bonded pair TB5 connections within TensorFold, which adds 83% uplift in throughput for Tensor and Pipeline parallelism. Thank you, @digitalix, for letting me try this on your two @apple M5U studios. t.co/VfyMCFNcqG t.co/WWS5q0NKoF t.co/Z2jobonlco

409 likes44 replies216 bookmarks24.3k views

View on X
More posts in the thread (10)
  1. @Soveryn_AI @digitalix @Apple Already did with not much improvement due to memory bandwidth limits.
  2. @CerebralCoding_ @digitalix @Apple It should work just ask your agent to set it up :)
  3. @cryptosibbers @digitalix @Apple yeah it's already in pushed on gh.
  4. @arthurlee @digitalix @Apple Nothing yet but we can create you something if you’d like?
  5. @shesaidmewakeup @digitalix @Apple I’ve only tested two cables so far
  6. @ElGuiscardo @digitalix @Apple Yes
  7. @JamesMcPherson @MiaAI_lab @digitalix @Apple t.co/qGQkBfHOFf
  8. @wong_wyz @digitalix @Apple Yes.
  9. @mHatak @digitalix @Apple Yeah should do, not tested it.
  10. @JoesInvestments @digitalix @Apple Just doing my bit to help local AI.
Selected replies (5)
  • @JoesInvestments @ashxhart @digitalix @Apple At some point, @Apple will need to consider hiring you. You're single handedly moving the needle on sales when you ship code like this. Incredible.
  • @JamesMcPherson @ashxhart @MiaAI_lab @digitalix @Apple This is a big deal. Well done. Just a week ago I had storms of people telling me that Apple had proven this was impossible. Impossible is a funny word in the age of AI.
  • @CerebralCoding_ @ashxhart @digitalix @Apple Would love to see how it behaves with an M5 Ultra + M5 Max (since that’s the setup I’m aiming for 🤣)
  • @JoshuaSWarren @ashxhart @volatilemarkts @digitalix @Apple I absolutely will be testing and/or PRing any tweaks needed to get this working with Omarchy Linux on M5U once my M5U arrives in (checks order status) 16-18 weeks 😒
  • @xLouisHein @ashxhart @digitalix @Apple This is amazing, great work!!

Mentioned links

Links and tools from this entry

Summary generated by AI (Gemini 3.8 Flash, 7 October 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:ashhart/MCDMA

    MCDMA: RDMA Memory Access Linking Mac Studio, NVIDIA Spark, and AMD Strix Halo

    MCDMA enables Remote Direct Memory Access (RDMA) across heterogeneous workstations. Built in C under Apache-2.0, it bridges Apple Silicon Macs via Thunderbolt with Linux hosts like NVIDIA DGX Spark and AMD Strix Halo to accelerate distributed AI workloads.

    178starsC

    AI & LLMs· Tool

  • Repository:ashhart/TensorFold

    TensorFold: LLM Inference Engine for Apple Silicon and Modern NVIDIA GPUs

    TensorFold is an open-source inference engine designed to serve large language models on Apple Silicon (via MLX/Metal) and newer NVIDIA GPUs (Compute Capability 8.9+). It provides OpenAI- and Anthropic-compatible endpoints with model-tailored kernels and speculative draft verification.

    1.1kstarsPython

    AI & LLMs· Tool

  • Link:Apple

    Apple Introduces New Mac mini Powered by M6 and M5 Pro

    Apple has announced a new generation of the Mac mini featuring M6 and M5 Pro chips. The ultra-compact 5-by-5-inch desktop focuses heavily on local AI inference, development workflows, and small-scale clustering.

    AI & LLMs· Announcement

  • X post:Xaden Ryan

    Hybrid LLM Inference Setup: DGX Sparks for Prefill and M3 Ultra for Decode

    Xaden Ryan is experimenting with a split LLM pipeline: two DGX Sparks will handle the compute-heavy prefill phase, followed by transferring the KV cache to a 512 GB Apple M3 Ultra for decoding, targeting 200k-token contexts in GLM-5.3.

    25bookmarks8.8kviews

    AI & LLMs· Discussion

  • Repository:drumih/turbo-fieldfare

    TurboFieldfare: Gemma 4 26B-A4B Inference in 2 GB RAM on Apple Silicon

    TurboFieldfare is a dedicated Swift and Metal runtime designed to run Google's Gemma 4 26B-A4B on Apple Silicon Macs within an approximately 2 GB RAM budget. It avoids loading the full 14.3 GB model into memory by keeping only the 1.35 GB core and KV cache in RAM while streaming the required mixture-of-experts weights from SSD on demand.

    6.6kstarsSwift

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.