Beyond VLAs: NVIDIA Cosmos 3 Introduces World Action Models for Robot Manipulation

ArticleSaeed BabamohamadiNews

NVIDIA details Cosmos 3, an open omni-model foundation that replaces conventional vision-language backbones with video world models to give robot policies predictive physical priors.

Key points

  1. World Action Models (WAMs) utilize video world models instead of language-centric backbones, allowing policies to anticipate physical changes rather than solely describing scenes.
  2. Cosmos 3 employs a Mixture-of-Transformers (MoT) architecture that pairs an autoregressive transformer for discrete tokens with a diffusion transformer for continuous modalities such as video and action trajectories.
  3. The foundation model is available in 4B (Cosmos Edge), 16B (Cosmos Nano), and 64B (Cosmos 3 Super) tiers, pretrained on 767 million images, 348 million dynamics videos, and 8 million action samples.
  4. Evaluations on the DROID platform demonstrate that post-training from the omni checkpoint improved RoboLab success rates from 28.1% to 36.8% compared to the standard base checkpoint.
  5. Deployment options span workstation inference using the 16B Nano model on an NVIDIA RTX PRO 6000 to on-device control with the 4B Edge model running at 15 Hz on NVIDIA Jetson Thor.

Why it matters

Standard Vision-Language-Action (VLA) models often fail when encountering altered physical conditions such as unseen friction or object geometry. By learning underlying physics from massive interaction video data, WAMs adapt to new robot embodiments with significantly fewer demonstration trajectories.

Analysis

Predicting physical actions alongside simulated visual outcomes within a unified multimodal model directly addresses the physical generalization bottleneck in manipulation tasks. Offering open weights, post-training recipes, and LeRobotDataset compatibility under a commercially viable license makes Cosmos 3 an immediately accessible foundation for robotics developers.

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Similar in content, found by the AI search.

  • Article:Moo Jin Kim et al.

    OpenVLA: An Open-Source Vision-Language-Action Model for Robot Control

    Researchers from Stanford, UC Berkeley, Google DeepMind, and partner institutions introduce OpenVLA, an open-source 7-billion-parameter vision-language-action model. Trained on 970k real-world robot demonstrations, it surpasses closed models such as RT-2-X while enabling parameter-efficient fine-tuning on consumer-grade hardware.

    AI & LLMs· Research

  • Link:Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti et al.

    OpenVLA: Open-Source Vision-Language-Action Model for Robotics

    OpenVLA is an open-source 7B parameter vision-language-action (VLA) model designed for generalist robot manipulation. Built on Prismatic-7B and Llama 2, it was trained on 970k episodes from the Open X-Embodiment dataset and outperforms earlier generalist policies such as Octo, RT-1-X, and in several benchmarks the closed 55B RT-2-X.

    AI & LLMs· Research

  • Article:Peiyan Li, Yuze Zhu, Yixiang Chen, Qisen Ma, Yuan Xu, Jiabing Yang, He Guan, Yan Huang, Hongtao Wu, Xiao Ma, Tao Kong, Liang Wang, Tieniu Tan

    BridgeVLA++: A Memory-Augmented 3D Vision-Language-Action Framework for Robotics

    Researchers introduce BridgeVLA++, an upgraded vision-language-action framework designed for 3D robotic manipulation. It introduces a unified spatio-temporal memory architecture into pre-trained vision-language models, enabling persistent spatial reasoning and temporal history tracking without losing data efficiency.

    AI & LLMs· Research

  • Link:Dyna Robotics

    Dyna Robotics Unveils DYNA-2: World-Action Model Driven by Human Video Data

    Dyna Robotics has launched DYNA-2, a World-Action Model (WAM) pre-trained on more than one million hours of egocentric human video. The model circumvents traditional teleoperation bottlenecks, transferring physical reasoning and action policies directly to diverse robotic hardware.

    AI & LLMs· Announcement

  • Link:Zhe Liu, Jinghua Hou, Yuxiang Lu, et al.

    StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models

    Researchers from The University of Hong Kong and ACE Robotics introduce StreamPI, a framework that equips single-frame Vision-Language-Action (VLA) models such as π0.5 with temporal reasoning without adding new parameters. By bundling images and instructions into atomic temporal units and using random-interval training, StreamPI improves precision, memory, and robustness in real-robot manipulation.

    AI & LLMs· Research

  • Repository:LMD0311/Awesome-World-Model

    Awesome World Models: Curated Research Collection for Autonomous Driving and Robotics

    The GitHub repository LMD0311/Awesome-World-Model tracks, records, and benchmarks recent research methods on world models for autonomous driving and robotics. It acts as an ongoing supplement to the survey paper 'The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey'.

    2.2kstars

    AI & LLMs· Resource

Let's talk about your project

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.