NVIDIA details Cosmos Framework and NIM for world model deployment

NVIDIA said it is expanding Cosmos with software aimed at two practical problems in physical AI: adapting world models to a developer’s sensors, tasks and data, and deploying those models reliably in applications.

Framework for training, simulation and optimization

In a developer blog published Nov. 2, 2026, NVIDIA described Cosmos Framework as an open, modular environment for world model training, model optimization and inference. The company said the framework provides agent skills and recipes for multimodal data processing, training, reinforcement learning, and deployment workflows. It also said Cosmos Framework is expanding beyond Cosmos 3 to support NVIDIA Alpamayo, GR00T, Qwen, pi0.5 and other compatible Hugging Face models.

NVIDIA said the framework supports training on text, images, video, audio and actions, with LoRA for parameter-efficient adaptation or full supervised fine-tuning to update all model parameters. It added that supported data inputs include JSONL, WebDataset, LeRobot and custom adapters, and that training can run across GPUs with checkpoints saved for resuming runs.

The company also said the platform now covers online and offline reinforcement learning paths, including DPO, GRPO, PPO, OPD and DiffusionNFT. According to the post, users can run rollout, reward and training roles on shared GPUs or separate pools, and connect supported simulators to collect experience.

Simulation workflows for vehicles, robots and surgery

NVIDIA highlighted several Cosmos simulation recipes built around Cosmos 3 Nano checkpoints. For autonomous vehicles, it said Cosmos3-Nano-Transfer-Auto uses bidirectional generation to denoise an entire multi-view clip at once for offline synthetic data generation. The post said it supports 30 fps input and output video, with up to 400 frames at 480p or 720p, and that longer sequences can be generated through autoregressive chunking. It also said a reference run produces seven 720p videos, or 297 frames at 30 FPS, plus 99 LiDAR sweeps.

For closed-loop driving simulation, NVIDIA said Cosmos3-Nano-Sim-Auto is a causal, few-step-distilled variant of Transfer-Auto and generates observations incrementally as an AV policy or human driving input updates the scenario through a world scenario map renderer. The company said the reference rollout writes to an outputs/sim_auto/daytime_scene/ directory and produces synchronized camera videos and LiDAR observations autoregressively.

On robotics, NVIDIA said Cosmos3-Nano-Sim-Transfer converts depth video and a prompt into RGB video while preserving geometry and motion, and that Cosmos3-Nano-Sim-Bimanual generates the next visual observation from scene and robot actions for two-arm simulation. For surgical simulation, the post said Cosmos3-H-Embodied-Nano can generate surgical video from robot actions, predict actions from video, or generate both together, while Cosmos3-H-Dreams is a distilled causal model for real-time, closed-loop simulation.

NIM microservices for accelerated inference

NVIDIA said Cosmos NIM packages supported models in optimized containers for accelerated inference and easier production deployment. The company said the package can use quantization, speculative decoding, caching, token pruning and KV-cache-aware routing, and noted that speculative decoding delivers measured mean speedups of roughly 2 to 3 times on H200.

The post also said Cosmos NIM supports custom fine-tuned checkpoints and live streams through streaming VLM inference, which it said can enable sub-second alerts with 10 times lower latency. It added that deployment includes automatic configuration selection, validated performance and quality across supported GPUs, licensing support, and quarterly feature, optimization and security updates.

NVIDIA included a GPU memory table showing Cosmos3-Edge requiring 16 GiB in BF16 and 14.5 GiB in FP8 for reasoning, Cosmos3-Nano requiring 23.1 GiB in BF16 and FP8 for reasoning, and Cosmos3-Super requiring 135 GiB in BF16 and 67 GiB in FP8. For generation, the table lists Cosmos3-Nano at 58 GiB in BF16, 44 GiB in FP8 and 31 GiB with FP8 plus offloading.

The company said it will host a livestream on Thursday, Nov. 5, on how to use Cosmos Framework. It also said developers can deploy Cosmos models for reasoning and generation using certified Cosmos NIM on more than 20 NVIDIA GPUs.