SSyncDev

Cosmos3-H-Dreams

SyncDev model directory · Profile updated Nov 19, 2026

Compare with another model

Cosmos3-H-Dreams is NVIDIA's open real-time surgical world model: it streams Versius robot video from one endoscope frame and two-arm commands.

Key specifications

Specification score
14.8
License
NVIDIA Open Model License

The specification score summarizes recorded features and limits. It is not a benchmark result.

Pricing conditions: Open weights; no hosted API price. GPU time is the cost, so confirm throughput on your own hardware.

Inputs and outputs

Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.

Accepts

  • Image input

Produces

  • Video output

Capabilities

VisionOpen weights

What is Cosmos3-H-Dreams?

Testing a surgical robot policy on patients is not an option, and testing it in a physics simulator rarely looks like real tissue. Cosmos3-H-Dreams takes a third route. Give it one endoscopic frame from a CMR Surgical Versius procedure, the recent command history and a stream of future commands for both arms, and it generates the video those commands would produce, fast enough to drive interactively.

NVIDIA released it on 20 October 2026, with weights and a streaming inference library.

What it was trained to imagine

The model is narrow on purpose. It was fine-tuned on Versius clinical cholecystectomy recordings, keeping only windows with exactly two instruments in view, typically a grasper on the left arm and a hook on the right. That came to 64 recordings and about 526 unique minutes of video. Each arm is described by 14 numbers per step: anchor-relative translation and 6-D rotation, jaw, electrosurgical energy, a thumbstick button and two thumbstick axes. Twenty-eight dimensions in all.

Lineage runs Cosmos3-Nano, then Cosmos3-H-Embodied-Nano, then this model. It replaces the earlier Cosmos-H-Dreams built on Cosmos-Predict2.5.

Making it real-time

A bidirectional 72-frame teacher needing 35 sampling steps was distilled into a block-causal student, first with causal teacher forcing and then with Self-Forcing plus distribution-matching distillation. At inference the starting frame and up to 16 past actions (1.6 seconds) go into a KV cache once. Every latent frame after that consumes four future actions and emits four video frames.

GPU Resolution Steps Reported FPS
RTX PRO 6000 Blackwell 832x480 4 30.0
RTX PRO 6000 Blackwell 832x480 2 43.2
RTX PRO 6000 Blackwell Max-Q 832x480 4 20.2
RTX PRO 6000 Blackwell Max-Q 832x480 2 31.2
DGX Spark 512x288 2 16.9

Default rollouts are 72 frames at 10 Hz, 7.2 seconds, and can be continued or re-anchored for longer sequences. It also runs on H100.

A parameter-count wrinkle

The model lists 15.17 billion parameters. Its base, Embodied-Nano, quotes 16 billion, the official Cosmos3-Nano size. Weights here are exported in FP32 while native inference is BF16, and other precisions are not qualified, so measure memory on your card before planning a fleet.

Using it responsibly

  • Treat output as simulation for policy evaluation and synthetic data, not as clinical evidence.
  • Validate on held-out cases from your own instrument set; anything other than two visible instruments is out of distribution.
  • Check action control separately from visual quality: does the jaw actually close when commanded?

The library supports replay of recorded trajectories to MP4 and live control from a headset browser. For other healthcare models from NVIDIA, see NV-Reason-CT; for general video generators, the image-to-video directory.

Licence: NVIDIA Open Model License, commercial use permitted.

How to evaluate Cosmos3-H-Dreams

Compare the model on the work you need to complete.

Tasks and inputs

Cosmos3-H-Dreams is listed for Image-to-video, Video-to-video. It accepts image and produces video. Use the publisher’s input format and preprocessing for your chosen deployment.

Capacity and cost

Choose duration, resolution and frame rate together. Compare the cost of an accepted clip, including repeat generations.

Choosing a deployment

The weights are downloadable under NVIDIA Open Model License. Check runtime support and size the complete model and working context for your expected concurrency.

Frequently asked questions

The CMR Surgical Versius system, specifically cholecystectomy procedures with exactly two instruments visible.

These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.

tbc

TBC mischief 1

TBC mischief 1 is an image-to-video model built on Wan 2.2, sold on fal and OpenRouter at about half a cent per second for 3-5 s clips at up to 720p.

— context

NVIDIA

Cosmos3-H-Embodied-Nano

Cosmos3-H-Embodied-Nano is NVIDIA's open 16B surgical world model that does forward dynamics, inverse dynamics and policy from one checkpoint.

— context

NVIDIA

Cosmos3-Nano-Sim-Bimanual

Cosmos3-Nano-Sim-Bimanual is NVIDIA's open 16B world model that simulates video of a two-armed AgiBot robot as it moves its camera and manipulates.

— context

noiz-ai

Helix World

Helix World (preview) is an open real-time world model from Noiz AI and HKUST that streams explorable 768x512 video with stereo sound that follows the camera.

— context

Alibaba

Wan-Animate-2

Wan-Animate-2 is Alibaba Wan's open 14B character-animation model that drives a character from video directly, with text-controlled camera angle.

— context

Z.ai / Zhipu AI

SCAIL-2

SCAIL-2 is Zhipu's open character-animation model that drives a reference character from any video, with no pose skeletons or masks. MIT licence.

— context