---
title: NVIDIA CMD Image to Video
emoji: 🎬
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
python_version: "3.12"
startup_duration_timeout: 1h
short_description: Animate an image with NVIDIA CMD autoregressive I2V
license: other
license_name: nvidia-oneway-noncommercial
license_link: LICENSE
tags:
  - image-to-video
  - video-generation
  - cosmos
  - autoregressive
  - nvidia
models:
  - nvidia/cmd
  - nvidia/Cosmos-Reason1-7B
  - Wan-AI/Wan2.1-T2V-1.3B
---

# NVIDIA CMD — image to video

Interactive demo of [**nvidia/cmd**](https://huggingface.co/nvidia/cmd) (*Context-Matched
Distillation*), an autoregressive image-to-video model distilled from
[Cosmos-Predict2.5-2B](https://huggingface.co/nvidia/Cosmos-Predict2.5-2B).

Given a first frame and a text prompt, the model generates **832×480 @ 16 fps** video one
latent frame at a time, using a 4-step causal student with a rolling KV cache
(`local_attn_size = 21`).

## What this Space runs

This demo follows the reference implementation at
[nv-tlabs/cmd](https://github.com/nv-tlabs/cmd) (`inference.py` plus
`examples/run_examples.sh chunk1-short`) and serves the released **`chunk1-short`**
variant:

| | |
|---|---|
| Checkpoint | `chunk1_short_t24_l21.safetensors` |
| Config | `configs/cosmos/t24_l21_student_context_distillation.yaml` |
| Latent frames (`t`) | 24 → 93 video frames (≈ 5.8 s at 16 fps) |
| Chunk size | 1 latent frame per autoregressive block |
| Local attention | 21 frames, `sink_size = 0` |
| Denoising steps | 4 (`[1000, 750, 500, 250]`, warped), `context_noise = 128` |
| Text encoder | `nvidia/Cosmos-Reason1-7B`, 28 concatenated mean-normalised hidden states |
| Tokenizer / VAE | Wan2.1 16-channel VAE (`Wan-AI/Wan2.1-T2V-1.3B`) |

The `cosmos/`, `pipeline/`, `utils/`, `wan/` and `configs/` directories are vendored from
the upstream repository (see `LICENSE`, `NOTICE` and `LICENSES/`). Three deliberate
deviations from the reference script:

1. **Input framing** — the reference resizes any image straight to 832×480, which
   distorts non-16:9 inputs. Because visitors upload arbitrary photos, this Space
   centre-crops to 16:9 first and then resizes. For a 16:9 input the two paths are
   identical.
2. **No base checkpoint** — the reference first materialises the DiT from the *gated*
   `nvidia/Cosmos-Predict2.5-2B` base checkpoint and then overwrites every weight with
   the CMD student export. The student export turns out to be complete: it supplies all
   572 of the network's 576 state-dict entries, the four absent ones being scalar
   training counters (`accum_iteration` and friends) that inference never reads. This
   Space therefore builds the network directly from `nvidia/cmd`, which removes a gated
   dependency and ~4 GB of startup download. The loader still fails loudly on any
   unexpected key, any shape mismatch, any missing tensor outside that counter
   allow-list, and any parameter left un-materialised.
3. **Ungated VAE** — the VAE weights Cosmos-Predict2.5 ships as `tokenizer.pth` are the
   Wan2.1 16-channel VAE, so they are loaded from the Apache-2.0
   [`Wan-AI/Wan2.1-T2V-1.3B`](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B) release
   instead. Verified to populate the vendored `_video_vae` with an exact state-dict
   match (no missing, unexpected or un-materialised tensors), and CMD's latent
   normalisation constants are byte-identical to Wan2.1's.

Checkpoint tensors are streamed one at a time with `safetensors.safe_open` and cast
fp32 → bf16 on the way in, so the 8 GB fp32 export is never fully resident in host
memory; the result matches the reference's `load_state_dict` + `.to(torch.bfloat16)`.

The other released variants are not exposed here: `chunk1-long` / `chunk4-long`
(`t126` / `t121`) need a KV cache far beyond a ZeroGPU slot, the `chunk4-*` variants take
a `torch.compile`d FlexAttention path that ZeroGPU cannot JIT-compile, and the
camera-control variants require an additional per-shot camera trajectory (`camera.npz`)
that has no natural UI here.

## Example assets

The two example first frames and prompts come from
[nvidia-cosmos/cosmos-predict2.5](https://github.com/nvidia-cosmos/cosmos-predict2.5)
(`assets/base/`), which is released under Apache-2.0 — the official Cosmos-Predict2.5
image-to-video showcase inputs, i.e. the base model this checkpoint is distilled from.

## License

The vendored CMD/Cosmos code and the `nvidia/cmd` weights are covered by the
**NVIDIA OneWay Noncommercial License** (`LICENSE`, `LICENSES/`). **Non-commercial use
only.** Files inherited from the Self-Forcing and Wan2.1 projects remain Apache-2.0; see
the per-file SPDX headers and `NOTICE`.