| --- |
| tags: |
| - blackhole |
| - p150 |
| - tt-dit-server |
| - tt-model-cache |
| - tt-model-catalog |
| - tt-model-container |
| --- |
| |
| # alpamayo2-super-p300x2 |
|
|
| nvidia/Alpamayo2-Super (34B vision-language-action driving model: Qwen3-VL-32B backbone + flow-matching action expert) running end to end on ONE Tenstorrent Blackhole chip via tt-nn with fused decode kernels; 6 cameras x 4 frames + ego history in, chain-of-causation text and a 6.4 s trajectory out in 3-5 s. |
|
|
| Runs on **p150** (mesh `P150`). |
|
|
| Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1). |
|
|
| ## Quickstart |
|
|
| ```bash |
| tt-model pull changh95/Alpamayo2-Super-p300x2 --with-weights |
| tt-model serve changh95/Alpamayo2-Super-p300x2 |
| ``` |
|
|
| `pull --with-weights` downloads the Docker image and the [`nvidia/Alpamayo2-Super`](https://huggingface.co/nvidia/Alpamayo2-Super) weights at `00554695e729a6ff0b6281fd2c81b18d06e33dbe` (into your HF cache; they are not in the image). `serve` starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`. |
|
|
| ### Run with tt-model-manager or tt-cli |
|
|
| ```bash |
| tt-model serve changh95/Alpamayo2-Super-p300x2 # pulls the image + the NVIDIA weights (73 GB) into your HF cache |
| python code/alpamayo_tt/server/client_example.py sample.pt --url http://localhost:20000 # or build the JSON below |
| tt-model stop changh95/Alpamayo2-Super-p300x2 |
| ``` |
|
|
| This is a `tt-dit-server` package, which needs tt-model-manager from 2026-09 or later |
| (`uv tool install git+https://github.com/tenstorrent/tt-model-manager`). `tt serve changh95/Alpamayo2-Super-p300x2` |
| (tt-cli) works once tt-cli's managed tt-model is at such a version: point it at a current one with |
| `TT_TOOL_BIN_TT_MODEL=$(which tt-model) tt serve changh95/Alpamayo2-Super-p300x2`, or upgrade the managed copy |
| (`uv pip install --python ~/.local/share/tenstorrent/tools/tt-model/bin/python -U git+https://github.com/tenstorrent/tt-model-manager`). |
|
|
| The first start compiles kernels and converts the weights to the device layout (a few minutes); later starts |
| are ready in about a minute. One chip of the box is used (any free one); the other chips stay free. |
|
|
| ### API |
|
|
| - `POST /predict` |
| - `images`: `images[camera][frame]` base64 PNG/JPEG, cameras in `camera_ids` order, 4 frames per camera, |
| oldest to newest (the last frame is t0). Default `camera_ids` `[0,1,2,3,5,6]` = cross-left, front-wide, |
| cross-right, rear-left, rear-right, front-tele (the release "trajectory" profile of the PhysicalAI-AV ring). |
| - `ego_history_xyz` `[16,3]` and `ego_history_rot` `[16,3,3]`: ego poses at t0-1.5 s ... t0 (0.1 s steps). |
| - `reasoning` (default true): generate the chain-of-causation before the trajectory; `false` prompts for the |
| trajectory only (about 1 s faster, same trajectory quality on the validation clips). |
| - sampling: `greedy`, `top_p` (0.98), `temperature` (0.6), `seed`; `num_traj_samples` (1-16) draws extra |
| flow-matching noise samples sharing the same prompt (+0.33 s each); `diffusion_steps` (10). |
| - optional `ego_future_xyz` `[64,3]` ground truth -> `ade_m` / `minADE_m` in the response. |
| - Response: `trajectory_xyz` `[64,3]` and `trajectory_rot` `[64,3,3]` (ego frame at t0, 0.1 s steps, 6.4 s), |
| `chain_of_causation`, `samples`, `timing_ms`, `seed`. |
| - `GET /health`, `GET /info`. |
| |
| ```bash |
| curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json | python -m json.tool | head |
| ``` |
|
|
| ### What runs on the chip |
|
|
| | stage | implementation | time | |
| |---|---|---:| |
| | vision tower, 24 images (17280 patches) | Qwen3-VL ViT, per-image attention, deepstack, bf16 | 0.32 s | |
| | prefill of the ~4.6K-token prompt | 64 Qwen3 layers, bfp8 attention / bfp4 MLP weights, whole-sequence SDPA | 2.1 s | |
| | chain-of-causation decode | traced step; per layer two fused programs (RMSNorm+QKV+q/k-norm+rope+KV write, o_proj+residual+RMSNorm+SwiGLU+residual) around `sdpa_decode` | 56 ms/token | |
| | action expert, 10 Euler steps | 64-layer 1536-wide transformer attending to the whole VLM KV cache, traced | 0.33 s | |
| | unicycle integration (host) | reference `action_to_traj` | 1 ms | |
|
|
| End to end: 2.9 s (trajectory only) to 3.5-5 s (10-30 token chain of causation) per request; ~25 GB of the |
| chip's 32 GB DRAM. Weights: bfp8 attention, bfp4 MLP, bf16 vision, bfp8 expert, bf16 KV cache. |
|
|
| ### Accuracy vs the CPU bf16 reference (PhysicalAI-AV validation clips, greedy, seed 42) |
|
|
| | clip | teacher-forced ADE vs reference | this package, ADE vs ground truth | reference, ADE vs ground truth | |
| |---|---:|---:|---:| |
| | 030c760c @5.1s | 0.04 m | 0.195 m | 0.266 m | |
| | 030c760c, trajectory only | 0.07 m | 0.183 m | 0.239 m | |
| | 0347d9f9 @5.1s | 0.30 m | 2.08 m | 2.36 m | |
| | 06b483cf @5.1s | 0.19 m | 3.17 m | 3.21 m | |
| | 09ad74a2 @5.1s | 0.09 m | 0.377 m | 0.411 m | |
| | 0a1ef808 @5.1s | 0.02 m | 1.83 m | 1.89 m | |
|
|
| Teacher-forced = same chain-of-causation tokens and the same flow-matching noise as the reference, so it |
| isolates the device's numerical error (bfp4 MLP weights dominate). Component PCCs: vision 0.996, text layers |
| 0.9999, expert layers 0.9999, fused vs plain decode logits 0.99998. The reference's first reasoning token is a |
| near-tie that bfp4 noise can flip, so the text can differ from the GPU/CPU run while staying sensible; the |
| flow-matching sampler is multi-modal, so single-sample ADE varies with `seed` (use `num_traj_samples`). |
|
|
| ### Caveats |
|
|
| - This is the 34B "Super" research/auto-labeling model, not a real-time in-vehicle policy: several seconds per |
| request on one chip (as on an H100). |
| - Fixed input profile: 6 cameras x 4 frames (1920x1080 or any size; the processor resizes to ~180 tokens per |
| image), 16 ego-history poses; batch 1; one request at a time. |
| - Not bit-exact run to run (vision tower kernels); trajectories vary at the centimetre level. |
|
|
| ### Licensing |
|
|
| - Weights: [nvidia/Alpamayo2-Super](https://huggingface.co/nvidia/Alpamayo2-Super) under OpenMDW-1.1, pulled |
| from NVIDIA's repo at serve time, not redistributed here. |
| - `code/alpamayo2_super`: [NVlabs/alpamayo2](https://github.com/NVlabs/alpamayo2) (Apache-2.0), used for the |
| chat template, trajectory tokenizers and the unicycle action space. |
| - `code/alpamayo_tt` (the Tenstorrent port, fused kernels, server): Apache-2.0. |
|
|
| ### Links |
|
|
| [nvidia/Alpamayo2-Super](https://huggingface.co/nvidia/Alpamayo2-Super) · [NVlabs/alpamayo2](https://github.com/NVlabs/alpamayo2) · |
| [PhysicalAI-AV dataset](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles) · design notes and |
| measurements: `code/alpamayo_tt/README.md`, `code/alpamayo_tt/PLAN.md`, `code/alpamayo_tt/*_NOTES.md` |
|
|
| ## Provenance |
|
|
| The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image: |
|
|
| | component | built from | |
| | --- | --- | |
| | tt-metal | [`54a826b1d931cf5c70f1d933194cfc32ea96b2b6`](https://github.com/tenstorrent/tt-metal/commit/54a826b1d931cf5c70f1d933194cfc32ea96b2b6) | |
| | `code/` digest | `37c7c852d3b4fd85` (sha256, first 16 hex digits) | |
| | built | 2026-09-20T05:28:51+00:00 by tt-model 0.1.0 | |
|
|
|
|
