--- tags: - blackhole - p150 - tt-dit-server - tt-model-cache - tt-model-catalog - tt-model-container --- # alpamayo2-super-p300x2 nvidia/Alpamayo2-Super (34B vision-language-action driving model: Qwen3-VL-32B backbone + flow-matching action expert) running end to end on ONE Tenstorrent Blackhole chip via tt-nn with fused decode kernels; 6 cameras x 4 frames + ego history in, chain-of-causation text and a 6.4 s trajectory out in 3-5 s. Runs on **p150** (mesh `P150`). Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1). ## Quickstart ```bash tt-model pull changh95/Alpamayo2-Super-p300x2 --with-weights tt-model serve changh95/Alpamayo2-Super-p300x2 ``` `pull --with-weights` downloads the Docker image and the [`nvidia/Alpamayo2-Super`](https://huggingface.co/nvidia/Alpamayo2-Super) weights at `00554695e729a6ff0b6281fd2c81b18d06e33dbe` (into your HF cache; they are not in the image). `serve` starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`. ### Run with tt-model-manager or tt-cli ```bash tt-model serve changh95/Alpamayo2-Super-p300x2 # pulls the image + the NVIDIA weights (73 GB) into your HF cache python code/alpamayo_tt/server/client_example.py sample.pt --url http://localhost:20000 # or build the JSON below tt-model stop changh95/Alpamayo2-Super-p300x2 ``` This is a `tt-dit-server` package, which needs tt-model-manager from 2026-09 or later (`uv tool install git+https://github.com/tenstorrent/tt-model-manager`). `tt serve changh95/Alpamayo2-Super-p300x2` (tt-cli) works once tt-cli's managed tt-model is at such a version: point it at a current one with `TT_TOOL_BIN_TT_MODEL=$(which tt-model) tt serve changh95/Alpamayo2-Super-p300x2`, or upgrade the managed copy (`uv pip install --python ~/.local/share/tenstorrent/tools/tt-model/bin/python -U git+https://github.com/tenstorrent/tt-model-manager`). The first start compiles kernels and converts the weights to the device layout (a few minutes); later starts are ready in about a minute. One chip of the box is used (any free one); the other chips stay free. ### API - `POST /predict` - `images`: `images[camera][frame]` base64 PNG/JPEG, cameras in `camera_ids` order, 4 frames per camera, oldest to newest (the last frame is t0). Default `camera_ids` `[0,1,2,3,5,6]` = cross-left, front-wide, cross-right, rear-left, rear-right, front-tele (the release "trajectory" profile of the PhysicalAI-AV ring). - `ego_history_xyz` `[16,3]` and `ego_history_rot` `[16,3,3]`: ego poses at t0-1.5 s ... t0 (0.1 s steps). - `reasoning` (default true): generate the chain-of-causation before the trajectory; `false` prompts for the trajectory only (about 1 s faster, same trajectory quality on the validation clips). - sampling: `greedy`, `top_p` (0.98), `temperature` (0.6), `seed`; `num_traj_samples` (1-16) draws extra flow-matching noise samples sharing the same prompt (+0.33 s each); `diffusion_steps` (10). - optional `ego_future_xyz` `[64,3]` ground truth -> `ade_m` / `minADE_m` in the response. - Response: `trajectory_xyz` `[64,3]` and `trajectory_rot` `[64,3,3]` (ego frame at t0, 0.1 s steps, 6.4 s), `chain_of_causation`, `samples`, `timing_ms`, `seed`. - `GET /health`, `GET /info`. ```bash curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json | python -m json.tool | head ``` ### What runs on the chip | stage | implementation | time | |---|---|---:| | vision tower, 24 images (17280 patches) | Qwen3-VL ViT, per-image attention, deepstack, bf16 | 0.32 s | | prefill of the ~4.6K-token prompt | 64 Qwen3 layers, bfp8 attention / bfp4 MLP weights, whole-sequence SDPA | 2.1 s | | chain-of-causation decode | traced step; per layer two fused programs (RMSNorm+QKV+q/k-norm+rope+KV write, o_proj+residual+RMSNorm+SwiGLU+residual) around `sdpa_decode` | 56 ms/token | | action expert, 10 Euler steps | 64-layer 1536-wide transformer attending to the whole VLM KV cache, traced | 0.33 s | | unicycle integration (host) | reference `action_to_traj` | 1 ms | End to end: 2.9 s (trajectory only) to 3.5-5 s (10-30 token chain of causation) per request; ~25 GB of the chip's 32 GB DRAM. Weights: bfp8 attention, bfp4 MLP, bf16 vision, bfp8 expert, bf16 KV cache. ### Accuracy vs the CPU bf16 reference (PhysicalAI-AV validation clips, greedy, seed 42) | clip | teacher-forced ADE vs reference | this package, ADE vs ground truth | reference, ADE vs ground truth | |---|---:|---:|---:| | 030c760c @5.1s | 0.04 m | 0.195 m | 0.266 m | | 030c760c, trajectory only | 0.07 m | 0.183 m | 0.239 m | | 0347d9f9 @5.1s | 0.30 m | 2.08 m | 2.36 m | | 06b483cf @5.1s | 0.19 m | 3.17 m | 3.21 m | | 09ad74a2 @5.1s | 0.09 m | 0.377 m | 0.411 m | | 0a1ef808 @5.1s | 0.02 m | 1.83 m | 1.89 m | Teacher-forced = same chain-of-causation tokens and the same flow-matching noise as the reference, so it isolates the device's numerical error (bfp4 MLP weights dominate). Component PCCs: vision 0.996, text layers 0.9999, expert layers 0.9999, fused vs plain decode logits 0.99998. The reference's first reasoning token is a near-tie that bfp4 noise can flip, so the text can differ from the GPU/CPU run while staying sensible; the flow-matching sampler is multi-modal, so single-sample ADE varies with `seed` (use `num_traj_samples`). ### Caveats - This is the 34B "Super" research/auto-labeling model, not a real-time in-vehicle policy: several seconds per request on one chip (as on an H100). - Fixed input profile: 6 cameras x 4 frames (1920x1080 or any size; the processor resizes to ~180 tokens per image), 16 ego-history poses; batch 1; one request at a time. - Not bit-exact run to run (vision tower kernels); trajectories vary at the centimetre level. ### Licensing - Weights: [nvidia/Alpamayo2-Super](https://huggingface.co/nvidia/Alpamayo2-Super) under OpenMDW-1.1, pulled from NVIDIA's repo at serve time, not redistributed here. - `code/alpamayo2_super`: [NVlabs/alpamayo2](https://github.com/NVlabs/alpamayo2) (Apache-2.0), used for the chat template, trajectory tokenizers and the unicycle action space. - `code/alpamayo_tt` (the Tenstorrent port, fused kernels, server): Apache-2.0. ### Links [nvidia/Alpamayo2-Super](https://huggingface.co/nvidia/Alpamayo2-Super) · [NVlabs/alpamayo2](https://github.com/NVlabs/alpamayo2) · [PhysicalAI-AV dataset](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles) · design notes and measurements: `code/alpamayo_tt/README.md`, `code/alpamayo_tt/PLAN.md`, `code/alpamayo_tt/*_NOTES.md` ## Provenance The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image: | component | built from | | --- | --- | | tt-metal | [`54a826b1d931cf5c70f1d933194cfc32ea96b2b6`](https://github.com/tenstorrent/tt-metal/commit/54a826b1d931cf5c70f1d933194cfc32ea96b2b6) | | `code/` digest | `37c7c852d3b4fd85` (sha256, first 16 hex digits) | | built | 2026-09-20T05:28:51+00:00 by tt-model 0.1.0 |