tt-model.yaml

# SPDX-License-Identifier: Apache-2.0
# tt-model-manager container manifest (schema 5.1) for NVIDIA Isaac GR00T N1.7-3B (DROID layout) on Blackhole
# (Stage-2 release: the whole DiT denoise runs as ONE persistent megakernel generic_op on a bfp8_b DRAM weight arena;
# vision / LLM / adapter are TTNN ops in Metal traces; tuned Qwen3-VL vision tower). The Stage-1 TTNN denoise stays
# selectable with GR00T_DIT_BACKEND=ttnn.
#
# Run EVERY tt-model command from this directory: `source.tt_metal` and `extra_code[].root: code` resolve against the
# process CWD, not against this file. `--out` must lie outside any git checkout (`package` deletes <out>/<name> first).
# On the build host rootless Docker needs `source /home/deepgadget/experiments/tt-models/bin/docker-env.sh` first.
#
#   tt-model package --container tt-model.yaml --out /home/deepgadget/experiments/gr00t/publish/build   # minutes with the ccache mount, ~2 h cold
#   tt-model serve   /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.7-3b-p150/tt_kernel_manifest.json
#   python code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:20000      # PASS/FAIL one-liner (prints /info -> stage)
#   tt-model stop    /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.7-3b-p150/tt_kernel_manifest.json   # org/name works only after pull/push
#   tt-model push    /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.7-3b-p150 --publish
#
# The one Blackhole card of the build host is shared: `serve -> smoke -> bench -> stop` runs inside ONE bin/with-device.sh hold.
schema: "5.1"

repo: changh95/GR00T-N1.7-3B-p150
name: gr00t-n1.7-3b-p150

# A POINTER, pinned. `serve` pre-downloads exactly these files at this sha into the host HF cache (mounted at /hf);
# the server resolves the same snapshot with snapshot_download(revision=TT_WEIGHTS_REVISION) and asserts that the
# port's own common.configs.snapshot_dir("n17") points at it. The weights never enter the image.
weights:
  repo: nvidia/GR00T-N1.7-3B
  revision: 2fc962b973bccdd5d8ce4f67cc63b264d6886495
  allow_patterns:
    - "model-*.safetensors"                 # 2 bf16 shards: 6,910,361,856 B of bf16 tensors (1,031 tensors)
    - model.safetensors.index.json          # LazyCheckpoint needs it (common/checkpoint.py)
    - config.json
    - processor_config.json                 # modality configs + processor flags (common/normalization.py)
    - statistics.json                       # per-embodiment q01/q99 statistics incl. relative_action (common/normalization.py)
    - embodiment_id.json                    # tag -> slot cross-check (common/normalization.py)
    - LICENSE                               # NVIDIA License: keep it next to the weights it covers

kind: tt-dit-server
arch: blackhole

source:
  # Clean shared clone of tt-metal main 668c2907575 (v0.79.0-dev20260914-1), the commit the port was validated on;
  # git status is empty there, so the card records a clean tree. The image builds tt-metal from this source.
  tt_metal: /home/deepgadget/experiments/gr00t/publish/tt-metal-668c2907575
  # tt-metal-relative files the port imports from the tree OTHER than itself, with their import closure
  # (grep of every `from models.` / `import models.` outside models.experimental.gr00t in the snapshot).
  code:
    - models/common/utility_functions.py                        # models.tt_dit.utils.matmul imports is_blackhole
    - models/tt_dit/utils/matmul.py                             # tt/layers.py: MinimalMatmul config rows (lazy)
    - models/tt_dit/utils/agmm_rules.py                         # lazy from matmul.py
    - models/tt_dit/utils/mmrs_rules.py                         # lazy from matmul.py
    - models/tt_dit/utils/tensor.py                             # layers/linear.py, layers/module.py, lora.py, parallel/manager.py
    - models/tt_dit/utils/progress.py                           # layers/module.py
    - models/tt_dit/utils/walltime.py                           # utils/progress.py
    - models/tt_dit/utils/substate.py                           # layers/module.py
    - models/tt_dit/layers/linear.py                            # encoders/qwen3vl/vision_qwen3vl.py (the verbatim Qwen3-VL tower = the oracle, GR00T_VIT_FORK=0)
    - models/tt_dit/layers/module.py
    - models/tt_dit/layers/normalization.py
    - models/tt_dit/layers/lora.py                              # imported at the bottom of layers/linear.py
    - models/tt_dit/parallel/config.py
    - models/tt_dit/parallel/manager.py
    - models/tt_dit/encoders/qwen3vl/vision_qwen3vl.py          # tt/qwen3vl_vision.py: verbatim tower (lazy; the served default is the tuned fork in the port itself)
    - models/demos/deepseek_v3_b1/unified_kernel_descriptor.py  # tt/megakernel/descriptors.py: the megakernel's UnifiedKernelDescriptor (SERVED: builds the denoise generic_op)
    # The megakernel's device kernel (tt/megakernel/kernels/dit_kernel.cpp) #includes these four tree headers by relative path
    # (../../../../../demos/deepseek_v3_b1/unified_kernels/...); the JIT compiles them at boot, so they must be in the image
    # (round 0 of the megakernel package failed its first boot on exactly this: docs/publish/build-mk-n17.md).
    - models/demos/deepseek_v3_b1/unified_kernels/kernel_op_api.hpp
    - models/demos/deepseek_v3_b1/unified_kernels/kernel_utils.hpp
    - models/demos/deepseek_v3_b1/unified_kernels/dataflow_utils.hpp
    - models/demos/deepseek_v3_b1/unified_kernels/mcast.hpp
  # This repo's own code (lands at /opt/tt-metal/<path>; PYTHONPATH=/opt/tt-metal).
  extra_code:
    - root: code
      paths:
        - models        # code/models/experimental/gr00t: the frozen snapshot of the port (tt-metal ref gr00t-p150-snapshot-mk = 009de892c895, parent 668c2907575; K4b kernel tree 7df61c35e9dc7857)
        - gr00t_p150    # FastAPI server, schemas, smoke test, DROID demo observation, vendored Qwen2 tokenizer + Qwen2-VL processor config (n17)
        - scripts       # download_weights.sh, bench_http.py
  ubuntu: "22.04"
  python: "3.12"

runtime:
  app: gr00t_p150.server.app:app
  mesh_shape_env: TT_MESH_SHAPE
  # On top of the kind defaults (fastapi, uvicorn, pydantic>=2, pillow), torch==2.11.0+cpu (auto-pinned from the tree's
  # tt_metal/python_env/requirements-dev.txt) and ttnn's own deps. Every third-party module on the serve path, pinned
  # where the tree pins it. Resolved on Python 3.12 with uv (dry-run 2026-09-17): numpy 1.26.4, transformers 5.12.1,
  # opencv-python-headless 4.8.1.78, torchvision 0.26.0+cpu, safetensors 0.8.0, huggingface_hub 1.32.0, loguru 0.7.3.
  packages:
    - "numpy>=1.24.4,<2"
    - safetensors                          # common/checkpoint.py (module level)
    - huggingface_hub                      # server: snapshot_download(revision=..., local_files_only=True) first
    - transformers==5.12.1                 # common/prompts.get_tokenizer (Qwen2 tokenizer) + common/preprocessing.get_qwen2vl_image_processor (Qwen2VLImageProcessor); tree pin
    - opencv-python-headless==4.8.1.78     # common/preprocessing.py imports cv2 at module level (the DROID eval chain); tree pin (cp312 wheel exists)
    - torchvision==0.26.0                  # common/preprocessing.qwen2vl_pixel_patches (bicubic antialias resize + normalize, torchvision.transforms.v2); tree pin for torch 2.11.0
    - loguru                               # models/common/utility_functions.py (via models.tt_dit.utils.matmul)

serve:
  port: 20000
  hardware: p150
  mesh_device: P150
  env:
    TT_WEIGHTS_REVISION: "2fc962b973bccdd5d8ce4f67cc63b264d6886495"   # == weights.revision == configs.HF_SNAPSHOT_SHAS["n17"]
    TT_METAL_VISIBLE_DEVICES: "0"
    TT_DEVICE_ID: "0"
    GR00T_VERSION: "n17"                   # the server code is shared with the N1.5 / N1.6 packages and never guesses
    GR00T_EMBODIMENT: "oxe_droid_relative_eef_relative_joint"   # DROID (Franka arm, relative EEF + relative joints), embodiment slot 24; the only device-validated N1.7 embodiment
    GR00T_LAYOUT: "droid"                  # static layout: 2 cameras x 2 frames -> 4 images x 112 tokens (16x28 patches each), L_max 51 text tokens, S_pad 512
    GR00T_POLICY: "mixed_dit"              # TTPolicy.dtype_policy: DiT matmul weights bfp8_b, everything else bf16 (the validated default)
    GR00T_TRACE_LAYOUT: "per_stage"        # four Metal traces: vision / llm / adapter / denoise (the denoise trace holds the megakernel generic_op)
    GR00T_DIT_BACKEND: "megakernel"        # TTPolicy.dit_backend (the port default since 2026-09-18): the whole DiT denoise as ONE persistent generic_op; "ttnn" = the Stage-1 TTNN op sequence (A/B, tests/tt/results/mk_default_summary.md)
    GR00T_MK_ARENA_DTYPE: "auto"           # the megakernel's DRAM weight arena: auto -> the port's per-version table = bfp8_b for n17 (the validated configuration; bf16 selectable); /info -> stage.mk_arena_dtype reports the resolved value
    GR00T_NUM_CQS: "2"                     # command queues of the device (tt.model.open_model_device): per-call input writes on CQ 1 overlap the traces; bit-equal to 1 CQ (tests/tt/test_mk_2cq.py)
    GR00T_VIT_FORK: "1"                    # tuned Qwen3-VL tower (HiFi2 + fp32 acc, HiFi3 on blocks 0-5; the port default since 2026-09-17); "0" = the verbatim tt_dit tower (oracle, 5x slower)
    GR00T_TOKENIZER_DIR: "/opt/tt-metal/gr00t_p150/assets/tokenizer/n17"   # vendored Qwen2 tokenizer + Qwen2-VL preprocessor_config.json (not in the weights repo)
    GR00T_PROJECT_ROOT: "/nonexistent"     # kill the dev-box default the port's prompts.py would otherwise fall back to
    GR00T_GOLDEN_ROOT: "/nonexistent"      # same for common/golden.py (tests only; defensive)
    GR00T_TT_CACHE: "/weight-cache/gr00t-tt"   # host .pt plan tier + .tensorbin device tier (~2.9 GB under the megakernel: the ttnn DiT tensors are not converted; 8.4 GB on the Stage-1 path) -> ~/.cache/tt-model/gr00t-n1.7-3b-p150/weights
    GR00T_WARMUP_RUNS: "5"                 # traced predict() calls after trace capture, before READY
    GR00T_WARMUP_PCC_MIN: "0.999"          # boot FAILS when the warm-up actions' PCC vs the shipped fp32 golden is below this

# Build-time assertions, run INSIDE the finished image as uid 1000, no device, no weights, no HF_MODEL.
verify:
  - "import gr00t_p150.server.app as a; assert a.app"
  - "import sys, gr00t_p150.server.app; assert 'ttnn' not in sys.modules and not any(m.startswith('models.experimental.gr00t') for m in sys.modules), 'the server module must import without side effects'"
  - "import models.experimental.gr00t.tt.model as m; assert m.Gr00tTT"
  - "from models.experimental.gr00t.common import configs, preprocessing, normalization, prompts, checkpoint, weights; assert configs.HF_SNAPSHOT_SHAS['n17'] == '2fc962b973bccdd5d8ce4f67cc63b264d6886495' and configs.HF_REPOS['n17'] == 'nvidia/GR00T-N1.7-3B'"
  - "from models.experimental.gr00t.common.configs import get_config; c = get_config('n17'); l = c.canonical_layout; assert (l.name, c.canonical_embodiment, l.embodiment_id, l.n_cameras, l.n_images, l.grid_thw, l.tokens_per_image, l.l_max, l.s_pad_llm, l.noise_shape, l.valid_slice) == ('droid', 'oxe_droid_relative_eef_relative_joint', 24, 2, 4, (1, 16, 28), 112, 51, 512, (1, 40, 132), (40, 17)), l"
  - "from models.experimental.gr00t.tt.qwen3vl_vision import build_vision_tower, resolve_vit_fork, TunedVisionConfig; from models.experimental.gr00t.tt.policy import TTPolicy; assert TTPolicy().vit_fork is True and resolve_vit_fork(TTPolicy(), None, 'qwen3vl') is True"
  - "from models.experimental.gr00t.tt.policy import TTPolicy, DEFAULT_DIT_BACKEND, MK_ARENA_DTYPE_DEFAULT; p = TTPolicy(); assert DEFAULT_DIT_BACKEND == 'megakernel' and p.dit_backend == 'megakernel' and p.mk_arena_dtype == 'auto' and p.mk_arena_dtype_for('n17') == 'bfp8_b' == MK_ARENA_DTYPE_DEFAULT['n17'], p.to_dict()"
  - "from models.experimental.gr00t.tt.policy import TTPolicy; assert TTPolicy().with_env_overrides({'GR00T_DIT_BACKEND': 'ttnn'}).dit_backend == 'ttnn' and TTPolicy().with_env_overrides({'GR00T_MK_ARENA_DTYPE': 'bf16'}).mk_arena_dtype == 'bf16'"
  - "import models.experimental.gr00t.tt.megakernel.dit_program as dp; from models.experimental.gr00t.tt.megakernel import arena, core_map, descriptors; assert dp.DiTMegakernel and descriptors.L1_USABLE_BYTES == 1461248 and core_map.N_COMPUTE_DEFAULT == 96"
  - "from models.experimental.gr00t.tt.model import open_model_device, worker_l1_size_for, MK_L1_CUT_BYTES_DEFAULT, ADAPTER_MEM_BY_BACKEND; from models.experimental.gr00t.tt.policy import TTPolicy; assert MK_L1_CUT_BYTES_DEFAULT == 65536 and worker_l1_size_for(TTPolicy()) == 1395712 and worker_l1_size_for(TTPolicy(dit_backend='ttnn')) is None and ADAPTER_MEM_BY_BACKEND == {'ttnn': 'L1', 'megakernel': 'DRAM'}"
  - "import hashlib; from pathlib import Path; d = Path('/opt/tt-metal/models/experimental/gr00t'); rows = [l.split() for l in (d / 'tests/tt/results/mk_tree_20260918-041439.sha256').read_text().splitlines() if l.strip()]; assert len(rows) == 16 and all(hashlib.sha256((d / p).read_bytes()).hexdigest() == h for h, p in rows), 'tt/megakernel tree != the K4b checkpoint 7df61c35e9dc7857'"
  - "import os, re; from pathlib import Path; k = Path('/opt/tt-metal/models/experimental/gr00t/tt/megakernel/kernels/dit_kernel.cpp'); incs = re.findall(r'#include \"(\\.\\./[^\"]+)\"', k.read_text()); assert len(incs) == 4 and all(Path(os.path.normpath(k.parent / i)).is_file() for i in incs), incs"
  - "import inspect, gr00t_p150.server.app as a; src = inspect.getsource(a); assert a.STAGE_SERVED == {'megakernel': 'stage2-megakernel', 'ttnn': 'stage1-ttnn-traces'} and a.DEFAULT_NUM_CQS == 2 and 'device = open_model_device(' in src and 'GR00T_DIT_BACKEND' in src and 'GR00T_NUM_CQS' in src"
  - "import os; os.environ.update({'GR00T_VERSION': 'n17', 'TT_WEIGHTS_REVISION': '2fc962b973bccdd5d8ce4f67cc63b264d6886495', 'GR00T_DIT_BACKEND': 'megakernel', 'GR00T_MK_ARENA_DTYPE': 'auto', 'GR00T_NUM_CQS': '2', 'GR00T_VIT_FORK': '1'}); from gr00t_p150.server.app import load_config; c = load_config(); assert (c.version, c.dit_backend, c.mk_arena_dtype, c.num_cqs, c.vit_fork) == ('n17', 'megakernel', 'auto', 2, True), c"
  - "import models.tt_dit.utils.matmul as mm; assert mm.get_matmul_config"
  - "from models.tt_dit.encoders.qwen3vl.vision_qwen3vl import Qwen3VlVisionModel; assert Qwen3VlVisionModel"
  - "import models.demos.deepseek_v3_b1.unified_kernel_descriptor as u; assert u.UnifiedKernelDescriptor"
  - "import safetensors, huggingface_hub, cv2, numpy, PIL, loguru; assert int(numpy.__version__.split('.')[0]) < 2, numpy.__version__; assert cv2.__version__ == '4.8.1', cv2.__version__"
  - "import transformers; assert transformers.__version__ == '5.12.1', transformers.__version__"
  - "import torchvision; assert torchvision.__version__.startswith('0.26.0'), torchvision.__version__"
  - "from pathlib import Path; d = Path('/opt/tt-metal/gr00t_p150/assets/tokenizer/n17'); assert all((d / f).is_file() for f in ('vocab.json', 'merges.txt', 'tokenizer_config.json', 'preprocessor_config.json', 'PROVENANCE.md')), sorted(p.name for p in d.iterdir())"
  - "import os; os.environ['GR00T_TOKENIZER_DIR_N17'] = '/opt/tt-metal/gr00t_p150/assets/tokenizer/n17'; from models.experimental.gr00t.common.prompts import get_tokenizer; t = get_tokenizer('n17'); assert t.convert_tokens_to_ids('<|image_pad|>') == 151655 and t.convert_tokens_to_ids('<|vision_start|>') == 151652 and t.convert_tokens_to_ids('<|vision_end|>') == 151653"
  - "import os; os.environ['GR00T_TOKENIZER_DIR_N17'] = '/opt/tt-metal/gr00t_p150/assets/tokenizer/n17'; from models.experimental.gr00t.common.preprocessing import get_qwen2vl_image_processor, qwen2vl_smart_resize; ip = get_qwen2vl_image_processor(); assert ip.patch_size == 16 and ip.merge_size == 2 and ip.temporal_patch_size == 2; assert qwen2vl_smart_resize(256, 455, 32, 65536, 16777216) == (256, 448)"
  - "from pathlib import Path; d = Path('/opt/tt-metal/gr00t_p150/demo/n17'); assert all((d / f).is_file() for f in ('observation.json', 'noise.json', 'expected.json', 'frames/exterior_image_1_left_tm15.png', 'frames/exterior_image_1_left_t0.png', 'frames/wrist_image_left_tm15.png', 'frames/wrist_image_left_t0.png')), sorted(str(p) for p in d.rglob('*'))"
  - "from gr00t_p150.server.app import load_demo_request; import numpy as np; r = load_demo_request('n17'); assert set(r['images']) == {'exterior_image_1_left', 'wrist_image_left'} and all(len(v) == 2 for v in r['images'].values()) and np.asarray(r['noise']).squeeze().shape == (40, 132)"
  - "from pathlib import Path; assert Path('/opt/tt-metal/models/experimental/gr00t/common/golden_tap_map.json').is_file() and Path('/opt/tt-metal/models/experimental/gr00t/tests/tt/gates/gates_multi.json').is_file()"
  - "from pathlib import Path; assert Path('/opt/tt-metal/gr00t_p150/server/smoke_test.py').is_file() and Path('/opt/tt-metal/scripts/bench_http.py').is_file() and Path('/opt/tt-metal/scripts/download_weights.sh').is_file()"
  - "import ttnn; assert all(hasattr(ttnn, a) for a in ('begin_trace_capture', 'end_trace_capture', 'execute_trace', 'release_trace', 'copy_host_to_device_tensor', 'MinimalMatmulConfig', 'embedding', 'generic_op', 'allocate_tensor_on_device'))"
  - "from gr00t_p150.server.app import parse_mesh_shape as p; assert p('1x1') == p('(1, 1)') == p('1,1') == (1, 1)"

card:
  description: |
    NVIDIA Isaac GR00T N1.7 (3B vision-language-action policy) running on one Tenstorrent Blackhole p150a via tt-nn, in the DROID layout: two cameras × two frames (t−15 and t) at 16:9, a 17-dim Franka state and a task instruction in, a 40-step chunk of end-effector, gripper and joint actions out. Stage-2 release: the whole DiT denoise runs as one persistent megakernel streaming bfp8 weights from a DRAM arena; vision (tuned Qwen3-VL tower), LLM and adapter are TTNN ops in Metal traces; `GR00T_DIT_BACKEND=ttnn` selects the Stage-1 denoise.
    Weights: [nvidia/GR00T-N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B) · Paper: [arXiv:2503.14734](https://arxiv.org/abs/2503.14734) · Upstream code: [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) (`main`, the `n1.7-release` line) · Port: `code/models/experimental/gr00t` and the server `code/gr00t_p150` in this repo
  quickstart: |
    ### Run with tt-cli

    ```bash
    tt serve changh95/GR00T-N1.7-3B-p150
    python3 - <<'EOF'
    import base64, json
    b64 = lambda f: base64.b64encode(open(f"media/{f}.png", "rb").read()).decode()
    obs = json.load(open("media/demo_observation.json"))   # DROID demo step: raw state + instruction
    obs["images"] = {cam: [b64(f"{cam}_tm15"), b64(f"{cam}_t0")]          # per camera: the frame at t-15, then the frame at t
                     for cam in ("exterior_image_1_left", "wrist_image_left")}
    json.dump(obs, open("req.json", "w"))
    EOF
    curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
    tt model stop changh95/GR00T-N1.7-3B-p150
    ```

    - `POST /predict`: `images` (`{"exterior_image_1_left": [<base64 PNG/JPEG at t−15>, <at t>], "wrist_image_left": [...]}`, exactly two frames per camera, 16:9 frames), `state` (`{group: [floats]}`: `eef_9d` 9 = xyz + rot6d, `gripper_position` 1, `joint_position` 7, raw values), `instruction` (≤ 51 tokens); optional `seed` (default 42) or `noise` (`[40, 132]`), `return_normalized`.
    - `GET /health`, `GET /info` (`stage` says what is served), `GET /demo` (the shipped demo request with its reference noise, and its fp32 reference actions).

    ### Response

    ```json
    {"actions": {"eef_9d":           [[0.5284, 0.0181, 0.4569, -0.0786, 0.3927, -0.9163, 0.9923, 0.1193, -0.034], "... 40 steps"],
                 "gripper_position": [[0.0], [0.002], "..."],
                 "joint_position":   [[-0.0051, 0.034, 0.0013, -1.917, 0.1641, 1.6101, 0.0844], "..."]},
     "action_horizon": 40, "embodiment": "oxe_droid_relative_eef_relative_joint", "seq_len": 469, "noise_source": "client", "seed": null,
     "timing_ms": {"decode": 3.4, "encode": 10.9, "device": 55.9, "decode_actions": 1.1, "total": 71.3}}
    ```

    - The example is the demo request with the reference noise, as `GET /demo` returns it (`noise_source: "client"`, `seed: null`); `timing_ms` from a warm request of the same serve.
    - `actions` are physical, un-normalised targets, 40 steps × [9, 1, 7] dims, as `Gr00tPolicy.get_action` returns them for the DROID data config: `eef_9d` is the **absolute** end-effector pose (the relative SE(3) output composed onto the request's pose), `joint_position` the relative output added to the request's joints, `gripper_position` absolute; `timing_ms.device` covers upload, the four trace replays and the readback.

    ### Demo

    | Input (`media/demo_frames.png`: both cameras at t−15 and t; DROID sample episode 1, step 40, 320×180) | Instruction | p150a actions vs the fp32 reference (`media/demo_actions_vs_golden.png`) |
    |:---:|:---|:---:|
    | ![](media/demo_frames.png) | `Put the blue block in the green bowl` | ![](media/demo_actions_vs_golden.png) |

    ### Accuracy and speed

    | Metric | Value |
    |---|---:|
    | Action chunk vs the fp32 reference (same observation, same initial noise) | PCC `eef_9d` 0.99998 · `joint_position` 0.99996 · `gripper_position` max\|d\| 0.0031 (near-constant group) · all 17 dims × 40 steps 0.99996 · normalised valid slice 0.99995 |
    | Intermediate taps vs the fp32 reference (27 gated taps, from pixels; DiT internals are not observable inside the megakernel) | 30 of 30 rows pass; tightest `vit_deepstack_mergers[0]` 0.999849 vs gate 0.9998. On six more DROID goldens the actions pass on five: `t2_s120` has two pre-existing misses (`llm_layer_first` 0.98998 vs 0.991, `joint_position` PCC 0.99976 / max\|d\| 0.107 vs 0.9998 / 0.083); `t1_s120` adds one megakernel-only marginal tap, `pred_velocity[k=2]` dim 9 per-dim PCC 0.99588 vs 0.996 (its actions pass) |
    | Megakernel vs Stage-1 TTNN denoise (same device, 7 DROID samples) | `action_pred_normalized` PCC ≥ 0.999992 between the two backends; per-step velocities ≥ 0.99996 |
    | Traced vs untraced, determinism, stale state, 2 vs 1 command queue | bit-identical; 5 trace replays bit-identical; A→B→A→B reproduces A and B exactly; 2-CQ == 1-CQ on every probe (26 rows) |
    | Inference on p150a (port benchmark, warm, batch 1, median of 50) | **62.8 ms end-to-end** (p90 68.6) ≈ host encode 5.6 + upload 3.2 + traces 52.4 (vision 17.5 · LLM 12.4 · adapter 6.2 · denoise 16.5 = 4 × 4.12) + readback and decode 1.3. Stage-1 path on the same tree: 79.7 ms (denoise 30.7) |
    | Served over HTTP by this image (warm, batch 1, 50 requests) | 55.9 ms device (p90 56.3) · 69.1 ms server-side incl. four PNG decodes (p90 72.7) |
    | Same policy on an RTX 5090 (same host, NVIDIA's unmodified N1.7 policy, eager PyTorch bf16, idle machine) | `get_action` 59.8–65.6 ms model, 73–79 ms incl. host pre/post-processing → p150a 1.22× faster end-to-end; NVIDIA's published H100 TensorRT full pipeline: 27.9 ms (one camera) |

    ### Caveats

    - Stage-2 path: one persistent `generic_op` runs the whole denoise inside the fourth Metal trace, streaming the 32 blocks' bfp8 weights once per step from a ~1 GB DRAM arena over the 11×10 Tensix grid (96 compute cores, 8 DRAM readers, hubs), with distributed LayerNorm, per-head flash SDPA and the cross attention over the 448-key image set split three ways. Costs: 64 KiB less allocatable L1 per core (`worker_l1_size` 1,395,712), the VL adapter's intermediates in DRAM (+0.6 ms), compute binaries of ~131 KB sitting ~4.5 KB under the 136 KB kernel-config ring, and no DiT-intermediate taps. Gain vs the Stage-1 path on the same tree: −19 % end-to-end, −46 % denoise (the plan's −30 % e2e was not reached). Everything outside the DiT matmuls is bf16.
    - The vision tower is the port's tuned Qwen3-VL fork (HiFi3 matmuls on blocks 0–5; 17.5 ms vs 87 ms for the verbatim tower, `GR00T_VIT_FORK=0` selects it).
    - One embodiment and layout: DROID (`oxe_droid_relative_eef_relative_joint`, embodiment slot 24), exactly two cameras × two frames (t−15 and t), landscape frames that land on the 16×28-patch grid (width / height in [1.686, 1.814]: 16:9, e.g. 320×180), instruction ≤ 51 tokens; anything else is refused with 400. Batch 1.
    - Noise: default `seed` 42 (CPU generator, deterministic). The demo's reference noise (`code/gr00t_p150/demo/n17/noise.json`) was drawn on CUDA and is not seed-reproducible; send it as `noise` to reproduce the reference actions.
    - Not an OpenAI-compatible API; `GET /v1/models` is a stub.
    - First boot is cold (weight conversion + kernel JIT, 64 s); later boots take 14 s. Validated on tt-metal `668c2907575` (port snapshot `009de892c895`), single p150a only. Details: [`SERVING.md`](SERVING.md), [`GPU_COMPARISON.md`](GPU_COMPARISON.md).

    ### Licensing

    - Weights: [nvidia/GR00T-N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B), **NVIDIA License** (non-commercial; copy in [`LICENSE`](LICENSE)). Not redistributed here.
    - Tokenizer and image-processor files (`code/gr00t_p150/assets/tokenizer/n17/`) are byte-identical to [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) (Apache-2.0; hashes in `PROVENANCE.md` next to them).
    - Port and serving code (`code/`): Apache-2.0, © 2026 Tenstorrent, distributed under the same non-commercial limitation as a derivative of the weights. See [`LICENSE-NOTICE.md`](LICENSE-NOTICE.md).