--- license: apache-2.0 language: - en pipeline_tag: robotics tags: - robotics - vla - vision-language-action - libero - gr00t - isaac-gr00t - imitation-learning - sae - mechanistic-interpretability - simulation pretty_name: "VLA-SAE — GR00T-N1.7-LIBERO LIBERO smoke-test tooling & run artifacts" --- # vla-sae-libero — LIBERO smoke-test tooling & run artifacts for `nvidia/GR00T-N1.7-LIBERO` This repo holds **only** the smoke-test *tooling* and the *artifacts* of one completed run from deploying **[`nvidia/GR00T-N1.7-LIBERO`](https://huggingface.co/nvidia/GR00T-N1.7-LIBERO)** (the `libero_10` checkpoint) and running **10 short, simulation-only LIBERO rollouts** to verify the deploy/eval plumbing — a stepping stone toward activation-capture / SAE work on the GR00T VLA. > ## Built on NVIDIA Isaac-GR00T — *not redistributed here* > The tooling in this repo runs **on top of** [**NVIDIA Isaac-GR00T**](https://github.com/NVIDIA/Isaac-GR00T) > (Apache-2.0, © NVIDIA CORPORATION & AFFILIATES). **This repo does NOT contain the upstream Isaac-GR00T > source tree, the model weights, or the gated VLM backbone.** To use the tooling: clone Isaac-GR00T, install > its `gr00t` package, drop `examples/LIBERO/smoke_tests/` (and `tests/test_libero_smoke_tests.py`) into the > checkout, download the checkpoint, and follow the steps below. > > Specifically, **not included** (get them from upstream): > - the Isaac-GR00T source tree (`gr00t/`, `examples/` except `examples/LIBERO/smoke_tests/`, `getting_started/`, `scripts/`, `docker/`, `demo_data/`, `media/`, …) — > - the model checkpoint `nvidia/GR00T-N1.7-LIBERO` (~6.5 GB) — > - the **gated** VLM backbone `nvidia/Cosmos-Reason2-2B` — > - Python virtualenvs (`.venv/`, `gr00t/eval/sim/LIBERO/libero_uv/`) and the LIBERO submodule (`external_dependencies/LIBERO/`) ## Contents ``` examples/LIBERO/smoke_tests/ # the smoke-test tooling ├── scenarios_10.yaml # 10-scenario manifest (8 normal + 2 abnormal_probe) ├── run_10_smoke_tests.py # runner: validates model/server, runs each scenario via the official eval path, writes summaries ├── _libero_rollout_worker.py # single-episode worker (runs in the LIBERO venv): records action chunks, optional sim-only probes ├── visualize_smoke_run.py # builds visual_report.html + per-scenario action plots + visual_summary.{csv,json} ├── make_video_montage.py # builds review_montage.mp4 + per-scenario captioned clips + review_playlist.html (needs ffmpeg) ├── review_smoke_tests.py # CLI: list/play the rollouts of a run ├── ACTIVATION_HOOK_NOTES.md # where to insert future SAE activation capture (module names, hook points) └── README.md # setup & run instructions tests/test_libero_smoke_tests.py # lightweight tests for the manifest + summary writer outputs/libero_smoke_tests/20260512_122756/ # the completed run (10/10 rollouts OK, 0 errors; success=false everywhere — these are 50-step smoke rollouts, not a benchmark) ├── summary.json, summary.md # run-level results + the printed comparison table ├── visual_report.html # self-contained report: embedded videos + plots + comparison table ├── visual_summary.csv, visual_summary.json # per-scenario action-stat table ├── review_montage.mp4 # ~22 s: all 10 scenarios back-to-back with captions ├── review_clips/.mp4 # 10 short captioned clips ├── review_playlist.html # the montage + each clip + metadata + links ├── plots// # action_norm / action_mean_per_dof / gripper_over_time / action_delta_norm (.png) └── / # ×10 ├── video.mp4 # rendered rollout (agentview | wrist, 512×256) ├── frames/ # decoded PNG frames (25 each; 8 for the short-timeout probe) ├── actions.npy # recorded action chunks, shape (n_calls, 1, 16, 7) float32 [DoF column order: gripper, pitch, roll, x, y, yaw, z] ├── metadata.json, rollout_summary.json # scenario config + results └── stdout.log, stderr.log # the worker subprocess logs outputs/_setup_logs/ # deployment record: uv install / LIBERO setup / server / smoke-run logs (no secrets) LICENSE # Apache-2.0 (same license as upstream Isaac-GR00T) NOTICE # attribution ``` ## The run, in one line 10 LIBERO simulation rollouts of `nvidia/GR00T-N1.7-LIBERO` (`libero_10`), `max_episode_steps = 50` (16 for the short-timeout probe), `n_action_steps = 8`, `n_envs = 1`, **no physical hardware**. 8 `normal` long-horizon tasks + 2 `abnormal_probe` (mild Gaussian observation noise; deliberately shortened timeout). All 10 ran cleanly with actions + video saved; `success = false` everywhere — a 50-step rollout cannot complete these long-horizon tasks, and that's the point: this verifies the deploy/serve/client/sim/render/save plumbing, not task performance. ## Reproduce Full instructions: [`examples/LIBERO/smoke_tests/README.md`](examples/LIBERO/smoke_tests/README.md). Summary: ```bash # 0. start from a clone of NVIDIA Isaac-GR00T and copy this repo's files into it git clone https://github.com/NVIDIA/Isaac-GR00T.git && cd Isaac-GR00T git submodule update --init external_dependencies/LIBERO # ... then place examples/LIBERO/smoke_tests/ and tests/test_libero_smoke_tests.py from THIS repo here ... # install gr00t (uv sync, or the minimal recipe in examples/LIBERO/smoke_tests/README.md), then: sudo apt install libegl1-mesa-dev libglu1-mesa cmake && bash gr00t/eval/sim/LIBERO/setup_libero.sh uv run hf download nvidia/GR00T-N1.7-LIBERO --include "libero_10/*" --local-dir checkpoints/GR00T-N1.7-LIBERO # request access to https://huggingface.co/nvidia/Cosmos-Reason2-2B and export HF_TOKEN=hf_... # 1. server (terminal 1) — this pulls the gated nvidia/Cosmos-Reason2-2B backbone uv run python gr00t/eval/run_gr00t_server.py \ --model-path checkpoints/GR00T-N1.7-LIBERO/libero_10 \ --embodiment-tag LIBERO_PANDA --use-sim-policy-wrapper # 2. smoke tests (terminal 2) uv run python examples/LIBERO/smoke_tests/run_10_smoke_tests.py \ --model-path checkpoints/GR00T-N1.7-LIBERO/libero_10 --host 127.0.0.1 --port 5555 \ --manifest examples/LIBERO/smoke_tests/scenarios_10.yaml --output-dir outputs/libero_smoke_tests \ --max-episode-steps 50 --save-video --render # 3. review artifacts (no GPU, no rerun) python examples/LIBERO/smoke_tests/visualize_smoke_run.py --run-dir outputs/libero_smoke_tests/ --open python examples/LIBERO/smoke_tests/make_video_montage.py --run-dir outputs/libero_smoke_tests/ --mode sequential --open ``` ## Next: activation capture / SAE See [`examples/LIBERO/smoke_tests/ACTIVATION_HOOK_NOTES.md`](examples/LIBERO/smoke_tests/ACTIVATION_HOOK_NOTES.md). Suggested first step: register a `forward_hook` on `policy.model.backbone` (the `Qwen3Backbone` — its `forward` returns `backbone_features`, the last Qwen3-VL hidden state = the fused image+instruction representation that conditions action generation), dump that per `get_action` call alongside the already-saved `actions.npy` + frames, and train an SAE on it. ## License & attribution This repo is released under **Apache-2.0**, matching upstream **NVIDIA Isaac-GR00T** (, © NVIDIA CORPORATION & AFFILIATES) — see `LICENSE` and `NOTICE`. The added tooling (`examples/LIBERO/smoke_tests/`, `tests/test_libero_smoke_tests.py`) is Apache-2.0 and imports/wraps the upstream `gr00t` package. The artifacts under `outputs/` are outputs of the `nvidia/GR00T-N1.7-LIBERO` model run in LIBERO simulation; **no model weights or upstream source files (other than this `LICENSE`) are redistributed here.**