fanqi-robo/groot_n17_insert_gear_in_gripper_lr0p5x_s1000

groot_n17 on insert_gear_in_gripper (learning rate 0p5x, seed 1000)

This branch (main) holds the final checkpoint (update 20000). main is the final checkpoint and the benchmark's comparison point; best is the checkpoint with the lowest validation loss.

Task	insert_gear_in_gripper, bimanual YAM, 14-D joint state and action, three 720x1280 cameras
Training data	fanqi-robo/insert_gear_in_gripper @ acdc9ac8 (50 episodes, 27,228 frames)
Validation data	villekuosmanen/insert_gear_in_gripper_val @ 7c4d62f3 (5 held-out episodes, 2,245 frames)
Initialization	nvidia/GR00T-N1.7-3B@2fc962b9, addressed by Hub id through a pinned offline cache
Trainable	language model, vision tower, projector, diffusion action head and VL layer norm (all five tune flags); fp32 parameters under bf16 autocast
Action chunk	40 actions predicted, 40 executed
Normalization	state and action min/max from the training set, clipped to [-1, 1] (what the lerobot 0.6.1 port does for new_embodiment; not q01/q99)
Embodiment-specific pretraining	trained under the generic new_embodiment slot; no YAM-specific pretraining is known
Optimizer and schedule	AdamW, diffusers cosine schedule with warmup of 5% of the updates, reaching 0 at the last update
Learning rate	0p5x of the starting rate: optimizer_lr=5e-06
Updates	20000 optimizer updates at effective batch 32 (32 x 1, one GPU)
Seed	1000
Image augmentation	none
lerobot	0.6.1
GPU	1 x NVIDIA H200, 11.1 GPU-hours
Parameters	3,144,016,000 trainable of 3,144,016,000
Peak VRAM	87.2 GB
Validation loss	0.079605 at the final update; best 0.0546447 at update 4255
Training loss (last window)	0.0126846
Inference latency ms	187.7
W&B	https://wandb.ai/fanqi-robo-saferobotics/insert_gear_in_gripper_benchmark/runs/2kjm1sg0

Validation loss: masked flow-matching MSE of policy.forward in eval mode, one noise draw per batch under a fixed seed; episode tails without a full 40-step chunk are not sampled. Eval mode removes dropout, so it sits below the training loss by construction. It ranks checkpoints of this run; it is not comparable across policies.

benchmark_run.json holds the benchmark settings, metrics.jsonl the training and validation curves, run_summary.json the end-of-run numbers.

Licence

Fine-tuned from nvidia/GR00T-N1.7-3B and distributed under the NVIDIA License included in this repository (LICENSE): research or evaluation use only, not commercial use.

Offline evaluation on the held-out episodes

Open-loop action-chunk error on villekuosmanen/insert_gear_in_gripper_val (423 queries, every 5th frame), in the dataset's joint units; hold (keeping the current pose) scores MAE@30 2.60.

Update	MAE@10	MAE@30	k=1	k=30	arm	grip rec	grip dt
851	8.33	8.40 +/- 0.43	9.12	8.94	0.89	0.54	4.7
1702	5.57	5.63 +/- 0.47	5.59	5.89	0.97	0.43	3.7
2553	4.34	4.59 +/- 0.34	4.30	5.05	0.98	0.52	4.0
3404	4.01	4.25 +/- 0.28	3.93	4.78	0.98	0.70	4.0
4000	3.67	3.99 +/- 0.39	3.62	4.51	0.98	0.60	4.8
4255	3.58	3.92 +/- 0.37	3.56	4.58	0.95	0.78	4.5
5106	3.44	3.87 +/- 0.46	3.38	4.64	0.98	0.78	3.8
5957	3.15	3.50 +/- 0.38	3.11	4.10	0.97	0.73	3.8
6808	3.25	3.64 +/- 0.54	3.15	4.26	0.99	0.78	4.4
7659	3.08	3.42 +/- 0.51	3.02	3.96	0.97	0.76	3.7
8000	3.21	3.52 +/- 0.45	3.22	4.10	0.97	0.79	3.6
8510	3.11	3.45 +/- 0.46	3.15	4.08	0.99	0.76	4.3
9361	3.12	3.39 +/- 0.53	3.05	3.90	0.97	0.76	4.5
10212	2.98	3.29 +/- 0.44	2.97	3.85	1.00	0.73	3.8
11063	2.90	3.29 +/- 0.50	2.79	3.94	0.99	0.81	4.1
11914	2.96	3.29 +/- 0.46	2.90	3.87	0.99	0.81	4.5
12000	3.05	3.37 +/- 0.39	2.92	3.95	0.99	0.81	4.7
12765	2.87	3.23 +/- 0.51	2.78	3.85	0.99	0.84	4.2
13616	2.91	3.23 +/- 0.60	2.84	3.77	0.99	0.79	4.7
14467	2.92	3.21 +/- 0.44	2.85	3.74	0.99	0.79	4.4
15318	2.89	3.22 +/- 0.56	2.82	3.80	0.99	0.78	4.3
16000	2.91	3.24 +/- 0.55	2.85	3.84	0.99	0.81	4.5
16169	2.90	3.23 +/- 0.54	2.84	3.83	0.99	0.79	4.5
17020	2.91	3.24 +/- 0.52	2.85	3.83	0.99	0.79	4.3
17871	2.87	3.21 +/- 0.49	2.79	3.78	0.99	0.81	4.5
18722	2.85	3.18 +/- 0.50	2.77	3.76	0.99	0.83	4.6
19573	2.86	3.19 +/- 0.50	2.78	3.77	0.99	0.83	4.7
20000	2.85	3.19 +/- 0.50	2.78	3.76	0.99	0.83	4.7