fanqi-robo/groot_n17_insert_gear_in_gripper_lr2x_s1000

groot_n17 on insert_gear_in_gripper (learning rate 2x, seed 1000)

This branch (main) holds the final checkpoint (update 20000). main is the final checkpoint and the benchmark's comparison point; best is the checkpoint with the lowest validation loss.

Task	insert_gear_in_gripper, bimanual YAM, 14-D joint state and action, three 720x1280 cameras
Training data	fanqi-robo/insert_gear_in_gripper @ acdc9ac8 (50 episodes, 27,228 frames)
Validation data	villekuosmanen/insert_gear_in_gripper_val @ 7c4d62f3 (5 held-out episodes, 2,245 frames)
Initialization	nvidia/GR00T-N1.7-3B@2fc962b9, addressed by Hub id through a pinned offline cache
Trainable	language model, vision tower, projector, diffusion action head and VL layer norm (all five tune flags); fp32 parameters under bf16 autocast
Action chunk	40 actions predicted, 40 executed
Normalization	state and action min/max from the training set, clipped to [-1, 1] (what the lerobot 0.6.1 port does for new_embodiment; not q01/q99)
Embodiment-specific pretraining	trained under the generic new_embodiment slot; no YAM-specific pretraining is known
Optimizer and schedule	AdamW, diffusers cosine schedule with warmup of 5% of the updates, reaching 0 at the last update
Learning rate	2x of the starting rate: optimizer_lr=2e-05
Updates	20000 optimizer updates at effective batch 32 (32 x 1, one GPU)
Seed	1000
Image augmentation	none
lerobot	0.6.1
GPU	1 x NVIDIA H200, 10.2 GPU-hours
Parameters	3,144,016,000 trainable of 3,144,016,000
Peak VRAM	87.2 GB
Validation loss	0.0917025 at the final update; best 0.0478538 at update 7659
Training loss (last window)	0.00479408
Inference latency ms	178.1
W&B	https://wandb.ai/fanqi-robo-saferobotics/insert_gear_in_gripper_benchmark/runs/6uuvhcnq

Validation loss: masked flow-matching MSE of policy.forward in eval mode, one noise draw per batch under a fixed seed; episode tails without a full 40-step chunk are not sampled. Eval mode removes dropout, so it sits below the training loss by construction. It ranks checkpoints of this run; it is not comparable across policies.

benchmark_run.json holds the benchmark settings, metrics.jsonl the training and validation curves, run_summary.json the end-of-run numbers.

Licence

Fine-tuned from nvidia/GR00T-N1.7-3B and distributed under the NVIDIA License included in this repository (LICENSE): research or evaluation use only, not commercial use.

Offline evaluation on the held-out episodes

Open-loop action-chunk error on villekuosmanen/insert_gear_in_gripper_val (423 queries, every 5th frame), in the dataset's joint units; hold (keeping the current pose) scores MAE@30 2.60.

Update	MAE@10	MAE@30	k=1	k=30	arm	grip rec	grip dt
851	6.14	6.39 +/- 0.31	5.96	6.86	0.92	0.35	3.4
1702	4.34	4.59 +/- 0.47	4.40	5.12	0.91	0.59	4.5
2553	3.72	4.05 +/- 0.43	3.60	4.64	0.99	0.60	3.8
3404	3.42	3.82 +/- 0.50	3.38	4.52	0.96	0.67	4.1
4000	3.25	3.63 +/- 0.39	3.21	4.23	0.98	0.67	4.2
4255	3.50	3.95 +/- 0.69	3.42	4.73	0.99	0.70	4.4
5106	3.13	3.61 +/- 0.50	2.97	4.37	0.98	0.78	4.4
5957	2.76	3.15 +/- 0.40	2.57	3.74	0.99	0.76	4.0
6808	2.66	3.04 +/- 0.48	2.54	3.65	0.99	0.83	3.9
7659	2.63	3.02 +/- 0.58	2.50	3.70	0.99	0.79	4.6
8000	2.56	3.05 +/- 0.52	2.41	3.89	0.99	0.75	3.8
8510	2.64	3.07 +/- 0.51	2.45	3.76	1.00	0.76	3.6
9361	2.46	2.97 +/- 0.51	2.31	3.72	1.00	0.73	4.7
10212	2.43	2.93 +/- 0.48	2.26	3.76	1.00	0.62	3.9
11063	2.44	2.97 +/- 0.53	2.24	3.83	1.00	0.71	4.4
11914	2.58	3.03 +/- 0.73	2.44	3.77	1.00	0.71	5.0
12000	2.48	2.93 +/- 0.56	2.35	3.66	1.00	0.71	4.0
12765	2.44	2.91 +/- 0.62	2.27	3.67	1.00	0.73	4.3
13616	2.38	2.86 +/- 0.60	2.22	3.62	1.00	0.65	4.3
14467	2.43	2.89 +/- 0.57	2.30	3.65	1.00	0.75	5.1
15318	2.37	2.84 +/- 0.57	2.23	3.60	1.00	0.70	4.4
16000	2.44	2.88 +/- 0.64	2.28	3.61	1.00	0.70	4.6
16169	2.47	2.93 +/- 0.66	2.34	3.69	1.00	0.71	4.8
17020	2.43	2.87 +/- 0.61	2.29	3.62	1.00	0.65	4.5
17871	2.43	2.88 +/- 0.59	2.30	3.63	1.00	0.70	4.8
18722	2.42	2.88 +/- 0.61	2.29	3.63	1.00	0.68	4.8
19573	2.41	2.86 +/- 0.60	2.27	3.62	1.00	0.70	4.7
20000	2.41	2.86 +/- 0.60	2.27	3.62	1.00	0.70	4.7