RyanL22/pi05-rby1-wujihand2-realsynth-rt2v6-15k

pi0.5 — RB-Y1 + Wujihand2, real teleop + synthetic (human-video) data, step 15k

LeRobot-native pi05 (v0.6.1) fine-tuned on real teleop v1 (209 ep, 30→20 fps) + synthetic RB-Y1 videos generated from human demonstrations (RoboTryOn-human-2, rt2 v6 eeray-pass, 366 ep, 20 fps): 575 episodes / 152,250 frames, 15 cells (4 real categories + 11 synthetic tasks), shares ∝ sqrt(frames).

Cells: real:ball 51, real:bottle 51, real:box 55, real:doll 52, synth:airfryer 26, synth:ball 47, synth:ball_in_cup 31, synth:bottle 46, synth:bottle_over_shelf 39, synth:bottle_pour 29, synth:box 39, synth:box_convey 15, synth:coffee_pot 26, synth:doll 37, synth:brush 31.

Checkpoint: step 15,000 of a 30,000-step run (NAVER MLXP, 2x H200, batch 32 x 2 = 64). Intermediate checkpoint for early rollout tests.

54-dim state / action

Same layout as RyanL22/pi05-rby1-wujihand2-teleopv1-baseline-20k: joint_position[10:64] = right_arm_0..6 (7) | right hand 20 (thumb_cmc_flex, thumb_cmc_abd, thumb_mcp, thumb_ip, index_finger_mcp_flex, index_finger_mcp_abd, index_finger_pip, index_finger_dip, middle_finger_mcp_flex, middle_finger_mcp_abd, middle_finger_pip, middle_finger_dip, ring_finger_mcp_flex, ring_finger_mcp_abd, ring_finger_pip, ring_finger_dip, pinky_mcp_flex, pinky_mcp_abd, pinky_pip, pinky_dip) | left_arm_0..6 (7) | left hand 20. Wheels / torso / head are dropped (constant in the data); hold them at the recording pose at rollout. max_state_dim = max_action_dim = 54, tokenizer max_length = 320 (both already in this repo's config).

Training settings

	value
vision encoder (SigLIP, 412.4M)	fine-tuned (not frozen)
image augmentation	photometric + affine, one draw replayed across the stereo pair
cell shares	∝ sqrt(frames), 15 cells
mirror augmentation	off
optimizer	AdamW, peak lr 2.5e-5, cosine decay to 2.5e-6, warmup 1000, 30k schedule
precision	bfloat16, gradient checkpointing
chunk	50 actions @ 20 fps (2.5 s), n_obs_steps=1
normalization stats	quantiles; constant dims widened (gate: q99 - q01 >= 1e-3)

Inputs

• observation.images.base_0_rgb <- left ZED view, 512x288 (real 1280x720 and synthetic 1024x576 both resized)
• observation.images.left_wrist_0_rgb <- right ZED view, 512x288
• observation.state — 54 dims; action — 54 dims, absolute joint targets

Tasks (11): move the box to the right, starting with the left hand and finishing with the right hand / move the cup aside, then pick up the ball from table and put it in the white box / open the air fryer with the left hand, pick up the apple from the table with the right hand and put it in the air fryer, then close the air fryer with the left hand / pick up the ball from table and put it in the white box / pick up the bottle from table and put it in the white box / pick up the bottle from table and put it on the shelf at front / pick up the brush with the right hand and wipe away the black dot / pick up the coffee pot from table and put it in the white box / pick up the doll from table and put it in the white box / pick up the small box from table and put it in the white box / pour the coffee from the bottle into the red cup

Initial state (frame 0) — set the robot here before a rollout

The synthetic episodes all start from one pose (per-cell arm means within 0.2 rad of each other; std ≤ 0.17 rad), so use this synthetic common start for every task. Values are rad in the 66-dim robot order; the 54 policy dims are joint_position[10:64]. Wheels/torso/head are the recording constants and are not fed to the policy.

group	values
wheels (0:4)	0 0 0 0
torso (4:10)	+0.000 +0.524 -1.047 +0.524 +0.000 -0.000
right_arm (10:17)	+0.349 -0.129 +0.244 -1.845 -0.459 -0.107 -1.313
right hand (17:37)	+0.360 -0.156 +0.006 +0.054 +0.147 +0.050 +0.055 +0.157 +0.109 +0.072 +0.057 +0.150 +0.071 +0.052 +0.060 +0.164 +0.040 +0.086 +0.068 +0.182
left_arm (37:44)	+0.317 +0.019 -0.291 -1.712 +0.363 -0.228 +1.261
left hand (44:64)	+0.200 +0.016 +0.013 +0.065 +0.105 +0.057 +0.040 +0.089 +0.093 +0.067 +0.037 +0.090 +0.043 +0.055 +0.026 +0.091 -0.010 +0.098 +0.032 +0.082
head (64:66)	+0.000 +0.870

Real teleop episodes start from a different, more extended pose (arm L2 distance 0.63 / 0.68 rad from the synthetic start, right/left; e.g. elbow −2.07 vs −1.85 and wrist ≈ 0 vs −0.46 / +0.36): right_arm +0.522 -0.020 +0.267 -2.078 -0.005 +0.006 -1.592, left_arm +0.528 +0.015 -0.268 -2.067 +0.008 +0.001 +1.591, hands ≈ open (all < 0.05). Both starts are in-distribution for the 4 real tasks; for the 7 synthetic-only tasks use the synthetic start. Per-cell means/std: init_state.json in the run notes (~/data/anyh2r/rollout_logs/init_state_rby1_0921/).

Load

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
policy = PI05Policy.from_pretrained("RyanL22/pi05-rby1-wujihand2-realsynth-rt2v6-15k")