---
license: apache-2.0
tags:
- robotics
- lerobot
- pi05
- rby1
base_model: lerobot/pi05_base
---

# pi0.5 — RB-Y1 + Wujihand2, synthetic only (human-video, box_full idm-pass), step 30k (final)

LeRobot-native `pi05` (v0.6.1) fine-tuned on **only synthetic RB-Y1 videos generated from human demonstrations (RoboTryOn-human-2, `lerobot_box_full_idm_pass`, 384 ep, 20 fps)**, no real teleop data: 11 cells, shares ∝ sqrt(frames).

Checkpoint: **step 30,000** (final) of a 30,000-step run (NAVER MLXP, 2x H200, batch 32 x 2 = 64).

## 54-dim state / action

Same layout as `RyanL22/pi05-rby1-wujihand2-teleopv1-baseline-20k`: `joint_position[10:64]` = right_arm_0..6 (7) | right hand 20 |
left_arm_0..6 (7) | left hand 20. Wheels / torso / head dropped (constant); hold them at the recording pose at rollout.
`max_state_dim = max_action_dim = 54`, tokenizer `max_length = 320` (already in this repo's config).

## Training settings

| | value |
|---|---|
| vision encoder (SigLIP, 412.4M) | fine-tuned (not frozen) |
| image augmentation | photometric + affine, one draw replayed across the stereo pair |
| mirror augmentation | off |
| optimizer | AdamW, peak lr 2.5e-5, cosine decay to 2.5e-6, warmup 1000 |
| precision | bfloat16, gradient checkpointing |
| chunk | 50 actions @ 20 fps (2.5 s), `n_obs_steps=1` |
| normalization stats | quantiles; constant dims widened (gate: q99 - q01 >= 1e-3) |

## Inputs

- `observation.images.base_0_rgb` <- left ZED view, 512x288; `observation.images.left_wrist_0_rgb` <- right ZED view, 512x288
- `observation.state` — 54 dims; `action` — 54 dims, absolute joint targets

Tasks (11): move the box to the right, starting with the left hand and finishing with the right hand / move the cup aside, then pick up the ball from table and put it in the white box / open the air fryer with the left hand, pick up the apple from the table with the right hand and put it in the air fryer, then close the air fryer with the left hand / pick up the ball from table and put it in the white box / pick up the bottle from table and put it in the white box / pick up the bottle from table and put it on the shelf at front / pick up the brush with the right hand and wipe away the black dot / pick up the coffee pot from table and put it in the white box / pick up the doll from table and put it in the white box / pick up the small box from table and put it in the white box / pour the coffee from the bottle into the red cup

## Initial state (frame 0) — set the robot here before a rollout

All synthetic episodes start from one pose (per-task arm means within 0.26 rad of the common mean, std ≤ 0.19 rad),
so use this **synthetic common start** for every task. Values are rad in the 66-dim robot order; the 54 policy dims are
`joint_position[10:64]`. Wheels/torso/head are the recording constants and are not fed to the policy.

| group | values |
|---|---|
| wheels (0:4) | 0 0 0 0 |
| torso (4:10) | +0.000 +0.524 -1.047 +0.524 +0.000 -0.000 |
| **right_arm (10:17)** | +0.407 -0.158 +0.267 -1.878 -0.496 -0.149 -1.304 |
| right hand (17:37) | +0.356 -0.132 +0.004 +0.059 +0.180 +0.060 +0.056 +0.162 +0.134 +0.074 +0.059 +0.157 +0.086 +0.055 +0.063 +0.169 +0.045 +0.086 +0.071 +0.188 |
| **left_arm (37:44)** | +0.479 +0.044 -0.370 -1.891 +0.312 -0.299 +1.245 |
| left hand (44:64) | +0.210 +0.027 +0.012 +0.059 +0.116 +0.073 +0.044 +0.101 +0.128 +0.077 +0.047 +0.111 +0.064 +0.056 +0.030 +0.110 +0.007 +0.074 +0.035 +0.091 |
| head (64:66) | +0.000 +0.870 |

Real teleop episodes start from a more extended pose: right_arm `+0.522 -0.020 +0.267 -2.078 -0.005 +0.006 -1.592`, left_arm `+0.528 +0.015 -0.268 -2.067 +0.008 +0.001 +1.591`, hands ≈ open.
Both starts are in-distribution for the 4 real tasks; for the synthetic-only tasks use the synthetic start.

## Load

```python
from lerobot.policies.pi05.modeling_pi05 import PI05Policy
policy = PI05Policy.from_pretrained("RyanL22/pi05-rby1-wujihand2-synth-boxfull-30k")
```