RyanL22/pi05-openarm-rh56f1-egomimic-baseline-qstate-20k

pi0.5 baseline -- egomimic (OpenArm + RH56F1), human state = retargeted pose, step 20k

Corrected rebuild of RyanL22/pi05-openarm-rh56f1-egomimic-baseline-20k. Same data, labels, actions, images, cells and hyper-parameters; only the human-frame observation.state changed.

EgoMimic style: SAM3 arm mask filled black plus a red line (line_on_hand for human frames, FK palm-to-elbow for teleop frames).

What was fixed

The original build synthesised the human-frame state from the action with a lag + gain/offset model fitted on teleop (command -> measured). The retargeted labels are already in the measured convention, so the correction was applied twice: the neck became 0.8973 instead of 0.889 (a perfect teleop-vs-human flag after quantile normalisation) and hand joints were compressed by gains as low as 0.17. At rollout the real robot's state therefore selected the teleop mode, and tasks that exist only as human data failed. Here the human state is the retargeted pose q itself, matched to each label file by its exact action sequence (466/466).

Rollout

• Policy state neck: 0.889 / 0.001 (teleop and human frames now share it; the measured 0.881 also normalises to -0.08).
• For human-only tasks start from the human-retarget initial pose (init_states_by_task_qstate.yaml); the hand convention (little finger curled) still differs from teleop.
• Task strings must match training exactly.

Training

teleop v4 20 Hz (4 tasks) + anyh2r human video (12 tasks), 16 cells with sqrt(frames) shares, 28-dim state/action, 20 fps, stereo 288x512. 20,000 steps, global batch 64 (2 x 32, H200), bfloat16, vision encoder unfrozen, stereo-consistent augmentation, mirror off.