EdwardoSunny/pi05_xarm_tube_binmarker9_lora

pi05 xArm tube - binmarker9 (9 arrows/finger, contact bit only)

pi0.5 LoRA fine-tune for the xArm tube task ("pick up the tube and put it in the nearest slot"), trained on frames with the binmarker9 tactile overlay burned in.

What the overlay encodes

Only the per-finger binary contact bit. Both force direction and force magnitude are omitted:

Signal	Present?
Binary contact bit	YES
Force direction	NO - fixed to the sensor-frame pad normal
Force magnitude	NO - fixed length (0.12 m)

9 arrows per finger (one at each of the 9 forward-kinematics-projected sensor pad cells), drawn iff that finger's contact state is ON (max-cell metric, hysteresis ON >= 0.25 / OFF < 0.18, 2-on/3-off debounce, state reset per episode).

Verified tactile-independence: 0 max pixel change across 300 randomized force readings at a fixed pose - the pixels provably carry no force information.

Ablation pair

Task	1 arrow	9 arrows
tube	binmarker	binmarker9

Hyperparameters are identical across both arms (LoRA gemma_2b_lora + gemma_300m_lora, batch size 8, cosine schedule warmup 500 / peak 1e-4 / decay 20k to 1e-5, AdamW grad-clip 1.0, no EMA, 20k max steps with early stopping), so differences are attributable to the overlay alone. Trained 5800 steps.

Contents

Orbax checkpoint (params/) plus assets/ norm stats - inference-ready. train_state/ (optimizer state) is intentionally excluded.

Inference

python examples/xarm/inference/run_xarm_inference.py \
  --checkpoint <local-path> \
  --overlay binmarker9_tube

The overlay must be redrawn at deployment exactly as in training, or the policy is out of distribution. The inference renderer (FixedMarkerOverlay, variant binmarker9) is verified byte-exact against the training renderer: 0 max pixel error over 3,566 camera images across all 4 tasks (cube 450, tube 760, charger 1,122, dishwasher 1,234), 0 contact-state mismatches. The --overlay flag requires the per-task suffix because the per-finger p99 scales differ up to ~6x across tasks.

Observation/action convention: state 8-dim [ee_pos_m(3), ee_axis_angle_rad(3), grasp, grasp], action 7-dim [dxyz_m, d_axis_angle_rad, grasp_pm1], 224x224 RGB uint8, 10 Hz, action chunk 10 (consume 5).

Paired dataset

EdwardoSunny/xarm_tube_binmarker9