| --- |
| license: apache-2.0 |
| library_name: openpi |
| tags: |
| - robotics |
| - tactile |
| - vla |
| - pi05 |
| - openpi |
| - xarm |
| - ablation |
| --- |
| |
| # pi05 xArm tube - `binmarker9` (9 arrows/finger, contact bit only) |
|
|
| pi0.5 LoRA fine-tune for the xArm **tube** task (`"pick up the tube and put it in the nearest slot"`), trained on frames |
| with the **`binmarker9`** tactile overlay burned in. |
|
|
| ## What the overlay encodes |
|
|
| **Only the per-finger binary contact bit.** Both force **direction** and force |
| **magnitude** are omitted: |
|
|
| | Signal | Present? | |
| |---|---| |
| | Binary contact bit | YES | |
| | Force direction | NO - fixed to the sensor-frame pad normal | |
| | Force magnitude | NO - fixed length (0.12 m) | |
|
|
| 9 arrows per finger (one at each of the 9 forward-kinematics-projected sensor pad cells), drawn iff that finger's contact |
| state is ON (max-cell metric, hysteresis ON >= 0.25 / OFF < 0.18, 2-on/3-off |
| debounce, state reset per episode). |
|
|
| Verified tactile-independence: **0 max pixel change across 300 randomized force |
| readings at a fixed pose** - the pixels provably carry no force information. |
|
|
| ## Ablation pair |
|
|
| | Task | 1 arrow | 9 arrows | |
| |---|---|---| |
| | tube | [`binmarker`](https://huggingface.co/EdwardoSunny/pi05_xarm_tube_binmarker_lora) | [`binmarker9`](https://huggingface.co/EdwardoSunny/pi05_xarm_tube_binmarker9_lora) | |
|
|
| Hyperparameters are identical across both arms (LoRA `gemma_2b_lora` + |
| `gemma_300m_lora`, batch size 8, cosine schedule warmup 500 / peak 1e-4 / |
| decay 20k to 1e-5, AdamW grad-clip 1.0, no EMA, 20k max steps with early |
| stopping), so differences are attributable to the overlay alone. |
| Trained 5800 steps. |
|
|
| ## Contents |
|
|
| Orbax checkpoint (`params/`) plus `assets/` norm stats - inference-ready. |
| `train_state/` (optimizer state) is intentionally excluded. |
|
|
| ## Inference |
|
|
| ```bash |
| python examples/xarm/inference/run_xarm_inference.py \ |
| --checkpoint <local-path> \ |
| --overlay binmarker9_tube |
| ``` |
|
|
| The overlay **must** be redrawn at deployment exactly as in training, or the |
| policy is out of distribution. The inference renderer (`FixedMarkerOverlay`, |
| variant `binmarker9`) is verified **byte-exact** against the training renderer: |
| 0 max pixel error over 3,566 camera images across all 4 tasks (cube 450, |
| tube 760, charger 1,122, dishwasher 1,234), 0 contact-state mismatches. The `--overlay` flag requires the per-task suffix |
| because the per-finger p99 scales differ up to ~6x across tasks. |
|
|
| Observation/action convention: state 8-dim |
| `[ee_pos_m(3), ee_axis_angle_rad(3), grasp, grasp]`, action 7-dim |
| `[dxyz_m, d_axis_angle_rad, grasp_pm1]`, 224x224 RGB uint8, 10 Hz, action |
| chunk 10 (consume 5). |
|
|
| ## Paired dataset |
|
|
| [EdwardoSunny/xarm_tube_binmarker9](https://huggingface.co/datasets/EdwardoSunny/xarm_tube_binmarker9) |
|
|
