File size: 3,284 Bytes
049f5bd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
tags:
- robotics
- pi0.5
- openpi
- lora
- tactile
- xarm
base_model: physical-intelligence/pi05_droid
---
# pi05_xarm_tube_binmarker_lora
pi0.5 (openpi) LoRA fine-tune for the xArm **tube** task (`"pick up the tube and put it in the nearest slot"`),
trained on frames with the **binmarker** overlay: a fixed-length,
fixed-direction arrow per finger (sensor-frame pad normal), shown iff that
finger is in binary contact. NO force magnitude and NO force direction — the
overlay carries only the per-finger contact bit. Ported from
LIBERO_contact_overlay's binbars style.
## Training
| | |
|---|---|
| Base | `pi05_droid` |
| Recipe | LoRA r16/a16 backbone + r32/a32 action expert, attn+ffn |
| Batch / LR | 8 / cosine 1e-4 -> 1e-5, 500 warmup, AdamW, grad-clip 1.0 |
| Data | [EdwardoSunny/xarm_tube_binmarker](https://huggingface.co/datasets/EdwardoSunny/xarm_tube_binmarker) (97 eps, 18408 frames) |
| Early stop | step 5800 |
| Final train loss | **0.0412** |
Variant comparison for this task (identical protocol, only overlay pixels differ):
baseline 0.0411 / points9_arrow_len0 0.0359 / **binmarker 0.0412** / points9_arrow 0.0275.
## Contact detection (how the threshold was tuned)
Per-finger binary contact from the 3x3x3-axis tactile array:
- **Normalization offset** = median over known-idle frames (first 10 frames of
each episode, post-homing, gripper open). The naive whole-dataset median is
contact-polluted on tasks that spend most frames in contact (charger: 60%
closed frames made idle read as a fake magnitude of ~1.2).
- **Metric** = max over the 9 cells of the normalized cell-vector L2 norm
("max-cell"). The cell-sum alternative cannot separate charger idle noise
(p99.5 = 0.45) from real light holds (0.5 on tube/dishwasher's weaker finger).
Max-cell: idle p99.5 <= 0.24 on every task, real contacts >= ~0.3.
- **Hysteresis + debounce** (Schmitt trigger): ON after 2 consecutive frames
>= 0.25, OFF after 3 consecutive frames < 0.18, state reset at episode
boundaries. Light-but-real holds (dips to 0.23) stay latched; single-frame
noise spikes never trigger.
Validated on all 400 episodes (zero idle false positives in every post-homing
window) and **visually verified on 5 episodes per task** (timelines + frame
montages at approach / contact onset / mid-hold / post-release), plus a final
spot check on the converted LeRobot frames: no arrows during approach, arrows
at contact onset, latched through holds, cleared within <=3 frames of release,
and honest negatives (gripper closed around nothing draws nothing).
## Contents
- `params/` — merged weights (orbax), self-contained for inference
- `assets/local/xarm_tube_binmarker/norm_stats.json`
- Config name: `pi05_xarm_tube_binmarker_lora`
## Deployment note
At inference, reproduce the SAME overlay on live frames: run the tuned contact
detector (max-cell metric, ON>=0.25 / OFF<0.18, 2-on/3-off debounce, idle-window
offset) and draw the fixed-direction marker (FIXED_DIRECTION, arrow_length_scale=0.12,
thickness 8, dot 22) — see `tactile-data-collection/scripts/render_binmarker.py`
at commit c64c562 or later (earlier revisions used the measured force direction).
|