pi05-so101-grab-ball-30k

pi0.5 fine-tuned on an SO-101 single-arm teleoperation dataset. The policy takes two camera views plus the current joint state and returns a chunk of 16 absolute joint targets at 15 Hz.

Task prompt: "Grab the ball" — the only task string in the training set.

Training

Base	pi05_base (gs://openpi-assets/checkpoints/pi05_base/params)
Dataset	Ibuprofene/grab_ball_2cam_v1 — 100 episodes, 68,458 frames
Steps	30,000, batch 32
LR	cosine, peak 2.5e-5 -> 2.5e-6, 1k warmup
Hardware	4x A100-64GB, FSDP over 4 devices
Final loss	0.00261
W&B	run

Model config is Pi0Config(pi05=True, action_dim=32, action_horizon=16). Actions are trained as deltas relative to the current state on the five arm joints, absolute on the gripper; the server adds the state back before replying, so what you receive is absolute. Normalization is quantile (q01/q99 -> [-1,1]).

Interface

Send:

Key	Type
images.fixed	uint8 HxWx3 RGB, the table camera
images.handeye	uint8 HxWx3 RGB, the wrist camera
state	float32 (6,)
prompt	"Grab the ball"

Receive: actions, float32 (16, 6).

The two cameras are not interchangeable — swapping them produces confident nonsense.

Units, for both state and actions, in order: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll in degrees, then gripper on 0-100. This is a LeRobot so_follower with use_degrees=True. If your control code reads the arm in the -100..100 servo percent convention, convert first.

Rate: the 16 actions are spaced at 15 Hz, not 30. The data was recorded at 30 fps and strided by 2 in training, so one chunk covers ~1.07 s. Replay at 15 Hz or interpolate up; replaying at 30 Hz halves the intended speed.

Usage

hf download mattewg/pi05-so101-grab-ball-30k --local-dir pi05-so101-grab-ball-30k

uv run scripts/serve_policy.py policy:checkpoint \
    --policy.config=<your config name> \
    --policy.dir=/absolute/path/to/pi05-so101-grab-ball-30k

--policy.dir must be the directory containing params/ and assets/, not params/ itself. Norm stats are read from assets/so101/grab_ball_2cam_v1/, so the server uses exactly the statistics the model was trained with.

Requires openpi at commit 215abfb217dbac7d5f1273282331b9b1866c0479 plus the SO-101 policy/config files, which are not in upstream openpi. Later commits may have changed the transform pipeline, which would silently produce wrong actions.

Known limitation

These policies can stall at the end-of-episode pose — arm up, gripper closed, roughly [25, -17, -79, 68, 36] in the joint order above. Every demonstration ends there and stops recording, so the data contains no frames showing what follows, and a policy that reaches that state has only "hold" available to it. Recovery demonstrations that start from that pose are needed to fix it; more training steps do not.

Optimizer state is not included, so these can be warm started but not resumed.