---
license: other
tags:
  - robotics
  - manipulation
  - dexterous-manipulation
  - dex-point-policy
  - openarm
  - inspire-rh56f1
---

# DPP 6-point — `human` keypoints (100k)

Dexterous Point Policy (DPP) finetuned on 80 anyh2r pick-and-place episodes, using
**HaWoR human** as the source of the 6 hand keypoints.

> One of three models that differ **only** in how the 6 keypoints
> (wrist + 5 fingertips) were derived. Everything else — object point clouds,
> per-finger contact, camera pose, language, validity mask, episode set,
> hyper-parameters, pretrained init — is byte-identical across the three.

| repo | keypoint source |
|---|---|
| [`dpp-6pt-anyh2r-fk-100k`](https://huggingface.co/RyanL22/dpp-6pt-anyh2r-fk-100k) | IDM wristIK robot state → MuJoCo FK **(ours)** |
| [`dpp-6pt-anyh2r-human-100k`](https://huggingface.co/RyanL22/dpp-6pt-anyh2r-human-100k) | HaWoR human hand (baseline) |
| [`dpp-6pt-anyh2r-retgt-100k`](https://huggingface.co/RyanL22/dpp-6pt-anyh2r-retgt-100k) | dex-retargeting → Inspire RH56F1 (baseline) |

**This model:** HaWoR human hand keypoints, used as-is (baseline)

## Files

| file | what |
|---|---|
| `100000.pt` | final snapshot (323 MB) — `state_dict`s + optimizer + normalisation stats |
| `train_config.yaml` / `train_overrides.yaml` | the exact resolved Hydra config of the run |
| `train_log.csv` | per-100-step training log |

## Training

Code: [`beomjun02/dex-point-policy`](https://github.com/beomjun02/dex-point-policy) @ `e972ba5`
(private). Init: `vitra_6points_final/snapshot/100000.pt` (VITRA video pretrain, 6points,
no contact head — the head is added at finetune and loaded with `strict=False`).

```
agent=dpp  suite.action_mode=6points  suite.history_len=1  num_queries=16
dataloader.bc_dataset.normalize=false          # raw metres, matches the pretrain run
dataloader.bc_dataset.hand=right
dataloader.bc_dataset.max_num_objects=4
teacher_forcing=true
use_contact_head=true  contact_loss_coef=1.0  contact_detach=true
uniform_contact_pos_weight=true                # scalar pos_weight 4.38
geom_drop_p=0  contact_drop_p=0  robot_noise_std=0
batch_size=64  lr=1e-4  steps=100000  finetune=true
```

1× GPU, ~55 min. Final training loss (mean over steps 95k–100k): **0.1083**.

> Training losses are **not** comparable across the three models — each fits a
> different target sequence. Use rollout success, not this number.

## Data

80 episodes (ball 22 / bottle 20 / box 22 / doll 16), 22,454 frames @ 20 fps, 95.6% kept.
Built from `dpp_ego3d_v1/openarm_inspire` (pt512 pack) for objects / contact / camera /
language, intersected with the anyh2r wristIK synthetic set
(`idm_v4form_rh56f1_vla_stereo-depth-wristIK_20260916`) so all three variants cover
exactly the same clips.

- Frame: `ego_right_optical_frame` (x right, y down, z forward), metres, static camera.
- Keypoints: MANO-21 indices `[0, 4, 8, 12, 16, 20]` = wrist, thumb, index, middle, ring, little tip.
- Objects: 128 points per slot, 2 slots (task object + white container).
- Contact: per-finger binary, thresholded at 0.3 on the HaWoR-tactile probability.

Verification of the FK path: our forward kinematics reproduces ego3d's own
`robot/*/hands21.npz` to **0.00 mm** (dohyeon calibrated ego extrinsic, rig in MJCF body
`pitch_motor_rotor_2`). Mean disagreement between the human and FK keypoints on the same
frames: wrist 36 mm, thumb 42, index 43, middle 46, ring 47, little 41 — note the wrist
points are different physical landmarks (MANO wrist joint vs RH56F1 palm site), so a
steady offset there is geometry, not error.

## License

Inherits the `dex-point-policy` / NVIDIA Source Code License terms of the upstream
`point_bridge` project. Research use.