Robotics
LeRobot
diffusion-policy
pusht
imitation-learning
zengxy0624 commited on
Commit
95143b0
·
verified ·
1 Parent(s): 2af637e

Add model card

Browse files
Files changed (1) hide show
  1. README.md +100 -0
README.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: lerobot
4
+ tags:
5
+ - diffusion-policy
6
+ - pusht
7
+ - imitation-learning
8
+ - robotics
9
+ base_model: lerobot/diffusion_pusht
10
+ datasets:
11
+ - zengxy0624/pusht_obstacles
12
+ ---
13
+
14
+ # pusht-obstacles-lerobot-finetune
15
+
16
+ Diffusion Policy checkpoints finetuned from
17
+ [`lerobot/diffusion_pusht`](https://huggingface.co/lerobot/diffusion_pusht) on a
18
+ custom 101-episode PushT-with-obstacles dataset (legacy obstacle preset:
19
+ circle radius=15, 1-3 obstacles per episode).
20
+
21
+ Source code:
22
+ [github.com/Zengxy0624/xarm7_collect](https://github.com/Zengxy0624/xarm7_collect)
23
+ (see `push_t/` and `push_t/doc/`).
24
+
25
+ ## Files
26
+
27
+ | File | Source run | Pipeline | Best success (n=50) | Best hit | Step |
28
+ |------|------------|----------|---------------------|----------|------|
29
+ | `lerobot_run3.5_best.pt` | jid2238141 | fixed | **0.62** | **0.12** | 44000 |
30
+ | `lerobot_run3.5_final.pt` | jid2238141 | fixed | 0.62 (final eval) | 0.12 | 50000 |
31
+ | `lerobot_run2_best.pt` | jid2235676 | broken (pre-fix) | 0.60 | 0.22 | 38000 |
32
+
33
+ The "fixed" pipeline applies three corrections matching LeRobot's official training
34
+ recipe (ImageNet image stats, padding-loss mask, `diffusers.get_scheduler('cosine')`)
35
+ that the original code accidentally bypassed by constructing `LeRobotDataset`
36
+ outside `make_dataset()`.
37
+
38
+ See [`push_t/doc/ft_jid2237070_2238141.md`](https://github.com/Zengxy0624/xarm7_collect/blob/main/push_t/doc/ft_jid2237070_2238141.md)
39
+ for the full eval timeline (25 evals across 50k training steps), phase analysis, and
40
+ A/B comparison vs the broken-pipeline run.
41
+
42
+ ## When to use which file
43
+
44
+ - **`lerobot_run3.5_best.pt`** — default; best success and lowest obstacle-hit rate.
45
+ - **`lerobot_run3.5_final.pt`** — only for resume training. Same model performance
46
+ as best.pt (62% at both step 44k and step 50k); carries end-of-cosine-schedule state.
47
+ - **`lerobot_run2_best.pt`** — A/B reference. Trained with the buggy pre-fix pipeline,
48
+ useful for ablations.
49
+
50
+ ## Loading
51
+
52
+ All ckpts contain numpy `stats` arrays, so PyTorch 2.6+ requires `weights_only=False`:
53
+
54
+ ```python
55
+ import torch
56
+ from huggingface_hub import hf_hub_download
57
+
58
+ ckpt_path = hf_hub_download(
59
+ repo_id="zengxy0624/pusht-obstacles-lerobot-finetune",
60
+ filename="lerobot_run3.5_best.pt",
61
+ )
62
+ ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)
63
+
64
+ print(ckpt.keys()) # ['model', 'model_cfg', 'stats', 'success']
65
+ print(ckpt["success"]) # 0.62
66
+ print(ckpt["model_cfg"]) # full DiffusionConfig dict (ResNet-18 + GroupNorm + 96x96)
67
+ ```
68
+
69
+ For full evaluation, use `push_t/test.py` from the source repo, which handles the
70
+ processor pipeline (ImageNet stats override, deterministic cudnn) correctly.
71
+
72
+ ## Training details
73
+
74
+ - **Base model:** `lerobot/diffusion_pusht` (ImageNet-pretrained ResNet-18 backbone,
75
+ GroupNorm, spatial-softmax 32 keypoints, U-Net 1D, DDPM 100 timesteps)
76
+ - **Architecture:** image_shape=[3, 96, 96], crop_shape=[84, 84], n_obs_steps=2,
77
+ horizon=16, n_action_steps=8, down_dims=[512, 1024, 2048]
78
+ - **Hyperparameters:** bs=64, lr=3e-5, warmup_steps=100, weight_decay=1e-6,
79
+ grad_clip=10.0, AMP fp16
80
+ - **Schedule:** linear warmup -> cosine decay over 50000 steps
81
+ - **Eval:** 50 episodes per checkpoint, every 2000 training steps
82
+ - **Hardware:** 1x H100 SXM5 on NCSA Delta AI
83
+
84
+ ## Limitations
85
+
86
+ - Eval n=50 gives Wilson 95% CI of approximately +/-14%. Reported success rates
87
+ should be treated as point estimates with this uncertainty.
88
+ - Trained only on the legacy circular-obstacle preset. Will not generalize to the
89
+ alternative `mixed` preset (rectangles, triangles, walls) without retraining.
90
+ - Both runs show val_loss divergence consistent with small-dataset overfitting; the
91
+ 101-episode dataset is the bottleneck, not the training stack.
92
+
93
+ ## Citation
94
+
95
+ If you use these checkpoints, please cite:
96
+
97
+ - The base Diffusion Policy paper: Cheng Chi et al. "Diffusion Policy: Visuomotor
98
+ Policy Learning via Action Diffusion" (RSS 2023, IJRR 2024).
99
+ - The LeRobot framework that produced the base ckpt and was used for these
100
+ experiments.