Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- starvla
|
| 5 |
+
- vla
|
| 6 |
+
- robotics
|
| 7 |
+
- unitree-g1
|
| 8 |
+
- wholebody
|
| 9 |
+
- qwen-oft
|
| 10 |
+
- sonic
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# StarVLA QwenOFT — Unitree G1 WholeBody (SONIC 78D action)
|
| 14 |
+
|
| 15 |
+
Smoke checkpoint for the StarVLA `examples/realRobots/UnitreeG1_WholeBody` workflow.
|
| 16 |
+
|
| 17 |
+
## What this is
|
| 18 |
+
- **Model**: QwenOFT — Qwen3.5-0.8B backbone + L1-regression MLP action head
|
| 19 |
+
- **Action contract**: SONIC-style **78D** = 64D `motion_token` + 7D `left_hand_joints` + 7D `right_hand_joints`
|
| 20 |
+
- **State**: **72D** proprioception = 43D `observation.state` + 14D `observation.eef_state` + 4D root_orientation + 3D projected_gravity + 4D cpp_rotation_offset + 4D init_base_quat
|
| 21 |
+
- **Normalization**: q99 (`dataset_statistics.json`; unnorm_key = `new_embodiment`)
|
| 22 |
+
- **Image**: single ego view, 224×224
|
| 23 |
+
- **Horizon**: 8
|
| 24 |
+
|
| 25 |
+
## Scope / caveats
|
| 26 |
+
- **1000-step SMOKE run** — not converged (final L1 loss ≈ 0.60). Do not expect task-level performance.
|
| 27 |
+
- Trained on the public `cloudwalk-research/gr00t-g1-grab-bottle-right-hand-v11` dataset (355 episodes), **not** the private `test_sonic` example data.
|
| 28 |
+
- Validates the data → train → policy-server → 78D-action-split engineering chain. It does **not** imply real-robot closed-loop control or task success (no physical G1 / SONIC / WBC used).
|
| 29 |
+
|
| 30 |
+
## Files
|
| 31 |
+
- `checkpoints/steps_1000_pytorch_model.pt` — StarVLA checkpoint (2.24 GB)
|
| 32 |
+
- `config.yaml` / `config.full.yaml` — model + training config
|
| 33 |
+
- `dataset_statistics.json` — q99 normalization stats (required for serving)
|
| 34 |
+
- `summary.jsonl` — training checkpoint-save log
|
| 35 |
+
|
| 36 |
+
## Serving (StarVLA)
|
| 37 |
+
Requires the StarVLA repo and the Qwen3.5-0.8B backbone (`base_vlm` in `config.yaml`):
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
cd starVLA
|
| 41 |
+
export PYTHONPATH=$PWD:${PYTHONPATH}
|
| 42 |
+
CUDA_VISIBLE_DEVICES=0 python deployment/model_server/server_policy.py \
|
| 43 |
+
--ckpt_path <this_repo>/checkpoints/steps_1000_pytorch_model.pt \
|
| 44 |
+
--port 5694 \
|
| 45 |
+
--use_bf16
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
## Reproduction
|
| 49 |
+
Training config / scripts live in the StarVLA example:
|
| 50 |
+
`examples/realRobots/UnitreeG1_WholeBody/step2_training/train_files/`.
|
| 51 |
+
|
| 52 |
+
- Dataset: `cloudwalk-research/gr00t-g1-grab-bottle-right-hand-v11` (LeRobot v2.1)
|
| 53 |
+
- Backbone: `Qwen/Qwen3.5-0.8B`
|
| 54 |
+
- `dataset_statistics.json` top-level key `new_embodiment` = `EmbodimentTag.NEW_EMBODIMENT`
|