| --- |
| license: other |
| library_name: rsl-rl |
| tags: |
| - reinforcement-learning |
| - robotics |
| - humanoid |
| - locomotion |
| - isaac-lab |
| --- |
| |
| # Humanoid V1 — Velocity Locomotion Policy |
|
|
| A velocity-tracking locomotion policy for the ETHRC Humanoid V1 robot, trained in |
| [Isaac Lab](https://github.com/isaac-sim/IsaacLab) with [rsl-rl](https://github.com/leggedrobotics/rsl_rl) |
| (PPO) on the `ETHRC-humanoidv1-v0` task. The policy maps proprioceptive observations plus a target |
| base-velocity command to joint position targets, and tracks that command while keeping the robot balanced. |
|
|
| ## Velocity command |
|
|
| The policy is driven by a 3-D `base_velocity` command `[lin_vel_x, lin_vel_y, ang_vel_z]`: |
|
|
| | Component | Controls | Sign | |
| | --- | --- | --- | |
| | `lin_vel_x` | lateral (strafe) | `+` = right, `−` = left | |
| | `lin_vel_y` | forward / backward | `+` = forward, `−` = backward | |
| | `ang_vel_z` | yaw (turn) | `+` = turn left | |
|
|
| Linear components are in m/s, angular in rad/s. |
|
|
| ## Capabilities |
|
|
| Measured as the mean over 300 steps under fixed commands: |
|
|
| | Command | Achieved | Tracking | |
| | --- | --- | --- | |
| | Forward (`lin_vel_y = 0.5`) | ~0.36 m/s | ~71% | |
| | Sidestep (`lin_vel_x = ±0.5`) | ~0.36 m/s | ~71% | |
| | Turn (`ang_vel_z = 0.5`) | 0.34 rad/s | 68% | |
| | Diagonal (`lin_vel_y = 0.4`, `lin_vel_x = 0.3`) | 0.33 / 0.21 m/s | 70–82% | |
| | Top speed (`lin_vel_y = 1.0`) | 0.71 m/s | 71% | |
|
|
| The policy is omnidirectional — forward, backward, lateral, turning, and combined commands |
| (translation + yaw together) all work, and it stays upright under disturbances. Tracking undershoots |
| roughly uniformly (~70%), so a practical rule is to **command ≈ 1.4× the target speed** to reach it. |
| Sustained top forward speed is about **0.7 m/s**. |
|
|
| Example rollouts for each command are in [`videos/`](./videos). |
|
|
| ## Files |
|
|
| | Path | Description | |
| | --- | --- | |
| | `model_3999.pt` | rsl-rl checkpoint (final, iteration 3999) | |
| | `exported/policy.pt` | TorchScript policy (deployment) | |
| | `exported/policy.onnx` (+ `.data`) | ONNX policy (deployment) | |
| | `params/agent.yaml`, `params/env.yaml` | training / environment configs | |
| | `videos/` | evaluation clips per command | |
|
|
| ## Usage |
|
|
| Evaluate or record a rollout with Isaac Lab (requires rsl-rl ≥ 5.0): |
|
|
| ```bash |
| cd <rc_humanoid_rl_lab> |
| isaaclab.sh -p scripts/rsl_rl/play.py \ |
| --task=ETHRC-humanoidv1-v0 --num_envs=4 --headless --video --video_length=300 \ |
| --checkpoint model_3999.pt \ |
| env.commands.base_velocity.ranges.lin_vel_y="[0.5, 0.5]" \ |
| env.commands.base_velocity.ranges.lin_vel_x="[0.0, 0.0]" \ |
| env.commands.base_velocity.ranges.ang_vel_z="[0.0, 0.0]" |
| ``` |
|
|
| Set `lin_vel_y` for forward/backward, `lin_vel_x` for strafing, and `ang_vel_z` for turning. |
| For deployment, load `exported/policy.pt` (TorchScript) or `exported/policy.onnx` (ONNX) — both take |
| the observation vector and return joint position targets. |
|
|
| ## Training |
|
|
| - **Framework:** Isaac Lab + rsl-rl (PPO) |
| - **Task:** `ETHRC-humanoidv1-v0` — velocity-tracking locomotion |
| - **Observations:** proprioception + base-velocity command |
| - **Actions:** joint position targets |
| - **Curriculum:** `lin_vel_cmd_levels` progressively widens the commanded velocity range as the |
| `track_lin_vel_xy` reward improves, so the policy is trained on harder commands only once it tracks |
| the easier ones. |
|
|
| Exact hyperparameters and reward terms are in `params/agent.yaml` and `params/env.yaml`. |
|
|
