Add r16_latency_g200 (report-16 one-change-at-a-time, measured gait in physical units)

r16_latency_g200/README.md

@@ -0,0 +1,72 @@
+# r16_latency_g200 — Report 16 one-change-at-a-time walker
+
+**Measured:  the gains are free.:0.342 m/s — ✅ walks (= arm A)**
+
+A.2 — control + ~65 ms latency + leg gains 180/5→200/10. Paired with arm A, it isolates the gain change from the latency
+
+| | |
+|---|---|
+| Task | `ETHRC-humanoidv1-walk-r16-latency-g200-psu-v0` |
+| Branch | `test/SjohnU_remodel_actuators` @ `8f9f058` |
+| Checkpoint | `model_9999.pt` (10,000 iters, completed) |
+| Envs / seed | 4096 / 42 |
+| rsl-rl | 5.0.1 (native actor/critic state_dict) |
+| Actions | 26 @ 50 Hz |
+| wandb | [`k28tg52k`](https://wandb.ai/sjohnu-eth-z-rich/rc_humanoid_rl/runs/k28tg52k) |
+| Hardware | AWS us-east-2, 1x L40S (g6e.xlarge) |
+## Report 16: one-change-at-a-time, measured in physical units
+
+All arms = `u4_walk_crouch_real` + the r16 shared layer (25% standing envs, `track_lin_vel_xy`
+1.5 -> 2.2) + **exactly one** further change. 10k iters, 4096 envs, seed 42, branch
+`test/SjohnU_remodel_actuators` @ `8f9f058`.
+
+Measured with `scripts/rsl_rl/measure_gait.py` (1000 steps x 64 envs, forward command forced).
+**Physical units only.** Reward-term values are NOT comparable across configs with different
+weight/std, and they actively mislead here: `r16_footlift` posts the best curves of the entire
+set (tvel 1.194, mean_reward 11.79) while moving **0.001 m/s**.
+
+| policy | change | cmd | **fwd m/s** | cadence | lean | walks? |
+|---|---|---|---|---|---|---|
+| `r16_upright` @0.8 | upright price + spawn-tilt DR | 0.8 | **0.634** | 3.30/s | 1.5° | ✅ best |
+| `crouch_nogait` | (ref) | 0.5 | 0.572 | 8.14/s | 5.3° | ✅ |
+| `r16_upright` @0.5 | upright price + spawn-tilt DR | 0.5 | 0.406 | 3.37/s | 0.7° | ✅ |
+| `u4_walk_crouch_real` | (ref, 22k) | 0.5 | 0.397 | 3.48/s | 9.9° | ✅ |
+| **`r16_latency`** | +~65 ms delay | 0.5 | **0.347** | 3.39/s | 13.3° | ✅ |
+| **`r16_latency_g200`** | +~65 ms + legs 200/10 | 0.5 | **0.342** | 3.47/s | 10.7° | ✅ |
+| **`r16_control`** | — (baseline) | 0.5 | **0.026** | 0.24/s | 9.2° | ❌ |
+| `u4_walk_bigstep_v1` | (ref, 4 changes, 25k) | 0.5 | 0.009 | 1.71/s | 20.9° | ❌ |
+| **`r16_footlift`** | clearance 1.0@0.15 -> 2.4@0.17 | 0.5 | **0.001** | 0.28/s | 11.9° | ❌ worst |
+
+## The rule: standing must be made untenable
+
+The policy's default attractor is a **stand-and-micro-correct local optimum** — that is what
+`r16_control` is stuck in at 10k (0.026 m/s, 0.24 steps/env/s). Every result follows from
+whether a change makes standing tenable or not:
+
+- **Makes standing untenable -> a real gait emerges:** `latency` (cannot twitch-correct at
+  50 Hz -> 0.347), spawn-tilt DR (must recover every reset -> 0.406), or simply training to
+  22k (`crouch_real` -> 0.397; it only learns to translate between 15k and 22k).
+- **Enriches standing -> it never walks:** `footlift` (0.001, worse than its own control) and
+  a slower gait clock (0.004).
+
+**Latency is load-bearing, not a deploy tax.** `control + latency` walks at 0.347 m/s where the
+identical `control` marches at 0.026 -- same code, same 10k iters, one knob, 14x the stepping.
+Not a confound: the crouch-envelope chain already swaps `legs_waist` to `IdealPDActuatorCfg` and
+moves torque-speed into the PhysX spawn func; `_apply_latency_only` turns that into a DelayedPD
+with identical effort/velocity limits, gains and armature, and never touches `spawn.func`.
+
+**Leg gains 180/5 -> 200/10 are free** (0.342 vs 0.347).
+
+**Foot-lift is what broke `bigstep`** -- bigstep carried exactly this layer. It perversely reached
+*lower* clearance than control (0.1003 vs 0.1567 m) while paying 2.4x for height: it did not lift
+higher, it stopped stepping.
+
+**Lean is not causal.** `r16_latency` walks at 13.3° BACK, `r16_upright` at 0.7°, `bigstep` fails
+at 20.9°. Lean predicts nothing -- though *pricing* it still helps, via the spawn-tilt/untenable
+-standing mechanism rather than via the lean itself.
+
+## Files
+
+- `policy.pt`, `policy.onnx` — exported via `scripts/rsl_rl/play_export.py`
+- `deploy.yaml` — deploy config (obs/action layout, gains, 50 Hz)
+- `model_9999.pt` — raw rsl-rl checkpoint (re-export / resume)

r16_latency_g200/deploy.yaml

@@ -0,0 +1,425 @@
+experiment_name: r16_latency_g200
+task: ETHRC-humanoidv1-walk-r16-latency-g200-psu-v0
+interface: lo
+sim2sim: true
+msg_type: go
+imu_type: base_link
+lowcmd_topic: rt/lowcmd
+lowstate_topic: rt/lowstate
+policy_path: policy.pt
+num_actions: 26
+action_scale: 0.25
+use_default_offset: true
+control_dt: 0.02
+policy_rate_hz: 50.0
+decimation: 4
+sim_dt: 0.005
+joint_names:
+- waist_upperbody_yaw
+- hip_central_rotation
+- neck_yaw
+- shoulder_flexion_l
+- shoulder_flexion_r
+- leg_flexion_l
+- leg_flexion_r
+- neck_pitch
+- shoulder_abduction_l
+- shoulder_abduction_r
+- leg_abduction_l
+- leg_abduction_r
+- shoulder_rotation_l
+- shoulder_rotation_r
+- hip_yaw_l
+- hip_yaw_r
+- elbow_flexion_l
+- elbow_flexion_r
+- knee_l
+- knee_r
+- wrist_rotation_l
+- wrist_rotation_r
+- foot_pitch_l
+- foot_pitch_r
+- foot_roll_l
+- foot_roll_r
+joint2motor_idx:
+- 0
+- 1
+- 2
+- 3
+- 4
+- 5
+- 6
+- 7
+- 8
+- 9
+- 10
+- 11
+- 12
+- 13
+- 14
+- 15
+- 16
+- 17
+- 18
+- 19
+- 20
+- 21
+- 22
+- 23
+- 24
+- 25
+default_angles:
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.18000000715255737
+- -0.18000000715255737
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- -0.3499999940395355
+- -0.3499999940395355
+- 0.0
+- 0.0
+- 0.18000000715255737
+- 0.18000000715255737
+- 0.0
+- 0.0
+kps:
+- 189.63851928710938
+- 189.63851928710938
+- 40.2313117980957
+- 68.69254302978516
+- 68.69254302978516
+- 189.63851928710938
+- 189.63851928710938
+- 40.2313117980957
+- 68.69254302978516
+- 68.69254302978516
+- 189.63851928710938
+- 189.63851928710938
+- 40.2313117980957
+- 40.2313117980957
+- 189.63851928710938
+- 189.63851928710938
+- 40.2313117980957
+- 40.2313117980957
+- 189.63851928710938
+- 189.63851928710938
+- 40.2313117980957
+- 40.2313117980957
+- 35.159873962402344
+- 35.159873962402344
+- 35.159873962402344
+- 35.159873962402344
+kds:
+- 9.641944885253906
+- 9.641944885253906
+- 0.9752398133277893
+- 1.6067781448364258
+- 1.6067781448364258
+- 9.641944885253906
+- 9.641944885253906
+- 0.9752398133277893
+- 1.6067781448364258
+- 1.6067781448364258
+- 9.641944885253906
+- 9.641944885253906
+- 0.9752398133277893
+- 0.9752398133277893
+- 9.641944885253906
+- 9.641944885253906
+- 0.9752398133277893
+- 0.9752398133277893
+- 9.641944885253906
+- 9.641944885253906
+- 0.9752398133277893
+- 0.9752398133277893
+- 1.054377555847168
+- 1.054377555847168
+- 1.054377555847168
+- 1.054377555847168
+effort_limits:
+- 128.0
+- 128.0
+- 40.0
+- 80.0
+- 80.0
+- 128.0
+- 128.0
+- 40.0
+- 80.0
+- 80.0
+- 128.0
+- 128.0
+- 40.0
+- 40.0
+- 128.0
+- 128.0
+- 40.0
+- 40.0
+- 128.0
+- 128.0
+- 40.0
+- 40.0
+- 60.0
+- 60.0
+- 60.0
+- 60.0
+articulation_joint_names:
+- waist_upperbody_yaw
+- hip_central_rotation
+- neck_yaw
+- shoulder_flexion_l
+- shoulder_flexion_r
+- leg_flexion_l
+- leg_flexion_r
+- neck_pitch
+- shoulder_abduction_l
+- shoulder_abduction_r
+- leg_abduction_l
+- leg_abduction_r
+- shoulder_rotation_l
+- shoulder_rotation_r
+- hip_yaw_l
+- hip_yaw_r
+- elbow_flexion_l
+- elbow_flexion_r
+- knee_l
+- knee_r
+- wrist_rotation_l
+- wrist_rotation_r
+- foot_pitch_l
+- foot_pitch_r
+- foot_roll_l
+- foot_roll_r
+articulation_default_angles:
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.18000000715255737
+- -0.18000000715255737
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- 0.0
+- -0.3499999940395355
+- -0.3499999940395355
+- 0.0
+- 0.0
+- 0.18000000715255737
+- 0.18000000715255737
+- 0.0
+- 0.0
+pd_per_joint:
+  waist_upperbody_yaw:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  hip_central_rotation:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  leg_flexion_l:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  leg_flexion_r:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  leg_abduction_l:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  leg_abduction_r:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  hip_yaw_l:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  hip_yaw_r:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  knee_l:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  knee_r:
+    kp: 189.63851928710938
+    kd: 9.641944885253906
+    effort_limit: 128.0
+    velocity_limit: 14.65999984741211
+  shoulder_flexion_l:
+    kp: 68.69254302978516
+    kd: 1.6067781448364258
+    effort_limit: 80.0
+    velocity_limit: 18.0
+  shoulder_flexion_r:
+    kp: 68.69254302978516
+    kd: 1.6067781448364258
+    effort_limit: 80.0
+    velocity_limit: 18.0
+  shoulder_abduction_l:
+    kp: 68.69254302978516
+    kd: 1.6067781448364258
+    effort_limit: 80.0
+    velocity_limit: 18.0
+  shoulder_abduction_r:
+    kp: 68.69254302978516
+    kd: 1.6067781448364258
+    effort_limit: 80.0
+    velocity_limit: 18.0
+  neck_yaw:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  neck_pitch:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  shoulder_rotation_l:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  shoulder_rotation_r:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  elbow_flexion_l:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  elbow_flexion_r:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  wrist_rotation_l:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  wrist_rotation_r:
+    kp: 40.2313117980957
+    kd: 0.9752398133277893
+    effort_limit: 40.0
+    velocity_limit: 20.0
+  foot_pitch_l:
+    kp: 35.159873962402344
+    kd: 1.054377555847168
+    effort_limit: 60.0
+    velocity_limit: 50.0
+  foot_pitch_r:
+    kp: 35.159873962402344
+    kd: 1.054377555847168
+    effort_limit: 60.0
+    velocity_limit: 50.0
+  foot_roll_l:
+    kp: 35.159873962402344
+    kd: 1.054377555847168
+    effort_limit: 60.0
+    velocity_limit: 50.0
+  foot_roll_r:
+    kp: 35.159873962402344
+    kd: 1.054377555847168
+    effort_limit: 60.0
+    velocity_limit: 50.0
+obs_history_length: 5
+obs_single_frame_dim: 89
+obs_total_dim: 445
+obs_layout:
+- name: base_ang_vel
+  start: 0
+  end: 15
+  base_dim: 3
+  block_dim: 15
+  scale: 0.2
+- name: projected_gravity
+  start: 15
+  end: 30
+  base_dim: 3
+  block_dim: 15
+  scale: 1.0
+- name: velocity_commands
+  start: 30
+  end: 45
+  base_dim: 3
+  block_dim: 15
+  scale: 1.0
+  params:
+    command_name: base_velocity
+- name: joint_pos_rel
+  start: 45
+  end: 175
+  base_dim: 26
+  block_dim: 130
+  scale: 1.0
+- name: joint_vel_rel
+  start: 175
+  end: 305
+  base_dim: 26
+  block_dim: 130
+  scale: 0.05
+- name: last_action
+  start: 305
+  end: 435
+  base_dim: 26
+  block_dim: 130
+  scale: 1.0
+- name: gait_phase
+  start: 435
+  end: 445
+  base_dim: 2
+  block_dim: 10
+  scale: 1.0
+  params:
+    period: 0.6
+    command_name: base_velocity
+command_ranges:
+  lin_vel_x:
+  - -0.1
+  - 0.1
+  lin_vel_y:
+  - 0.1
+  - 0.8
+  ang_vel_z:
+  - -0.1
+  - 0.1
+checkpoint: /root/rc/logs/rsl_rl/humanoid_v1_locomotion/r16_latency_g200/model_9999.pt

r16_latency_g200/model_9999.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7374076ba3b6ccc8b3f7cca25f72e1bc2a610f435cc66464770c1ee11d6deea3
+size 7475573

r16_latency_g200/policy.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56654a3899b887d2a0c9da8c8a994eae7b11979c75422f186c84aa2e21fe0b03
+size 1584867

r16_latency_g200/policy.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edd4b8a84bc3ceed23bfe6da1f5b1c15e6ca804b3f72f1cf08d90a875e9f6131
+size 1596822