# Isaac-Reach-Franka PPO sweep: none of the 6 runs clearly learned

All 6 runs finished the full 300 iterations, one at a time, and none was stopped early. The best run by final mean reward is **lr=1e-3, seed 1 (final +0.02)**. But it's the only run of the six whose reward ended above where it started, and that's weak evidence for 1e-3 as a setting.

| lr | seed | job_id | status | iterations | final reward | best reward | steps/s | stopped early? | checkpoint |
|---|---|---|---|---|---|---|---|---|---|
| 3e-4 | 1 | isaaclab-20260929-045345-edcec0d28a0b | success | 300/300 | -0.35 | -0.25 (iteration 0) | 117,943 | no | `3e-4_s1/logs/rsl_rl/reach_franka/2026-09-29_04-53-49_…edcec0d28a0b/model_299.pt` |
| 3e-4 | 2 | isaaclab-20260929-045905-b06eca8dd857 | success | 300/300 | -0.29 | -0.11 | 120,219 | no | `3e-4_s2/logs/rsl_rl/reach_franka/2026-09-29_04-59-09_…b06eca8dd857/model_299.pt` |
| **1e-3** | **1** | **isaaclab-20260929-050417-05a8c9c0dbae** | success | 300/300 | **+0.02** | **+0.02** | 119,700 | no | `1e-3_s1/logs/rsl_rl/reach_franka/2026-09-29_05-04-21_…05a8c9c0dbae/model_299.pt` |
| 1e-3 | 2 | isaaclab-20260929-050939-116af3a5a1bb | success | 300/300 | -0.51 | -0.24 (iteration 0) | 115,623 | no | `1e-3_s2/logs/rsl_rl/reach_franka/2026-09-29_05-09-43_…116af3a5a1bb/model_299.pt` |
| 3e-3 | 1 | isaaclab-20260929-051452-41a69ccff38b | success | 300/300 | -0.34 | -0.18 | 209,099 | no | `3e-3_s1/logs/rsl_rl/reach_franka/2026-09-29_05-14-56_…41a69ccff38b/model_299.pt` |
| 3e-3 | 2 | isaaclab-20260929-051847-e67c06646018 | success | 300/300 | -0.47 | -0.14 | 206,862 | no | `3e-3_s2/logs/rsl_rl/reach_franka/2026-09-29_05-18-51_…e67c06646018/model_299.pt` |

Checkpoint paths are relative to `/home/ubuntu/strands-robots-isaac-test/isaaclab-extreme/runs/c3_agent_sweep/`.

- **Why nothing was stopped early:** the gap to the leader was never bigger than the spread between seeds. The two 1e-3 seeds differ by 0.53 in final reward, which is larger than any gap between learning rates. So no learning rate was "clearly worse", and all seeds ran to the end.
- **Rewards mostly went down:** in 5 of 6 runs the final reward is below the first-iteration reward (about -0.25). The status tool marked `learning: False` for all of them.
  - Two runs' "best reward" is just their first iteration.
  - Average final reward per learning rate: 3e-4 = -0.32, 1e-3 = -0.245, 3e-3 = -0.405. These are within seed noise of each other.
- **Speed:** steps/s for the 3e-3 runs aren't comparable to the others. Another process was using the GPU (about 7.5 GB, 55–67% utilisation) during the first four runs and was gone by the fifth, so those runs were slower. The 1-run limit and the 8 GB free rule were kept at every launch.

**Recommendation:**
1. Use 1e-3 / seed 1 (`…05a8c9c0dbae/model_299.pt`) only as a starting point. This sweep can't tell the three learning rates apart.
2. Before retuning, find out why rewards fell over 300 iterations (my guess: a penalty that ramps up during training, or runs that are just too short). Try one long run (1000+ iterations) at 1e-3 to see if it keeps improving.
3. Then repeat the sweep with at least 3–5 seeds per learning rate, since the seed-to-seed differences here are bigger than the differences between learning rates.