---
license: mit
library_name: onnx
pipeline_tag: reinforcement-learning
tags:
- strands-arm
- so-101
- nicla-vision
- mujoco
- ppo
- onnx
- robotics
- cameraman
---

# strands-arm `frame`: the whole leader arm keeps a subject framed

Six servo deltas per 0.1 s (ids 1..6, clipped +-5 deg) from what the Nicla reports (FOMO box centre,
size, visible, ToF), the six servo readings, the last command and a shot spec (where in the frame,
how big). Trained with PPO in a
MuJoCo twin of this arm (SO-101 leader + printed head, lens as a camera at the firmware FOV, identified
STS3215 servo model, calibrated limits) with occlusion rays, one-frame detector delay, pixel noise and
randomized FOV.
Baseline: the head-only P step `watch` uses today, arm frozen at the fold.

![frame](media/frame.gif)

Top: head-only P. Bottom: this policy, same subject. Left = the Nicla's view (green = shot spec, yellow =
detection), right = third person. [MP4](media/frame.mp4).

## Eval (`rl/eval_frame.py`, 4 seeds x 64 envs x 15 s, 256 episodes)

| controller | visible | framed | centre err | size err | contacts/step | travel deg/step |
|---|---|---|---|---|---|---|
| zero action | 0.29 | 0.00 | 0.945 | 0.191 | 0.020 | 0.10 |
| head-only P (watch.py) | 0.39 | 0.01 | 0.660 | 0.211 | 0.846 | 2.06 |
| **this policy** | 0.69 | 0.20 | 0.198 | 0.184 | 0.036 | 5.44 |
| this policy, arm frozen | 0.64 | 0.03 | 0.396 | 0.192 | 1.670 | 1.39 |

**PASS: keeps the subject visible more often than the head-only P step.** framed = centre error < 0.15 and size error < 0.1. The arm-frozen row is the same
network with servos 1..4 held: the difference is what the arm buys.

## Contract

Input `obs` [1, 30] float32, output `act` [1, 6] float32 = servo deg per 0.1 s step, ids 1..6.

| i | obs |
|---|---|
| 0 | `s1_n` |
| 1 | `s2_n` |
| 2 | `s3_n` |
| 3 | `s4_n` |
| 4 | `s5_n` |
| 5 | `s6_n` |
| 6 | `g1_n` |
| 7 | `g2_n` |
| 8 | `g3_n` |
| 9 | `g4_n` |
| 10 | `g5_n` |
| 11 | `g6_n` |
| 12 | `ex` |
| 13 | `ey` |
| 14 | `size` |
| 15 | `vis` |
| 16 | `last_ex` |
| 17 | `last_ey` |
| 18 | `last_size` |
| 19 | `age_n` |
| 20 | `ex_star` |
| 21 | `ey_star` |
| 22 | `size_star` |
| 23 | `tof_n` |
| 24 | `a1_prev` |
| 25 | `a2_prev` |
| 26 | `a3_prev` |
| 27 | `a4_prev` |
| 28 | `a5_prev` |
| 29 | `a6_prev` |

Runner: `strands_arm.policy.FramePolicy` (image ey flipped to the sim's up-positive frame, size = box
height / half frame, ToF mm -> m, targets clamped through `kine.clamp_pose`). **Never run on the real
arm yet**; shadow mode (real pose, detections and ToF, nothing written) is logged in the repo's RL.md.
Source: github.com/cagataycali/strands-arm `e6d5d31`, `rl/envs/frame_env.py`, `rl/sim/`. PPO: {"n_envs": 256, "steps_per_env": 50, "epochs": 4, "minibatches": 8, "lr": 0.0003, "gamma": 0.98, "lam": 0.95, "clip": 0.2, "ent_coef": 0.002, "vf_coef": 0.5, "max_grad_norm": 0.5, "hidden": 128, "act_scale": 5.0, "act_dim": 6, "total_updates": 2500, "device": "cpu", "seed": 0}
Env overrides: {"w_act": 0.1, "w_jerk": 0.1}