yxma's picture
rebuild testset/ from the probe package
404666a verified
Raw
History Blame
14.8 kB
<!doctype html><html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>React probe test set</title><style>
:root{--bg:#0b1020;--fg:#e8eefb;--dim:#8ea0c2;--line:#1e2a45;--card:#111a2e;--accent:#ffc46b}
*{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--fg);
font:16px/1.6 'IBM Plex Sans',system-ui,sans-serif}
.wrap{max-width:1400px;margin:0 auto;padding:28px 20px 70px}
h1{font-size:28px;margin:0 0 8px}h2{font-size:16px;color:var(--accent);margin:30px 0 8px}
h2 small{color:var(--dim);font-weight:400}
p{color:var(--dim);max-width:78ch}code{background:#0d1526;padding:1px 5px;border-radius:4px}
a{color:var(--accent)}
.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(170px,1fr));gap:10px;margin:20px 0}
.card{background:var(--card);border:1px solid var(--line);border-radius:10px;padding:12px 14px}
.card b{display:block;font-size:22px;color:var(--accent);font-weight:650}
.card span{color:var(--dim);font-size:12px}
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(300px,1fr));gap:10px}
figure{margin:0;background:var(--card);border:1px solid var(--line);border-radius:9px;padding:7px}
img{width:100%;border-radius:5px;display:block}
figcaption{color:var(--dim);font-size:12px;margin-top:5px}
.hero{background:var(--card);border:1px solid var(--line);border-radius:12px;padding:10px;margin:18px 0}
</style></head><body><div class="wrap">
<h1>React probe test set</h1>
<p>72 commanded action sequences over 6 start frames, for scoring a tactile
world model's rollouts against ground truth that is <b>geometric, not photometric</b>.
Nobody performed these motions, so there is no ground-truth future image &mdash; what
is ground truth is where the sensor <i>would be</i>, and its projection into each
camera. Full method and usage in the
<a href="README.md">README</a>; per-probe clips are under
<a href="../probes/index.html">/probes/</a>.</p>
<div class="hero"><img src="overlay_example.jpg">
<figcaption style="color:var(--dim);font-size:13px;margin-top:8px">Yellow: commanded
ground truth. Red: a deliberately wrong rollout, offset 25&nbsp;mm in world x &mdash;
it reads 18&ndash;19&nbsp;px against a ~6&nbsp;px noise floor. Dimmed: the hand that
must stay still.</figcaption></div>
<div class="cards"><div class='card'><b>72</b><span>probes &middot; 6 start frames</span></div><div class='card'><b>0.11&ndash;0.38 m</b><span>translation &middot; 23&ndash;90&deg; rotation</span></div><div class='card'><b>p40&ndash;p79</b><span>speed vs the dataset</span></div><div class='card'><b>~6 px</b><span>worst reprojection noise floor</span></div><div class='card'><b>0.133 m</b><span>closest the hands come (rule 0.12)</span></div></div>
<p>Start frames are drawn from <b>held-out intervals</b> of <code>splits.json</code> &mdash; never training frames. Eligible sessions: 2026-05-10, 2026-05-11, 2026-05-19. This draw happened to select 2026-05-10, 2026-05-11; 2026-05-19 was not sampled this time, which is chance, not a judgement about it. <b>2026-05-19</b> had its OptiTrack world redefined mid-collection; the release applies a translation-only correction and the residual yaw about the table normal is <b>unmeasured</b> (about 16 px at the workspace). It is included with that stated in <code>world_residual</code> rather than dropped &mdash; a bounded, declared error is not a reason to discard a fifth of the sessions.</p>
<h2>run0 &mdash; 2026-05-11/episode_007, rows 1322&ndash;1325 &nbsp;<small>moving right, holding left</small></h2><div class='grid'><figure><img loading='lazy' src='overlays/run0_trans+x.jpg'><figcaption>trans+x &middot; 0.1087m &middot; 1.53s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run0_trans-x.jpg'><figcaption>trans-x &middot; 0.2134m &middot; 2.70s &middot; p48</figcaption></figure><figure><img loading='lazy' src='overlays/run0_trans+y.jpg'><figcaption>trans+y &middot; 0.1908m &middot; 1.67s &middot; p60</figcaption></figure><figure><img loading='lazy' src='overlays/run0_trans-y.jpg'><figcaption>trans-y &middot; 0.1736m &middot; 1.70s &middot; p56</figcaption></figure><figure><img loading='lazy' src='overlays/run0_trans+z.jpg'><figcaption>trans+z &middot; 0.3653m &middot; 1.70s &middot; p79</figcaption></figure><figure><img loading='lazy' src='overlays/run0_trans-z.jpg'><figcaption>trans-z &middot; 0.3543m &middot; 2.13s &middot; p71</figcaption></figure><figure><img loading='lazy' src='overlays/run0_rot+x.jpg'><figcaption>rot+x &middot; 63.3655&deg; &middot; 2.60s &middot; p55</figcaption></figure><figure><img loading='lazy' src='overlays/run0_rot-x.jpg'><figcaption>rot-x &middot; 25.1764&deg; &middot; 1.50s &middot; p43</figcaption></figure><figure><img loading='lazy' src='overlays/run0_rot+y.jpg'><figcaption>rot+y &middot; 71.2313&deg; &middot; 3.93s &middot; p45</figcaption></figure><figure><img loading='lazy' src='overlays/run0_rot-y.jpg'><figcaption>rot-y &middot; 33.3475&deg; &middot; 1.73s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run0_rot+z.jpg'><figcaption>rot+z &middot; 31.9447&deg; &middot; 1.50s &middot; p50</figcaption></figure><figure><img loading='lazy' src='overlays/run0_rot-z.jpg'><figcaption>rot-z &middot; 51.4259&deg; &middot; 1.53s &middot; p66</figcaption></figure></div><h2>run1 &mdash; 2026-05-10/episode_009, rows 760&ndash;763 &nbsp;<small>moving left, holding right</small></h2><div class='grid'><figure><img loading='lazy' src='overlays/run1_trans+x.jpg'><figcaption>trans+x &middot; 0.1916m &middot; 2.60s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run1_trans-x.jpg'><figcaption>trans-x &middot; 0.1404m &middot; 1.57s &middot; p52</figcaption></figure><figure><img loading='lazy' src='overlays/run1_trans+y.jpg'><figcaption>trans+y &middot; 0.1078m &middot; 1.53s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run1_trans-y.jpg'><figcaption>trans-y &middot; 0.3249m &middot; 1.70s &middot; p76</figcaption></figure><figure><img loading='lazy' src='overlays/run1_trans+z.jpg'><figcaption>trans+z &middot; 0.3773m &middot; 3.37s &middot; p59</figcaption></figure><figure><img loading='lazy' src='overlays/run1_trans-z.jpg'><figcaption>trans-z &middot; 0.23m &middot; 3.00s &middot; p48</figcaption></figure><figure><img loading='lazy' src='overlays/run1_rot+x.jpg'><figcaption>rot+x &middot; 79.3507&deg; &middot; 4.03s &middot; p48</figcaption></figure><figure><img loading='lazy' src='overlays/run1_rot-x.jpg'><figcaption>rot-x &middot; 60.6016&deg; &middot; 2.80s &middot; p51</figcaption></figure><figure><img loading='lazy' src='overlays/run1_rot+y.jpg'><figcaption>rot+y &middot; 33.8293&deg; &middot; 1.87s &middot; p45</figcaption></figure><figure><img loading='lazy' src='overlays/run1_rot-y.jpg'><figcaption>rot-y &middot; 46.4903&deg; &middot; 1.77s &middot; p58</figcaption></figure><figure><img loading='lazy' src='overlays/run1_rot+z.jpg'><figcaption>rot+z &middot; 78.7558&deg; &middot; 1.77s &middot; p75</figcaption></figure><figure><img loading='lazy' src='overlays/run1_rot-z.jpg'><figcaption>rot-z &middot; 52.0133&deg; &middot; 2.10s &middot; p56</figcaption></figure></div><h2>run2 &mdash; 2026-05-10/episode_002, rows 5845&ndash;5848 &nbsp;<small>moving left, holding right</small></h2><div class='grid'><figure><img loading='lazy' src='overlays/run2_trans+x.jpg'><figcaption>trans+x &middot; 0.1056m &middot; 1.50s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run2_trans-x.jpg'><figcaption>trans-x &middot; 0.3057m &middot; 1.77s &middot; p72</figcaption></figure><figure><img loading='lazy' src='overlays/run2_trans+y.jpg'><figcaption>trans+y &middot; 0.1933m &middot; 1.93s &middot; p55</figcaption></figure><figure><img loading='lazy' src='overlays/run2_trans-y.jpg'><figcaption>trans-y &middot; 0.1357m &middot; 1.53s &middot; p52</figcaption></figure><figure><img loading='lazy' src='overlays/run2_trans+z.jpg'><figcaption>trans+z &middot; 0.1328m &middot; 1.77s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run2_trans-z.jpg'><figcaption>trans-z &middot; 0.2835m &middot; 2.93s &middot; p54</figcaption></figure><figure><img loading='lazy' src='overlays/run2_rot+x.jpg'><figcaption>rot+x &middot; 40.0498&deg; &middot; 1.53s &middot; p57</figcaption></figure><figure><img loading='lazy' src='overlays/run2_rot-x.jpg'><figcaption>rot-x &middot; 82.3862&deg; &middot; 2.67s &middot; p63</figcaption></figure><figure><img loading='lazy' src='overlays/run2_rot+y.jpg'><figcaption>rot+y &middot; 71.2197&deg; &middot; 1.57s &middot; p76</figcaption></figure><figure><img loading='lazy' src='overlays/run2_rot-y.jpg'><figcaption>rot-y &middot; 72.1978&deg; &middot; 2.17s &middot; p66</figcaption></figure><figure><img loading='lazy' src='overlays/run2_rot+z.jpg'><figcaption>rot+z &middot; 58.6594&deg; &middot; 2.33s &middot; p56</figcaption></figure><figure><img loading='lazy' src='overlays/run2_rot-z.jpg'><figcaption>rot-z &middot; 24.6532&deg; &middot; 1.50s &middot; p42</figcaption></figure></div><h2>run3 &mdash; 2026-05-10/episode_005, rows 401&ndash;404 &nbsp;<small>moving right, holding left</small></h2><div class='grid'><figure><img loading='lazy' src='overlays/run3_trans+x.jpg'><figcaption>trans+x &middot; 0.1212m &middot; 1.70s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run3_trans-x.jpg'><figcaption>trans-x &middot; 0.1639m &middot; 1.77s &middot; p53</figcaption></figure><figure><img loading='lazy' src='overlays/run3_trans+y.jpg'><figcaption>trans+y &middot; 0.3207m &middot; 2.50s &middot; p63</figcaption></figure><figure><img loading='lazy' src='overlays/run3_trans-y.jpg'><figcaption>trans-y &middot; 0.1668m &middot; 1.83s &middot; p52</figcaption></figure><figure><img loading='lazy' src='overlays/run3_trans+z.jpg'><figcaption>trans+z &middot; 0.2874m &middot; 4.00s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run3_trans-z.jpg'><figcaption>trans-z &middot; 0.1555m &middot; 2.10s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run3_rot+x.jpg'><figcaption>rot+x &middot; 55.638&deg; &middot; 1.83s &middot; p62</figcaption></figure><figure><img loading='lazy' src='overlays/run3_rot-x.jpg'><figcaption>rot-x &middot; 75.9153&deg; &middot; 3.70s &middot; p49</figcaption></figure><figure><img loading='lazy' src='overlays/run3_rot+y.jpg'><figcaption>rot+y &middot; 23.4508&deg; &middot; 1.50s &middot; p41</figcaption></figure><figure><img loading='lazy' src='overlays/run3_rot-y.jpg'><figcaption>rot-y &middot; 25.9304&deg; &middot; 1.50s &middot; p44</figcaption></figure><figure><img loading='lazy' src='overlays/run3_rot+z.jpg'><figcaption>rot+z &middot; 34.2098&deg; &middot; 1.50s &middot; p53</figcaption></figure><figure><img loading='lazy' src='overlays/run3_rot-z.jpg'><figcaption>rot-z &middot; 45.5395&deg; &middot; 1.67s &middot; p59</figcaption></figure></div><h2>run4 &mdash; 2026-05-11/episode_011, rows 2377&ndash;2380 &nbsp;<small>moving right, holding left</small></h2><div class='grid'><figure><img loading='lazy' src='overlays/run4_trans+x.jpg'><figcaption>trans+x &middot; 0.2533m &middot; 1.87s &middot; p65</figcaption></figure><figure><img loading='lazy' src='overlays/run4_trans-x.jpg'><figcaption>trans-x &middot; 0.2352m &middot; 1.97s &middot; p61</figcaption></figure><figure><img loading='lazy' src='overlays/run4_trans+y.jpg'><figcaption>trans+y &middot; 0.2915m &middot; 1.63s &middot; p73</figcaption></figure><figure><img loading='lazy' src='overlays/run4_trans-y.jpg'><figcaption>trans-y &middot; 0.1628m &middot; 1.67s &middot; p55</figcaption></figure><figure><img loading='lazy' src='overlays/run4_trans+z.jpg'><figcaption>trans+z &middot; 0.1941m &middot; 2.77s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run4_trans-z.jpg'><figcaption>trans-z &middot; 0.3085m &middot; 2.90s &middot; p57</figcaption></figure><figure><img loading='lazy' src='overlays/run4_rot+x.jpg'><figcaption>rot+x &middot; 25.9953&deg; &middot; 1.50s &middot; p44</figcaption></figure><figure><img loading='lazy' src='overlays/run4_rot-x.jpg'><figcaption>rot-x &middot; 47.495&deg; &middot; 2.33s &middot; p49</figcaption></figure><figure><img loading='lazy' src='overlays/run4_rot+y.jpg'><figcaption>rot+y &middot; 89.5097&deg; &middot; 1.83s &middot; p78</figcaption></figure><figure><img loading='lazy' src='overlays/run4_rot-y.jpg'><figcaption>rot-y &middot; 39.8404&deg; &middot; 1.80s &middot; p52</figcaption></figure><figure><img loading='lazy' src='overlays/run4_rot+z.jpg'><figcaption>rot+z &middot; 28.5261&deg; &middot; 1.50s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run4_rot-z.jpg'><figcaption>rot-z &middot; 83.6116&deg; &middot; 2.67s &middot; p64</figcaption></figure></div><h2>run5 &mdash; 2026-05-11/episode_007, rows 2365&ndash;2368 &nbsp;<small>moving right, holding left</small></h2><div class='grid'><figure><img loading='lazy' src='overlays/run5_trans+x.jpg'><figcaption>trans+x &middot; 0.1139m &middot; 1.57s &middot; p46</figcaption></figure><figure><img loading='lazy' src='overlays/run5_trans-x.jpg'><figcaption>trans-x &middot; 0.2181m &middot; 2.43s &middot; p52</figcaption></figure><figure><img loading='lazy' src='overlays/run5_trans+y.jpg'><figcaption>trans+y &middot; 0.201m &middot; 1.63s &middot; p62</figcaption></figure><figure><img loading='lazy' src='overlays/run5_trans-y.jpg'><figcaption>trans-y &middot; 0.1837m &middot; 2.03s &middot; p52</figcaption></figure><figure><img loading='lazy' src='overlays/run5_trans+z.jpg'><figcaption>trans+z &middot; 0.2822m &middot; 3.53s &middot; p49</figcaption></figure><figure><img loading='lazy' src='overlays/run5_trans-z.jpg'><figcaption>trans-z &middot; 0.2499m &middot; 3.40s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run5_rot+x.jpg'><figcaption>rot+x &middot; 75.7041&deg; &middot; 2.57s &middot; p62</figcaption></figure><figure><img loading='lazy' src='overlays/run5_rot-x.jpg'><figcaption>rot-x &middot; 32.7892&deg; &middot; 1.70s &middot; p47</figcaption></figure><figure><img loading='lazy' src='overlays/run5_rot+y.jpg'><figcaption>rot+y &middot; 70.4553&deg; &middot; 2.47s &middot; p60</figcaption></figure><figure><img loading='lazy' src='overlays/run5_rot-y.jpg'><figcaption>rot-y &middot; 39.7622&deg; &middot; 1.53s &middot; p57</figcaption></figure><figure><img loading='lazy' src='overlays/run5_rot+z.jpg'><figcaption>rot+z &middot; 49.5627&deg; &middot; 2.33s &middot; p50</figcaption></figure><figure><img loading='lazy' src='overlays/run5_rot-z.jpg'><figcaption>rot-z &middot; 22.6513&deg; &middot; 1.50s &middot; p40</figcaption></figure></div>
</div></body></html>