Spaces:

yxma
/

visuotactile-stage1-gate-cosmos

Running

File size: 4,963 Bytes

<!doctype html>
<html lang=en><meta charset=utf-8>
<meta name=viewport content="width=device-width,initial-scale=1">
<title>Stage-1 gate — Cosmos</title>
<style>
 body{font:15px/1.6 -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;
      max-width:820px;margin:40px auto;padding:0 20px;color:#1f2328}
 h1{font-size:22px;margin-bottom:4px} .sub{color:#57606a;margin-bottom:24px}
 .verdict{display:inline-block;padding:6px 14px;border-radius:6px;color:#fff;
      font-weight:600;background:#bc4c00}
 table{border-collapse:collapse;width:100%;margin:22px 0}
 th,td{padding:9px 10px;border-bottom:1px solid #d8dee4;text-align:left;vertical-align:top}
 th{font-size:12px;text-transform:uppercase;color:#57606a;letter-spacing:.04em}
 .num{text-align:right;font-variant-numeric:tabular-nums}
 .muted{color:#57606a;font-size:13px}
 .win{color:#1a7f37;font-weight:600} .lose{color:#cf222e;font-weight:600}
 .note{background:#f6f8fa;border-left:3px solid #d8dee4;padding:12px 16px;
      margin:20px 0;font-size:14px}
</style>
<h1>Stage-1 gate — does the action-motion encoder transfer to Cosmos?</h1>
<div class=sub>tokenizer <code>cosmos-cv4x8x8</code> &middot; generated 2026-07-31 07:55:33Z</div>
<p><span class=verdict>NO TRANSFER</span> &nbsp; the encoder learns well (D beats zero by 32.3%, C by 33.0%), but the action map adds NOTHING over tactile-only (D is 1.0% worse than C; Wan2.2 had D +3.3% over C). The conditioning signal Stages 2-3 inject is redundant here.</p>
<div class=note>
 The encoder predicts per-latent-transition tactile flow. <b>Active-pixel EPE</b> is the
 endpoint error over pixels whose ground-truth motion exceeds the active threshold;
 lower is better. The zero-motion baseline on this val split is
 <b>1.2195</b> (Wan2.2: 1.612).
 An ablation is only meaningful if it beats that.
</div>
<table>
 <tr><th>ablation</th><th>inputs</th><th class=num>EPE (Cosmos)</th>
     <th class=num>EPE (Wan2.2)</th><th class=num>&Delta;</th><th class=num>trained</th></tr>
 <tr><td><b>D</b><div class=muted>(z_ctx, a) -> F</div></td><td class=muted>action map + tactile state</td><td class='num win'>0.8254</td><td class='num muted'>1.130</td><td class='num muted'>-0.305</td><td class='num muted'>12 ep</td></tr><tr><td><b>B</b><div class=muted>action only</div></td><td class=muted>action map only</td><td class='num win'>1.0553</td><td class='num muted'>1.362</td><td class='num muted'>-0.307</td><td class='num muted'>12 ep</td></tr><tr><td><b>C</b><div class=muted>tactile only</div></td><td class=muted>tactile state only</td><td class='num win'>0.8170</td><td class='num muted'>1.168</td><td class='num muted'>-0.351</td><td class='num muted'>12 ep</td></tr>
</table>
<div class=note>
 <b>&Delta; is context, not a result.</b> The two columns are not measured over the same
 interval: a Wan2.2 latent transition spans 5 frame pairs at 6&nbsp;fps (0.83&nbsp;s),
 a Cosmos one spans 4 pairs at 30&nbsp;fps (0.13&nbsp;s) — <b>6.2&times; less motion
 time</b> — on top of a different tokenizer and dataset. The magnitudes happen to
 land in a similar range because gel deformation saturates rather than accumulating,
 but that is a coincidence of contact mechanics, not a controlled comparison.
 <b>Judge the gate on the ordering</b> (D &lt; C &lt; B &lt; zero) <b>and on D's
 relative margin over zero</b>, both measured entirely within the Cosmos column.
</div>
<div class=note><b>Seeds.</b> 3 seeds per ablation. The D-vs-C increment is -0.9% &plusmn; 0.6% (std). The sign is stable across seeds.</div>
<h2 style="font-size:17px;margin-top:32px">What the two ablations actually see</h2>
<p>D's input is a strict superset of C's: the same tactile latent, plus the action
map. <b>D &minus; C is therefore exactly what knowing the sensor's own motion buys
you beyond looking at the gel</b> — the signal Stages 2-3 inject.</p>
<img src="stage1_cosmos.png" style="width:100%;border:1px solid #d8dee4;border-radius:6px">
<div class=note>
 The action map (purple) is a near-<b>constant</b> field: 99.5% of its energy is
 global translation, so it carries about one 2-D vector. The AllTracker target
 (red) is only 33% global translation — two thirds is spatial structure a constant
 field cannot express. That is why B (action only) is far behind, and why D adds
 nothing C does not already have.
</div>
<h2 style="font-size:17px;margin-top:32px">Cosmos vs Wan2.2</h2>
<img src="stage1_cosmos_vs_wan.png" style="width:100%;border:1px solid #d8dee4;border-radius:6px">
<div class=note>
 The ordering broadly agrees — action-only weakest, tactile dominant. What differs
 is the increment the recipe rests on. Note <b>D did not get worse</b> on Cosmos
 (29.9 &rarr; 32.3): <b>C got better</b> (27.5 &rarr; 33.0). The Cosmos tactile
 latent already carries more of the motion, leaving no headroom. And the increment
 was only +3.3% on Wan2.2 to begin with.
</div>
<p class=muted>Raw metrics: <a href="data.json">data.json</a></p>
</html>