Spaces:
Running
Running
add Part 4: reference refinements (clean bank + difference-image)
Browse files- index.html +19 -1
- media/diffimg/train/sample_000/tl.mp4 +0 -0
- media/diffimg/train/sample_000/tr.mp4 +0 -0
- media/diffimg/train/sample_000/view.mp4 +0 -0
- media/diffimg/train/sample_001/tl.mp4 +0 -0
- media/diffimg/train/sample_001/tr.mp4 +0 -0
- media/diffimg/train/sample_001/view.mp4 +0 -0
- media/diffimg/train/sample_002/tl.mp4 +0 -0
- media/diffimg/train/sample_002/tr.mp4 +0 -0
- media/diffimg/train/sample_002/view.mp4 +0 -0
- media/diffimg/train/sample_003/tl.mp4 +0 -0
- media/diffimg/train/sample_003/tr.mp4 +0 -0
- media/diffimg/train/sample_003/view.mp4 +0 -0
- media/diffimg/val/sample_000/tl.mp4 +0 -0
- media/diffimg/val/sample_000/tr.mp4 +0 -0
- media/diffimg/val/sample_000/view.mp4 +0 -0
- media/diffimg/val/sample_001/tl.mp4 +0 -0
- media/diffimg/val/sample_001/tr.mp4 +0 -0
- media/diffimg/val/sample_001/view.mp4 +0 -0
- media/diffimg/val/sample_002/tl.mp4 +0 -0
- media/diffimg/val/sample_002/tr.mp4 +0 -0
- media/diffimg/val/sample_002/view.mp4 +0 -0
- media/diffimg/val/sample_003/tl.mp4 +0 -0
- media/diffimg/val/sample_003/tr.mp4 +0 -0
- media/diffimg/val/sample_003/view.mp4 +0 -0
- media/difflat/train/sample_000/tl.mp4 +0 -0
- media/difflat/train/sample_000/tr.mp4 +0 -0
- media/difflat/train/sample_000/view.mp4 +0 -0
- media/difflat/train/sample_001/tl.mp4 +0 -0
- media/difflat/train/sample_001/tr.mp4 +0 -0
- media/difflat/train/sample_001/view.mp4 +0 -0
- media/difflat/train/sample_002/tl.mp4 +0 -0
- media/difflat/train/sample_002/tr.mp4 +0 -0
- media/difflat/train/sample_002/view.mp4 +0 -0
- media/difflat/train/sample_003/tl.mp4 +0 -0
- media/difflat/train/sample_003/tr.mp4 +0 -0
- media/difflat/train/sample_003/view.mp4 +0 -0
- media/difflat/val/sample_000/tl.mp4 +0 -0
- media/difflat/val/sample_000/tr.mp4 +0 -0
- media/difflat/val/sample_000/view.mp4 +0 -0
- media/difflat/val/sample_001/tl.mp4 +0 -0
- media/difflat/val/sample_001/tr.mp4 +0 -0
- media/difflat/val/sample_001/view.mp4 +0 -0
- media/difflat/val/sample_002/tl.mp4 +0 -0
- media/difflat/val/sample_002/tr.mp4 +0 -0
- media/difflat/val/sample_002/view.mp4 +0 -0
- media/difflat/val/sample_003/tl.mp4 +0 -0
- media/difflat/val/sample_003/tr.mp4 +0 -0
- media/difflat/val/sample_003/view.mp4 +0 -0
index.html
CHANGED
|
@@ -110,7 +110,7 @@ footer{border-top:1px solid var(--line);padding:26px 0 50px;color:var(--sub);fon
|
|
| 110 |
</div>
|
| 111 |
<div class="legend"><div class="chip"><b>v3 · baseline</b><span>Bidirectional temporal attention (full-window). The original Cosmos-tokenizer world model.</span></div><div class="chip"><b>v3 · causal</b><span>Causal temporal attention (per-frame tril mask) — matches the causal tokenizer and AR rollout regime.</span></div><div class="chip"><b>v3 · contact-aware</b><span>Adds the contact-aware tactile aux loss (x0-space, alpha_bar-weighted) on top of v3.</span></div></div>
|
| 112 |
</div></header>
|
| 113 |
-
<nav class="sticky"><div class="wrap"><span class="ngrp">Overfit</span><a href="#v3">v3 · baseline</a><a href="#causal">v3 · causal</a><a href="#contact">v3 · contact-aware</a><span class="ngrp">Cross-arch</span><a href="#x-vt">Full V + T</a><a href="#x-vision">Vision-only</a><a href="#x-tactile">Tactile-only</a><span class="ngrp">Ref frame</span><a href="#ref-v3">fixed p01 ref · baseline</a><a href="#ref-p01rand">random no-contact ref</a><a href="#ref-p01rand_v2t">random ref + V→T aux</a></div></nav>
|
| 114 |
<main class="wrap">
|
| 115 |
<div class="band-h"><div class="t">Part 1</div><h2 class="big">Overfit test — full V + T</h2>
|
| 116 |
<p>Same recipe, three backbones. Does the model memorize the ~30 training episodes?</p></div>
|
|
@@ -166,6 +166,24 @@ footer{border-top:1px solid var(--line);padding:26px 0 50px;color:var(--sub);fon
|
|
| 166 |
<div class="col-q"><h3>Overfit test — train vs val</h3><table class="qt"><thead><tr><th>metric</th><th>train (n=16)</th><th>val (n=16)</th><th>gap</th></tr></thead><tbody><tr class="grp"><td colspan="4">Visual — middle view</td></tr><tr><td class="mname">LPIPS <span class="arrow">↓</span></td><td class="num">0.0473</td><td class="num">0.0460</td><td class="num gap-ok">-3%</td></tr><tr><td class="mname">SSIM <span class="arrow">↑</span></td><td class="num">0.8962</td><td class="num">0.8977</td><td class="num gap-ok">-0%</td></tr><tr><td class="mname">PSNR <span class="arrow">↑</span></td><td class="num">28.5951</td><td class="num">28.8901</td><td class="num gap-ok">-1%</td></tr><tr class="grp"><td colspan="4">Tactile — perceptual</td></tr><tr><td class="mname">LPIPS · left <span class="arrow">↓</span></td><td class="num">0.1010</td><td class="num">0.0765</td><td class="num gap-ok">-24%</td></tr><tr><td class="mname">LPIPS · right <span class="arrow">↓</span></td><td class="num">0.1582</td><td class="num">0.0687</td><td class="num gap-ok">-57%</td></tr><tr class="grp"><td colspan="4">Tactile — contact / physics</td></tr><tr><td class="mname">mask IoU (τ8) <span class="arrow">↑</span></td><td class="num">0.4096</td><td class="num">0.1762</td><td class="num gap-bad">+57%</td></tr><tr><td class="mname">contact MSE <span class="arrow">↓</span></td><td class="num">115.7422</td><td class="num">34.2311</td><td class="num gap-ok">-70%</td></tr><tr><td class="mname">onset err <span class="arrow">↓</span></td><td class="num">27.3989</td><td class="num">0.0000</td><td class="num gap-ok">-100%</td></tr><tr><td class="mname">intensity spearman <span class="arrow">↑</span></td><td class="num">0.3625</td><td class="num">0.5844</td><td class="num gap-ok">-61%</td></tr></tbody></table></div>
|
| 167 |
<div class="col-v"><h3>Short-horizon rollouts</h3><div class="qual"><div class="seg"><button class="seg-b active" data-t="train">Train set</button><button class="seg-b" data-t="val">Val set</button></div><div class="pane" data-split="train"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div><div class="pane" data-split="val"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div></div></div>
|
| 168 |
</div></section>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 169 |
</main>
|
| 170 |
<footer><div class="wrap">Visuo-tactile world model · decoded with Cosmos CV4x8x8 · GT | pred are future-half predictions (history skipped for metrics).</div></footer>
|
| 171 |
<script>
|
|
|
|
| 110 |
</div>
|
| 111 |
<div class="legend"><div class="chip"><b>v3 · baseline</b><span>Bidirectional temporal attention (full-window). The original Cosmos-tokenizer world model.</span></div><div class="chip"><b>v3 · causal</b><span>Causal temporal attention (per-frame tril mask) — matches the causal tokenizer and AR rollout regime.</span></div><div class="chip"><b>v3 · contact-aware</b><span>Adds the contact-aware tactile aux loss (x0-space, alpha_bar-weighted) on top of v3.</span></div></div>
|
| 112 |
</div></header>
|
| 113 |
+
<nav class="sticky"><div class="wrap"><span class="ngrp">Overfit</span><a href="#v3">v3 · baseline</a><a href="#causal">v3 · causal</a><a href="#contact">v3 · contact-aware</a><span class="ngrp">Cross-arch</span><a href="#x-vt">Full V + T</a><a href="#x-vision">Vision-only</a><a href="#x-tactile">Tactile-only</a><span class="ngrp">Ref frame</span><a href="#ref-v3">fixed p01 ref · baseline</a><a href="#ref-p01rand">random no-contact ref</a><a href="#ref-p01rand_v2t">random ref + V→T aux</a><span class="ngrp">Refinements</span><a href="#rf-p01rand">random ref (in-video)</a><a href="#rf-difflat">clean reference bank</a><a href="#rf-diffimg">difference image</a></div></nav>
|
| 114 |
<main class="wrap">
|
| 115 |
<div class="band-h"><div class="t">Part 1</div><h2 class="big">Overfit test — full V + T</h2>
|
| 116 |
<p>Same recipe, three backbones. Does the model memorize the ~30 training episodes?</p></div>
|
|
|
|
| 166 |
<div class="col-q"><h3>Overfit test — train vs val</h3><table class="qt"><thead><tr><th>metric</th><th>train (n=16)</th><th>val (n=16)</th><th>gap</th></tr></thead><tbody><tr class="grp"><td colspan="4">Visual — middle view</td></tr><tr><td class="mname">LPIPS <span class="arrow">↓</span></td><td class="num">0.0473</td><td class="num">0.0460</td><td class="num gap-ok">-3%</td></tr><tr><td class="mname">SSIM <span class="arrow">↑</span></td><td class="num">0.8962</td><td class="num">0.8977</td><td class="num gap-ok">-0%</td></tr><tr><td class="mname">PSNR <span class="arrow">↑</span></td><td class="num">28.5951</td><td class="num">28.8901</td><td class="num gap-ok">-1%</td></tr><tr class="grp"><td colspan="4">Tactile — perceptual</td></tr><tr><td class="mname">LPIPS · left <span class="arrow">↓</span></td><td class="num">0.1010</td><td class="num">0.0765</td><td class="num gap-ok">-24%</td></tr><tr><td class="mname">LPIPS · right <span class="arrow">↓</span></td><td class="num">0.1582</td><td class="num">0.0687</td><td class="num gap-ok">-57%</td></tr><tr class="grp"><td colspan="4">Tactile — contact / physics</td></tr><tr><td class="mname">mask IoU (τ8) <span class="arrow">↑</span></td><td class="num">0.4096</td><td class="num">0.1762</td><td class="num gap-bad">+57%</td></tr><tr><td class="mname">contact MSE <span class="arrow">↓</span></td><td class="num">115.7422</td><td class="num">34.2311</td><td class="num gap-ok">-70%</td></tr><tr><td class="mname">onset err <span class="arrow">↓</span></td><td class="num">27.3989</td><td class="num">0.0000</td><td class="num gap-ok">-100%</td></tr><tr><td class="mname">intensity spearman <span class="arrow">↑</span></td><td class="num">0.3625</td><td class="num">0.5844</td><td class="num gap-ok">-61%</td></tr></tbody></table></div>
|
| 167 |
<div class="col-v"><h3>Short-horizon rollouts</h3><div class="qual"><div class="seg"><button class="seg-b active" data-t="train">Train set</button><button class="seg-b" data-t="val">Val set</button></div><div class="pane" data-split="train"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/p01rand_v2t/train/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/train/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div><div class="pane" data-split="val"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/p01rand_v2t/val/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand_v2t/val/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div></div></div>
|
| 168 |
</div></section>
|
| 169 |
+
<div class="band band-h"><div class="t">Part 4</div><h2 class="big">Reference refinements — clean bank & difference-image</h2>
|
| 170 |
+
<p>Two principled attempts to push past the random-reference result: (1) guarantee the reference latent is clean via a replicate-16 bank; (2) target the encoded pixel-difference instead of the latent-difference. Neither beats plain random-reference — clean-bank is a wash (pool contamination was negligible), and difference-image is worst (its decode+ref reconstruction adds error, even though the model predicts the diff latent near-perfectly). The tactile ceiling is data + tokenizer fidelity, not reference cleanliness or diff-space.</p></div>
|
| 171 |
+
<section class="run" id="rf-p01rand">
|
| 172 |
+
<div class="run-head"><h2>random ref (in-video)</h2><p>The winner so far: random in-video no-contact reference, difference-LATENT target. Baseline for the two refinements.</p></div>
|
| 173 |
+
<div class="cols">
|
| 174 |
+
<div class="col-q"><h3>Overfit test — train vs val</h3><table class="qt"><thead><tr><th>metric</th><th>train (n=16)</th><th>val (n=16)</th><th>gap</th></tr></thead><tbody><tr class="grp"><td colspan="4">Visual — middle view</td></tr><tr><td class="mname">LPIPS <span class="arrow">↓</span></td><td class="num">0.0395</td><td class="num">0.0419</td><td class="num gap-ok">+6%</td></tr><tr><td class="mname">SSIM <span class="arrow">↑</span></td><td class="num">0.9061</td><td class="num">0.9001</td><td class="num gap-ok">+1%</td></tr><tr><td class="mname">PSNR <span class="arrow">↑</span></td><td class="num">29.2115</td><td class="num">29.1317</td><td class="num gap-ok">+0%</td></tr><tr class="grp"><td colspan="4">Tactile — perceptual</td></tr><tr><td class="mname">LPIPS · left <span class="arrow">↓</span></td><td class="num">0.0839</td><td class="num">0.0813</td><td class="num gap-ok">-3%</td></tr><tr><td class="mname">LPIPS · right <span class="arrow">↓</span></td><td class="num">0.1151</td><td class="num">0.0698</td><td class="num gap-ok">-39%</td></tr><tr class="grp"><td colspan="4">Tactile — contact / physics</td></tr><tr><td class="mname">mask IoU (τ8) <span class="arrow">↑</span></td><td class="num">0.4778</td><td class="num">0.1734</td><td class="num gap-bad">+64%</td></tr><tr><td class="mname">contact MSE <span class="arrow">↓</span></td><td class="num">92.2412</td><td class="num">35.5656</td><td class="num gap-ok">-61%</td></tr><tr><td class="mname">onset err <span class="arrow">↓</span></td><td class="num">0.0000</td><td class="num">0.0000</td><td class="num ">—</td></tr><tr><td class="mname">intensity spearman <span class="arrow">↑</span></td><td class="num">0.3500</td><td class="num">0.5219</td><td class="num gap-ok">-49%</td></tr></tbody></table></div>
|
| 175 |
+
<div class="col-v"><h3>Short-horizon rollouts</h3><div class="qual"><div class="seg"><button class="seg-b active" data-t="train">Train set</button><button class="seg-b" data-t="val">Val set</button></div><div class="pane" data-split="train"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/p01rand/train/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/p01rand/train/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/p01rand/train/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/p01rand/train/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/train/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div><div class="pane" data-split="val"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/p01rand/val/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/p01rand/val/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/p01rand/val/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/p01rand/val/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/p01rand/val/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div></div></div>
|
| 176 |
+
</div></section><section class="run" id="rf-difflat">
|
| 177 |
+
<div class="run-head"><h2>clean reference bank</h2><p>Guarantee the reference latent is clean via a replicate-16 bank (no in-video ti≥1 contamination). No gain — decoded LPIPS slightly worse, contact IoU tied: pool contamination was negligible in practice.</p></div>
|
| 178 |
+
<div class="cols">
|
| 179 |
+
<div class="col-q"><h3>Overfit test — train vs val</h3><table class="qt"><thead><tr><th>metric</th><th>train (n=16)</th><th>val (n=16)</th><th>gap</th></tr></thead><tbody><tr class="grp"><td colspan="4">Visual — middle view</td></tr><tr><td class="mname">LPIPS <span class="arrow">↓</span></td><td class="num">0.0476</td><td class="num">0.0433</td><td class="num gap-ok">-9%</td></tr><tr><td class="mname">SSIM <span class="arrow">↑</span></td><td class="num">0.8970</td><td class="num">0.9040</td><td class="num gap-ok">-1%</td></tr><tr><td class="mname">PSNR <span class="arrow">↑</span></td><td class="num">28.8151</td><td class="num">29.3747</td><td class="num gap-ok">-2%</td></tr><tr class="grp"><td colspan="4">Tactile — perceptual</td></tr><tr><td class="mname">LPIPS · left <span class="arrow">↓</span></td><td class="num">0.1007</td><td class="num">0.0876</td><td class="num gap-ok">-13%</td></tr><tr><td class="mname">LPIPS · right <span class="arrow">↓</span></td><td class="num">0.1427</td><td class="num">0.0743</td><td class="num gap-ok">-48%</td></tr><tr class="grp"><td colspan="4">Tactile — contact / physics</td></tr><tr><td class="mname">mask IoU (τ8) <span class="arrow">↑</span></td><td class="num">0.4012</td><td class="num">0.1763</td><td class="num gap-bad">+56%</td></tr><tr><td class="mname">contact MSE <span class="arrow">↓</span></td><td class="num">122.4160</td><td class="num">35.8125</td><td class="num gap-ok">-71%</td></tr><tr><td class="mname">onset err <span class="arrow">↓</span></td><td class="num">0.0000</td><td class="num">51.4189</td><td class="num ">—</td></tr><tr><td class="mname">intensity spearman <span class="arrow">↑</span></td><td class="num">0.3344</td><td class="num">0.4750</td><td class="num gap-ok">-42%</td></tr></tbody></table></div>
|
| 180 |
+
<div class="col-v"><h3>Short-horizon rollouts</h3><div class="qual"><div class="seg"><button class="seg-b active" data-t="train">Train set</button><button class="seg-b" data-t="val">Val set</button></div><div class="pane" data-split="train"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/difflat/train/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/difflat/train/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/difflat/train/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/difflat/train/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/train/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div><div class="pane" data-split="val"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/difflat/val/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/difflat/val/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/difflat/val/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/difflat/val/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/difflat/val/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div></div></div>
|
| 181 |
+
</div></section><section class="run" id="rf-diffimg">
|
| 182 |
+
<div class="run-head"><h2>difference image</h2><p>Target = encode((tac−ref)/255) on the fly; reconstruct tac ≈ decode(pred)+ref. The model predicts the diff latent near-perfectly, but the decode+ref roundtrip adds error → WORST decoded quality. (Its latent contact metrics are in diff-space, not comparable.)</p></div>
|
| 183 |
+
<div class="cols">
|
| 184 |
+
<div class="col-q"><h3>Overfit test — train vs val</h3><table class="qt"><thead><tr><th>metric</th><th>train (n=16)</th><th>val (n=16)</th><th>gap</th></tr></thead><tbody><tr class="grp"><td colspan="4">Visual — middle view</td></tr><tr><td class="mname">LPIPS <span class="arrow">↓</span></td><td class="num">0.0505</td><td class="num">0.0453</td><td class="num gap-ok">-10%</td></tr><tr><td class="mname">SSIM <span class="arrow">↑</span></td><td class="num">0.8939</td><td class="num">0.8965</td><td class="num gap-ok">-0%</td></tr><tr><td class="mname">PSNR <span class="arrow">↑</span></td><td class="num">28.7100</td><td class="num">28.8675</td><td class="num gap-ok">-1%</td></tr><tr class="grp"><td colspan="4">Tactile — perceptual</td></tr><tr><td class="mname">LPIPS · left <span class="arrow">↓</span></td><td class="num">0.1271</td><td class="num">0.0928</td><td class="num gap-ok">-27%</td></tr><tr><td class="mname">LPIPS · right <span class="arrow">↓</span></td><td class="num">0.1670</td><td class="num">0.0779</td><td class="num gap-ok">-53%</td></tr><tr class="grp"><td colspan="4">Tactile — contact / physics</td></tr><tr><td class="mname">mask IoU (τ8) <span class="arrow">↑</span></td><td class="num">0.2804</td><td class="num">0.0672</td><td class="num gap-bad">+76%</td></tr><tr><td class="mname">contact MSE <span class="arrow">↓</span></td><td class="num">149.4884</td><td class="num">46.5032</td><td class="num gap-ok">-69%</td></tr><tr><td class="mname">onset err <span class="arrow">↓</span></td><td class="num">0.0000</td><td class="num">—</td><td class="num ">—</td></tr><tr><td class="mname">intensity spearman <span class="arrow">↑</span></td><td class="num">0.2656</td><td class="num">0.5062</td><td class="num gap-ok">-91%</td></tr></tbody></table></div>
|
| 185 |
+
<div class="col-v"><h3>Short-horizon rollouts</h3><div class="qual"><div class="seg"><button class="seg-b active" data-t="train">Train set</button><button class="seg-b" data-t="val">Val set</button></div><div class="pane" data-split="train"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/diffimg/train/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/diffimg/train/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/diffimg/train/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/diffimg/train/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/train/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div><div class="pane" data-split="val"><div class="grid"><div class="scard"><div class="scard-h">sample 000</div><figure class="vid"><video src="media/diffimg/val/sample_000/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_000/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_000/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 001</div><figure class="vid"><video src="media/diffimg/val/sample_001/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_001/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_001/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 002</div><figure class="vid"><video src="media/diffimg/val/sample_002/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_002/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_002/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div><div class="scard"><div class="scard-h">sample 003</div><figure class="vid"><video src="media/diffimg/val/sample_003/view.mp4" muted playsinline preload="auto" data-lead="1"></video><figcaption>middle view <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_003/tl.mp4" muted playsinline preload="auto"></video><figcaption>tactile · left <span class="gtp">GT · pred</span></figcaption></figure><figure class="vid"><video src="media/diffimg/val/sample_003/tr.mp4" muted playsinline preload="auto"></video><figcaption>tactile · right <span class="gtp">GT · pred</span></figcaption></figure><div class="ctrl"><button class="pp" aria-label="play/pause">▶</button><input class="scrub" type="range" min="0" max="1000" value="0" step="1" aria-label="scrub"><span class="tlabel">0/16</span></div></div></div></div></div></div>
|
| 186 |
+
</div></section>
|
| 187 |
</main>
|
| 188 |
<footer><div class="wrap">Visuo-tactile world model · decoded with Cosmos CV4x8x8 · GT | pred are future-half predictions (history skipped for metrics).</div></footer>
|
| 189 |
<script>
|
media/diffimg/train/sample_000/tl.mp4
ADDED
|
Binary file (10.5 kB). View file
|
|
|
media/diffimg/train/sample_000/tr.mp4
ADDED
|
Binary file (10 kB). View file
|
|
|
media/diffimg/train/sample_000/view.mp4
ADDED
|
Binary file (13.2 kB). View file
|
|
|
media/diffimg/train/sample_001/tl.mp4
ADDED
|
Binary file (12 kB). View file
|
|
|
media/diffimg/train/sample_001/tr.mp4
ADDED
|
Binary file (12.3 kB). View file
|
|
|
media/diffimg/train/sample_001/view.mp4
ADDED
|
Binary file (12.9 kB). View file
|
|
|
media/diffimg/train/sample_002/tl.mp4
ADDED
|
Binary file (9.25 kB). View file
|
|
|
media/diffimg/train/sample_002/tr.mp4
ADDED
|
Binary file (11.6 kB). View file
|
|
|
media/diffimg/train/sample_002/view.mp4
ADDED
|
Binary file (13.5 kB). View file
|
|
|
media/diffimg/train/sample_003/tl.mp4
ADDED
|
Binary file (8.48 kB). View file
|
|
|
media/diffimg/train/sample_003/tr.mp4
ADDED
|
Binary file (9.47 kB). View file
|
|
|
media/diffimg/train/sample_003/view.mp4
ADDED
|
Binary file (16.7 kB). View file
|
|
|
media/diffimg/val/sample_000/tl.mp4
ADDED
|
Binary file (6.31 kB). View file
|
|
|
media/diffimg/val/sample_000/tr.mp4
ADDED
|
Binary file (6.17 kB). View file
|
|
|
media/diffimg/val/sample_000/view.mp4
ADDED
|
Binary file (13.8 kB). View file
|
|
|
media/diffimg/val/sample_001/tl.mp4
ADDED
|
Binary file (5.69 kB). View file
|
|
|
media/diffimg/val/sample_001/tr.mp4
ADDED
|
Binary file (6.97 kB). View file
|
|
|
media/diffimg/val/sample_001/view.mp4
ADDED
|
Binary file (14.3 kB). View file
|
|
|
media/diffimg/val/sample_002/tl.mp4
ADDED
|
Binary file (6.79 kB). View file
|
|
|
media/diffimg/val/sample_002/tr.mp4
ADDED
|
Binary file (6.15 kB). View file
|
|
|
media/diffimg/val/sample_002/view.mp4
ADDED
|
Binary file (14.8 kB). View file
|
|
|
media/diffimg/val/sample_003/tl.mp4
ADDED
|
Binary file (5.7 kB). View file
|
|
|
media/diffimg/val/sample_003/tr.mp4
ADDED
|
Binary file (5.71 kB). View file
|
|
|
media/diffimg/val/sample_003/view.mp4
ADDED
|
Binary file (12.1 kB). View file
|
|
|
media/difflat/train/sample_000/tl.mp4
ADDED
|
Binary file (9.64 kB). View file
|
|
|
media/difflat/train/sample_000/tr.mp4
ADDED
|
Binary file (9.51 kB). View file
|
|
|
media/difflat/train/sample_000/view.mp4
ADDED
|
Binary file (13.3 kB). View file
|
|
|
media/difflat/train/sample_001/tl.mp4
ADDED
|
Binary file (11.2 kB). View file
|
|
|
media/difflat/train/sample_001/tr.mp4
ADDED
|
Binary file (11.5 kB). View file
|
|
|
media/difflat/train/sample_001/view.mp4
ADDED
|
Binary file (12.8 kB). View file
|
|
|
media/difflat/train/sample_002/tl.mp4
ADDED
|
Binary file (9.09 kB). View file
|
|
|
media/difflat/train/sample_002/tr.mp4
ADDED
|
Binary file (11.2 kB). View file
|
|
|
media/difflat/train/sample_002/view.mp4
ADDED
|
Binary file (13.8 kB). View file
|
|
|
media/difflat/train/sample_003/tl.mp4
ADDED
|
Binary file (8.42 kB). View file
|
|
|
media/difflat/train/sample_003/tr.mp4
ADDED
|
Binary file (9.27 kB). View file
|
|
|
media/difflat/train/sample_003/view.mp4
ADDED
|
Binary file (16.8 kB). View file
|
|
|
media/difflat/val/sample_000/tl.mp4
ADDED
|
Binary file (5.83 kB). View file
|
|
|
media/difflat/val/sample_000/tr.mp4
ADDED
|
Binary file (5.37 kB). View file
|
|
|
media/difflat/val/sample_000/view.mp4
ADDED
|
Binary file (13.7 kB). View file
|
|
|
media/difflat/val/sample_001/tl.mp4
ADDED
|
Binary file (5.27 kB). View file
|
|
|
media/difflat/val/sample_001/tr.mp4
ADDED
|
Binary file (6.39 kB). View file
|
|
|
media/difflat/val/sample_001/view.mp4
ADDED
|
Binary file (13.8 kB). View file
|
|
|
media/difflat/val/sample_002/tl.mp4
ADDED
|
Binary file (6.25 kB). View file
|
|
|
media/difflat/val/sample_002/tr.mp4
ADDED
|
Binary file (5.62 kB). View file
|
|
|
media/difflat/val/sample_002/view.mp4
ADDED
|
Binary file (14.6 kB). View file
|
|
|
media/difflat/val/sample_003/tl.mp4
ADDED
|
Binary file (5.34 kB). View file
|
|
|
media/difflat/val/sample_003/tr.mp4
ADDED
|
Binary file (5.06 kB). View file
|
|
|
media/difflat/val/sample_003/view.mp4
ADDED
|
Binary file (12 kB). View file
|
|
|