Part 1
Overfit test — full V + T
Same recipe, three backbones. Does the model memorize the ~30 training episodes?
v3 · baseline
Bidirectional temporal attention (full-window). The original Cosmos-tokenizer world model.
Overfit test — train vs val
| metric | train (n=10) | val (n=10) | gap |
|---|---|---|---|
| Visual — middle view | |||
| LPIPS ↓ | 0.0985 | 0.0735 | -25% |
| SSIM ↑ | 0.8107 | 0.8555 | -6% |
| PSNR ↑ | 25.0741 | 26.5707 | -6% |
| Tactile — perceptual | |||
| LPIPS · left ↓ | 0.0618 | 0.0457 | -26% |
| LPIPS · right ↓ | 0.0217 | 0.0665 | +206% |
| Tactile — contact / physics | |||
| mask IoU (τ8) ↑ | 0.2959 | 0.2469 | +17% |
| contact MSE ↓ | 29.4892 | 37.3549 | +27% |
| onset err ↓ | 94.0515 | 4.9556 | -95% |
| intensity spearman ↑ | 0.5550 | 0.7600 | -37% |
Short-horizon rollouts
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
v3 · causal
Causal temporal attention (per-frame tril mask) — matches the causal tokenizer and AR rollout regime.
Overfit test — train vs val
| metric | train (n=10) | val (n=10) | gap |
|---|---|---|---|
| Visual — middle view | |||
| LPIPS ↓ | 0.0531 | 0.0441 | -17% |
| SSIM ↑ | 0.8772 | 0.9008 | -3% |
| PSNR ↑ | 28.0965 | 29.1089 | -4% |
| Tactile — perceptual | |||
| LPIPS · left ↓ | 0.0630 | 0.0531 | -16% |
| LPIPS · right ↓ | 0.0203 | 0.0761 | +274% |
| Tactile — contact / physics | |||
| mask IoU (τ8) ↑ | 0.2870 | 0.2067 | +28% |
| contact MSE ↓ | 32.1382 | 38.6478 | +20% |
| onset err ↓ | 133.9037 | 59.3774 | -56% |
| intensity spearman ↑ | 0.5750 | 0.6100 | -6% |
Short-horizon rollouts
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
v3 · contact-aware
Adds the contact-aware tactile aux loss (x0-space, alpha_bar-weighted) on top of v3.
Overfit test — train vs val
| metric | train (n=10) | val (n=10) | gap |
|---|---|---|---|
| Visual — middle view | |||
| LPIPS ↓ | 0.0674 | 0.0460 | -32% |
| SSIM ↑ | 0.8538 | 0.8894 | -4% |
| PSNR ↑ | 26.9559 | 28.2838 | -5% |
| Tactile — perceptual | |||
| LPIPS · left ↓ | 0.0593 | 0.0430 | -28% |
| LPIPS · right ↓ | 0.0270 | 0.0816 | +203% |
| Tactile — contact / physics | |||
| mask IoU (τ8) ↑ | 0.2849 | 0.2296 | +19% |
| contact MSE ↓ | 30.9060 | 38.3891 | +24% |
| onset err ↓ | 41.5967 | 14.6901 | -65% |
| intensity spearman ↑ | 0.7150 | 0.6800 | +5% |
Short-horizon rollouts
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
Part 2
Cross-architecture — causal vs noncausal
The same causal-backbone swap under three modality configurations. Val metrics; the winning backbone per metric is highlighted.
Full V + T — causal vs noncausal
Three views + two tactile sensors. Swapping the backbone from bidirectional (v3) to causal.
noncausal vs causal (val)
| metric (val) | noncausal | causal |
|---|---|---|
| Visual — middle view | ||
| LPIPS ↓ | 0.0735 | 0.0441 |
| SSIM ↑ | 0.8555 | 0.9008 |
| PSNR ↑ | 26.5707 | 29.1089 |
| Tactile | ||
| LPIPS · left ↓ | 0.0457 | 0.0531 |
| LPIPS · right ↓ | 0.0665 | 0.0761 |
| mask IoU (τ8) ↑ | 0.2469 | 0.2067 |
| contact MSE ↓ | 37.3549 | 38.6478 |
| onset err ms ↓ | 4.9556 | 59.3774 |
Rollouts — noncausal | causal
noncausal · val_loss 0.1234
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
causal · val_loss 0.1647
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
Vision-only — causal vs noncausal
Tactile stream removed — isolates how the causal backbone affects pure visual prediction.
noncausal vs causal (val)
| metric (val) | noncausal | causal |
|---|---|---|
| Visual — middle view | ||
| LPIPS ↓ | 0.0498 | 0.0343 |
| SSIM ↑ | 0.8870 | 0.9162 |
| PSNR ↑ | 28.3848 | 30.1398 |
Rollouts — noncausal | causal
noncausal · val_loss 0.0297
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
causal · val_loss 0.0347
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
Tactile-only — causal vs noncausal
Visual stream removed — isolates the causal backbone's effect on tactile / contact prediction.
noncausal vs causal (val)
| metric (val) | noncausal | causal |
|---|---|---|
| Tactile | ||
| LPIPS · left ↓ | 0.0514 | 0.0499 |
| LPIPS · right ↓ | 0.0698 | 0.0713 |
| mask IoU (τ8) ↑ | 0.2448 | 0.2183 |
| contact MSE ↓ | 38.4460 | 38.8099 |
| onset err ms ↓ | 0.0000 | 0.0000 |
Rollouts — noncausal | causal
noncausal · val_loss 0.1084
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
causal · val_loss 0.1208
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16
sample 000
0/16
sample 001
0/16
sample 002
0/16
sample 003
0/16