# Live Gen Probe 8-GPU

Updated: 2026-08-17T19:29:56.573368+00:00

- ckpt: 
- loaded tensors: 391

## Prompt


- unique tokens: 5



## Prompt


- unique tokens: 6



## Prompt


- unique tokens: 5



## Prompt


- unique tokens: 4



## Prompt


- unique tokens: 6




## Training snapshot (same time)

| GPU | tokens | ema_tf | tok/s |
|-----|--------|--------|-------|
| 0 | ~8.9M | ~9.1 | ~955 |
| 1 | ~8.7M | ~8.9 | ~933 |
| 2 | ~9.9M | ~8.0 | ~1062 |
| 3 | ~8.7M | ~8.7 | ~932 |
| 4 | ~8.8M | ~9.4 | ~941 |
| 5 | ~9.2M | ~9.2 | ~991 |
| 6 | ~9.0M | ~8.9 | ~960 |
| 7 | ~9.4M | ~8.6 | ~1010 |

## Interpretation

- Teacher-forced loss continues to fall on all 8 GPUs (ema_tf ~8-9 after ~9M tokens/shard).
- Free-run greedy generation still shows **single-token attractor collapse** (repeat last word / commas).
- This matches prior stage findings: knowledge in weights (TF) != free-run coherence yet (AR gap / decode dynamics).
- Unique tokens per prompt ~4-6 out of 48 generated.

## Next

- Keep SS training running.
- Optional: re-apply decode surgery / temperature / n-gram ban at inference.
- Do not read TF loss alone as "model speaks".