Live Gen Probe 8-GPU
Updated: 2026-08-17T19:29:56.573368+00:00
- ckpt:
- loaded tensors: 391
Prompt
- unique tokens: 5
Prompt
- unique tokens: 6
Prompt
- unique tokens: 5
Prompt
- unique tokens: 4
Prompt
- unique tokens: 6
Training snapshot (same time)
| GPU | tokens | ema_tf | tok/s |
|---|---|---|---|
| 0 | ~8.9M | ~9.1 | ~955 |
| 1 | ~8.7M | ~8.9 | ~933 |
| 2 | ~9.9M | ~8.0 | ~1062 |
| 3 | ~8.7M | ~8.7 | ~932 |
| 4 | ~8.8M | ~9.4 | ~941 |
| 5 | ~9.2M | ~9.2 | ~991 |
| 6 | ~9.0M | ~8.9 | ~960 |
| 7 | ~9.4M | ~8.6 | ~1010 |
Interpretation
- Teacher-forced loss continues to fall on all 8 GPUs (ema_tf ~8-9 after ~9M tokens/shard).
- Free-run greedy generation still shows single-token attractor collapse (repeat last word / commas).
- This matches prior stage findings: knowledge in weights (TF) != free-run coherence yet (AR gap / decode dynamics).
- Unique tokens per prompt ~4-6 out of 48 generated.
Next
- Keep SS training running.
- Optional: re-apply decode surgery / temperature / n-gram ban at inference.
- Do not read TF loss alone as "model speaks".