Upload docs/POD_STATE_LIVE.md with huggingface_hub
Browse files- docs/POD_STATE_LIVE.md +19 -0
docs/POD_STATE_LIVE.md
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Pod state live — 2026-08-17T23:50:38.680454+00:00
|
| 2 |
+
|
| 3 |
+
## Training
|
| 4 |
+
- Phase: **phase2** (raw HF dataset tokenized)
|
| 5 |
+
- 8x RTX 5090, BATCH=2, SEQ=128, LR=7e-4, SS_RATE=0.25
|
| 6 |
+
- Shard: `data/shard_phase2_gpu{0-7}.npy` (~430M tokens each, memmap)
|
| 7 |
+
- Total phase2 corpus: **3,439,171,703** tokens
|
| 8 |
+
- Weights continuous from phase1 (~11M tokens on 1.52B .pt corpus)
|
| 9 |
+
|
| 10 |
+
## Anti re-ingest
|
| 11 |
+
- Phase1 stream (1.52B pt) separate from phase2 (raw jsonl/gz)
|
| 12 |
+
- RESUME_MANIFEST_8GPU.json tracks offsets
|
| 13 |
+
- PHASE_SWITCH.json records switch timestamp
|
| 14 |
+
|
| 15 |
+
## Artifacts on HF
|
| 16 |
+
- Model repo `thefinalboss/fractus-cte`: checkpoints, scripts, docs
|
| 17 |
+
- Dataset repo `thefinalboss/fractus-datasets`: original + phase2 tokenized npy shards
|
| 18 |
+
|
| 19 |
+
## Do not destroy pod
|