File size: 1,854 Bytes
896b90d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | # large v1
- **preset:** large
- **training corpus:** data/large (see corpus_index.md / corpus_stats.txt)
- **trained:** 2026-06-18 13:58 (checkpoint mtime)
- **training wall-clock:** 208.2 min
- **final val loss:** 1.1111 (iter 23000)
- **parameters:** 49.72M | vocab 198
- **note:** v1 trained on the large corpus as of 2026-06-18 BEFORE the xlarge expansion: 1,085,579,228 chars / 248 authors / 22 categories / vocab 198 / best val 1.1111. The corpus_index.md + corpus_stats.txt in this dir have been CORRECTED to that true training set (the live data/large has since grown toward the xlarge ~2B target).
## Training hyperparameters
| param | value |
|---|---|
| block_size | 384 |
| n_embd | 640 |
| n_head | 10 |
| n_layer | 10 |
| dropout | 0.0 |
| batch_size | 16 |
| max_iters | 24000 |
| eval_interval | 1000 |
| eval_iters | 80 |
| learning_rate | 0.00025 |
| vocab_size | 198 |
## Validation curve
| step | train | val |
|---:|---:|---:|
| 0 | 5.5531 | 5.5478 |
| 1000 | 2.0925 | 2.0524 |
| 2000 | 1.5805 | 1.5329 |
| 3000 | 1.4392 | 1.3825 |
| 4000 | 1.3836 | 1.3366 |
| 5000 | 1.3361 | 1.2880 |
| 6000 | 1.3129 | 1.2693 |
| 7000 | 1.2897 | 1.2436 |
| 8000 | 1.2580 | 1.2296 |
| 9000 | 1.2388 | 1.2164 |
| 10000 | 1.2397 | 1.1977 |
| 11000 | 1.2146 | 1.1828 |
| 12000 | 1.2159 | 1.1787 |
| 13000 | 1.2004 | 1.1669 |
| 14000 | 1.1967 | 1.1636 |
| 15000 | 1.1915 | 1.1576 |
| 16000 | 1.1814 | 1.1457 |
| 17000 | 1.1793 | 1.1388 |
| 18000 | 1.1634 | 1.1461 |
| 19000 | 1.1557 | 1.1360 |
| 20000 | 1.1552 | 1.1365 |
| 21000 | 1.1573 | 1.1241 |
| 22000 | 1.1556 | 1.1260 |
| 23000 | 1.1388 | 1.1111 |
| 23999 | 1.1367 | 1.1189 |
## Reproducing the corpus
`corpus_index.md` lists every author and work in the training set; re-run `python -m corpus add-author` / `add-topic` per that index and `make finalize` to rebuild an equivalent corpus.
|