Release Pollock 1.4 (r006)

CHANGELOG.md

@@ -2,6 +2,15 @@
 
 Revision numbers identify published model states independently of release names.
 
+## r006 — Pollock 1.4
+
+- Returned from the r005 5B corpus to V2 of `SlayerLab/minimal-en-corpus-2.5b`, including its corrected cleaning and record boundaries.
+- Restored the tokenizer used by r003-r004 and trained the unchanged 12/14/896 architecture for approximately 4.02 data passes at the same 491,520-token effective batch and 4e-4→4e-5 learning-rate range.
+- Replaced the legacy inclusive training loop with the exact configured 22,003-update budget while preserving the training, data, and evaluation seeds.
+- Updated training evaluation, the seven-task English zero-shot suite, Transformers weights, and fixed inference samples.
+
+Full record: [`training-history/r006.md`](./training-history/r006.md)
+
 ## r005 — Pollock 1.3
 
 - Returned to the 12/14/896 architecture used by r003 while keeping the model below 128M native trainable parameters.

LICENSE.md

@@ -16,14 +16,14 @@ Code components distributed with the project remain subject to their respective
 
 ## Korpus i wagi / Corpus and weights
 
-Model został wytrenowany na `SlayerLab/minimal-en-corpus-5b`, agregacie danych z wielu źródeł. Korpus nie nadaje dokumentom jednej wspólnej licencji; każdy dokument zachowuje identyfikator źródła i podlega warunkom, licencjom oraz ograniczeniom właściwego upstreamowego datasetu.
+Model został wytrenowany na `SlayerLab/minimal-en-corpus-2.5b`, agregacie danych z wielu źródeł. Korpus nie nadaje dokumentom jednej wspólnej licencji; każdy dokument zachowuje identyfikator źródła i podlega warunkom, licencjom oraz ograniczeniom właściwego upstreamowego datasetu.
 
 Ze względu na mieszany charakter tych warunków repozytorium modelu używa metadanej Hugging Face `license: other`. Nie jest to przyznanie dodatkowych praw do materiałów źródłowych. Użytkownik powinien przed użyciem, redystrybucją lub zastosowaniem komercyjnym zapoznać się z kartą korpusu i warunkami wszystkich właściwych źródeł:
 
-<https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-5b>
+<https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-2.5b>
 
-The model was trained on `SlayerLab/minimal-en-corpus-5b`, an aggregate of multiple data sources. The corpus does not apply a single common license to its documents; each document retains its source identifier and remains subject to the terms, licenses, and restrictions of the applicable upstream dataset.
+The model was trained on `SlayerLab/minimal-en-corpus-2.5b`, an aggregate of multiple data sources. The corpus does not apply a single common license to its documents; each document retains its source identifier and remains subject to the terms, licenses, and restrictions of the applicable upstream dataset.
 
 Because these terms are mixed, the model repository uses the Hugging Face metadata value `license: other`. This notice does not grant additional rights to upstream materials. Before use, redistribution, or commercial application, users should review the corpus card and the terms of every applicable source:
 
-<https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-5b>
+<https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-2.5b>

README.md

@@ -5,38 +5,38 @@ pipeline_tag: text-generation
 license: other
 license_name: mixed-upstream-dataset-terms
 license_link: https://huggingface.co/SlayerLab/pollock-mini-lm-125m/blob/main/LICENSE.md
-datasets: [SlayerLab/minimal-en-corpus-5b]
+datasets: [SlayerLab/minimal-en-corpus-2.5b]
 tags: [causal-lm, gpt2, nanogpt, bpe, educational, base-model]
 model-index:
-  - name: Pollock 1.3
+  - name: Pollock 1.4
     results:
       - task: {type: text-generation, name: Language modeling}
-        dataset: {type: SlayerLab/minimal-en-corpus-5b, name: Minimal EN validation (sampled), split: validation}
-        metrics: [{type: loss, value: 2.5406661582, name: Final sampled validation loss}]
+        dataset: {type: SlayerLab/minimal-en-corpus-2.5b, name: Minimal EN 2.5B V2 validation (fixed subset), split: validation}
+        metrics: [{type: loss, value: 2.5363565445, name: Final fixed-subset validation loss}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: blimp, name: BLiMP, split: train}
-        metrics: [{type: acc, value: 0.7753134328}]
+        metrics: [{type: acc, value: 0.7691641791}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: EleutherAI/lambada_openai, name: LAMBADA OpenAI, split: test}
-        metrics: [{type: acc, value: 0.2974966039}, {type: perplexity, value: 47.32861410}]
+        metrics: [{type: acc, value: 0.2769260625}, {type: perplexity, value: 49.9072065519}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: hellaswag, name: HellaSwag, split: validation}
-        metrics: [{type: acc_norm, value: 0.3045210118}]
+        metrics: [{type: acc_norm, value: 0.3013343955}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: piqa, name: PIQA, split: validation}
-        metrics: [{type: acc_norm, value: 0.6033732318}]
+        metrics: [{type: acc_norm, value: 0.5968443961}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: sciq, name: SciQ, split: test}
-        metrics: [{type: acc_norm, value: 0.6760000000}]
+        metrics: [{type: acc_norm, value: 0.6640000000}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: allenai/ai2_arc, config: ARC-Easy, name: ARC-Easy, split: test}
-        metrics: [{type: acc_norm, value: 0.4297138047}]
+        metrics: [{type: acc_norm, value: 0.4343434343}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: allenai/ai2_arc, config: ARC-Challenge, name: ARC-Challenge, split: test}
-        metrics: [{type: acc_norm, value: 0.2320819113}]
+        metrics: [{type: acc_norm, value: 0.2414675768}]
 ---
 
-# Pollock 1.3 — r005
+# Pollock 1.4 — r006
 
 ![Pollock avatar](./assets/pollock-mini-lm-avatar-320.png)
 
@@ -48,13 +48,13 @@ model-index:
 
 Pollock to niewielki, anglojęzyczny model bazowy typu decoder-only, wytrenowany od zera jako czytelny eksperyment edukacyjny. Implementacja bazuje na [nanoGPT](https://github.com/karpathy/nanoGPT) i własnym tokenizerze byte-level BPE. Jest to model do uzupełniania tekstu, nie asystent konwersacyjny.
 
-Nazwa luźno nawiązuje do gestu malarskiego Jacksona Pollocka: nanoGPT jest płótnem, na którym dane, konfiguracja i decyzje treningowe tworzą różne wzorce zachowania. Pełne dane techniczne tej wersji znajdują się w [`training-history/r005.md`](./training-history/r005.md), a różnice między wydaniami w [`CHANGELOG.md`](./CHANGELOG.md).
+Nazwa luźno nawiązuje do gestu malarskiego Jacksona Pollocka: nanoGPT jest płótnem, na którym dane, konfiguracja i decyzje treningowe tworzą różne wzorce zachowania. Pełne dane techniczne tej wersji znajdują się w [`training-history/r006.md`](./training-history/r006.md), a różnice między wydaniami w [`CHANGELOG.md`](./CHANGELOG.md).
 
 ### Architektura i tokenizer
 
 | Właściwość | Wartość |
 |---|---:|
-| Rewizja / wydanie | r005 / Pollock 1.3 |
+| Rewizja / wydanie | r006 / Pollock 1.4 |
 | Typ | decoder-only Transformer w stylu GPT-2 |
 | Warstwy / głowy / embedding | 12 / 14 / 896 |
 | Maksymalny kontekst | 1024 tokeny |
@@ -68,40 +68,40 @@ Artefakt Transformers ma 127 674 624 parametry, w tym 119 168 zerowych parametr
 
 ### Dane i trening
 
-Model wytrenowano na [`SlayerLab/minimal-en-corpus-5b`](https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-5b), subiektywnie dobranej mieszance 15 anglojęzycznych źródeł. Przygotowanie obejmowało deduplikację dokładną i przybliżoną, filtrowanie języka oraz decontaminację benchmarków. Tokenizer BPE został przebudowany dla nowego korpusu; zachowuje rozmiar słownika i identyfikatory tokenów specjalnych z r004, ale pozostałe identyfikatory tokenów nie są zgodne między tymi rewizjami.
+Model wytrenowano na wersji V2 datasetu [`SlayerLab/minimal-en-corpus-2.5b`](https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-2.5b) przypiętej do commita `d68d992`. Jest to subiektywnie dobrana mieszanka 15 anglojęzycznych źródeł po filtrowaniu języka, deduplikacji dokładnej i przybliżonej, decontaminacji benchmarków oraz dodatkowym czyszczeniu boilerplate'u i błędnie połączonych rekordów. R006 wraca do tokenizera używanego przez r003-r004; nie jest on zgodny z tokenizerem r005 poza rozmiarem słownika i identyfikatorami tokenów specjalnych.
 
 | Parametr | Wartość |
 |---|---:|
-| Tokeny treningowe / walidacyjne | 5 425 901 705 / 5 269 579 |
-| Finalny checkpoint | aktualizacja 22 078 |
-| Przetworzone tokeny | 10 851 778 560 (około 2 epoki) |
+| Tokeny treningowe / walidacyjne | 2 689 323 439 / 5 236 486 |
+| Finalny checkpoint | aktualizacja 22 003 |
+| Przetworzone tokeny | 10 814 914 560 (około 4,02 epoki) |
 | Sekwencja / micro-batch na GPU | 1024 / 12 |
-| Akumulacja globalna / na GPU | 40 / 40 micro-stepów |
+| Akumulacja globalna / na GPU | 40 / 20 micro-stepów |
 | Effective batch | 491 520 tokenów |
 | Optymalizator | fused AdamW, betas 0.9/0.95 |
 | Learning rate | 4e-4 → 4e-5, cosine decay |
 | Warmup / weight decay / grad clip | 440 / 0.1 / 1.0 |
 | Precyzja | BF16 |
-| Sprzęt | 1× NVIDIA GeForce RTX 5090 |
+| Sprzęt | 2× NVIDIA GeForce RTX 4090 |
 | Framework | PyTorch 2.8.0+cu128, nanoGPT commit `3adf61e` |
 
 ### Ewaluacja
 
-Loss treningowy szacowano na stałych podzbiorach po 1 228 800 tokenów na split. Finalny checkpoint uzyskał validation loss **2.540666**; najlepszy wynik to **2.540605** po 10 813 440 000 przetworzonych tokenów. Nie należy porównywać tych wartości bezpośrednio z r001-r004, ponieważ r005 używa innego korpusu, tokenizera i zbioru walidacyjnego.
+Loss treningowy szacowano na stałych podzbiorach po 1 228 800 tokenów na split. Finalny checkpoint uzyskał validation loss **2.536357**; najlepszy wynik to **2.536231** po 10 813 440 000 przetworzonych tokenów. Nie należy porównywać tych wartości bezpośrednio z r005, ponieważ r006 używa innego korpusu, tokenizera i zbioru walidacyjnego.
 
 Benchmarki wykonano zero-shot na pełnych splitach przy użyciu `lm-evaluation-harness` 0.4.12, batch size 8 i BF16.
 
 | Benchmark | Główna metryka | Wynik | Próbki |
 |---|---|---:|---:|
-| BLiMP | acc | 0.775313 | 67 000 |
-| LAMBADA | acc | 0.297497 | 5 153 |
-| HellaSwag | acc_norm | 0.304521 | 10 042 |
-| PIQA | acc_norm | 0.603373 | 1 838 |
-| SciQ | acc_norm | 0.676000 | 1 000 |
-| ARC-Easy | acc_norm | 0.429714 | 2 376 |
-| ARC-Challenge | acc_norm | 0.232082 | 1 172 |
+| BLiMP | acc | 0.769164 | 67 000 |
+| LAMBADA | acc | 0.276926 | 5 153 |
+| HellaSwag | acc_norm | 0.301334 | 10 042 |
+| PIQA | acc_norm | 0.596844 | 1 838 |
+| SciQ | acc_norm | 0.664000 | 1 000 |
+| ARC-Easy | acc_norm | 0.434343 | 2 376 |
+| ARC-Challenge | acc_norm | 0.241468 | 1 172 |
 
-LAMBADA osiągnęła perplexity 47.328614. Pełne metryki i protokół zapisano w [`benchmarks/english.json`](./benchmarks/english.json).
+LAMBADA osiągnęła perplexity 49.907207. Pełne metryki i protokół zapisano w [`benchmarks/english.json`](./benchmarks/english.json).
 
 ### Użycie z Transformers
 
@@ -123,11 +123,12 @@ Model używa standardowego `GPT2LMHeadModel`; `trust_remote_code=True` nie jest
 
 ### Stałe próbki inferencji
 
-Wspólny zestaw [`fixed-sampling-v1`](./inference-samples/README.md) pokazuje te same cztery prompty wygenerowane przez r001-r005. Wszystkie rewizje są ładowane z pełnych, niezmiennych SHA commitów, a SHA-256 każdego pliku z wagami jest sprawdzane przed inferencją. Referencyjny protokół używa macOS 26.2 na arm64, CPU, float32, jednego wątku, Transformers 5.15.1, seed 1337 resetowanego dla każdego promptu, temperature 0.7, top-k 50 i limitu 100 nowych tokenów.
+Wspólny zestaw [`fixed-sampling-v1`](./inference-samples/README.md) pokazuje te same cztery prompty wygenerowane przez r001-r006. Wszystkie opublikowane rewizje są ładowane z pełnych, niezmiennych SHA commitów, a SHA-256 każdego pliku z wagami jest sprawdzane przed inferencją. Referencyjny protokół używa macOS 26.2 na arm64, CPU, float32, jednego wątku, Transformers 5.15.1, seed 1337 resetowanego dla każdego promptu, temperature 0.7, top-k 50 i limitu 100 nowych tokenów.
 
 | Rewizja | Wydanie | Wagi | Historia |
 |---|---|---|---|
-| **r005** | **Pollock 1.3** | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
+| **r006** | **Pollock 1.4** | [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9) | [`r006.md`](./training-history/r006.md) |
+| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
 | r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | [`r004.md`](./training-history/r004.md) |
 | r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | [`r003.md`](./training-history/r003.md) |
 | r002 | Pollock 1.0 | [`5e571cd`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/5e571cde09ef2d1124e13d1c31d0b75f8bd0e513) | [`r002.md`](./training-history/r002.md) |
@@ -150,13 +151,13 @@ Pełne teksty są przeznaczone do porównywania zachowania, nie są benchmarkiem
 
 Pollock is a small English decoder-only base language model trained from scratch as a readable educational experiment. It is based on [nanoGPT](https://github.com/karpathy/nanoGPT) and a custom byte-level BPE tokenizer. It is a completion model, not a conversational assistant.
 
-The name loosely refers to Jackson Pollock's painterly gesture: nanoGPT is the canvas on which data, configuration, and training decisions create different behavioral patterns. See [`training-history/r005.md`](./training-history/r005.md) for the complete technical record and [`CHANGELOG.md`](./CHANGELOG.md) for release-to-release changes.
+The name loosely refers to Jackson Pollock's painterly gesture: nanoGPT is the canvas on which data, configuration, and training decisions create different behavioral patterns. See [`training-history/r006.md`](./training-history/r006.md) for the complete technical record and [`CHANGELOG.md`](./CHANGELOG.md) for release-to-release changes.
 
 ### Architecture and tokenizer
 
 | Property | Value |
 |---|---:|
-| Revision / release | r005 / Pollock 1.3 |
+| Revision / release | r006 / Pollock 1.4 |
 | Type | GPT-2-style decoder-only Transformer |
 | Layers / heads / width | 12 / 14 / 896 |
 | Maximum context | 1,024 tokens |
@@ -170,40 +171,40 @@ The Transformers artifact has 127,674,624 parameters, including 119,168 zero-val
 
 ### Data and training
 
-The model was trained on [`SlayerLab/minimal-en-corpus-5b`](https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-5b), a subjectively selected mixture of 15 English-language sources. Preparation included exact and approximate deduplication, language filtering, and benchmark decontamination. The BPE tokenizer was rebuilt for the new corpus; it retains r004's vocabulary size and special-token IDs, but other token IDs are not compatible across the two revisions.
+The model was trained on V2 of [`SlayerLab/minimal-en-corpus-2.5b`](https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-2.5b), pinned to commit `d68d992`. It is a subjectively selected mixture of 15 English-language sources after language filtering, exact and approximate deduplication, benchmark decontamination, and additional cleanup of boilerplate and incorrectly concatenated records. R006 returns to the tokenizer used by r003-r004; it is not compatible with r005's tokenizer beyond the vocabulary size and special-token IDs.
 
 | Setting | Value |
 |---|---:|
-| Training / validation tokens | 5,425,901,705 / 5,269,579 |
-| Final checkpoint | update 22,078 |
-| Token presentations | 10,851,778,560 (approximately 2 epochs) |
+| Training / validation tokens | 2,689,323,439 / 5,236,486 |
+| Final checkpoint | update 22,003 |
+| Token presentations | 10,814,914,560 (approximately 4.02 epochs) |
 | Sequence / micro-batch per GPU | 1,024 / 12 |
-| Global / per-GPU accumulation | 40 / 40 micro-steps |
+| Global / per-GPU accumulation | 40 / 20 micro-steps |
 | Effective batch | 491,520 tokens |
 | Optimizer | fused AdamW, betas 0.9/0.95 |
 | Learning rate | 4e-4 → 4e-5, cosine decay |
 | Warmup / weight decay / grad clip | 440 / 0.1 / 1.0 |
 | Precision | BF16 |
-| Hardware | 1× NVIDIA GeForce RTX 5090 |
+| Hardware | 2× NVIDIA GeForce RTX 4090 |
 | Framework | PyTorch 2.8.0+cu128, nanoGPT commit `3adf61e` |
 
 ### Evaluation
 
-Training-time loss was estimated on fixed subsets of 1,228,800 tokens per split. The final checkpoint achieved validation loss **2.540666**; the best result was **2.540605** after 10,813,440,000 token presentations. These values are not directly comparable with r001-r004 because r005 uses a different corpus, tokenizer, and validation set.
+Training-time loss was estimated on fixed subsets of 1,228,800 tokens per split. The final checkpoint achieved validation loss **2.536357**; the best result was **2.536231** after 10,813,440,000 token presentations. These values are not directly comparable with r005 because r006 uses a different corpus, tokenizer, and validation set.
 
 Benchmarks used complete splits with `lm-evaluation-harness` 0.4.12, zero few-shot examples, batch size 8, and BF16.
 
 | Benchmark | Primary metric | Score | Samples |
 |---|---|---:|---:|
-| BLiMP | acc | 0.775313 | 67,000 |
-| LAMBADA | acc | 0.297497 | 5,153 |
-| HellaSwag | acc_norm | 0.304521 | 10,042 |
-| PIQA | acc_norm | 0.603373 | 1,838 |
-| SciQ | acc_norm | 0.676000 | 1,000 |
-| ARC-Easy | acc_norm | 0.429714 | 2,376 |
-| ARC-Challenge | acc_norm | 0.232082 | 1,172 |
+| BLiMP | acc | 0.769164 | 67,000 |
+| LAMBADA | acc | 0.276926 | 5,153 |
+| HellaSwag | acc_norm | 0.301334 | 10,042 |
+| PIQA | acc_norm | 0.596844 | 1,838 |
+| SciQ | acc_norm | 0.664000 | 1,000 |
+| ARC-Easy | acc_norm | 0.434343 | 2,376 |
+| ARC-Challenge | acc_norm | 0.241468 | 1,172 |
 
-LAMBADA perplexity was 47.328614. Full metrics and protocol details are recorded in [`benchmarks/english.json`](./benchmarks/english.json).
+LAMBADA perplexity was 49.907207. Full metrics and protocol details are recorded in [`benchmarks/english.json`](./benchmarks/english.json).
 
 ### Usage
 
@@ -211,11 +212,12 @@ Use the Transformers example in the Polish section. The artifact uses standard `
 
 ### Fixed inference samples
 
-The shared [`fixed-sampling-v1`](./inference-samples/README.md) suite runs the same four prompts on r001-r005. All revisions are loaded from full immutable commit SHAs, and every weight-file SHA-256 is verified before inference. The reference protocol uses macOS 26.2 on arm64, CPU float32 with one thread, Transformers 5.15.1, seed 1337 reset for every prompt, temperature 0.7, top-k 50, and a 100-new-token limit.
+The shared [`fixed-sampling-v1`](./inference-samples/README.md) suite runs the same four prompts on r001-r006. All published revisions are loaded from full immutable commit SHAs, and every weight-file SHA-256 is verified before inference. The reference protocol uses macOS 26.2 on arm64, CPU float32 with one thread, Transformers 5.15.1, seed 1337 reset for every prompt, temperature 0.7, top-k 50, and a 100-new-token limit.
 
 | Revision | Release | Weights | History |
 |---|---|---|---|
-| **r005** | **Pollock 1.3** | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
+| **r006** | **Pollock 1.4** | [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9) | [`r006.md`](./training-history/r006.md) |
+| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
 | r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | [`r004.md`](./training-history/r004.md) |
 | r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | [`r003.md`](./training-history/r003.md) |
 | r002 | Pollock 1.0 | [`5e571cd`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/5e571cde09ef2d1124e13d1c31d0b75f8bd0e513) | [`r002.md`](./training-history/r002.md) |

benchmarks/english.json

@@ -1,12 +1,22 @@
 {
   "schema_version": 1,
-  "revision_id": "r005",
-  "release": "Pollock 1.3",
-  "source_result": "runs/pollock-r005-v1p3/benchmarks/results.json",
+  "revision_id": "r006",
+  "release": "Pollock 1.4",
+  "source_result": "runs/r006-corrected-corpus-v2-s1337-lr0.0004/benchmarks/results.json",
   "checkpoint": {
-    "path": "runs/pollock-r005-v1p3/checkpoints/ckpt-final.pt",
-    "sha256": "bc68b2cb8a736fb00fa26b40c425dc1944b2ea02eb9514937ce5ca0908bbe9d2",
-    "source_iteration": 22078
+    "path": "runs/r006-corrected-corpus-v2-s1337-lr0.0004/checkpoints/ckpt-final.pt",
+    "sha256": "580fbb3c039ea1f349285a9bf7a67731fb6d455de931b6b7601d4ed09fce9437",
+    "source_iteration": 22003
+  },
+  "benchmark_checkpoint": {
+    "path": "/workspace/cache/dmpod-tokenizers/r006-final-benchmark.pt",
+    "sha256": "9339d20ea32d24bc9777518a2f0ec553e4a3871647d342db92d7c5b6c18c7b56",
+    "trusted_input_required": true
+  },
+  "tokenizer": {
+    "dataset_revision": "d68d992622e9fc11f19e7d7fb8547c4e653439a4",
+    "sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51",
+    "version": "V2.0@d68d992622e9fc11f19e7d7fb8547c4e653439a4"
   },
   "execution": {
     "harness": "lm-evaluation-harness 0.4.12",
@@ -19,43 +29,43 @@
   },
   "results": {
     "blimp": {
-      "acc,none": 0.7753134328358209,
+      "acc,none": 0.7691641791044775,
       "sample_len": 67000.0,
       "samples": 67000
     },
     "lambada_openai": {
-      "acc,none": 0.29749660392004657,
-      "perplexity,none": 47.32861410229832,
+      "acc,none": 0.27692606248787116,
+      "perplexity,none": 49.907206551888116,
       "sample_len": 5153.0,
       "samples": 5153
     },
     "hellaswag": {
-      "acc,none": 0.28450507866958774,
-      "acc_norm,none": 0.3045210117506473,
+      "acc,none": 0.2830113523202549,
+      "acc_norm,none": 0.3013343955387373,
       "sample_len": 10042.0,
       "samples": 10042
     },
     "piqa": {
-      "acc,none": 0.6099020674646355,
-      "acc_norm,none": 0.6033732317736671,
+      "acc,none": 0.6071817192600653,
+      "acc_norm,none": 0.5968443960826986,
       "sample_len": 1838.0,
       "samples": 1838
     },
     "sciq": {
-      "acc,none": 0.772,
-      "acc_norm,none": 0.676,
+      "acc,none": 0.768,
+      "acc_norm,none": 0.664,
       "sample_len": 1000.0,
       "samples": 1000
     },
     "arc_easy": {
-      "acc,none": 0.48358585858585856,
-      "acc_norm,none": 0.4297138047138047,
+      "acc,none": 0.4877946127946128,
+      "acc_norm,none": 0.43434343434343436,
       "sample_len": 2376.0,
       "samples": 2376
     },
     "arc_challenge": {
-      "acc,none": 0.19197952218430034,
-      "acc_norm,none": 0.23208191126279865,
+      "acc,none": 0.21075085324232082,
+      "acc_norm,none": 0.24146757679180889,
       "sample_len": 1172.0,
       "samples": 1172
     }

config.json

@@ -12,7 +12,7 @@
   "initializer_range": 0.02,
   "layer_norm_epsilon": 1e-05,
   "model_type": "gpt2",
-  "model_version": "1.3",
+  "model_version": "1.4",
   "n_ctx": 1024,
   "n_embd": 896,
   "n_head": 14,

inference-samples/README.md

@@ -23,6 +23,7 @@ Exact token replay was verified in the recorded environment. Sampling may diverg
 | r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | `3a9964a0ba5f6d20ef6061d2a7e4442c1b001209ebe80f7b7148a25573decaa9` |
 | r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | `3c453f22d4bb70228e5cca79e183a425e0f0221e54bd782d12027c42c943c880` |
 | r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | `3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c` |
+| r006 | Pollock 1.4 | [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9) | `8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43` |
 
 ## Results
 
@@ -110,6 +111,15 @@ social
 
 Output SHA-256: `ed9cdec69775563f011b56008d68f9f49f6aa8b74a23e806a8a1e2f7f333557c`
 
+#### r006 - Pollock 1.4
+
+````text
+The capital of France is the capital city of England. It is on the banks of the River Liguria, and is divided into four provinces. Its capital is Paris, and is situated at the end of the Aisne River.
+The municipality of Aisne is situated on the banks of the Neuve-Brieu river, which flows through the north of France. The capital of the city is Aisne-sur-Ligne, on the banks of
+````
+
+Output SHA-256: `b47fcb42354b8f735c7b98cfdf93605c6e095ed8290e2ddb3329886df244e99f`
+
 ### Explanation
 
 Prompt: `Photosynthesis is the process by which`
@@ -175,6 +185,18 @@ Carbon dioxide is a gas, meaning it contains a trace amount of energy. Carbon di
 
 Output SHA-256: `1f68683a0156d242a283349bdc053bb92d0d41a1a6c94daf5cfad67b88fb23d5`
 
+#### r006 - Pollock 1.4
+
+````text
+Photosynthesis is the process by which plants and animals obtain oxygen from the air they breathe.
+Chemical change: A chemical reaction is the change in state of matter or substance that occurs in a substance.
+Chemical reaction: The combining of two substances to produce a new substance or a new compound.
+Chemical reaction: The chemical reaction between two substances to give a new compound.
+Carbon gas: A mixture of gases in which carbon dioxide is replaced by a gas; in which carbon dioxide
+````
+
+Output SHA-256: `c09d560713828851410d37fcd7c648d49c2bedf6e9c8325bb47cb32a8015bfd4`
+
 ### Story
 
 Prompt: `In a distant future, humanity discovered`
@@ -244,6 +266,16 @@ The ability to read and write at the same time was something the world
 
 Output SHA-256: `7feee4648e0b22f7fdf4365efea4ac48c21fd9e1d87afc7c46825ff11808e198`
 
+#### r006 - Pollock 1.4
+
+````text
+In a distant future, humanity discovered that the human race is not only a highly advanced civilization, but also a civilization founded on a shared philosophy that believes in the power of living and the dignity of the earth. The concept of humanity transcends the limits of human understanding, but the human spirit is built on a complex interplay of human intelligence and cultural heritage.
+user
+This is fascinating! Can you tell me more about the specific cultural values that have shaped the concept of humanity?
+````
+
+Output SHA-256: `6455cdbe15a71ae11ff476739071f31b3f86b8bf4d97f3dabe99a7d1595c5c7c`
+
 ### Code
 
 Prompt: `def fibonacci(n):`
@@ -354,6 +386,28 @@ while True
 
 Output SHA-256: `5767898a26ba2104eb5ede9f29195d39715573f09d9d415f58881e5000cd688f`
 
+#### r006 - Pollock 1.4
+
+````text
+def fibonacci(n):
+    return n * fibonacci(n)
+#    if n > 1:
+#        return n + fibonacci(n - 1)
+#    return fibonacci(n)
+#
+#    return fibonacci(2)
+#
+#    return fibonacci(4)
+#
+#    return fibonacci(6)
+#
+#    return fibonacci(10)
+#
+
+````
+
+Output SHA-256: `ad3aa2ea2cab9a46ae5f52a9cc109bcf5f7362e1d214ceb791ee2dd6995a345f`
+
 ## Reproduce or verify
 
 Run from this directory. The check downloads approximately 2.4 GB of published model artifacts if they are not already cached.

inference-samples/config.json

@@ -54,6 +54,12 @@
       "release": "Pollock 1.3",
       "commit": "e780025f15e06bc3765a74d973906eb8a11c022c",
       "model_sha256": "3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c"
+    },
+    {
+      "revision_id": "r006",
+      "release": "Pollock 1.4",
+      "commit": "a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9",
+      "model_sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43"
     }
   ],
   "prompts": [

inference-samples/results.json

@@ -2454,6 +2454,497 @@
           "full_text_sha256": "5767898a26ba2104eb5ede9f29195d39715573f09d9d415f58881e5000cd688f"
         }
       ]
+    },
+    {
+      "revision_id": "r006",
+      "release": "Pollock 1.4",
+      "commit": "a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9",
+      "model_sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43",
+      "loaded_model_version": "1.4",
+      "results": [
+        {
+          "prompt_id": "general_knowledge",
+          "prompt": "The capital of France is",
+          "prompt_token_ids": [
+            492,
+            4514,
+            286,
+            4413,
+            311
+          ],
+          "completion_token_ids": [
+            263,
+            4514,
+            2329,
+            286,
+            3717,
+            46,
+            665,
+            311,
+            336,
+            263,
+            8700,
+            286,
+            263,
+            4286,
+            440,
+            325,
+            329,
+            572,
+            44,
+            292,
+            311,
+            5664,
+            685,
+            1689,
+            881,
+            260,
+            633,
+            46,
+            5646,
+            4514,
+            311,
+            6703,
+            44,
+            292,
+            311,
+            1640,
+            5446,
+            405,
+            263,
+            1009,
+            286,
+            263,
+            335,
+            271,
+            618,
+            4286,
+            46,
+            10,
+            492,
+            10715,
+            1523,
+            286,
+            335,
+            271,
+            618,
+            311,
+            1640,
+            5446,
+            336,
+            263,
+            8700,
+            286,
+            263,
+            2775,
+            117,
+            314,
+            45,
+            66,
+            3388,
+            117,
+            4952,
+            44,
+            506,
+            9271,
+            857,
+            263,
+            4097,
+            286,
+            4413,
+            46,
+            382,
+            4514,
+            286,
+            263,
+            2329,
+            311,
+            335,
+            271,
+            618,
+            45,
+            9745,
+            45,
+            76,
+            579,
+            101,
+            44,
+            336,
+            263,
+            8700,
+            286
+          ],
+          "completion_tokens": 100,
+          "stopped_on_eos": false,
+          "completion": " the capital city of England. It is on the banks of the River Liguria, and is divided into four provinces. Its capital is Paris, and is situated at the end of the Aisne River.\nThe municipality of Aisne is situated on the banks of the Neuve-Brieu river, which flows through the north of France. The capital of the city is Aisne-sur-Ligne, on the banks of",
+          "full_text": "The capital of France is the capital city of England. It is on the banks of the River Liguria, and is divided into four provinces. Its capital is Paris, and is situated at the end of the Aisne River.\nThe municipality of Aisne is situated on the banks of the Neuve-Brieu river, which flows through the north of France. The capital of the city is Aisne-sur-Ligne, on the banks of",
+          "full_text_sha256": "b47fcb42354b8f735c7b98cfdf93605c6e095ed8290e2ddb3329886df244e99f"
+        },
+        {
+          "prompt_id": "explanation",
+          "prompt": "Photosynthesis is the process by which",
+          "prompt_token_ids": [
+            3712,
+            316,
+            394,
+            11580,
+            8310,
+            311,
+            263,
+            1170,
+            411,
+            506
+          ],
+          "completion_token_ids": [
+            3335,
+            292,
+            3620,
+            2663,
+            5715,
+            433,
+            263,
+            1773,
+            548,
+            280,
+            674,
+            258,
+            46,
+            10,
+            67,
+            1601,
+            481,
+            1477,
+            58,
+            335,
+            4144,
+            5372,
+            311,
+            263,
+            1477,
+            287,
+            1158,
+            286,
+            2578,
+            378,
+            8562,
+            342,
+            4682,
+            287,
+            257,
+            8562,
+            46,
+            10,
+            67,
+            1601,
+            481,
+            5372,
+            58,
+            382,
+            1984,
+            1806,
+            286,
+            734,
+            11163,
+            290,
+            3371,
+            257,
+            712,
+            8562,
+            378,
+            257,
+            712,
+            9715,
+            46,
+            10,
+            67,
+            1601,
+            481,
+            5372,
+            58,
+            382,
+            4144,
+            5372,
+            907,
+            734,
+            11163,
+            290,
+            1775,
+            257,
+            712,
+            9715,
+            46,
+            10,
+            7658,
+            2743,
+            2924,
+            58,
+            335,
+            7910,
+            286,
+            10458,
+            287,
+            506,
+            3781,
+            8726,
+            311,
+            6713,
+            411,
+            257,
+            2924,
+            59,
+            287,
+            506,
+            3781,
+            8726
+          ],
+          "completion_tokens": 100,
+          "stopped_on_eos": false,
+          "completion": " plants and animals obtain oxygen from the air they breathe.\nChemical change: A chemical reaction is the change in state of matter or substance that occurs in a substance.\nChemical reaction: The combining of two substances to produce a new substance or a new compound.\nChemical reaction: The chemical reaction between two substances to give a new compound.\nCarbon gas: A mixture of gases in which carbon dioxide is replaced by a gas; in which carbon dioxide",
+          "full_text": "Photosynthesis is the process by which plants and animals obtain oxygen from the air they breathe.\nChemical change: A chemical reaction is the change in state of matter or substance that occurs in a substance.\nChemical reaction: The combining of two substances to produce a new substance or a new compound.\nChemical reaction: The chemical reaction between two substances to give a new compound.\nCarbon gas: A mixture of gases in which carbon dioxide is replaced by a gas; in which carbon dioxide",
+          "full_text_sha256": "c09d560713828851410d37fcd7c648d49c2bedf6e9c8325bb47cb32a8015bfd4"
+        },
+        {
+          "prompt_id": "story",
+          "prompt": "In a distant future, humanity discovered",
+          "prompt_token_ids": [
+            750,
+            257,
+            10103,
+            2629,
+            44,
+            12187,
+            4850
+          ],
+          "completion_token_ids": [
+            342,
+            263,
+            1811,
+            4909,
+            311,
+            434,
+            766,
+            257,
+            4295,
+            5787,
+            4161,
+            1353,
+            44,
+            520,
+            638,
+            257,
+            4161,
+            1353,
+            7887,
+            336,
+            257,
+            5354,
+            10102,
+            342,
+            11980,
+            287,
+            263,
+            1319,
+            286,
+            3099,
+            292,
+            263,
+            289,
+            579,
+            420,
+            286,
+            263,
+            3411,
+            46,
+            382,
+            2713,
+            286,
+            12187,
+            11261,
+            1628,
+            263,
+            7031,
+            286,
+            1811,
+            3137,
+            44,
+            520,
+            263,
+            1811,
+            4042,
+            311,
+            3063,
+            336,
+            257,
+            2481,
+            910,
+            1617,
+            286,
+            1811,
+            8096,
+            292,
+            5289,
+            699,
+            9595,
+            46,
+            12287,
+            10,
+            12286,
+            1512,
+            10,
+            1336,
+            311,
+            9311,
+            7152,
+            33,
+            1885,
+            360,
+            2081,
+            533,
+            583,
+            654,
+            263,
+            1936,
+            5289,
+            1940,
+            342,
+            451,
+            447,
+            5442,
+            263,
+            2713,
+            286,
+            12187,
+            63,
+            12287
+          ],
+          "completion_tokens": 100,
+          "stopped_on_eos": false,
+          "completion": " that the human race is not only a highly advanced civilization, but also a civilization founded on a shared philosophy that believes in the power of living and the dignity of the earth. The concept of humanity transcends the limits of human understanding, but the human spirit is built on a complex interplay of human intelligence and cultural heritage.\nuser\nThis is fascinating! Can you tell me more about the specific cultural values that have shaped the concept of humanity?",
+          "full_text": "In a distant future, humanity discovered that the human race is not only a highly advanced civilization, but also a civilization founded on a shared philosophy that believes in the power of living and the dignity of the earth. The concept of humanity transcends the limits of human understanding, but the human spirit is built on a complex interplay of human intelligence and cultural heritage.\nuser\nThis is fascinating! Can you tell me more about the specific cultural values that have shaped the concept of humanity?",
+          "full_text_sha256": "6455cdbe15a71ae11ff476739071f31b3f86b8bf4d97f3dabe99a7d1595c5c7c"
+        },
+        {
+          "prompt_id": "code",
+          "prompt": "def fibonacci(n):",
+          "prompt_token_ids": [
+            1830,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            110,
+            1000
+          ],
+          "completion_token_ids": [
+            470,
+            982,
+            303,
+            1001,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            110,
+            41,
+            10,
+            35,
+            332,
+            573,
+            303,
+            2035,
+            346,
+            58,
+            10,
+            35,
+            445,
+            982,
+            303,
+            784,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            110,
+            486,
+            346,
+            41,
+            10,
+            35,
+            332,
+            982,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            110,
+            41,
+            10,
+            35,
+            10,
+            35,
+            332,
+            982,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            50,
+            41,
+            10,
+            35,
+            10,
+            35,
+            332,
+            982,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            52,
+            41,
+            10,
+            35,
+            10,
+            35,
+            332,
+            982,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            54,
+            41,
+            10,
+            35,
+            10,
+            35,
+            332,
+            982,
+            7545,
+            262,
+            387,
+            1659,
+            40,
+            929,
+            41,
+            10,
+            35,
+            10
+          ],
+          "completion_tokens": 100,
+          "stopped_on_eos": false,
+          "completion": "\n    return n * fibonacci(n)\n#    if n > 1:\n#        return n + fibonacci(n - 1)\n#    return fibonacci(n)\n#\n#    return fibonacci(2)\n#\n#    return fibonacci(4)\n#\n#    return fibonacci(6)\n#\n#    return fibonacci(10)\n#\n",
+          "full_text": "def fibonacci(n):\n    return n * fibonacci(n)\n#    if n > 1:\n#        return n + fibonacci(n - 1)\n#    return fibonacci(n)\n#\n#    return fibonacci(2)\n#\n#    return fibonacci(4)\n#\n#    return fibonacci(6)\n#\n#    return fibonacci(10)\n#\n",
+          "full_text_sha256": "ad3aa2ea2cab9a46ae5f52a9cc109bcf5f7362e1d214ceb791ee2dd6995a345f"
+        }
+      ]
     }
   ]
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c
+oid sha256:8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43
 size 510713512

release_manifest.json

@@ -1,19 +1,19 @@
 {
   "schema_version": 3,
-  "revision": 5,
-  "revision_id": "r005",
-  "release": "Pollock 1.3",
+  "revision": 6,
+  "revision_id": "r006",
+  "release": "Pollock 1.4",
   "model_id": "SlayerLab/pollock-mini-lm-125m",
   "publication": {
-    "weights_commit": "e780025f15e06bc3765a74d973906eb8a11c022c",
-    "version_tag": "v1.3",
-    "publication_commit_pending": false
+    "weights_commit": "a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9",
+    "version_tag": "v1.4",
+    "publication_commit_pending": true
   },
   "source_checkpoint": {
-    "path_in_training_workspace": "runs/pollock-r005-v1p3/checkpoints/ckpt-final.pt",
-    "sha256": "bc68b2cb8a736fb00fa26b40c425dc1944b2ea02eb9514937ce5ca0908bbe9d2",
-    "iteration": 22078,
-    "tokens_seen": 10851778560,
+    "path_in_training_workspace": "runs/r006-corrected-corpus-v2-s1337-lr0.0004/checkpoints/ckpt-final.pt",
+    "sha256": "580fbb3c039ea1f349285a9bf7a67731fb6d455de931b6b7601d4ed09fce9437",
+    "iteration": 22003,
+    "tokens_seen": 10814914560,
     "native_nanogpt_parameters": 126637952,
     "native_unique_trainable_parameters": 127555456
   },
@@ -28,14 +28,17 @@
     "tied_word_embeddings": true
   },
   "training": {
-    "dataset": "SlayerLab/minimal-en-corpus-5b",
-    "training_tokens": 5425901705,
-    "validation_tokens": 5269579,
-    "tokenizer_sha256": "3733307577230bb4802d2c774d8e2323f7f64e4139c13712736a57daf91bdda1",
+    "dataset": "SlayerLab/minimal-en-corpus-2.5b",
+    "dataset_revision": "d68d992622e9fc11f19e7d7fb8547c4e653439a4",
+    "training_tokens": 2689323439,
+    "validation_tokens": 5236486,
+    "tokenizer_sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51",
+    "native_tokenizer_sha256": "d0d126f0a59e51e8cb1b26d77b5527870be0e8ea1e84adefd730fff906c48234",
     "init_from": "scratch",
     "micro_batch_per_gpu": 12,
     "gradient_accumulation_global": 40,
-    "ddp_world_size": 1,
+    "gradient_accumulation_per_gpu": 20,
+    "ddp_world_size": 2,
     "effective_batch_tokens": 491520,
     "optimizer": "fused AdamW",
     "learning_rate": 0.0004,
@@ -47,26 +50,28 @@
     "weight_decay": 0.1,
     "grad_clip": 1.0,
     "precision": "bfloat16",
-    "hardware": "1x NVIDIA GeForce RTX 5090",
+    "hardware": "2x NVIDIA GeForce RTX 4090",
     "framework": "PyTorch 2.8.0+cu128",
     "nanogpt_commit": "3adf61e154c3fe3fca428ad6bc3818b27a3b8291",
-    "data_pass_equivalent": 1.9999954201160746,
-    "runtime_hours": 16.855,
-    "mean_tokens_per_second": 182508.3
+    "data_pass_equivalent": 4.021425762020438,
+    "runtime_hours": 15.226956854563,
+    "mean_tokens_per_second": 199894.6379482684
   },
   "evaluation": {
     "protocol": "fixed sampled subset",
     "subset_tokens_per_split": 1228800,
-    "final_train_loss": 2.447470265626907,
-    "final_validation_loss": 2.5406661581993104,
-    "best_validation_loss": 2.5406045150756835,
+    "final_train_loss": 2.48974818944931,
+    "final_validation_loss": 2.536356544494629,
+    "best_validation_loss": 2.5362311387062073,
     "best_validation_update": 22000,
     "best_validation_tokens_seen": 10813440000,
-    "final_validation_perplexity": 12.68812045164865,
+    "final_validation_perplexity": 12.633557211556399,
     "benchmark_harness": "lm-evaluation-harness 0.4.12",
     "benchmark_num_fewshot": 0,
     "benchmark_batch_size": 8,
     "truncated_benchmark_requests": 0,
+    "benchmark_checkpoint_sha256": "9339d20ea32d24bc9777518a2f0ec553e4a3871647d342db92d7c5b6c18c7b56",
+    "benchmark_tokenizer_sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51",
     "results_file": "benchmarks/english.json"
   },
   "fixed_inference": {
@@ -81,43 +86,44 @@
     "dtype": "float32",
     "seed": 1337,
     "prompt_count": 4,
-    "revision_count": 5,
+    "revision_count": 6,
     "local_review_artifact": false,
     "exact_replay_verified": true,
-    "publication_commit_pending": false
+    "publication_commit_pending": true
   },
   "conversion": {
     "target_class": "GPT2LMHeadModel",
     "transformers_version": "5.15.1",
-    "checkpoint_sha256": "bc68b2cb8a736fb00fa26b40c425dc1944b2ea02eb9514937ce5ca0908bbe9d2",
-    "model_sha256": "3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c",
+    "checkpoint_sha256": "580fbb3c039ea1f349285a9bf7a67731fb6d455de931b6b7601d4ed09fce9437",
+    "model_sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43",
     "unique_serialized_parameters": 127674624,
     "compatibility_zero_bias_parameters": 119168,
     "validation_probe_shape": [2, 64],
-    "max_absolute_logit_error": 4.76837158203125e-06
+    "max_absolute_logit_error": 0.0
   },
   "artifacts": {
     ".gitattributes": {"sha256": "c759491a998899dbefaec4d51cc791e68c714a37dd9f9829020a368788fe3063"},
-    "CHANGELOG.md": {"sha256": "74a87cefef26ff31a9b9828df875d8c3a68383e13e360ff7ce201cfceb9af915"},
-    "LICENSE.md": {"sha256": "162b97898a956d43d0a5428d1999bac3e42e2a97eb01a79f73ad719a4a15c412"},
-    "README.md": {"sha256": "1958e268e5ac6838941b8225d27f506bbce4e37c8f76a9045d5913129e874ad7"},
+    "CHANGELOG.md": {"sha256": "8422ad337b22be7adb561d1b2388cf53e5a6d1c6d49d628468498c72f078769a"},
+    "LICENSE.md": {"sha256": "46cbe928ed0aa24875f02f774313b27ec9d8abdf41f0c9ef67e0adb4e0614de4"},
+    "README.md": {"sha256": "875fa9cd748691e376df905e036a0c471db77aaa1272d71c5ea10d0f76fa888d"},
     "assets/pollock-mini-lm-avatar-320.png": {"sha256": "7be10cc9d0f4aedeb298d9a5d722b2b2ac5b2e219b3884916dbca16198f70750"},
-    "benchmarks/english.json": {"sha256": "c9210b71334898140b543ce684e9209c1919bbcdceb43224f4c5770f4fb93db7"},
-    "config.json": {"sha256": "ba6775a805995b19d738604bcb035fccf012a0f4596b156d9945fd3a55c50d0b"},
+    "benchmarks/english.json": {"sha256": "646a3aff7c97859e6974baa2c191702829a982af87675870dd1a339f8073a146"},
+    "config.json": {"sha256": "1529f8a8fc31b4fa68fc18fffbad7e155dc8f9e8663a40b4f3306b4012ea38c0"},
     "generation_config.json": {"sha256": "435beb27be51f0ed054f4a011e5109d125cdadc118b8799b18b155cc798d94d2"},
-    "inference-samples/README.md": {"sha256": "178c85ec0e003754371bde7aea0af55fe918c8d45f8cad79ec371264f52210f1"},
-    "inference-samples/config.json": {"sha256": "fb504c65e8c6e01aa8d7f235c7f2ab0a88b95b2acbd884ec1e1d0e083669cdec"},
+    "inference-samples/README.md": {"sha256": "6559c85e1ed70f47f2d2978f28151fdd9e3355e397c5591cbe0b8ab767fa5ac5"},
+    "inference-samples/config.json": {"sha256": "a1b306a1e1f3a70942706d00107a83dd98289a534d17af78adbb95d48c9b69c1"},
     "inference-samples/generate.py": {"sha256": "807a9946472bd0087dd2b9260ee90d1dea09785d24d4f923f028667c53a23933"},
     "inference-samples/requirements.txt": {"sha256": "584583335ffb3061aa62058aee725b0de25fed7523904fbd13fb4622f601943d"},
-    "inference-samples/results.json": {"sha256": "c2eee21056ddd95eb3ece7d5bcd93242e62d90302814c0178c6beb4425d81553"},
-    "model.safetensors": {"sha256": "3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c"},
+    "inference-samples/results.json": {"sha256": "a279806ea225130b0e0aa36d9bd13653401e236395fe64b5f0aa7c825d445620"},
+    "model.safetensors": {"sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43"},
     "special_tokens_map.json": {"sha256": "8b2257a17ea997bb038f43b133aefec82344ad2b8abc2b8a02a6c0a994ed624e"},
-    "tokenizer.json": {"sha256": "3733307577230bb4802d2c774d8e2323f7f64e4139c13712736a57daf91bdda1"},
+    "tokenizer.json": {"sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51"},
     "tokenizer_config.json": {"sha256": "4cdabe37dbdc1adfcc017ee9a1f86ab89bdf184827d2d9f05181cae0f8af19bf"},
-    "training-history/r001.md": {"sha256": "b914562e1b67c92d279cdda55dda93965c35737475b000a758db95564a8c895e"},
-    "training-history/r002.md": {"sha256": "2b777a46e34fad856f323742dd0adf7f2c8ba6ee73a2ba2106d727644b1c8ca8"},
-    "training-history/r003.md": {"sha256": "9fb6a1e04bdadada6f8760cc64a96e538ee7d23b7fe112959d8ba45b89ca8ada"},
-    "training-history/r004.md": {"sha256": "968191c3a314989e5bfc749cd79e105ae06d5622c24b0f870435e698c969ac63"},
-    "training-history/r005.md": {"sha256": "a25c2932701695dd7402582510adbe71e50a6fcc2da0fb3c6224a846dfdb22df"}
+    "training-history/r001.md": {"sha256": "5b0ac39265921ed3604e7dfe8181b00d07902640d8024f591462eeb508bcfece"},
+    "training-history/r002.md": {"sha256": "80031a84b2b67dc4cb4f81a178fa354feea15cb184d17368ee7e64ec3d0a2629"},
+    "training-history/r003.md": {"sha256": "7ea9e6b15a0af635ff238417b9fff2a98c12a9090800a0923d622253b7eab02d"},
+    "training-history/r004.md": {"sha256": "e260bfba69f154caf16f405d6944c2f963f467da9ed1e6c3c94e08932feae9b9"},
+    "training-history/r005.md": {"sha256": "a25c2932701695dd7402582510adbe71e50a6fcc2da0fb3c6224a846dfdb22df"},
+    "training-history/r006.md": {"sha256": "47d53f9aa8a233eb7a319f559209936df988e83d96f6f6517fa3670b5dd2cdd3"}
   }
 }

training-history/r001.md

@@ -2,7 +2,7 @@
 
 ## Identity
 
-`r001` is the first published experimental predecessor of Pollock 1.0. All values and samples in this file describe the r001 weights, not the current r005 weights.
+`r001` is the first published experimental predecessor of Pollock 1.0. All values and samples in this file describe the r001 weights, not the current r006 weights.
 
 | Field | Value |
 |---|---|
@@ -101,7 +101,7 @@ def fibonacci(n):)) = (n - 1) + (n - 1) + ( n - 1) + ( n - 2) + ( n - 1) + ( n -
 
 ## Fixed cross-revision inference reference
 
-The later [`fixed-sampling-v1`](../inference-samples/README.md) suite reruns the same four prompts on r001-r005 using immutable commits for published revisions, verified weight hashes, CPU float32, a pinned software environment, and exact output token IDs. Use that suite, rather than the historical samples above, for direct revision-to-revision comparison.
+The later [`fixed-sampling-v1`](../inference-samples/README.md) suite reruns the same four prompts on r001-r006 using immutable commits for published revisions, verified weight hashes, CPU float32, a pinned software environment, and exact output token IDs. Use that suite, rather than the historical samples above, for direct revision-to-revision comparison.
 
 ## Limitations
 

training-history/r002.md

@@ -104,7 +104,7 @@ Structured results: [`../benchmarks/english.json`](../benchmarks/english.json).
 
 ## Fixed inference reference
 
-The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r002 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r005.
+The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r002 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.
 
 ## Licensing and limitations
 

training-history/r003.md

@@ -123,7 +123,7 @@ Structured English results: [`../benchmarks/english.json`](../benchmarks/english
 
 ## Fixed inference reference
 
-The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r003 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r005.
+The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r003 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.
 
 ## Release preparation status
 

training-history/r004.md

@@ -132,7 +132,7 @@ Structured English results: [`../benchmarks/english.json`](../benchmarks/english
 
 ## Fixed inference reference
 
-The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r004 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r005.
+The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r004 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.
 
 ## Release preparation status
 

training-history/r006.md

@@ -0,0 +1,49 @@
+# r006 — Pollock 1.4 training record
+
+## Identity and provenance
+
+| Field | Value |
+|---|---|
+| Revision / release | `r006` / Pollock 1.4 |
+| Source run | `r006-corrected-corpus-v2-s1337-lr0.0004` |
+| Final checkpoint SHA-256 | `580fbb3c039ea1f349285a9bf7a67731fb6d455de931b6b7601d4ed09fce9437` |
+| `model.safetensors` SHA-256 | `8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43` |
+
+## Architecture and training
+
+| Setting | Value |
+|---|---:|
+| Layers / heads / width | 12 / 14 / 896 |
+| Context / vocabulary | 1024 / 12288 |
+| Native trainable parameters | 127,555,456 |
+| Dataset | SlayerLab/minimal-en-corpus-2.5b V2 (`d68d992`) |
+| Tokenizer SHA-256 | `6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51` |
+| Token presentations / passes | 10,814,914,560 / 4.021426 |
+| Effective batch | 491,520 tokens |
+| LR range | 4e-04 → 4e-05 |
+| Precision / hardware | BF16 / 2× NVIDIA GeForce RTX 4090 |
+| Runtime / mean throughput | 15.227 h / 199,894.6 tokens/s |
+
+The frozen training source stores the same tokenizer mapping in nanoGPT's native merge format, whose SHA-256 is `d0d126f0a59e51e8cb1b26d77b5527870be0e8ea1e84adefd730fff906c48234`.
+
+## Training-time evaluation
+
+Final train-eval loss: **2.489748**. Final validation loss: **2.536357** (perplexity 12.633557). Minimum validation loss: **2.536231** at 10,813,440,000 tokens.
+
+## English zero-shot benchmarks
+
+| Benchmark | Primary metric | Score | Samples |
+|---|---|---:|---:|
+| BLiMP | acc | 0.769164 | 67,000 |
+| LAMBADA | acc | 0.276926 | 5,153 |
+| HellaSwag | acc_norm | 0.301334 | 10,042 |
+| PIQA | acc_norm | 0.596844 | 1,838 |
+| SciQ | acc_norm | 0.664000 | 1,000 |
+| ARC-Easy | acc_norm | 0.434343 | 2,376 |
+| ARC-Challenge | acc_norm | 0.241468 | 1,172 |
+
+LAMBADA perplexity was **49.907207**. Benchmarks used complete splits, zero few-shot examples, `lm-evaluation-harness` 0.4.12, BF16, batch size 8, and a 1,024-token maximum context. No requests were truncated. The exact temporary benchmark-checkpoint and tokenizer hashes are preserved in [`../benchmarks/english.json`](../benchmarks/english.json).
+
+## Conversion and release status
+
+The native final checkpoint was converted to `GPT2LMHeadModel` with Transformers 5.15.1. A deterministic `[2, 64]` parity probe produced maximum absolute logit error **0.0**. The fixed inference suite replayed exactly against immutable Hugging Face weights commit [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9), with the expected model-file SHA-256 verified before loading. Merge and version tag `v1.4` remain pending.