Text Generation
Transformers
Safetensors
English
gpt2
causal-lm
nanogpt
bpe
educational
base-model
Eval Results (legacy)
text-generation-inference
Instructions to use SlayerLab/pollock-mini-lm-125m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SlayerLab/pollock-mini-lm-125m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="SlayerLab/pollock-mini-lm-125m")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("SlayerLab/pollock-mini-lm-125m") model = AutoModelForCausalLM.from_pretrained("SlayerLab/pollock-mini-lm-125m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use SlayerLab/pollock-mini-lm-125m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SlayerLab/pollock-mini-lm-125m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SlayerLab/pollock-mini-lm-125m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/SlayerLab/pollock-mini-lm-125m
- SGLang
How to use SlayerLab/pollock-mini-lm-125m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "SlayerLab/pollock-mini-lm-125m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SlayerLab/pollock-mini-lm-125m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "SlayerLab/pollock-mini-lm-125m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SlayerLab/pollock-mini-lm-125m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use SlayerLab/pollock-mini-lm-125m with Docker Model Runner:
docker model run hf.co/SlayerLab/pollock-mini-lm-125m
Finalize Pollock 1.4 release metadata
Browse files- README.md +2 -2
- inference-samples/README.md +54 -0
- inference-samples/config.json +6 -0
- inference-samples/results.json +491 -0
- release_manifest.json +49 -43
- training-history/r001.md +2 -2
- training-history/r002.md +1 -1
- training-history/r003.md +1 -1
- training-history/r004.md +1 -1
- training-history/r006.md +1 -1
README.md
CHANGED
|
@@ -127,7 +127,7 @@ Wspólny zestaw [`fixed-sampling-v1`](./inference-samples/README.md) pokazuje te
|
|
| 127 |
|
| 128 |
| Rewizja | Wydanie | Wagi | Historia |
|
| 129 |
|---|---|---|---|
|
| 130 |
-
| **r006** | **Pollock 1.4** |
|
| 131 |
| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
|
| 132 |
| r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | [`r004.md`](./training-history/r004.md) |
|
| 133 |
| r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | [`r003.md`](./training-history/r003.md) |
|
|
@@ -216,7 +216,7 @@ The shared [`fixed-sampling-v1`](./inference-samples/README.md) suite runs the s
|
|
| 216 |
|
| 217 |
| Revision | Release | Weights | History |
|
| 218 |
|---|---|---|---|
|
| 219 |
-
| **r006** | **Pollock 1.4** |
|
| 220 |
| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
|
| 221 |
| r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | [`r004.md`](./training-history/r004.md) |
|
| 222 |
| r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | [`r003.md`](./training-history/r003.md) |
|
|
|
|
| 127 |
|
| 128 |
| Rewizja | Wydanie | Wagi | Historia |
|
| 129 |
|---|---|---|---|
|
| 130 |
+
| **r006** | **Pollock 1.4** | [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9) | [`r006.md`](./training-history/r006.md) |
|
| 131 |
| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
|
| 132 |
| r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | [`r004.md`](./training-history/r004.md) |
|
| 133 |
| r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | [`r003.md`](./training-history/r003.md) |
|
|
|
|
| 216 |
|
| 217 |
| Revision | Release | Weights | History |
|
| 218 |
|---|---|---|---|
|
| 219 |
+
| **r006** | **Pollock 1.4** | [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9) | [`r006.md`](./training-history/r006.md) |
|
| 220 |
| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | [`r005.md`](./training-history/r005.md) |
|
| 221 |
| r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | [`r004.md`](./training-history/r004.md) |
|
| 222 |
| r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | [`r003.md`](./training-history/r003.md) |
|
inference-samples/README.md
CHANGED
|
@@ -23,6 +23,7 @@ Exact token replay was verified in the recorded environment. Sampling may diverg
|
|
| 23 |
| r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | `3a9964a0ba5f6d20ef6061d2a7e4442c1b001209ebe80f7b7148a25573decaa9` |
|
| 24 |
| r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | `3c453f22d4bb70228e5cca79e183a425e0f0221e54bd782d12027c42c943c880` |
|
| 25 |
| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | `3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c` |
|
|
|
|
| 26 |
|
| 27 |
## Results
|
| 28 |
|
|
@@ -110,6 +111,15 @@ social
|
|
| 110 |
|
| 111 |
Output SHA-256: `ed9cdec69775563f011b56008d68f9f49f6aa8b74a23e806a8a1e2f7f333557c`
|
| 112 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 113 |
### Explanation
|
| 114 |
|
| 115 |
Prompt: `Photosynthesis is the process by which`
|
|
@@ -175,6 +185,18 @@ Carbon dioxide is a gas, meaning it contains a trace amount of energy. Carbon di
|
|
| 175 |
|
| 176 |
Output SHA-256: `1f68683a0156d242a283349bdc053bb92d0d41a1a6c94daf5cfad67b88fb23d5`
|
| 177 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 178 |
### Story
|
| 179 |
|
| 180 |
Prompt: `In a distant future, humanity discovered`
|
|
@@ -244,6 +266,16 @@ The ability to read and write at the same time was something the world
|
|
| 244 |
|
| 245 |
Output SHA-256: `7feee4648e0b22f7fdf4365efea4ac48c21fd9e1d87afc7c46825ff11808e198`
|
| 246 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 247 |
### Code
|
| 248 |
|
| 249 |
Prompt: `def fibonacci(n):`
|
|
@@ -354,6 +386,28 @@ while True
|
|
| 354 |
|
| 355 |
Output SHA-256: `5767898a26ba2104eb5ede9f29195d39715573f09d9d415f58881e5000cd688f`
|
| 356 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 357 |
## Reproduce or verify
|
| 358 |
|
| 359 |
Run from this directory. The check downloads approximately 2.4 GB of published model artifacts if they are not already cached.
|
|
|
|
| 23 |
| r003 | Pollock 1.1 | [`698984b`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/698984b1d1b96c9b6ffaff00c7fc2e78e140e842) | `3a9964a0ba5f6d20ef6061d2a7e4442c1b001209ebe80f7b7148a25573decaa9` |
|
| 24 |
| r004 | Pollock 1.2 | [`30feb81`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/30feb81e5097eef13b939e1827d98e0458bf602d) | `3c453f22d4bb70228e5cca79e183a425e0f0221e54bd782d12027c42c943c880` |
|
| 25 |
| r005 | Pollock 1.3 | [`e780025`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/e780025f15e06bc3765a74d973906eb8a11c022c) | `3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c` |
|
| 26 |
+
| r006 | Pollock 1.4 | [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9) | `8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43` |
|
| 27 |
|
| 28 |
## Results
|
| 29 |
|
|
|
|
| 111 |
|
| 112 |
Output SHA-256: `ed9cdec69775563f011b56008d68f9f49f6aa8b74a23e806a8a1e2f7f333557c`
|
| 113 |
|
| 114 |
+
#### r006 - Pollock 1.4
|
| 115 |
+
|
| 116 |
+
````text
|
| 117 |
+
The capital of France is the capital city of England. It is on the banks of the River Liguria, and is divided into four provinces. Its capital is Paris, and is situated at the end of the Aisne River.
|
| 118 |
+
The municipality of Aisne is situated on the banks of the Neuve-Brieu river, which flows through the north of France. The capital of the city is Aisne-sur-Ligne, on the banks of
|
| 119 |
+
````
|
| 120 |
+
|
| 121 |
+
Output SHA-256: `b47fcb42354b8f735c7b98cfdf93605c6e095ed8290e2ddb3329886df244e99f`
|
| 122 |
+
|
| 123 |
### Explanation
|
| 124 |
|
| 125 |
Prompt: `Photosynthesis is the process by which`
|
|
|
|
| 185 |
|
| 186 |
Output SHA-256: `1f68683a0156d242a283349bdc053bb92d0d41a1a6c94daf5cfad67b88fb23d5`
|
| 187 |
|
| 188 |
+
#### r006 - Pollock 1.4
|
| 189 |
+
|
| 190 |
+
````text
|
| 191 |
+
Photosynthesis is the process by which plants and animals obtain oxygen from the air they breathe.
|
| 192 |
+
Chemical change: A chemical reaction is the change in state of matter or substance that occurs in a substance.
|
| 193 |
+
Chemical reaction: The combining of two substances to produce a new substance or a new compound.
|
| 194 |
+
Chemical reaction: The chemical reaction between two substances to give a new compound.
|
| 195 |
+
Carbon gas: A mixture of gases in which carbon dioxide is replaced by a gas; in which carbon dioxide
|
| 196 |
+
````
|
| 197 |
+
|
| 198 |
+
Output SHA-256: `c09d560713828851410d37fcd7c648d49c2bedf6e9c8325bb47cb32a8015bfd4`
|
| 199 |
+
|
| 200 |
### Story
|
| 201 |
|
| 202 |
Prompt: `In a distant future, humanity discovered`
|
|
|
|
| 266 |
|
| 267 |
Output SHA-256: `7feee4648e0b22f7fdf4365efea4ac48c21fd9e1d87afc7c46825ff11808e198`
|
| 268 |
|
| 269 |
+
#### r006 - Pollock 1.4
|
| 270 |
+
|
| 271 |
+
````text
|
| 272 |
+
In a distant future, humanity discovered that the human race is not only a highly advanced civilization, but also a civilization founded on a shared philosophy that believes in the power of living and the dignity of the earth. The concept of humanity transcends the limits of human understanding, but the human spirit is built on a complex interplay of human intelligence and cultural heritage.
|
| 273 |
+
user
|
| 274 |
+
This is fascinating! Can you tell me more about the specific cultural values that have shaped the concept of humanity?
|
| 275 |
+
````
|
| 276 |
+
|
| 277 |
+
Output SHA-256: `6455cdbe15a71ae11ff476739071f31b3f86b8bf4d97f3dabe99a7d1595c5c7c`
|
| 278 |
+
|
| 279 |
### Code
|
| 280 |
|
| 281 |
Prompt: `def fibonacci(n):`
|
|
|
|
| 386 |
|
| 387 |
Output SHA-256: `5767898a26ba2104eb5ede9f29195d39715573f09d9d415f58881e5000cd688f`
|
| 388 |
|
| 389 |
+
#### r006 - Pollock 1.4
|
| 390 |
+
|
| 391 |
+
````text
|
| 392 |
+
def fibonacci(n):
|
| 393 |
+
return n * fibonacci(n)
|
| 394 |
+
# if n > 1:
|
| 395 |
+
# return n + fibonacci(n - 1)
|
| 396 |
+
# return fibonacci(n)
|
| 397 |
+
#
|
| 398 |
+
# return fibonacci(2)
|
| 399 |
+
#
|
| 400 |
+
# return fibonacci(4)
|
| 401 |
+
#
|
| 402 |
+
# return fibonacci(6)
|
| 403 |
+
#
|
| 404 |
+
# return fibonacci(10)
|
| 405 |
+
#
|
| 406 |
+
|
| 407 |
+
````
|
| 408 |
+
|
| 409 |
+
Output SHA-256: `ad3aa2ea2cab9a46ae5f52a9cc109bcf5f7362e1d214ceb791ee2dd6995a345f`
|
| 410 |
+
|
| 411 |
## Reproduce or verify
|
| 412 |
|
| 413 |
Run from this directory. The check downloads approximately 2.4 GB of published model artifacts if they are not already cached.
|
inference-samples/config.json
CHANGED
|
@@ -54,6 +54,12 @@
|
|
| 54 |
"release": "Pollock 1.3",
|
| 55 |
"commit": "e780025f15e06bc3765a74d973906eb8a11c022c",
|
| 56 |
"model_sha256": "3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
}
|
| 58 |
],
|
| 59 |
"prompts": [
|
|
|
|
| 54 |
"release": "Pollock 1.3",
|
| 55 |
"commit": "e780025f15e06bc3765a74d973906eb8a11c022c",
|
| 56 |
"model_sha256": "3db43dfa44622e8156f060fee6f16875e108db4f5f949969cb91c0ca8cdce32c"
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"revision_id": "r006",
|
| 60 |
+
"release": "Pollock 1.4",
|
| 61 |
+
"commit": "a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9",
|
| 62 |
+
"model_sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43"
|
| 63 |
}
|
| 64 |
],
|
| 65 |
"prompts": [
|
inference-samples/results.json
CHANGED
|
@@ -2454,6 +2454,497 @@
|
|
| 2454 |
"full_text_sha256": "5767898a26ba2104eb5ede9f29195d39715573f09d9d415f58881e5000cd688f"
|
| 2455 |
}
|
| 2456 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2457 |
}
|
| 2458 |
]
|
| 2459 |
}
|
|
|
|
| 2454 |
"full_text_sha256": "5767898a26ba2104eb5ede9f29195d39715573f09d9d415f58881e5000cd688f"
|
| 2455 |
}
|
| 2456 |
]
|
| 2457 |
+
},
|
| 2458 |
+
{
|
| 2459 |
+
"revision_id": "r006",
|
| 2460 |
+
"release": "Pollock 1.4",
|
| 2461 |
+
"commit": "a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9",
|
| 2462 |
+
"model_sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43",
|
| 2463 |
+
"loaded_model_version": "1.4",
|
| 2464 |
+
"results": [
|
| 2465 |
+
{
|
| 2466 |
+
"prompt_id": "general_knowledge",
|
| 2467 |
+
"prompt": "The capital of France is",
|
| 2468 |
+
"prompt_token_ids": [
|
| 2469 |
+
492,
|
| 2470 |
+
4514,
|
| 2471 |
+
286,
|
| 2472 |
+
4413,
|
| 2473 |
+
311
|
| 2474 |
+
],
|
| 2475 |
+
"completion_token_ids": [
|
| 2476 |
+
263,
|
| 2477 |
+
4514,
|
| 2478 |
+
2329,
|
| 2479 |
+
286,
|
| 2480 |
+
3717,
|
| 2481 |
+
46,
|
| 2482 |
+
665,
|
| 2483 |
+
311,
|
| 2484 |
+
336,
|
| 2485 |
+
263,
|
| 2486 |
+
8700,
|
| 2487 |
+
286,
|
| 2488 |
+
263,
|
| 2489 |
+
4286,
|
| 2490 |
+
440,
|
| 2491 |
+
325,
|
| 2492 |
+
329,
|
| 2493 |
+
572,
|
| 2494 |
+
44,
|
| 2495 |
+
292,
|
| 2496 |
+
311,
|
| 2497 |
+
5664,
|
| 2498 |
+
685,
|
| 2499 |
+
1689,
|
| 2500 |
+
881,
|
| 2501 |
+
260,
|
| 2502 |
+
633,
|
| 2503 |
+
46,
|
| 2504 |
+
5646,
|
| 2505 |
+
4514,
|
| 2506 |
+
311,
|
| 2507 |
+
6703,
|
| 2508 |
+
44,
|
| 2509 |
+
292,
|
| 2510 |
+
311,
|
| 2511 |
+
1640,
|
| 2512 |
+
5446,
|
| 2513 |
+
405,
|
| 2514 |
+
263,
|
| 2515 |
+
1009,
|
| 2516 |
+
286,
|
| 2517 |
+
263,
|
| 2518 |
+
335,
|
| 2519 |
+
271,
|
| 2520 |
+
618,
|
| 2521 |
+
4286,
|
| 2522 |
+
46,
|
| 2523 |
+
10,
|
| 2524 |
+
492,
|
| 2525 |
+
10715,
|
| 2526 |
+
1523,
|
| 2527 |
+
286,
|
| 2528 |
+
335,
|
| 2529 |
+
271,
|
| 2530 |
+
618,
|
| 2531 |
+
311,
|
| 2532 |
+
1640,
|
| 2533 |
+
5446,
|
| 2534 |
+
336,
|
| 2535 |
+
263,
|
| 2536 |
+
8700,
|
| 2537 |
+
286,
|
| 2538 |
+
263,
|
| 2539 |
+
2775,
|
| 2540 |
+
117,
|
| 2541 |
+
314,
|
| 2542 |
+
45,
|
| 2543 |
+
66,
|
| 2544 |
+
3388,
|
| 2545 |
+
117,
|
| 2546 |
+
4952,
|
| 2547 |
+
44,
|
| 2548 |
+
506,
|
| 2549 |
+
9271,
|
| 2550 |
+
857,
|
| 2551 |
+
263,
|
| 2552 |
+
4097,
|
| 2553 |
+
286,
|
| 2554 |
+
4413,
|
| 2555 |
+
46,
|
| 2556 |
+
382,
|
| 2557 |
+
4514,
|
| 2558 |
+
286,
|
| 2559 |
+
263,
|
| 2560 |
+
2329,
|
| 2561 |
+
311,
|
| 2562 |
+
335,
|
| 2563 |
+
271,
|
| 2564 |
+
618,
|
| 2565 |
+
45,
|
| 2566 |
+
9745,
|
| 2567 |
+
45,
|
| 2568 |
+
76,
|
| 2569 |
+
579,
|
| 2570 |
+
101,
|
| 2571 |
+
44,
|
| 2572 |
+
336,
|
| 2573 |
+
263,
|
| 2574 |
+
8700,
|
| 2575 |
+
286
|
| 2576 |
+
],
|
| 2577 |
+
"completion_tokens": 100,
|
| 2578 |
+
"stopped_on_eos": false,
|
| 2579 |
+
"completion": " the capital city of England. It is on the banks of the River Liguria, and is divided into four provinces. Its capital is Paris, and is situated at the end of the Aisne River.\nThe municipality of Aisne is situated on the banks of the Neuve-Brieu river, which flows through the north of France. The capital of the city is Aisne-sur-Ligne, on the banks of",
|
| 2580 |
+
"full_text": "The capital of France is the capital city of England. It is on the banks of the River Liguria, and is divided into four provinces. Its capital is Paris, and is situated at the end of the Aisne River.\nThe municipality of Aisne is situated on the banks of the Neuve-Brieu river, which flows through the north of France. The capital of the city is Aisne-sur-Ligne, on the banks of",
|
| 2581 |
+
"full_text_sha256": "b47fcb42354b8f735c7b98cfdf93605c6e095ed8290e2ddb3329886df244e99f"
|
| 2582 |
+
},
|
| 2583 |
+
{
|
| 2584 |
+
"prompt_id": "explanation",
|
| 2585 |
+
"prompt": "Photosynthesis is the process by which",
|
| 2586 |
+
"prompt_token_ids": [
|
| 2587 |
+
3712,
|
| 2588 |
+
316,
|
| 2589 |
+
394,
|
| 2590 |
+
11580,
|
| 2591 |
+
8310,
|
| 2592 |
+
311,
|
| 2593 |
+
263,
|
| 2594 |
+
1170,
|
| 2595 |
+
411,
|
| 2596 |
+
506
|
| 2597 |
+
],
|
| 2598 |
+
"completion_token_ids": [
|
| 2599 |
+
3335,
|
| 2600 |
+
292,
|
| 2601 |
+
3620,
|
| 2602 |
+
2663,
|
| 2603 |
+
5715,
|
| 2604 |
+
433,
|
| 2605 |
+
263,
|
| 2606 |
+
1773,
|
| 2607 |
+
548,
|
| 2608 |
+
280,
|
| 2609 |
+
674,
|
| 2610 |
+
258,
|
| 2611 |
+
46,
|
| 2612 |
+
10,
|
| 2613 |
+
67,
|
| 2614 |
+
1601,
|
| 2615 |
+
481,
|
| 2616 |
+
1477,
|
| 2617 |
+
58,
|
| 2618 |
+
335,
|
| 2619 |
+
4144,
|
| 2620 |
+
5372,
|
| 2621 |
+
311,
|
| 2622 |
+
263,
|
| 2623 |
+
1477,
|
| 2624 |
+
287,
|
| 2625 |
+
1158,
|
| 2626 |
+
286,
|
| 2627 |
+
2578,
|
| 2628 |
+
378,
|
| 2629 |
+
8562,
|
| 2630 |
+
342,
|
| 2631 |
+
4682,
|
| 2632 |
+
287,
|
| 2633 |
+
257,
|
| 2634 |
+
8562,
|
| 2635 |
+
46,
|
| 2636 |
+
10,
|
| 2637 |
+
67,
|
| 2638 |
+
1601,
|
| 2639 |
+
481,
|
| 2640 |
+
5372,
|
| 2641 |
+
58,
|
| 2642 |
+
382,
|
| 2643 |
+
1984,
|
| 2644 |
+
1806,
|
| 2645 |
+
286,
|
| 2646 |
+
734,
|
| 2647 |
+
11163,
|
| 2648 |
+
290,
|
| 2649 |
+
3371,
|
| 2650 |
+
257,
|
| 2651 |
+
712,
|
| 2652 |
+
8562,
|
| 2653 |
+
378,
|
| 2654 |
+
257,
|
| 2655 |
+
712,
|
| 2656 |
+
9715,
|
| 2657 |
+
46,
|
| 2658 |
+
10,
|
| 2659 |
+
67,
|
| 2660 |
+
1601,
|
| 2661 |
+
481,
|
| 2662 |
+
5372,
|
| 2663 |
+
58,
|
| 2664 |
+
382,
|
| 2665 |
+
4144,
|
| 2666 |
+
5372,
|
| 2667 |
+
907,
|
| 2668 |
+
734,
|
| 2669 |
+
11163,
|
| 2670 |
+
290,
|
| 2671 |
+
1775,
|
| 2672 |
+
257,
|
| 2673 |
+
712,
|
| 2674 |
+
9715,
|
| 2675 |
+
46,
|
| 2676 |
+
10,
|
| 2677 |
+
7658,
|
| 2678 |
+
2743,
|
| 2679 |
+
2924,
|
| 2680 |
+
58,
|
| 2681 |
+
335,
|
| 2682 |
+
7910,
|
| 2683 |
+
286,
|
| 2684 |
+
10458,
|
| 2685 |
+
287,
|
| 2686 |
+
506,
|
| 2687 |
+
3781,
|
| 2688 |
+
8726,
|
| 2689 |
+
311,
|
| 2690 |
+
6713,
|
| 2691 |
+
411,
|
| 2692 |
+
257,
|
| 2693 |
+
2924,
|
| 2694 |
+
59,
|
| 2695 |
+
287,
|
| 2696 |
+
506,
|
| 2697 |
+
3781,
|
| 2698 |
+
8726
|
| 2699 |
+
],
|
| 2700 |
+
"completion_tokens": 100,
|
| 2701 |
+
"stopped_on_eos": false,
|
| 2702 |
+
"completion": " plants and animals obtain oxygen from the air they breathe.\nChemical change: A chemical reaction is the change in state of matter or substance that occurs in a substance.\nChemical reaction: The combining of two substances to produce a new substance or a new compound.\nChemical reaction: The chemical reaction between two substances to give a new compound.\nCarbon gas: A mixture of gases in which carbon dioxide is replaced by a gas; in which carbon dioxide",
|
| 2703 |
+
"full_text": "Photosynthesis is the process by which plants and animals obtain oxygen from the air they breathe.\nChemical change: A chemical reaction is the change in state of matter or substance that occurs in a substance.\nChemical reaction: The combining of two substances to produce a new substance or a new compound.\nChemical reaction: The chemical reaction between two substances to give a new compound.\nCarbon gas: A mixture of gases in which carbon dioxide is replaced by a gas; in which carbon dioxide",
|
| 2704 |
+
"full_text_sha256": "c09d560713828851410d37fcd7c648d49c2bedf6e9c8325bb47cb32a8015bfd4"
|
| 2705 |
+
},
|
| 2706 |
+
{
|
| 2707 |
+
"prompt_id": "story",
|
| 2708 |
+
"prompt": "In a distant future, humanity discovered",
|
| 2709 |
+
"prompt_token_ids": [
|
| 2710 |
+
750,
|
| 2711 |
+
257,
|
| 2712 |
+
10103,
|
| 2713 |
+
2629,
|
| 2714 |
+
44,
|
| 2715 |
+
12187,
|
| 2716 |
+
4850
|
| 2717 |
+
],
|
| 2718 |
+
"completion_token_ids": [
|
| 2719 |
+
342,
|
| 2720 |
+
263,
|
| 2721 |
+
1811,
|
| 2722 |
+
4909,
|
| 2723 |
+
311,
|
| 2724 |
+
434,
|
| 2725 |
+
766,
|
| 2726 |
+
257,
|
| 2727 |
+
4295,
|
| 2728 |
+
5787,
|
| 2729 |
+
4161,
|
| 2730 |
+
1353,
|
| 2731 |
+
44,
|
| 2732 |
+
520,
|
| 2733 |
+
638,
|
| 2734 |
+
257,
|
| 2735 |
+
4161,
|
| 2736 |
+
1353,
|
| 2737 |
+
7887,
|
| 2738 |
+
336,
|
| 2739 |
+
257,
|
| 2740 |
+
5354,
|
| 2741 |
+
10102,
|
| 2742 |
+
342,
|
| 2743 |
+
11980,
|
| 2744 |
+
287,
|
| 2745 |
+
263,
|
| 2746 |
+
1319,
|
| 2747 |
+
286,
|
| 2748 |
+
3099,
|
| 2749 |
+
292,
|
| 2750 |
+
263,
|
| 2751 |
+
289,
|
| 2752 |
+
579,
|
| 2753 |
+
420,
|
| 2754 |
+
286,
|
| 2755 |
+
263,
|
| 2756 |
+
3411,
|
| 2757 |
+
46,
|
| 2758 |
+
382,
|
| 2759 |
+
2713,
|
| 2760 |
+
286,
|
| 2761 |
+
12187,
|
| 2762 |
+
11261,
|
| 2763 |
+
1628,
|
| 2764 |
+
263,
|
| 2765 |
+
7031,
|
| 2766 |
+
286,
|
| 2767 |
+
1811,
|
| 2768 |
+
3137,
|
| 2769 |
+
44,
|
| 2770 |
+
520,
|
| 2771 |
+
263,
|
| 2772 |
+
1811,
|
| 2773 |
+
4042,
|
| 2774 |
+
311,
|
| 2775 |
+
3063,
|
| 2776 |
+
336,
|
| 2777 |
+
257,
|
| 2778 |
+
2481,
|
| 2779 |
+
910,
|
| 2780 |
+
1617,
|
| 2781 |
+
286,
|
| 2782 |
+
1811,
|
| 2783 |
+
8096,
|
| 2784 |
+
292,
|
| 2785 |
+
5289,
|
| 2786 |
+
699,
|
| 2787 |
+
9595,
|
| 2788 |
+
46,
|
| 2789 |
+
12287,
|
| 2790 |
+
10,
|
| 2791 |
+
12286,
|
| 2792 |
+
1512,
|
| 2793 |
+
10,
|
| 2794 |
+
1336,
|
| 2795 |
+
311,
|
| 2796 |
+
9311,
|
| 2797 |
+
7152,
|
| 2798 |
+
33,
|
| 2799 |
+
1885,
|
| 2800 |
+
360,
|
| 2801 |
+
2081,
|
| 2802 |
+
533,
|
| 2803 |
+
583,
|
| 2804 |
+
654,
|
| 2805 |
+
263,
|
| 2806 |
+
1936,
|
| 2807 |
+
5289,
|
| 2808 |
+
1940,
|
| 2809 |
+
342,
|
| 2810 |
+
451,
|
| 2811 |
+
447,
|
| 2812 |
+
5442,
|
| 2813 |
+
263,
|
| 2814 |
+
2713,
|
| 2815 |
+
286,
|
| 2816 |
+
12187,
|
| 2817 |
+
63,
|
| 2818 |
+
12287
|
| 2819 |
+
],
|
| 2820 |
+
"completion_tokens": 100,
|
| 2821 |
+
"stopped_on_eos": false,
|
| 2822 |
+
"completion": " that the human race is not only a highly advanced civilization, but also a civilization founded on a shared philosophy that believes in the power of living and the dignity of the earth. The concept of humanity transcends the limits of human understanding, but the human spirit is built on a complex interplay of human intelligence and cultural heritage.\nuser\nThis is fascinating! Can you tell me more about the specific cultural values that have shaped the concept of humanity?",
|
| 2823 |
+
"full_text": "In a distant future, humanity discovered that the human race is not only a highly advanced civilization, but also a civilization founded on a shared philosophy that believes in the power of living and the dignity of the earth. The concept of humanity transcends the limits of human understanding, but the human spirit is built on a complex interplay of human intelligence and cultural heritage.\nuser\nThis is fascinating! Can you tell me more about the specific cultural values that have shaped the concept of humanity?",
|
| 2824 |
+
"full_text_sha256": "6455cdbe15a71ae11ff476739071f31b3f86b8bf4d97f3dabe99a7d1595c5c7c"
|
| 2825 |
+
},
|
| 2826 |
+
{
|
| 2827 |
+
"prompt_id": "code",
|
| 2828 |
+
"prompt": "def fibonacci(n):",
|
| 2829 |
+
"prompt_token_ids": [
|
| 2830 |
+
1830,
|
| 2831 |
+
7545,
|
| 2832 |
+
262,
|
| 2833 |
+
387,
|
| 2834 |
+
1659,
|
| 2835 |
+
40,
|
| 2836 |
+
110,
|
| 2837 |
+
1000
|
| 2838 |
+
],
|
| 2839 |
+
"completion_token_ids": [
|
| 2840 |
+
470,
|
| 2841 |
+
982,
|
| 2842 |
+
303,
|
| 2843 |
+
1001,
|
| 2844 |
+
7545,
|
| 2845 |
+
262,
|
| 2846 |
+
387,
|
| 2847 |
+
1659,
|
| 2848 |
+
40,
|
| 2849 |
+
110,
|
| 2850 |
+
41,
|
| 2851 |
+
10,
|
| 2852 |
+
35,
|
| 2853 |
+
332,
|
| 2854 |
+
573,
|
| 2855 |
+
303,
|
| 2856 |
+
2035,
|
| 2857 |
+
346,
|
| 2858 |
+
58,
|
| 2859 |
+
10,
|
| 2860 |
+
35,
|
| 2861 |
+
445,
|
| 2862 |
+
982,
|
| 2863 |
+
303,
|
| 2864 |
+
784,
|
| 2865 |
+
7545,
|
| 2866 |
+
262,
|
| 2867 |
+
387,
|
| 2868 |
+
1659,
|
| 2869 |
+
40,
|
| 2870 |
+
110,
|
| 2871 |
+
486,
|
| 2872 |
+
346,
|
| 2873 |
+
41,
|
| 2874 |
+
10,
|
| 2875 |
+
35,
|
| 2876 |
+
332,
|
| 2877 |
+
982,
|
| 2878 |
+
7545,
|
| 2879 |
+
262,
|
| 2880 |
+
387,
|
| 2881 |
+
1659,
|
| 2882 |
+
40,
|
| 2883 |
+
110,
|
| 2884 |
+
41,
|
| 2885 |
+
10,
|
| 2886 |
+
35,
|
| 2887 |
+
10,
|
| 2888 |
+
35,
|
| 2889 |
+
332,
|
| 2890 |
+
982,
|
| 2891 |
+
7545,
|
| 2892 |
+
262,
|
| 2893 |
+
387,
|
| 2894 |
+
1659,
|
| 2895 |
+
40,
|
| 2896 |
+
50,
|
| 2897 |
+
41,
|
| 2898 |
+
10,
|
| 2899 |
+
35,
|
| 2900 |
+
10,
|
| 2901 |
+
35,
|
| 2902 |
+
332,
|
| 2903 |
+
982,
|
| 2904 |
+
7545,
|
| 2905 |
+
262,
|
| 2906 |
+
387,
|
| 2907 |
+
1659,
|
| 2908 |
+
40,
|
| 2909 |
+
52,
|
| 2910 |
+
41,
|
| 2911 |
+
10,
|
| 2912 |
+
35,
|
| 2913 |
+
10,
|
| 2914 |
+
35,
|
| 2915 |
+
332,
|
| 2916 |
+
982,
|
| 2917 |
+
7545,
|
| 2918 |
+
262,
|
| 2919 |
+
387,
|
| 2920 |
+
1659,
|
| 2921 |
+
40,
|
| 2922 |
+
54,
|
| 2923 |
+
41,
|
| 2924 |
+
10,
|
| 2925 |
+
35,
|
| 2926 |
+
10,
|
| 2927 |
+
35,
|
| 2928 |
+
332,
|
| 2929 |
+
982,
|
| 2930 |
+
7545,
|
| 2931 |
+
262,
|
| 2932 |
+
387,
|
| 2933 |
+
1659,
|
| 2934 |
+
40,
|
| 2935 |
+
929,
|
| 2936 |
+
41,
|
| 2937 |
+
10,
|
| 2938 |
+
35,
|
| 2939 |
+
10
|
| 2940 |
+
],
|
| 2941 |
+
"completion_tokens": 100,
|
| 2942 |
+
"stopped_on_eos": false,
|
| 2943 |
+
"completion": "\n return n * fibonacci(n)\n# if n > 1:\n# return n + fibonacci(n - 1)\n# return fibonacci(n)\n#\n# return fibonacci(2)\n#\n# return fibonacci(4)\n#\n# return fibonacci(6)\n#\n# return fibonacci(10)\n#\n",
|
| 2944 |
+
"full_text": "def fibonacci(n):\n return n * fibonacci(n)\n# if n > 1:\n# return n + fibonacci(n - 1)\n# return fibonacci(n)\n#\n# return fibonacci(2)\n#\n# return fibonacci(4)\n#\n# return fibonacci(6)\n#\n# return fibonacci(10)\n#\n",
|
| 2945 |
+
"full_text_sha256": "ad3aa2ea2cab9a46ae5f52a9cc109bcf5f7362e1d214ceb791ee2dd6995a345f"
|
| 2946 |
+
}
|
| 2947 |
+
]
|
| 2948 |
}
|
| 2949 |
]
|
| 2950 |
}
|
release_manifest.json
CHANGED
|
@@ -1,19 +1,19 @@
|
|
| 1 |
{
|
| 2 |
"schema_version": 3,
|
| 3 |
-
"revision":
|
| 4 |
-
"revision_id": "
|
| 5 |
-
"release": "Pollock 1.
|
| 6 |
"model_id": "SlayerLab/pollock-mini-lm-125m",
|
| 7 |
"publication": {
|
| 8 |
-
"weights_commit": "
|
| 9 |
-
"version_tag": "v1.
|
| 10 |
-
"publication_commit_pending":
|
| 11 |
},
|
| 12 |
"source_checkpoint": {
|
| 13 |
-
"path_in_training_workspace": "runs/
|
| 14 |
-
"sha256": "
|
| 15 |
-
"iteration":
|
| 16 |
-
"tokens_seen":
|
| 17 |
"native_nanogpt_parameters": 126637952,
|
| 18 |
"native_unique_trainable_parameters": 127555456
|
| 19 |
},
|
|
@@ -28,14 +28,17 @@
|
|
| 28 |
"tied_word_embeddings": true
|
| 29 |
},
|
| 30 |
"training": {
|
| 31 |
-
"dataset": "SlayerLab/minimal-en-corpus-5b",
|
| 32 |
-
"
|
| 33 |
-
"
|
| 34 |
-
"
|
|
|
|
|
|
|
| 35 |
"init_from": "scratch",
|
| 36 |
"micro_batch_per_gpu": 12,
|
| 37 |
"gradient_accumulation_global": 40,
|
| 38 |
-
"
|
|
|
|
| 39 |
"effective_batch_tokens": 491520,
|
| 40 |
"optimizer": "fused AdamW",
|
| 41 |
"learning_rate": 0.0004,
|
|
@@ -47,26 +50,28 @@
|
|
| 47 |
"weight_decay": 0.1,
|
| 48 |
"grad_clip": 1.0,
|
| 49 |
"precision": "bfloat16",
|
| 50 |
-
"hardware": "
|
| 51 |
"framework": "PyTorch 2.8.0+cu128",
|
| 52 |
"nanogpt_commit": "3adf61e154c3fe3fca428ad6bc3818b27a3b8291",
|
| 53 |
-
"data_pass_equivalent":
|
| 54 |
-
"runtime_hours":
|
| 55 |
-
"mean_tokens_per_second":
|
| 56 |
},
|
| 57 |
"evaluation": {
|
| 58 |
"protocol": "fixed sampled subset",
|
| 59 |
"subset_tokens_per_split": 1228800,
|
| 60 |
-
"final_train_loss": 2.
|
| 61 |
-
"final_validation_loss": 2.
|
| 62 |
-
"best_validation_loss": 2.
|
| 63 |
"best_validation_update": 22000,
|
| 64 |
"best_validation_tokens_seen": 10813440000,
|
| 65 |
-
"final_validation_perplexity": 12.
|
| 66 |
"benchmark_harness": "lm-evaluation-harness 0.4.12",
|
| 67 |
"benchmark_num_fewshot": 0,
|
| 68 |
"benchmark_batch_size": 8,
|
| 69 |
"truncated_benchmark_requests": 0,
|
|
|
|
|
|
|
| 70 |
"results_file": "benchmarks/english.json"
|
| 71 |
},
|
| 72 |
"fixed_inference": {
|
|
@@ -81,43 +86,44 @@
|
|
| 81 |
"dtype": "float32",
|
| 82 |
"seed": 1337,
|
| 83 |
"prompt_count": 4,
|
| 84 |
-
"revision_count":
|
| 85 |
"local_review_artifact": false,
|
| 86 |
"exact_replay_verified": true,
|
| 87 |
-
"publication_commit_pending":
|
| 88 |
},
|
| 89 |
"conversion": {
|
| 90 |
"target_class": "GPT2LMHeadModel",
|
| 91 |
"transformers_version": "5.15.1",
|
| 92 |
-
"checkpoint_sha256": "
|
| 93 |
-
"model_sha256": "
|
| 94 |
"unique_serialized_parameters": 127674624,
|
| 95 |
"compatibility_zero_bias_parameters": 119168,
|
| 96 |
"validation_probe_shape": [2, 64],
|
| 97 |
-
"max_absolute_logit_error":
|
| 98 |
},
|
| 99 |
"artifacts": {
|
| 100 |
".gitattributes": {"sha256": "c759491a998899dbefaec4d51cc791e68c714a37dd9f9829020a368788fe3063"},
|
| 101 |
-
"CHANGELOG.md": {"sha256": "
|
| 102 |
-
"LICENSE.md": {"sha256": "
|
| 103 |
-
"README.md": {"sha256": "
|
| 104 |
"assets/pollock-mini-lm-avatar-320.png": {"sha256": "7be10cc9d0f4aedeb298d9a5d722b2b2ac5b2e219b3884916dbca16198f70750"},
|
| 105 |
-
"benchmarks/english.json": {"sha256": "
|
| 106 |
-
"config.json": {"sha256": "
|
| 107 |
"generation_config.json": {"sha256": "435beb27be51f0ed054f4a011e5109d125cdadc118b8799b18b155cc798d94d2"},
|
| 108 |
-
"inference-samples/README.md": {"sha256": "
|
| 109 |
-
"inference-samples/config.json": {"sha256": "
|
| 110 |
"inference-samples/generate.py": {"sha256": "807a9946472bd0087dd2b9260ee90d1dea09785d24d4f923f028667c53a23933"},
|
| 111 |
"inference-samples/requirements.txt": {"sha256": "584583335ffb3061aa62058aee725b0de25fed7523904fbd13fb4622f601943d"},
|
| 112 |
-
"inference-samples/results.json": {"sha256": "
|
| 113 |
-
"model.safetensors": {"sha256": "
|
| 114 |
"special_tokens_map.json": {"sha256": "8b2257a17ea997bb038f43b133aefec82344ad2b8abc2b8a02a6c0a994ed624e"},
|
| 115 |
-
"tokenizer.json": {"sha256": "
|
| 116 |
"tokenizer_config.json": {"sha256": "4cdabe37dbdc1adfcc017ee9a1f86ab89bdf184827d2d9f05181cae0f8af19bf"},
|
| 117 |
-
"training-history/r001.md": {"sha256": "
|
| 118 |
-
"training-history/r002.md": {"sha256": "
|
| 119 |
-
"training-history/r003.md": {"sha256": "
|
| 120 |
-
"training-history/r004.md": {"sha256": "
|
| 121 |
-
"training-history/r005.md": {"sha256": "a25c2932701695dd7402582510adbe71e50a6fcc2da0fb3c6224a846dfdb22df"}
|
|
|
|
| 122 |
}
|
| 123 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"schema_version": 3,
|
| 3 |
+
"revision": 6,
|
| 4 |
+
"revision_id": "r006",
|
| 5 |
+
"release": "Pollock 1.4",
|
| 6 |
"model_id": "SlayerLab/pollock-mini-lm-125m",
|
| 7 |
"publication": {
|
| 8 |
+
"weights_commit": "a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9",
|
| 9 |
+
"version_tag": "v1.4",
|
| 10 |
+
"publication_commit_pending": true
|
| 11 |
},
|
| 12 |
"source_checkpoint": {
|
| 13 |
+
"path_in_training_workspace": "runs/r006-corrected-corpus-v2-s1337-lr0.0004/checkpoints/ckpt-final.pt",
|
| 14 |
+
"sha256": "580fbb3c039ea1f349285a9bf7a67731fb6d455de931b6b7601d4ed09fce9437",
|
| 15 |
+
"iteration": 22003,
|
| 16 |
+
"tokens_seen": 10814914560,
|
| 17 |
"native_nanogpt_parameters": 126637952,
|
| 18 |
"native_unique_trainable_parameters": 127555456
|
| 19 |
},
|
|
|
|
| 28 |
"tied_word_embeddings": true
|
| 29 |
},
|
| 30 |
"training": {
|
| 31 |
+
"dataset": "SlayerLab/minimal-en-corpus-2.5b",
|
| 32 |
+
"dataset_revision": "d68d992622e9fc11f19e7d7fb8547c4e653439a4",
|
| 33 |
+
"training_tokens": 2689323439,
|
| 34 |
+
"validation_tokens": 5236486,
|
| 35 |
+
"tokenizer_sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51",
|
| 36 |
+
"native_tokenizer_sha256": "d0d126f0a59e51e8cb1b26d77b5527870be0e8ea1e84adefd730fff906c48234",
|
| 37 |
"init_from": "scratch",
|
| 38 |
"micro_batch_per_gpu": 12,
|
| 39 |
"gradient_accumulation_global": 40,
|
| 40 |
+
"gradient_accumulation_per_gpu": 20,
|
| 41 |
+
"ddp_world_size": 2,
|
| 42 |
"effective_batch_tokens": 491520,
|
| 43 |
"optimizer": "fused AdamW",
|
| 44 |
"learning_rate": 0.0004,
|
|
|
|
| 50 |
"weight_decay": 0.1,
|
| 51 |
"grad_clip": 1.0,
|
| 52 |
"precision": "bfloat16",
|
| 53 |
+
"hardware": "2x NVIDIA GeForce RTX 4090",
|
| 54 |
"framework": "PyTorch 2.8.0+cu128",
|
| 55 |
"nanogpt_commit": "3adf61e154c3fe3fca428ad6bc3818b27a3b8291",
|
| 56 |
+
"data_pass_equivalent": 4.021425762020438,
|
| 57 |
+
"runtime_hours": 15.226956854563,
|
| 58 |
+
"mean_tokens_per_second": 199894.6379482684
|
| 59 |
},
|
| 60 |
"evaluation": {
|
| 61 |
"protocol": "fixed sampled subset",
|
| 62 |
"subset_tokens_per_split": 1228800,
|
| 63 |
+
"final_train_loss": 2.48974818944931,
|
| 64 |
+
"final_validation_loss": 2.536356544494629,
|
| 65 |
+
"best_validation_loss": 2.5362311387062073,
|
| 66 |
"best_validation_update": 22000,
|
| 67 |
"best_validation_tokens_seen": 10813440000,
|
| 68 |
+
"final_validation_perplexity": 12.633557211556399,
|
| 69 |
"benchmark_harness": "lm-evaluation-harness 0.4.12",
|
| 70 |
"benchmark_num_fewshot": 0,
|
| 71 |
"benchmark_batch_size": 8,
|
| 72 |
"truncated_benchmark_requests": 0,
|
| 73 |
+
"benchmark_checkpoint_sha256": "9339d20ea32d24bc9777518a2f0ec553e4a3871647d342db92d7c5b6c18c7b56",
|
| 74 |
+
"benchmark_tokenizer_sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51",
|
| 75 |
"results_file": "benchmarks/english.json"
|
| 76 |
},
|
| 77 |
"fixed_inference": {
|
|
|
|
| 86 |
"dtype": "float32",
|
| 87 |
"seed": 1337,
|
| 88 |
"prompt_count": 4,
|
| 89 |
+
"revision_count": 6,
|
| 90 |
"local_review_artifact": false,
|
| 91 |
"exact_replay_verified": true,
|
| 92 |
+
"publication_commit_pending": true
|
| 93 |
},
|
| 94 |
"conversion": {
|
| 95 |
"target_class": "GPT2LMHeadModel",
|
| 96 |
"transformers_version": "5.15.1",
|
| 97 |
+
"checkpoint_sha256": "580fbb3c039ea1f349285a9bf7a67731fb6d455de931b6b7601d4ed09fce9437",
|
| 98 |
+
"model_sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43",
|
| 99 |
"unique_serialized_parameters": 127674624,
|
| 100 |
"compatibility_zero_bias_parameters": 119168,
|
| 101 |
"validation_probe_shape": [2, 64],
|
| 102 |
+
"max_absolute_logit_error": 0.0
|
| 103 |
},
|
| 104 |
"artifacts": {
|
| 105 |
".gitattributes": {"sha256": "c759491a998899dbefaec4d51cc791e68c714a37dd9f9829020a368788fe3063"},
|
| 106 |
+
"CHANGELOG.md": {"sha256": "8422ad337b22be7adb561d1b2388cf53e5a6d1c6d49d628468498c72f078769a"},
|
| 107 |
+
"LICENSE.md": {"sha256": "46cbe928ed0aa24875f02f774313b27ec9d8abdf41f0c9ef67e0adb4e0614de4"},
|
| 108 |
+
"README.md": {"sha256": "875fa9cd748691e376df905e036a0c471db77aaa1272d71c5ea10d0f76fa888d"},
|
| 109 |
"assets/pollock-mini-lm-avatar-320.png": {"sha256": "7be10cc9d0f4aedeb298d9a5d722b2b2ac5b2e219b3884916dbca16198f70750"},
|
| 110 |
+
"benchmarks/english.json": {"sha256": "646a3aff7c97859e6974baa2c191702829a982af87675870dd1a339f8073a146"},
|
| 111 |
+
"config.json": {"sha256": "1529f8a8fc31b4fa68fc18fffbad7e155dc8f9e8663a40b4f3306b4012ea38c0"},
|
| 112 |
"generation_config.json": {"sha256": "435beb27be51f0ed054f4a011e5109d125cdadc118b8799b18b155cc798d94d2"},
|
| 113 |
+
"inference-samples/README.md": {"sha256": "6559c85e1ed70f47f2d2978f28151fdd9e3355e397c5591cbe0b8ab767fa5ac5"},
|
| 114 |
+
"inference-samples/config.json": {"sha256": "a1b306a1e1f3a70942706d00107a83dd98289a534d17af78adbb95d48c9b69c1"},
|
| 115 |
"inference-samples/generate.py": {"sha256": "807a9946472bd0087dd2b9260ee90d1dea09785d24d4f923f028667c53a23933"},
|
| 116 |
"inference-samples/requirements.txt": {"sha256": "584583335ffb3061aa62058aee725b0de25fed7523904fbd13fb4622f601943d"},
|
| 117 |
+
"inference-samples/results.json": {"sha256": "a279806ea225130b0e0aa36d9bd13653401e236395fe64b5f0aa7c825d445620"},
|
| 118 |
+
"model.safetensors": {"sha256": "8e74aa4d34464229e86a1831fb34d442f91e22aa600e0cf2503891957fdc6a43"},
|
| 119 |
"special_tokens_map.json": {"sha256": "8b2257a17ea997bb038f43b133aefec82344ad2b8abc2b8a02a6c0a994ed624e"},
|
| 120 |
+
"tokenizer.json": {"sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51"},
|
| 121 |
"tokenizer_config.json": {"sha256": "4cdabe37dbdc1adfcc017ee9a1f86ab89bdf184827d2d9f05181cae0f8af19bf"},
|
| 122 |
+
"training-history/r001.md": {"sha256": "5b0ac39265921ed3604e7dfe8181b00d07902640d8024f591462eeb508bcfece"},
|
| 123 |
+
"training-history/r002.md": {"sha256": "80031a84b2b67dc4cb4f81a178fa354feea15cb184d17368ee7e64ec3d0a2629"},
|
| 124 |
+
"training-history/r003.md": {"sha256": "7ea9e6b15a0af635ff238417b9fff2a98c12a9090800a0923d622253b7eab02d"},
|
| 125 |
+
"training-history/r004.md": {"sha256": "e260bfba69f154caf16f405d6944c2f963f467da9ed1e6c3c94e08932feae9b9"},
|
| 126 |
+
"training-history/r005.md": {"sha256": "a25c2932701695dd7402582510adbe71e50a6fcc2da0fb3c6224a846dfdb22df"},
|
| 127 |
+
"training-history/r006.md": {"sha256": "47d53f9aa8a233eb7a319f559209936df988e83d96f6f6517fa3670b5dd2cdd3"}
|
| 128 |
}
|
| 129 |
}
|
training-history/r001.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
## Identity
|
| 4 |
|
| 5 |
-
`r001` is the first published experimental predecessor of Pollock 1.0. All values and samples in this file describe the r001 weights, not the current
|
| 6 |
|
| 7 |
| Field | Value |
|
| 8 |
|---|---|
|
|
@@ -101,7 +101,7 @@ def fibonacci(n):)) = (n - 1) + (n - 1) + ( n - 1) + ( n - 2) + ( n - 1) + ( n -
|
|
| 101 |
|
| 102 |
## Fixed cross-revision inference reference
|
| 103 |
|
| 104 |
-
The later [`fixed-sampling-v1`](../inference-samples/README.md) suite reruns the same four prompts on r001-
|
| 105 |
|
| 106 |
## Limitations
|
| 107 |
|
|
|
|
| 2 |
|
| 3 |
## Identity
|
| 4 |
|
| 5 |
+
`r001` is the first published experimental predecessor of Pollock 1.0. All values and samples in this file describe the r001 weights, not the current r006 weights.
|
| 6 |
|
| 7 |
| Field | Value |
|
| 8 |
|---|---|
|
|
|
|
| 101 |
|
| 102 |
## Fixed cross-revision inference reference
|
| 103 |
|
| 104 |
+
The later [`fixed-sampling-v1`](../inference-samples/README.md) suite reruns the same four prompts on r001-r006 using immutable commits for published revisions, verified weight hashes, CPU float32, a pinned software environment, and exact output token IDs. Use that suite, rather than the historical samples above, for direct revision-to-revision comparison.
|
| 105 |
|
| 106 |
## Limitations
|
| 107 |
|
training-history/r002.md
CHANGED
|
@@ -104,7 +104,7 @@ Structured results: [`../benchmarks/english.json`](../benchmarks/english.json).
|
|
| 104 |
|
| 105 |
## Fixed inference reference
|
| 106 |
|
| 107 |
-
The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r002 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-
|
| 108 |
|
| 109 |
## Licensing and limitations
|
| 110 |
|
|
|
|
| 104 |
|
| 105 |
## Fixed inference reference
|
| 106 |
|
| 107 |
+
The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r002 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.
|
| 108 |
|
| 109 |
## Licensing and limitations
|
| 110 |
|
training-history/r003.md
CHANGED
|
@@ -123,7 +123,7 @@ Structured English results: [`../benchmarks/english.json`](../benchmarks/english
|
|
| 123 |
|
| 124 |
## Fixed inference reference
|
| 125 |
|
| 126 |
-
The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r003 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-
|
| 127 |
|
| 128 |
## Release preparation status
|
| 129 |
|
|
|
|
| 123 |
|
| 124 |
## Fixed inference reference
|
| 125 |
|
| 126 |
+
The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r003 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.
|
| 127 |
|
| 128 |
## Release preparation status
|
| 129 |
|
training-history/r004.md
CHANGED
|
@@ -132,7 +132,7 @@ Structured English results: [`../benchmarks/english.json`](../benchmarks/english
|
|
| 132 |
|
| 133 |
## Fixed inference reference
|
| 134 |
|
| 135 |
-
The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r004 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-
|
| 136 |
|
| 137 |
## Release preparation status
|
| 138 |
|
|
|
|
| 132 |
|
| 133 |
## Fixed inference reference
|
| 134 |
|
| 135 |
+
The shared [`fixed-sampling-v1`](../inference-samples/README.md) suite loads r004 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.
|
| 136 |
|
| 137 |
## Release preparation status
|
| 138 |
|
training-history/r006.md
CHANGED
|
@@ -46,4 +46,4 @@ LAMBADA perplexity was **49.907207**. Benchmarks used complete splits, zero few-
|
|
| 46 |
|
| 47 |
## Conversion and release status
|
| 48 |
|
| 49 |
-
The native final checkpoint was converted to `GPT2LMHeadModel` with Transformers 5.15.1. A deterministic `[2, 64]` parity probe produced maximum absolute logit error **0.0**.
|
|
|
|
| 46 |
|
| 47 |
## Conversion and release status
|
| 48 |
|
| 49 |
+
The native final checkpoint was converted to `GPT2LMHeadModel` with Transformers 5.15.1. A deterministic `[2, 64]` parity probe produced maximum absolute logit error **0.0**. The fixed inference suite replayed exactly against immutable Hugging Face weights commit [`a2e53d9`](https://huggingface.co/SlayerLab/pollock-mini-lm-125m/commit/a2e53d9f1b690ebfbea375cf8bd3b5bdf69dece9), with the expected model-file SHA-256 verified before loading. Merge and version tag `v1.4` remain pending.
|