Instructions to use SlayerLab/pollock-mini-lm-125m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SlayerLab/pollock-mini-lm-125m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SlayerLab/pollock-mini-lm-125m")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SlayerLab/pollock-mini-lm-125m")
model = AutoModelForCausalLM.from_pretrained("SlayerLab/pollock-mini-lm-125m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SlayerLab/pollock-mini-lm-125m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SlayerLab/pollock-mini-lm-125m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SlayerLab/pollock-mini-lm-125m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SlayerLab/pollock-mini-lm-125m

SGLang

How to use SlayerLab/pollock-mini-lm-125m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SlayerLab/pollock-mini-lm-125m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SlayerLab/pollock-mini-lm-125m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SlayerLab/pollock-mini-lm-125m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SlayerLab/pollock-mini-lm-125m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SlayerLab/pollock-mini-lm-125m with Docker Model Runner:
```
docker model run hf.co/SlayerLab/pollock-mini-lm-125m
```

pollock-mini-lm-125m / training-history /r002.md

dawidmajewski

Release Pollock 1.4 (r006) (#1)

9d9f729 7 days ago

preview code

Raw

History Blame Contribute Delete

5.69 kB

r002 — Pollock 1.0 training record

Identity and provenance

Field	Value
Revision / release	`r002` / Pollock 1.0
Model ID	`SlayerLab/pollock-mini-lm-125m`
Published weights	`5e571cde09ef2d1124e13d1c31d0b75f8bd0e513`
Source checkpoint	`runs/minimal-en-125m-4ep/ckpt.pt`
Checkpoint iteration / SHA-256	22,004 / `026f54a390036b35792aa8fb131c8b0d394efb6d933753fb0e1257e55a67374b`
`model.safetensors` SHA-256	`2aada5b26abe9ce3b70af393d77be4ed1a799a7205b9c0dd5cfaff8bcaabce68`
nanoGPT commit	`3adf61e`

Iteration 22,004 is one greater than configured max_iters=22003 because of nanoGPT's inclusive loop/checkpoint behavior.

Architecture and parameter accounting

Setting	Value
Type	GPT-2-style decoder-only Transformer
Layers / heads / width	12 / 14 / 896
Head dimension / MLP width	64 / 3,584
Context / vocabulary	1,024 / 12,288
Activation / dropout / bias	GELU / 0.0 / false
Position representation	learned absolute embeddings
Input/output token weights	tied
Transformer excluding embeddings	115,627,904
Tied token embedding/output matrix	11,010,048
nanoGPT reported count	126,637,952
Learned position embedding	917,504
Native unique trainable total	127,555,456
Transformers serialized total	127,674,624
Zero-valued compatibility bias parameters	119,168

Conversion to GPT2LMHeadModel produced a maximum absolute logit difference of 0 on a deterministic 2×64-token probe.

Tokenizer and dataset

The custom tokenizer is byte-level BPE with GPT-2-style pretokenization, 12,288 vocabulary entries, and 12,029 merges. <|endoftext|> (12285) is EOD/BOS/EOS/padding; <|im_start|> (12286) and <|im_end|> (12287) are additional special tokens. No chat template is defined. Tokenizer SHA-256: 6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51.

Training used Minimal EN Corpus 2.5B, a subjectively selected mixture of 15 English sources with exact and approximate deduplication, language filtering, and benchmark decontamination.

Split	Documents	BPE tokens	Binary SHA-256
Train	2,044,652	2,703,768,815	`5396834c25664080c14793a4880f86b86ae94074a6036f8f54a58686b02d30dc`
Validation	4,469	5,267,742	`bf07bd1fb1f956a92ee7788f2a8110e1837f5b880ce432f6cd2a22d05f8ac135`

Complete training configuration

Setting	Value
Initialization	scratch
Final checkpoint iteration	22,004
Sequence / micro-batch per GPU	1,024 / 12
Global / per-GPU accumulation	40 / 20 micro-steps
DDP world size / effective batch	2 / 491,520 tokens
Token presentations / passes	10,815,406,080 / approximately 4.0
Optimizer / betas	fused AdamW / 0.9, 0.95
Maximum / minimum LR	3e-4 / 3e-5
Schedule / warmup / decay	cosine / 440 / 22,003 iterations
Weight decay / grad clip	0.1 / 1.0
Evaluation interval / batches	1,000 / 100 per split
Precision / compilation / backend	BF16 / `torch.compile` / NCCL
Seed	1337 plus DDP-rank offset
Hardware	2× NVIDIA GeForce RTX 4090 24 GB
PyTorch / runtime	2.8.0+cu128 / approximately 12 h 41 min

Training-time validation

Loss estimates used 100 randomly sampled batches and are not a deterministic full-validation pass.

Step	Train loss	Validation loss
0	9.5797	9.5732
1,000	3.4769	3.5219
2,000	3.0943	3.1266
5,000	2.8030	2.8569
10,000	2.6098	2.6928
15,000	2.5469	2.6056
18,000	2.5104	2.5790
19,000	2.4960	2.5795
20,000	2.4705	2.5600
21,000	2.4877	2.5653
22,000	2.4545	2.5775

The released weights are the final checkpoint. The increase after step 20,000 is within a noisy sampled protocol and is not definitive evidence of overfitting.

English zero-shot benchmarks

All tasks used complete splits with lm-evaluation-harness 0.4.12, zero few-shot examples, batch size 8, BF16, 1,024 maximum context, and no sample limit. Random/NumPy/PyTorch/few-shot seeds were 0/1234/1234/1234. No requests were truncated.

Benchmark	Primary	Additional	Samples
BLiMP	acc 0.7669701493	—	67,000
LAMBADA OpenAI	acc 0.2780904328	perplexity 53.6675225106	5,153
HellaSwag	acc_norm 0.2987452699	acc 0.2818163712	10,042
PIQA	acc_norm 0.6033732318	acc 0.6137105550	1,838
SciQ	acc_norm 0.6580000000	acc 0.7370000000	1,000
ARC-Easy	acc_norm 0.4229797980	acc 0.4659090909	2,376
ARC-Challenge	acc_norm 0.2414675768	acc 0.2081911263	1,172

Structured results: ../benchmarks/english.json. Release metadata: ../release_manifest.json.

Fixed inference reference

The shared fixed-sampling-v1 suite loads r002 from its immutable published commit, verifies the weight hash above, and records the exact output token IDs for the same four prompts used across r001-r006.

Licensing and limitations

This is an English, unaligned base model, not an instruction-following assistant. It may hallucinate, generate harmful or biased content, and reproduce training patterns. It is unsuitable for high-risk or production decisions without independent safeguards. The dataset combines sources with different terms; see ../LICENSE.md and the dataset card.