---
license: apache-2.0
language:
- en
datasets:
- allenai/c4
tags:
- pytorch
- gpt
- gelu
- 261M
- chinchilla
- ablation
- seed2
library_name: transformers
---

# gelu 261M (d=12) — seed 2

Reproducibility seed for the [`gelu 261M`](https://huggingface.co/mlnomad/gelu-d12-chinchilla-261M-pytorch) ablation
(seed 0 is the canonical published checkpoint). Same architecture, same data, same
hyper-params — only the random seed differs. Useful for variance estimation
when comparing architectures.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained(
    "mlnomad/gelu-d12-chinchilla-261M-seed2-pytorch",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")
```

Apache 2.0.