---
license: mit
tags:
- text-generation
- character-level
- compression
- research
datasets:
- enwik8
---

# Compressed nanoGPT (enwik8)

## Outstanding Compression Results!

- **Performance**: 1.635 → 1.637 BPC (+0.002)
- **Parameters**: 28,801,536 → 27,359,744
- **Compression**: 1.053× smaller (5.0% reduction)
- **Quality Loss**: Only 0.1% degradation!

This demonstrates **near-perfect compression** of a character-level transformer.

## Usage

```python
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "prompterminal/nanogpt-enwik8-compressed-working",
    trust_remote_code=True
)

# Generate text
import torch
prompt = torch.randint(0, 6060, (1, 10))  # Random start
output = model.generate(prompt, max_new_tokens=100)
```

## Research Impact

First successful demonstration of high-quality compression on character-level transformers!