File size: 3,744 Bytes
79636b3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
license: apache-2.0
base_model: talkie-lm/talkie-1930-13b-it
tags:
- gptq
- 4-bit
- quantized
- instruction-tuned
- vintage-language-model
- chat
language:
- en
pipeline_tag: text-generation
---
# talkie-1930-13b-it — GPTQ int4
A 4-bit GPTQ quantization of [`talkie-lm/talkie-1930-13b-it`](https://huggingface.co/talkie-lm/talkie-1930-13b-it) — the **instruction-tuned** variant of talkie 13B, by Alec Radford, Nick Levine, and David Duvenaud.
The base model was trained on 260B tokens of pre-1931 English. This IT variant was further fine-tuned on a custom instruction-following dataset built entirely from pre-1931 reference works (etiquette manuals, letter-writing manuals, encyclopedias, poetry collections), then refined with online DPO. The result: a chatbot that answers in **early-20th-century formal English** and has no knowledge of anything after 1930.
This quantization shrinks the model from ~24.7 GB (bf16) to ~7.4 GB (int4), so it fits comfortably on a single 16 GB consumer GPU.
## Use it
```python
from gptqmodel import GPTQModel
import talkie_hf.talkie_qmodel # registers TalkieQModel
model = GPTQModel.load("dtestnyrr/talkie-1930-13b-it-gptq-int4", trust_remote_code=True)
# Use the talkie chat template
prompt = "<|user|>Write a brief letter declining a dinner invitation.<|end|><|assistant|>"
ids = model.tokenizer(prompt, return_tensors="pt").input_ids.cuda()
out = model.generate(input_ids=ids, max_new_tokens=300, do_sample=True, temperature=0.7)
print(model.tokenizer.decode(out[0], skip_special_tokens=False))
```
Stop generation at any of: `<|end|>`, `<|user|>`, `<|assistant|>`, `<|system|>`, `<|endoftext|>`.
## Chat template
```
<|system|>{optional system prompt}<|end|>
<|user|>{user message 1}<|end|>
<|assistant|>{model reply 1}<|end|>
<|user|>{user message 2}<|end|>
<|assistant|>
```
End the prompt with `<|assistant|>` (no trailing `<|end|>`) to signal the model should generate.
## Quantization details
Same recipe as the base-variant int4 release ([dtestnyrr/talkie-1930-13b-base-gptq-int4](https://huggingface.co/dtestnyrr/talkie-1930-13b-base-gptq-int4)):
| Parameter | Value |
|---|---|
| Method | GPTQ |
| Bits | 4 |
| Group size | 128 |
| Activation order | False |
| Symmetric | True |
| Effective bits per weight | ~4.29 BPW |
| Calibration corpus | 256 × 2048-token windows from 8 pre-1931 Project Gutenberg classics |
| Quantization framework | [GPTQModel](https://github.com/ModelCloud/GPTQModel) v6.0.3 |
## What it's good at
- Period-correct prose in any genre (letters, essays, sermons, news editorials, fiction)
- Encyclopedia-style explanations of anything pre-1930
- 1900-1930s-style poetry in proper meter and rhyme
- Edwardian / Georgian etiquette
- "Predicting" the future from a 1930 vantage point
## What it can't do
- Knowledge of anything post-1930 (no WWII, no computers, no internet, no DNA structure, no antibiotics beyond very early research, etc.)
- Modern slang or casual register — it will respond formally regardless of how casually you address it
- Technical assistance with modern tools, languages, or frameworks
- Knowledge of people who became famous after 1930
## Architecture
Custom decoder-only transformer (40 layers, 5120 hidden, 40 heads × 128 head_dim, 13,696 SwiGLU intermediate, vocab 65,540 = 65,536 BPE merges + 4 chat special tokens). Uses RoPE θ=10⁶, F.rms_norm everywhere, QK-norm, and per-residual learnable gain modules. See [`modeling_talkie.py`](modeling_talkie.py) for the full HF-compatible port.
## License & attribution
Apache 2.0. Original model credit:
- Authors: Alec Radford, Nick Levine, David Duvenaud
- Project: https://talkie-lm.com/
- Reference code: https://github.com/talkie-lm/talkie
|