dtestnyrr/talkie-1930-13b-it-gptq-int4

talkie-1930-13b-it — GPTQ int4

A 4-bit GPTQ quantization of talkie-lm/talkie-1930-13b-it — the instruction-tuned variant of talkie 13B, by Alec Radford, Nick Levine, and David Duvenaud.

The base model was trained on 260B tokens of pre-1931 English. This IT variant was further fine-tuned on a custom instruction-following dataset built entirely from pre-1931 reference works (etiquette manuals, letter-writing manuals, encyclopedias, poetry collections), then refined with online DPO. The result: a chatbot that answers in early-20th-century formal English and has no knowledge of anything after 1930.

This quantization shrinks the model from ~24.7 GB (bf16) to ~7.4 GB (int4), so it fits comfortably on a single 16 GB consumer GPU.

Use it

from gptqmodel import GPTQModel
import talkie_hf.talkie_qmodel  # registers TalkieQModel

model = GPTQModel.load("dtestnyrr/talkie-1930-13b-it-gptq-int4", trust_remote_code=True)

# Use the talkie chat template
prompt = "<|user|>Write a brief letter declining a dinner invitation.<|end|><|assistant|>"
ids = model.tokenizer(prompt, return_tensors="pt").input_ids.cuda()
out = model.generate(input_ids=ids, max_new_tokens=300, do_sample=True, temperature=0.7)
print(model.tokenizer.decode(out[0], skip_special_tokens=False))

Stop generation at any of: <|end|>, <|user|>, <|assistant|>, <|system|>, <|endoftext|>.

Chat template

<|system|>{optional system prompt}<|end|>
<|user|>{user message 1}<|end|>
<|assistant|>{model reply 1}<|end|>
<|user|>{user message 2}<|end|>
<|assistant|>

End the prompt with <|assistant|> (no trailing <|end|>) to signal the model should generate.

Quantization details

Same recipe as the base-variant int4 release (dtestnyrr/talkie-1930-13b-base-gptq-int4):

Parameter	Value
Method	GPTQ
Bits	4
Group size	128
Activation order	False
Symmetric	True
Effective bits per weight	~4.29 BPW
Calibration corpus	256 × 2048-token windows from 8 pre-1931 Project Gutenberg classics
Quantization framework	GPTQModel v6.0.3

What it's good at

• Period-correct prose in any genre (letters, essays, sermons, news editorials, fiction)
• Encyclopedia-style explanations of anything pre-1930
• 1900-1930s-style poetry in proper meter and rhyme
• Edwardian / Georgian etiquette
• "Predicting" the future from a 1930 vantage point

What it can't do

• Knowledge of anything post-1930 (no WWII, no computers, no internet, no DNA structure, no antibiotics beyond very early research, etc.)
• Modern slang or casual register — it will respond formally regardless of how casually you address it
• Technical assistance with modern tools, languages, or frameworks
• Knowledge of people who became famous after 1930

Architecture

Custom decoder-only transformer (40 layers, 5120 hidden, 40 heads × 128 head_dim, 13,696 SwiGLU intermediate, vocab 65,540 = 65,536 BPE merges + 4 chat special tokens). Uses RoPE θ=10⁶, F.rms_norm everywhere, QK-norm, and per-residual learnable gain modules. See modeling_talkie.py for the full HF-compatible port.

License & attribution

Apache 2.0. Original model credit:

• Authors: Alec Radford, Nick Levine, David Duvenaud
• Project: https://talkie-lm.com/
• Reference code: https://github.com/talkie-lm/talkie