Voice

Gemma 4 · 26B MoE · StyleTune · GGUF Q8_0

Your model, in your voice. One file to run—nothing extra to set up.

A full model is 26GB. A voice is 1.4GB — smaller models even less. You download the voice, not the model.

This is a tool — no .gguf is included. You create the voiced model locally with voice cast.

---

Two steps to start

One-time setup, then two commands. One model, one voice:

python3 voice.py path  # once - adds voice to PATH (then use voice directly)
voice get Gryphe/Gemma-4-26B-A4B-StyleTune-V2
# Name it styletune-v2 when prompted
voice cast ./model.gguf styletune-v2 --out ./voiced/

Run it:

llama serve -m ./voiced/model.gguf

We tried it on Orion (n=10 heuristic, not a guarantee): We cast Gryphe/Gemma-4-26B-A4B-StyleTune-V2 voice onto TheDrummer/Orion-26B-A4B-v1.4 — on our 10 prompts, phrasing was ~98% different (1.6% shared at 1.0, 3.2% at greedy via our 50-phrase heuristic). Same cast on its abliterated sibling llmfan46/gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic looped; re-cast as delta stopped looping and kept the voice. We got StyleTune on the QAT version with just voice cast delta — one file out, same family/shape required.

No adapter at runtime. voice cast bakes the voice into a complete .gguf — the head at Q8_0, everything else untouched. Just llama serve -m.

Example voice shown. Any compatible safetensors voice works the same way. Names are lowercased on disk—styletune-v2 is the stored form. Model and voice names here are examples; use any compatible pair that shares the same architecture and vocabulary.

---

Not just Gemma — any compatible model within the same family and shape

Voice is not tied to Gemma.

Same architecture, same vocab — the voice transfers. Qwen → Qwen, Gemma → Gemma, Llama → Llama. Take the voice from a finetune and cast it onto the stronger base you already have — reasoning stays, only how it sounds changes. Shape must match (we try [vocab, hidden] → [hidden, vocab]), GGUF head is Q8_0 when last dim %32==0 else F16 fallback (larger, compatible).

For example, a voice from Wiself/Holodeck-Lounge-MTP casts onto any qwen3.5-9b base, ReadyArt/For-Her-Darkside-12B-v1.4 onto any gemma4 12b base — even mixed with ReadyArt/Serenity-12B to see what happens. Gemma 4 26B MoE above is just one illustration. Mix and match within that family and shape: keep the more performant base, keep the better voice.

Qwen, Llama, Gemma, GLM, DeepSeek — if it has a voice, you can move it within that family and shape.

---

The Idea — a scalpel

One tensor holds voice. We lift it. We place it. Transpose and dtype handled. Nothing else changes.

Voice is a scalpel, not a pipeline — one idea made trivial: lift a model's character and place it elsewhere.

Backed by proven ideas: QAT-aware bases (Google: 5k steps, 54% less drop at Q4_0) stay intact; lm_head carries style (MythoMax, Gryphe StyleTune). We just make it portable.

What it trivializes (power users, same family & shape)

• Same family, any container: voice get from any safetensors repo / .gguf / URL via range requests; voice cast onto any compatible .gguf or .safetensors — same vocab + same head shape required, transposes BF16/F16/F32→Q8_0 auto (last dim %32==0 else F16 fallback).
• Any single tensor, precisely: voice tensor get --tensor <n> / tensor cast for any named tensor — bit-exact when quant types match, validated offsets, otherwise F32 round-trip.
• No surprises: atomic writes, validated header, ETag-aware resume, 0600 token, symlink-safe — resumes or fails cleanly.

Keep the stronger model for reasoning, the better voice for style. llama serve -m ready.

How it works

Two benefits, no extras to learn:

• Get what you need: Light and fast — pulled with range requests in seconds, saved as one tensor (voice.safetensors or voice.gguf for quant passthrough + voice.json). voice list to see, voice info to inspect.
• Run fused: Cast writes a complete model you can run anywhere. No extra setup. Quant voices stay bit-exact Q8_0→Q8_0.

You will see each step as it happens and get a ready-to-run file at the end.

Take the voice of any compatible model or finetune and cast it onto the file you already have. Mix and match. Keep the stronger model for reasoning and the better voice for style. You choose—no extra steps at runtime.

When you need a delta

Direct cast is the default. If an abliterated target loops, try delta — it blends only the difference, so the MoE router stays calm.

Why a delta helps when looping

A delta carries only the difference from its base, so it blends more gently with the target. On some abliterated targets, a tiny shift can change behavior. MoE routing is discrete, so a small change in the residual stream can tip which expert fires next. If direct casting loops, the delta path often avoids it.

You tried it. Leave a breadcrumb?

No one knows what Voice X does on Base Y until someone tries. You just tried.

voice report saves your note privately first. Then it shows you exactly what would be shared — and asks once.

voice report --voice styletune-v2 --target ./model.gguf -m "delta fixed loop, 1.6% shared"

Thanks for noting — saved locally (a3f8…) → ~/.voices/reports.jsonl (private)
Will share: voice: styletune-v2, tensor: lm_head.weight [262144,2816] BF16, target: model.gguf
Leave a breadcrumb to help the next person? [y/N] (N):  ← Enter = No

• Private by default. Your note stays in ~/.voices/reports.jsonl and ~/.voices/outbox/ until you say yes. N default. We never ask again for this report.
• If you say y, it shares to Wiself/voice-reports. We send your one-line note + preview + source/shape/arch we auto-pull + basic env. That's it. Preview is truth before you decide.
• Why bother? Voices are 1.4GB, models are 26GB. Trying is cheap, but someone else is about to waste an hour re-learning what you already learned — "sings" or "loops." One line saves them.

The notebook is still mostly empty. That's why the next line matters most.

---

Commands

Three you need daily — rest in DOCS.md (voice --help is truth, 13 commands v2.1.0):

Command	What it does
voice get <model>	Get a voice — HF safetensors repo / .gguf / direct URL with Range (same family, head must exist)
voice cast <target> <voice>	Cast it onto .gguf or .safetensors of same shape/vocab (Q8_0 when last dim %32==0 else F16 fallback)
voice report -m "note"	Save a one-line note locally, optionally share to community — small file, big iteration (we pull source/tensor/shape)

Extra commands

Command	What it does
voice add <safetensors> <json>	Add local voice.safetensors + voice.json to ~/.voices/ (both required, asks name)
voice tensor get --tensor <n>	Extract any named tensor (by name, validated header, range-fetched)
voice tensor cast <target> <file>	Cast any named tensor onto compatible target (must match shape, same family)
voice delta [name]	Make a portable voice − base style
voice list [base]	List voices or cached bases
voice info <voice>	Show voice details
voice remove <voice>	Remove a voice
voice cleanup [--force]	Remove incomplete .parts
voice auth [--show|--clear]	Save HF token (0600)
voice path	Put voice on PATH
voice doctor	Check setup

Full reference: DOCS.md · voice --help

Example:

voice get Gryphe/Gemma-4-26B-A4B-StyleTune-V2
# Name it styletune-v2 when prompted
voice list
#   Name               Source                                  Type
#   ────────────────── ─────────────────────────────────────── ────────
#   styletune-v2       Gryphe/Gemma-4-26B-A4B-StyleTune-V2      voice
#   delta-styletune-v2 Gryphe/Gemma-4-26B-A4B-StyleTune-V2      delta

voice info styletune-v2
voice list base
voice cast ./model.gguf styletune-v2 --out ./voiced/

delta- is reserved. Voice names cannot start with it. Deltas always do. Full CLI reference: DOCS.md (generated from voice --help, 13 commands). Names are lowercased when saved.

What StyleTune showed

Gryphe reports for StyleTune-V2—on their private 200-prompt roleplay eval (private regex, certified cliché-free, 1–20 turns, greedy 0.0)—52% fewer clichés per 100 words (1.141 → 0.551) and 19.9% shared trigram vocabulary vs base instruct.[^1]

[^1]: Source: Gryphe/Gemma-4-26B-A4B-StyleTune-V2 model card and discussion #1. We cite their numbers, not ours. No model-index is published here—this is a private metric, not a reproducible Hub leaderboard.

How Gryphe tested

• Single epoch, 100% narrative data; prompts certified cliché-free and not seen in training.
• 200 prompts sampled at intervals 1 to 20 turns, each ending on a user turn.
• Run at temperature 0.0 greedy deterministic.
• Clichés via their private regex (for example, calculating and predatory) counted per 100 words; phrasing via shared trigram vocabulary.

Our quick check — heuristic, n=10, not comparable

Small sample, heuristic, not comparable—direction only. With that caveat, the check suggests distinct phrasing.

We used 10 roleplay prompts in chat. Each was run at temp 1.0 and at greedy 0.0 (max_tokens 2000, seed 42). This is a small-sample heuristic using our own 50-phrase list, not Gryphe's private regex. Absolute per-100 numbers are not comparable to Gryphe's.

With that caveat, overall trigram overlap vs base was 1.6% at 1.0 and 3.2% at greedy. Every prompt was under 5%. We saw 5–5.7k words total. Thinking tokens averaged about 2.6k. There were no loops and no empties. The voice was clearly distinct. Greedy was slightly lower on clichés than 1.0. Direction and distinctness align with Gryphe's report.

Details, per-prompt tables, and harness: benchmark.md.

---

A smarter base: QAT

Google's QAT (5k steps, fake-quant + KL to BF16) cuts perplexity drop 54% at Q4_0 vs PTQ — we keep your QAT base intact. Love StyleTune-V2 but want QAT? Keep the QAT model, cast any voice onto it.

---

Evaluation

Gryphe (n=200): 52% fewer clichés (1.141 → 0.551) + 19.9% shared vocab at greedy 0.0. We see 1.6% at temp 1.0, 3.2% at greedy — all prompts <5%, direction matches. Not comparable (private regex vs our 50-phrase heuristic) — see benchmark.md for full tables.

---

Limitations

• Heuristic mismatch: Our 50-phrase list misses Gryphe's private regex (calculating xx, predatory xx, and others), so absolute 0.10–0.18 vs 1.141 → 0.551 is not comparable—only direction and low shared vocabulary are signal.
• Sample size and prompts: 10 single-turn prompts vs Gryphe's 200 multi-turn certified set (intervals 1–20, ends on user turn). Small sample, higher variance.
• Thinking overhead: Gemma 4 thinking mode (<|channel>thought) adds about 2.6k thinking tokens on average. Gryphe's eval predates thinking configs and did not report this split.
• Determinism: Greedy 0.0 reduces variance but still yields more canonical phrasing (shared 1.6%→3.2%)—expected, not a failure.
• If you see looping, use the delta path (voice delta → voice cast … delta-…). Tiny shifts can affect behavior on abliterated targets. See “Why a delta helps when looping” above.
• Scope: Style transfer changes phrasing and voice, not knowledge or factuality. Not evaluated for safety, bias, or instruction following. See Gemma 4 model card Limitations.

---

References

• Gemma 4 Technical Report—Gemma Team, 2026. arXiv:2607.02770—base model google/gemma-4-26B-A4B-it (MoE 26B A4B, 256K context, 262K vocab).
• Gemma 3 Quantization-Aware Training—Google Developers Blog, Apr 18, 2025: State-of-the-art AI to consumer GPUs—QAT recipe (5k steps, fake-quant + KL to BF16 teacher, 54% less perplexity drop at Q4_0).
• Gryphe StyleTune-V2—Model card and discussion #1—200-prompt eval, private regex, 1.141 → 0.551 and 19.9% shared vocabulary.

---

Use with Hugging Face

GGUF (this repo—local with llama.cpp)

This is the native path for the fused model above—no Python, no adapter at runtime:

llama serve -m ./voiced/model.gguf --jinja
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"voiced","messages":[{"role":"user","content":"Write a tavern scene, no clichés."}],"temperature":1.0}'

Transformers (HF-native snippet—for the base or other safetensors voices)

The Voice tool extracts the output-projection tensor (lm_head and friends) from safetensors or GGUF models. For any HF transformers model (including the base below), the canonical snippet is:

from transformers import pipeline

# Base model—valid, lowercased HF ID (link verified, HF is case-insensitive)
# https://huggingface.co/google/gemma-4-26b-a4b-it
pipe = pipeline("text-generation", model="google/gemma-4-26b-a4b-it")
print(pipe("Write a short tavern scene with varied diction:")[0]["generated_text"])

# For a StyleTune finetune example (illustrative):
# pipe = pipeline("text-generation", model="gryphe/gemma-4-26b-a4b-styletune-v2")

Base-model note: google/gemma-4-26b-a4b-it is the lowercased form of the real HF ID google/gemma-4-26B-A4B-it (verified live, Apache-2.0 via Gemma license). gryphe/gemma-4-26b-a4b-styletune-v2 is the lowercased illustrative voice source—replace with any compatible safetensors model that shares architecture and vocab. Relation: base_model_relation: quantized (fused GGUF head quantized to Q8_0, remainder preserved).

---

Demo

Local demo — same one-file run:

llama serve -m ./voiced/model.gguf

---

Choosing a style

Pick the voice you like. Cast it directly. That is the default. Make a delta only when you need it:

• Full voice: The exact voice from the source model. Use it when you want that model's voice, exactly. Two steps: get and cast.
• Delta: Make once — casts onto any compatible target of that family. One style, all your bases.

Both fuse the same way into the output file. Small, sharp, and complete.

If you are not sure, start direct:

voice cast ./model.gguf styletune-v2 --out ./voiced/

If a model loops, try delta:

voice delta styletune-v2
voice cast ./model.gguf delta-styletune-v2 --out ./voiced/

---

Compatibility — scalpel needs the same cut

Runs with compatible models that share the same architecture and vocabulary. Check shape before you cast — same family, same shape, or it won't fit. This is why voicepacks are inevitable: one tensor is one cast, so trying is cheap — what you mix is up to you.

Works with:

• Hugging Face safetensors models that expose an output projection — lm_head.weight, output.weight, or tied embed_tokens.weight auto-found
• GGUF models of the same architecture and vocab size—the rest of the model stays exactly as it was
• Transposed heads and BF16/F16/F32 — handled, no flag needed
• llama.cpp and anything that reads GGUF—run with llama serve -m <voiced.gguf>

Does not work with:

• Mismatched architectures or vocab sizes—shape must match
• Missing output projection

Your voiced GGUF is standalone. No extra files. No extra steps to run.

Split—quantized vs full precision: The source safetensors voice is full precision (BF16/F16/F32), full shape. The fused GGUF keeps all tensors at their original quantization except the output head, which is written at Q8_0 when last dim %32==0 (true for Gemma 4 26B: hidden 2816, vocab 262144) — otherwise it falls back to F16 (larger, ~2× Q8_0, but compatible; human-in-loop warned). For deltas, the math is abliterated_head (dequantized to F32) + delta (F16→F32) then quantized. All other tensors and metadata (tokenizer, vocab, arch) are byte-copied.

Disclosures: get and delta fetch from Hugging Face and need network access. cast writes a new GGUF to disk, so ensure you have free space for a full model copy.

---

Your files, where they are

Your library lives at ~/.voices/:

~/.voices/
  styletune-v2/voice.safetensors      # your voice, single tensor
  styletune-v2/voice.json             # metadata for that voice
  delta-styletune-v2/                 # portable style for reuse
  delta-styletune-v2/voice.safetensors
  delta-styletune-v2/voice.json
  base/<org>/<model>.safetensors      # cached base head
  base/<org>/<model>.json             # metadata for the base

Notes:

• Names are lowercased when saved. Input StyleTune-V2 is stored as styletune-v2.
• voice list shows voices and deltas. voice list base shows cached bases. voice info <name> shows details for any entry.

---

Troubleshooting

Voice not found:

Use voice list to see what is saved. Names match case-insensitively. StyleTune-V2 and styletune-v2 resolve to the same entry.

Shape mismatch. Does not fit this model:

The voice and the target must share the same head shape. A 26B head will not fit a 12B model. Try a voice from the same architecture and vocab size.

Tied weights:

Some models tie the output projection to the embedding. Voice finds the correct tensor automatically. You do not need to specify it.

Could not reach the model:

Check the org/model name and your connection. Voice fetches the header first and will tell you if the model is not accessible.

Output folder not writable / want a different location:

voice cast without --out saves to ~/Voiced/<voice>-<model>/<voice>-<model>.gguf (sane, no overwrite of your input). Want elsewhere?

voice cast ./model.gguf styletune-v2 --out ./voiced/
voice cast ./model.gguf styletune-v2 --out ./voiced/model.gguf
voice cast ./model.gguf styletune-v2 --out ~/my-voices/

Every wait shows a bar — → lm_head.weight 42% · 604/1437 MB · 87 MB/s · 9s; if it hits the net, you get MB/s + ETA. delta crafting shows → crafting delta 50% · 744/1476 MB · 420 MB/s too. Add -v for more detail. You can pass a direct .safetensors path to cast instead of a saved name when needed.

---

Details

What is inside the files

• Voice: single-tensor safetensors, lm_head.weight or output.weight, original dtype (BF16/F16/F32), full shape.
• Delta: single-tensor safetensors + tiny marker delta.voice.marker (U8 [1]) + __metadata__ {"type":"delta"}, F16, same shape, voice − base. Marker lets voice cast model.gguf delta.safetensors auto-detect and fuse (just works, no flag, 1 byte overhead, ignored except creation/cast); old delta- folders without marker still work via voice.json type:delta. Custom style.delta (like Lora/StyleTune/style.safetensors) is not our marker — intentionally not auto-fused (use delta- folder or voice tensor explicit if you want it).
• Voiced GGUF: copy of your model with the output head replaced and quantized to Q8_0. All other tensors and metadata are preserved. Vocab, tokenizer, and architecture stay exactly as they were.

How it works under the hood:

• get downloads only the head bytes with range requests, not the whole file, then saves to ~/.voices/<name>/.
• delta subtracts the base head from the voice head (voice − base) and saves an F16 delta as delta-<name>. The base head is cached once at ~/.voices/base/<org>/<model>.safetensors.
• cast replaces the head and writes a new GGUF with the head at Q8_0 (delta to Q8_0 for deltas, direct fuse for full voices). The result is a complete model.

A voice lives in the last layer. Changing that layer changes how the model sounds. Voice saves that layer and casts it onto a compatible model you already have—fused, standalone, and ready to run. Transposition between [vocab, hidden] and [hidden, vocab] is handled automatically. So is BF16 to F32 for the math and F32 to Q8_0 for the result (last dim %32==0 else F16 fallback, still compatible).

Scope: This card documents the Voice tool (adapter extraction) and its quantized GGUF output (head at Q8_0). Base weights remain under their source license; tool code is Apache-2.0—see License.

---

Collection

This voice builds on Gemma 4. Find all Gemma 4 models in the official collection:

• Google Gemma 4 Collection: https://huggingface.co/collections/google/gemma-4—includes google/gemma-4-26B-A4B-it (this card's base_model) and google/gemma-4-26B-A4B.

Related StyleTune voices (examples, any compatible safetensors works):

• StyleTune: 12B · 26B · 26B V2 · 31B

---

Credits

Gryphe proved one tensor changes everything. Anthracite and Latitude helped make it happen. The MythoMax lineage found that lm_head carries style. Hugging Face made the format portable. llama.cpp made it runnable.

StyleTune: 12B · 26B · 26B V2 · 31B

All StyleTune links are examples. Replace with any compatible voice you prefer.

---

License

Voice tooling in this repository is Apache-2.0—see LICENSE. This covers the voice code and packaging only.

Voices, deltas, and base heads keep their source model license. When you run voice get or voice delta, the saved voice.json records the source license for your reference. For example, google/gemma-4-26b-a4b-it (display: google/gemma-4-26B-A4B-it) remains under its Gemma license (Apache-2.0 with Gemma terms—see google/gemma-4-26B-A4B-it and ai.google.dev/gemma/docs/gemma_4_license). Check the Hugging Face model card before you share a voiced model. The google/gemma-4-26b-a4b-it name in the frontmatter is the lowercased, verifiable HF ID (link lives, case-insensitive); gryphe/gemma-4-26b-a4b-styletune-v2 and other StyleTune names are illustrative examples—use any compatible model you have access to.