leonsarmiento/Tiger-Gemma-12B-v3-4bit-mlx

Tiger-Gemma-12B-v3 — 4-bit (MLX)

Uniform 4-bit MLX quantization of TheDrummer/Tiger-Gemma-12B-v3 — TheDrummer's Gemma-3-12B tune that "unlocks more capabilities and less positivity": more neutral tone on harder/darker topics, better steerability, paragraph-style prose with fewer em-dashes.

Vision preserved — full multimodal (image input works in LM Studio / mlx_vlm).

Quantization

Recipe	Uniform 4-bit, group size 64, affine (vision tower included)
Bits per weight	4.6
Size	7.7 GB
Shards	2
Framework	mlx_vlm (gemma3, tie_word_embeddings handled)

Chat template shipped in both tokenizer_config.json and chat_template.jinja.

Note on generation config: the source model's generation_config.json ships baked-in sampling defaults (do_sample: true, top_k: 64, top_p: 0.95) that some runtimes (mlx_vlm) merge into their sampler, producing degenerate output. This build ships Google's canonical generation_config.json instead — behavior in transformers/vLLM is unchanged.

Usage

Works in LM Studio, oMLX, and mlx_vlm:

import mlx_vlm
model, processor = mlx_vlm.load("leonsarmiento/Tiger-Gemma-12B-v3-4bit-mlx")

Suggested starting settings (tune to taste — the source card is authoritative): temp 0.7–1.0 · min_p 0.05 · top_p 1.0 · 128K context

An 8-bit uniform variant is available at leonsarmiento/Tiger-Gemma-12B-v3-8bit-mlx.