ukint-vs/Tess-4-27B-MLX-mixed_4_6

Tess-4-27B MLX mixed_4_6

This is an MLX conversion of migtissera/Tess-4-27B, a Qwen3.6-27B based Qwen3.5-family multimodal reasoning model.

Conversion

• Source model: migtissera/Tess-4-27B
• Source revision: 65e1463e388bd2e94afc33a7d376d501cebe71b2
• Format: MLX safetensors
• Quantization: mixed_4_6
• Reported size: about 16 GB
• Reported average precision: 5.092 bits per weight
• Vision tower: kept unquantized by the mlx-vlm conversion predicate

The mixed quantization keeps embeddings, lm_head, selected v_proj, and selected down_proj layers at 6-bit while using 4-bit for most eligible language linear layers.

Requirements

Use a recent mlx-vlm that includes the Qwen3.5 norm sanitization fix from PR Blaizzy/mlx-vlm#1528. Older mlx-vlm builds can double-shift Qwen3.5 norm weights when loading converted MLX checkpoints, causing broken logits.

pip install -U mlx mlx-lm
pip install -U git+https://github.com/Blaizzy/mlx-vlm.git

Quickstart

from mlx_vlm.generate import generate
from mlx_vlm.utils import load

model_id = "ukint-vs/Tess-4-27B-MLX-mixed_4_6"

model, processor = load(model_id)

messages = [
    {"role": "user", "content": "Explain the tradeoffs of LoRA vs full fine-tuning."}
]

prompt = processor.tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

response = generate(
    model,
    processor,
    prompt,
    max_tokens=512,
    temp=0.6,
)

print(response.text)

For a concise answer without visible reasoning:

prompt = processor.tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is 2+2? Answer briefly."}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

Prompt Format

Tess-4 uses the Qwen3.5-family chat template with explicit thinking blocks:

<|im_start|>user
Your prompt here<|im_end|>
<|im_start|>assistant
<think>
... reasoning ...
</think>

... answer ...<|im_end|>

Prefer processor.tokenizer.apply_chat_template(...) instead of writing the template by hand.

Verification

The converted model was smoke-tested locally with greedy generation:

User: What is 2+2? Answer briefly.
Assistant: 4

Smoke-test stats on the conversion machine:

• Prompt tokens: 22
• Generated tokens: 2
• Peak memory: about 19.0 GB

Notes

This is a lossy quantized conversion intended for local Apple Silicon inference with MLX. For maximum fidelity, use the original BF16 checkpoint. For llama.cpp, LM Studio, or Ollama-style runtimes, see the upstream GGUF repo linked from the original model card.

Attribution

Tess-4-27B was created by Migel Tissera and is built on Qwen/Qwen3.6-27B. This conversion keeps the original Apache 2.0 license.

Original model card: migtissera/Tess-4-27B