m1rkocasu/Apertus-v1.5-8B-text-MLX-5bit

Apertus v1.5 8B text, MLX 5-bit

This repository holds the text branch of Apertus 1.5 8B, converted to the MLX format and quantized to 5-bit. It runs on Apple silicon through mlx-lm.

Apertus 1.5 is a fully open model from the Swiss AI Initiative, built at EPFL, ETH Zurich, and the Swiss National Supercomputing Centre on open data.

Model summary

Base model	swiss-ai/Apertus-v1.5-8B
Parameters	8.05B (8,053,338,240)
Architecture	ApertusForCausalLM, 32 layers, hidden size 4096, 32 attention heads, 8 key-value heads
Activation	xIELU
Vocabulary	131072 text tokens
Context length	262144
Quantization	5-bit, 5.500 bits per weight
Size on disk	5.54 GB
Format	MLX safetensors
Modality	text in, text out
License	Apache 2.0, with the Apertus 1.5 acceptable use policy

The model size that the sidebar of this page reports is smaller than 8.05B. That figure counts the elements of the stored tensors, and MLX packs quantized weights into 32-bit containers, eight weights to a container at 4 bits and four at 8 bits. The parameter count of the model is the one in the table.

Run it

pip install mlx-lm
mlx_lm.generate --model m1rkocasu/Apertus-v1.5-8B-text-MLX-5bit \
  --prompt "Name the capital of Switzerland and say one sentence about it." \
  --max-tokens 200

From Python:

from mlx_lm import load, generate

model, tokenizer = load("m1rkocasu/Apertus-v1.5-8B-text-MLX-5bit")
messages = [{"role": "user", "content": "Name the capital of Switzerland."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

Behind an OpenAI-compatible endpoint:

mlx_lm.server --model m1rkocasu/Apertus-v1.5-8B-text-MLX-5bit

This quantization

Every quantizable layer holds 5 bits with a group size of 64.

mlx_lm.convert --hf-path <text branch> \
  --mlx-path Apertus-v1.5-8B-text-MLX-5bit -q --q-bits 5 --q-group-size 64

The work ran with mlx-lm 0.31.3 and mlx 0.32.0 on macOS 26.5.

Quality

Perplexity is 10.5261 on the test split of Salesforce/wikitext, configuration wikitext-2-raw-v1, scored over 200 non-overlapping windows of 512 tokens, which is 102400 scored tokens.

Variant	Bits per weight	Size	Perplexity	Against the 8-bit build
8-bit	8.500	8.54 GB	10.41	reference
6-bit	6.500	6.54 GB	10.46	+0.5%
5-bit (this repository)	5.500	5.54 GB	10.53	+1.1%
4-bit DWQ	4.500	4.53 GB	10.97	+5.4%
MXFP4	4.250	4.28 GB	11.71	+12.5%

The 8-bit build is the reference because a bfloat16 run does not fit the 24 GB machine these were made on. Its 15 GB of weights page to disk, so no bfloat16 number is reported here. The step from 8 bits to 6 costs half a percent, which makes a large gap between bfloat16 and 8 bits unlikely, but that remains an inference and not a measurement.

Perplexity compares quantizations of one model on one corpus. It says nothing about how this model compares to a different model, and nothing about how well it follows instructions.

Reproduce the number

"""Token-level perplexity of an MLX model on the wikitext-2 raw test split.

The split is downloaded once from the Hub and read with pyarrow, so the run
needs no dataset library. The text of every row is joined with newlines,
tokenized once, then scored in non-overlapping windows.
"""
import sys, json, math
import mlx.core as mx
from mlx_lm import load
from huggingface_hub import hf_hub_download
import pyarrow.parquet as pq

import os
SEQ, BATCH, MAX_WINDOWS = 512, int(os.environ.get("PPL_BATCH", 4)), 200

def corpus(tokenizer):
    p = hf_hub_download("Salesforce/wikitext",
                        "wikitext-2-raw-v1/test-00000-of-00001.parquet",
                        repo_type="dataset")
    rows = pq.read_table(p).column("text").to_pylist()
    return tokenizer.encode("".join(rows))

def perplexity(path):
    model, tokenizer = load(path)
    ids = corpus(tokenizer)
    n = min(MAX_WINDOWS, (len(ids) - 1) // SEQ)
    total_nll, total_tok = 0.0, 0
    for start in range(0, n, BATCH):
        chunk = [ids[(start + j) * SEQ:(start + j) * SEQ + SEQ + 1]
                 for j in range(min(BATCH, n - start))]
        x = mx.array([c[:-1] for c in chunk])
        y = mx.array([c[1:] for c in chunk])
        logits = model(x).astype(mx.float32)
        nll = mx.take_along_axis(
            -mx.log(mx.softmax(logits, axis=-1)), y[..., None], axis=-1)
        total_nll += float(nll.sum())
        total_tok += y.size
        mx.clear_cache()
    return math.exp(total_nll / total_tok), total_tok

if __name__ == "__main__":
    ppl, tok = perplexity(sys.argv[1])
    print(json.dumps({"model": sys.argv[1].rstrip("/").split("/")[-1],
                      "perplexity": round(ppl, 4), "tokens": tok}))

What did not work

These builds were measured on the same corpus and are not published. The table is here so that nobody repeats the work.

Build	Bits per weight	Size	Perplexity	Why it is not here
mixed 4/6, affine	5.000	5.03 GB	11.24	larger than the 4-bit DWQ build and weaker
4-bit, affine, no distillation	4.500	4.53 GB	11.59	same size as the 4-bit DWQ build and weaker
3-bit, group size 32	4.000	4.03 GB	66.06	unusable
3-bit DWQ, group size 64	3.500	3.52 GB	50.91	unusable
mixed 3/6, affine	4.250	4.28 GB	140.93	unusable
3-bit, group size 64	3.500	3.52 GB	169.79	unusable

Three bits break this model. Uniform 3-bit affine quantization at a group size of 64 answers fluently and wrongly, completing "The capital of Switzerland is" with "not a good idea". A group size of 32 answers "Bern" and still reaches a perplexity of 66. The stock mixed_3_6 recipe of mlx-lm returns nothing at all, and forcing its token embedding to 6 bits changes little, which locates the damage in the 3-bit layers and not in the embedding. Distillation recovers a large share of the loss, moving the 3-bit build from 169.79 to 50.91, and the result is still five times the reference.

What the text branch is

The upstream release is multimodal. It reads images and audio as well as text, and its architecture is Apertus1p5ForConditionalGeneration, which neither mlx-lm 0.31.3 nor mlx-vlm 0.6.17 implements.

This repository holds the text decoder on its own, declared as ApertusForCausalLM so that mlx-lm loads it. The token embedding goes from 266752 rows to 131072. The rows that go are the image and audio codebooks, which start at index 131272 and 262344 in the upstream vocabulary. The upstream output_vocab_size is already 131072, so the language modelling head is untouched and no text token is lost.

The model reads and writes text. It does not accept images or audio.

Limitations

The quantization inherits every limitation of the upstream model, which its model card describes. No output filter ships with these weights.

Quality here is measured by perplexity on one English corpus. Apertus 1.5 is multilingual, and the effect of quantization on languages other than English is not measured in this collection. Instruction following, reasoning, and tool use are also unmeasured.

License and acceptable use

The weights stay under the Apache 2.0 license of the upstream release. Use is also subject to the Apertus 1.5 acceptable use policy and privacy policy:

• Acceptable use policy
• Privacy policy
• EU AI Act public summary
• EU AI Act code of practice

For removal of personal or copyrighted data, write to the Swiss AI Initiative at llm-privacy-requests@swiss-ai.org or llm-copyright-requests@swiss-ai.org.

Credits

The model is the work of the Swiss AI Initiative. This repository adds the MLX conversion, the quantization, and the measurements above.

@misc{ApertusV15,
  author       = {{Swiss AI Initiative}},
  title        = {Apertus v1.5},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/swiss-ai/Apertus-v1.5-8B}},
  note         = {EPFL, ETH Zurich, and the Swiss National Supercomputing Centre}
}