---
language:
  - en
  - zh
  - ja
  - ko
  - de
  - fr
  - es
  - pt
  - it
  - ar
  - hi
license: gemma
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
library_name: mlx
pipeline_tag: image-text-to-text
base_model: 0xSero/gemma-4-19b-a4b-it-REAP
tags:
- mlx
- gemma4
- reap
- ple-safe
- quantized
- apple-silicon
- vision
- moe
---

# gemma-4-19b-a4b-it-REAP-MLX-4bit

**PLE-safe** MLX 4-bit weights for [0xSero/gemma-4-19b-a4b-it-REAP](https://huggingface.co/0xSero/gemma-4-19b-a4b-it-REAP) on Apple Silicon.

[REAP](https://arxiv.org/abs/2510.13999) (Router-weighted Expert Activation Pruning) removes 30% of MoE experts while keeping the same active parameters per token (8 of 90 experts selected). Combined with PLE-safe 4-bit quantization, this model runs in **12.6 GB** — fits on 24GB+ Macs.

| | Original 26B | REAP (19B MoE (30% pruned)) | **This model** |
|---|---|---|---|
| Experts/layer | 128 | 90 | 90 |
| Precision | BF16 | BF16 | **4-bit** |
| Disk size | ~52 GB | ~36 GB | **12.6 GB** |

## Also available

- [gemma-4-21b-a4b-it-REAP-MLX-4bit](https://huggingface.co/ukint-vs/gemma-4-21b-a4b-it-REAP-MLX-4bit) — 21B MoE (20% pruned), 13.9 GB

## Accuracy Benchmarks

0-shot generative, thinking enabled, 50 samples per task, identical eval harness. Apple M4 Max 36GB.

| Task | 26B-A4B 4-bit (16.4 GB) | **This model (12.6 GB)** |
|---|---|---|
| Elementary Mathematics | 84% | **44%** |
| Philosophy | 66% | **54%** |
| World Religions | 66% | **34%** |
| College Computer Science | 58% | **34%** |
| High School Mathematics | 26% | **22%** |
| Abstract Algebra | 44% | **36%** |
| College Mathematics | 36% | **16%** |
| Gsm8K | 64% | **62%** |

The 30% expert pruning compounds with 4-bit quantization. Note: high extraction failure rates (up to 60%) on some tasks — the model generates verbose explanations instead of single-letter answers, so true accuracy may be higher than reported. Consider the [21B variant](https://huggingface.co/ukint-vs/gemma-4-21b-a4b-it-REAP-MLX-4bit) for better accuracy.

Extraction failures (unparseable responses) are counted as incorrect. REAP-19B: 113/400 (28%). True accuracy may be higher. Full methodology: [GitHub](https://github.com/ukint-vs/mlx-gemma4-reap).

## Quantization Details

- **Bits:** 4
- **Group size:** 64
- **Strategy:** PLE-safe — only large `nn.Linear` and `SwitchLinear` (MoE expert) layers are quantized. All PLE/ScaledLinear/vision layers stay in bf16.

| Quantized (4-bit) | Kept in bf16 |
|---|---|
| Attention projections (q/k/v/o_proj) | ScaledEmbedding (embed_tokens) |
| MLP layers (gate/up/down_proj) | ScaledLinear (PLE pathway) |
| MoE expert layers (SwitchLinear) | Per-layer embeddings (per_layer_*) |
| | Vision encoder |
| | All norms and scalars |

## Usage

```bash
pip install -U mlx-vlm
```

### Vision

```python
from mlx_vlm import load, generate

model, processor = load("ukint-vs/gemma-4-19b-a4b-it-REAP-MLX-4bit")
tokenizer = processor.tokenizer

messages = [{"role": "user", "content": [
    {"type": "image", "url": "photo.jpg"},
    {"type": "text", "text": "Describe this image in detail."},
]}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
out = generate(model, processor, prompt, ["photo.jpg"],
    max_tokens=200, repetition_penalty=1.2, temperature=0.7)
print(out.text)
```

### Text

```python
messages = [{"role": "user", "content": "What is the capital of France?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
out = generate(model, processor, prompt, max_tokens=100, temperature=0.0)
print(out.text)
```

## Validation

Trimodal validation: 10/10 vision, 3/3 chat (EN/ZH/JA). Full results: [GitHub](https://github.com/ukint-vs/mlx-gemma4-reap).

## Bugs Fixed in mlx-vlm

| # | Bug | Fix |
|---|---|---|
| 1 | `ScaledLinear` inherits `nn.Module` not `nn.Linear` | Change to `ScaledLinear(nn.Linear)` |
| 2 | Standard quantization quantizes PLE layers | PLE-safe `class_predicate` |
| 3 | `processor.save_pretrained()` strips audio config | Copy `processor_config.json` from source |
| 4 | `SwitchLinear` (MoE) not quantized | Check `hasattr(module, 'to_quantized')` |
| 5 | `embed_scale` double-scaling (mlx-vlm 0.4.4+) | Set `Gemma4TextModel.embed_scale = 1.0` |

## License

Model weights: [Google Gemma License](https://ai.google.dev/gemma/terms). Quantization scripts: MIT.