README.md

---
base_model: google/gemma-4-E2B-it-assistant
language:
- en
library_name: gguf
license: gemma
tags:
- gguf
- quantized
- mtp
- gemma-4
---

# Gemma 4 E2B IT Assistant — MTP GGUF

This repository contains a GGUF conversion of [google/gemma-4-E2B-it-assistant](https://huggingface.co/google/gemma-4-E2B-it-assistant) to use as MTP, including the centroid-based embedding tensors (`masked_embedding.centroids` and `masked_embedding.token_ordering`) used by the E2B/E4B sparse variants.

Do not use with llama.cpp as they do not offer support. This model is only compatible with [ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp).

~~Use only with [PR 1744](https://github.com/ikawrakow/ik_llama.cpp/pull/1744), which enables the gemma 4 MTP to work, otherwise you will encounter errors such as the model failing to load.~~

**New:** PR is merged.

## Usage

The assistant is a **draft model** that requires a Gemma 4 target GGUF alongside it.

```bash
./build/bin/llama-server \
  --model google_gemma-4-E2B-it-Q8_0.gguf \
  --ctx-size 32768 ctk q8_0 -ctv q8_0 --n-gpu-layers 99 \
  -b 1024 -ub 1024 --jinja \
  --spec-type mtp -md gemma-4-E2B-it-assistant-F16.gguf -ngld 99 \
  --draft-max 3 --draft-p-min 0.0
```

> **Note:** `--draft-max 3` is a good starting point.