| --- |
| license: apache-2.0 |
| base_model: google/embeddinggemma-2 |
| base_model_relation: quantized |
| library_name: llama.cpp |
| pipeline_tag: feature-extraction |
| tags: |
| - gguf |
| - llama.cpp |
| - embedding |
| - multimodal |
| - multilingual |
| - quantized |
| - sentence-transformers |
| --- |
| |
| # EmbeddingGemma 2 GGUF |
|
|
| Community GGUF quantizations of [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2). |
|
|
| <div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;"> |
| ☕ If this GGUF made your day easier, a coffee would make mine.<br> |
| <a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br> |
| I build and test these releases myself. Your coffee helps keep me going.<br> |
| Thank you for supporting this work. |
| </div> |
|
|
| ## About EmbeddingGemma 2 |
|
|
| [EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) is a multilingual, multimodal embedding model from Google DeepMind. It maps text and code, images, video, and audio into a shared 768-dimensional vector space and has an 8,192-token shared context window. The upstream checkpoint has a 270M-parameter text path plus optional vision and audio encoders. This release contains a quantized text backbone and a shared BF16 vision/audio projector. Text, image, and audio inputs returned normalized 768-dimensional vectors in local `llama-server` CPU smoke checks. See the [official model card](https://huggingface.co/google/embeddinggemma-2) for supported task prefixes, modalities, and input guidance. |
|
|
| ## Embedding evaluation |
|
|
| Every value below comes from local measurements of the locked BF16 GGUF and quantized files; the numbers are not copied from the upstream model card. The fixed task evaluation uses the 1,379-pair test split of [MTEB STSBenchmark STS](https://huggingface.co/datasets/mteb/stsbenchmark-sts), dataset revision `96943a16ea6a35129e253c659081cb59daf81b30`. It covers 2,552 unique sentences with the `task: sentence similarity | query:` prefix, an 8,192-token context, and the CPU `llama.cpp` runtime at commit `9c2e0e491a822adae1f0b1c831adb4160057d24f`. Spearman and Pearson measure correlation between cosine similarity and human scores. Mean and 5th-percentile cosine measure vector agreement with the BF16 GGUF reference. Higher values indicate stronger task correlation or closer vector agreement; smaller files use less disk. This is a task-specific embedding evaluation, not the full MTEB benchmark. Token-level KLD, next-token Top-1, PPL, and RMS probability deltas do not apply to this embedding model. |
|
|
| There are 26 quantized text artifacts in this release. Hold-out measurements are available for 24; the two TQ variants passed runtime smoke checks but were not included in this evaluation checkpoint, so no quality results are claimed for them. |
|
|
| ### BF16 reference baseline |
|
|
| | Reference | Size (GB) | Spearman vs human ↑ | Pearson vs human ↑ | Mean cosine vs BF16 | 5th-percentile cosine vs BF16 | |
| |---|---:|---:|---:|---:|---:| |
| | Local BF16 text GGUF | 0.557950 | 0.875414 | 0.865994 | 1.000000 | 1.000000 | |
|
|
| ### Quantized variants compared with BF16 |
|
|
| | File | Size (GB) | Spearman vs human ↑ | Pearson vs human ↑ | Mean cosine vs BF16 ↑ | 5th-percentile cosine vs BF16 ↑ | |
| |---|---:|---:|---:|---:|---:| |
| | `EmbeddingGemma-2-Q8_0.gguf` | 0.309856 | 0.875125 | 0.865776 | 0.999930 | 0.999903 | |
| | `EmbeddingGemma-2-Q6_K.gguf` | 0.245765 | 0.874311 | 0.864975 | 0.999548 | 0.999373 | |
| | `EmbeddingGemma-2-Q5_K_M.gguf` | 0.212759 | 0.874081 | 0.864863 | 0.999190 | 0.998841 | |
| | **`EmbeddingGemma-2-Q5_K_S.gguf`** | **0.210670** | **0.874892** | **0.865614** | **0.999104** | **0.998727** | |
| | `EmbeddingGemma-2-Q4_K_M.gguf` | 0.181695 | 0.872686 | 0.864659 | 0.997824 | 0.996776 | |
| | **`EmbeddingGemma-2-Q4_K_S.gguf`** | **0.178164** | **0.873112** | **0.864908** | **0.997379** | **0.996201** | |
| | `EmbeddingGemma-2-IQ4_NL.gguf` | 0.177640 | 0.871242 | 0.862865 | 0.996884 | 0.995270 | |
| | `EmbeddingGemma-2-IQ4_XS.gguf` | 0.169383 | 0.871200 | 0.863202 | 0.996687 | 0.994900 | |
| | `EmbeddingGemma-2-Q3_K_L.gguf` | 0.154440 | 0.869874 | 0.860846 | 0.993184 | 0.989422 | |
| | `EmbeddingGemma-2-Q3_K_M.gguf` | 0.148870 | 0.869604 | 0.860705 | 0.992364 | 0.988323 | |
| | `EmbeddingGemma-2-IQ3_M.gguf` | 0.145749 | 0.870880 | 0.862364 | 0.992638 | 0.988771 | |
| | `EmbeddingGemma-2-Q3_K_S.gguf` | 0.142546 | 0.866108 | 0.853203 | 0.987335 | 0.981859 | |
| | `EmbeddingGemma-2-IQ3_S.gguf` | 0.142546 | 0.870980 | 0.863233 | 0.990906 | 0.986058 | |
| | `EmbeddingGemma-2-IQ3_XS.gguf` | 0.139793 | 0.867124 | 0.859804 | 0.989128 | 0.983518 | |
| | `EmbeddingGemma-2-IQ3_XXS.gguf` | 0.135776 | 0.862974 | 0.858534 | 0.983963 | 0.976505 | |
| | `EmbeddingGemma-2-IQ2_M.gguf` | 0.130910 | 0.862532 | 0.861373 | 0.971570 | 0.960803 | |
| | `EmbeddingGemma-2-IQ2_S.gguf` | 0.127600 | 0.856632 | 0.851405 | 0.963077 | 0.949864 | |
| | `EmbeddingGemma-2-Q2_K.gguf` | 0.120394 | 0.856144 | 0.854341 | 0.972891 | 0.960440 | |
| | `EmbeddingGemma-2-Q2_K_S.gguf` | 0.116446 | 0.843346 | 0.843931 | 0.962411 | 0.948042 | |
| | `EmbeddingGemma-2-IQ2_XS.gguf` | 0.110946 | 0.846829 | 0.838447 | 0.944608 | 0.927359 | |
| | `EmbeddingGemma-2-IQ2_XXS.gguf` | 0.107178 | 0.834294 | 0.827123 | 0.919032 | 0.899266 | |
| | `EmbeddingGemma-2-IQ1_M.gguf` | 0.103041 | 0.797642 | 0.801111 | 0.819494 | 0.778721 | |
| | `EmbeddingGemma-2-IQ1_S.gguf` | 0.100558 | 0.761951 | 0.772373 | 0.768243 | 0.728349 | |
| | `EmbeddingGemma-2-Q1_0.gguf` | 0.066163 | 0.061457 | 0.062574 | 0.513763 | 0.425907 | |
|
|
| Q5_K_S and Q4_K_S are the current memory/fidelity candidates on this measured task: their Spearman scores are 0.874892 and 0.873112 with sizes 0.210670 GB and 0.178164 GB. No VRAM-fit or throughput claim is made. Q1_0 and IQ1_S return valid vectors but show substantial loss on this task; the TQ formats have smoke results only. The [machine-readable evaluation summary](reproducibility/embedding-evaluation.tsv) and [reproducibility manifest](reproducibility/manifest.md) include the measurements and method. The Q8_0 artifact and BF16 vision/audio projector also passed local text, image, and audio embedding smoke checks. |
| |
| ## Quick start |
| |
| Start an embedding endpoint with `llama.cpp`: |
| |
| ```bash |
| llama-server \ |
| -m EmbeddingGemma-2-Q8_0.gguf \ |
| --mmproj mmproj-EmbeddingGemma-2-BF16.gguf \ |
| --embedding \ |
| --pooling mean \ |
| --ctx-size 8192 \ |
| --port 8080 |
| ``` |
| |
| Send an English search query using the model's task prefix: |
| |
| ```bash |
| curl http://localhost:8080/v1/embeddings \ |
| -H 'Content-Type: application/json' \ |
| -d '{"model":"EmbeddingGemma-2","input":"task: search result | query: What causes the northern lights?"}' |
| ``` |
| |
| For text-only workloads, omit `--mmproj`. For documents, use `title: {title} | text: {content}` or `title: none | text: {content}` when there is no title. Keep query and document vectors at the same dimension. The server returns normalized vectors by default. |
| |
| ## Reproducibility and validation |
| |
| See the [reproducibility manifest](reproducibility/manifest.md), [artifact measurements](reproducibility/artifact-measurements.tsv), and [SHA256 checksums](SHA256SUMS.txt). The manifest records the locked upstream revision and model hash, BF16 GGUF source, converter/runtime revision, model-specific calibration/imatrix, quantization commands, and smoke profile. Raw conversion, imatrix, quantization, and validation logs are kept locally and are not part of this repository. |
| |
| ## License and attribution |
| |
| The upstream model is licensed under [Apache 2.0](LICENSE). These are community GGUF quantizations, not an official Google or Google DeepMind release or endorsement. |
| |
