---
license: apache-2.0
base_model: google/embeddinggemma-2
library_name: sentence-transformers
pipeline_tag: sentence-similarity
language:
  - tr
  - multilingual
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - embedding
  - turkish
  - retrieval
  - tokenizer-surgery
  - magibu-ai
---

# embeddingmagibu2

**embeddingmagibu2** is a Turkish-focused multilingual embedding model built from
[google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) and the Turkish-optimized
[magibu/Altus](https://huggingface.co/magibu/Altus) tokenizer.

> **Status: training in progress.** See *Training progress* below for the latest metrics.

<!-- training-progress:start -->
## Training progress

Offline distillation from `google/embeddinggemma-2` is **in progress**; this table is updated automatically at every release.
Latest release: **phase6-train-full-step8243**. Phases: test/new → test/full → validation/new → validation/full → train/new → train/full
(`new`: only the new Turkish token embeddings train; `full`: all embeddings + text backbone at a low learning rate).
Metrics on 2,048 held-out validation rows: cosine to the teacher's document embeddings for Turkish and for the other
languages, their mean (score), and STSbTR test Spearman.

| Release | Step | cos_doc_tr | cos_doc_other | score | STSbTR Spearman | Time (UTC) |
|---|---|---|---|---|---|---|
| initial (untrained) | 0 | 0.9314 | 0.9789 | 0.9551 | 0.5867 | 2026-10-10 14:21 |
| phase1-test-new-done | 600 | 0.9328 | 0.9802 | 0.9565 | 0.5900 | 2026-10-10 14:46 |
| phase2-test-full-step1600 | 1600 | 0.9478 | 0.9837 | 0.9658 | 0.5946 | 2026-10-10 16:58 |
| phase2-test-full-done | 1812 | 0.9497 | 0.9839 | 0.9668 | 0.5987 | 2026-10-10 17:26 |
| phase3-validation-new-done | 2212 | 0.9497 | 0.9839 | 0.9668 | 0.5987 | 2026-10-10 17:45 |
| phase4-validation-full-step3212 | 3212 | 0.9553 | 0.9848 | 0.9700 | 0.6076 | 2026-10-10 19:59 |
| phase4-validation-full-step4212 | 4212 | 0.9591 | 0.9850 | 0.9721 | 0.6159 | 2026-10-10 23:16 |
| phase4-validation-full-step5212 | 5212 | 0.9621 | 0.9854 | 0.9737 | 0.6226 | 2026-10-11 01:25 |
| phase4-validation-full-done | 6049 | 0.9635 | 0.9855 | 0.9745 | 0.6303 | 2026-10-11 03:13 |
| phase5-train-new-done | 6449 | 0.9635 | 0.9855 | 0.9745 | 0.6303 | 2026-10-11 03:35 |
| phase6-train-full-done | 7757 | 0.9641 | 0.9854 | 0.9748 | 0.6304 | 2026-10-11 07:59 |
| phase6-train-full-step8243 | 8243 | 0.9667 | 0.9863 | 0.9765 | 0.6309 | 2026-10-11 11:06 |
<!-- training-progress:end -->

## How it was built

It follows the tokenizer-surgery + offline-distillation pipeline of
[Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation](https://arxiv.org/abs/2605.29992).

1. **Tokenizer swap.** Altus is the Gemma-4 vocabulary (identical to the teacher's) in which
   42,484 rarely used tokens were replaced by Turkish subwords from Magibu's 65,536-token BPE vocabulary.
2. **Mean-composition initialization.** The 219,660 unchanged token ids keep their teacher
   embeddings exactly. Each of the 42,484 new tokens is initialized as the mean of the teacher
   embeddings of the teacher tokens that spell it (using Altus' `merged-to-original-token-ids.json`),
   e.g. `ağ` ← mean(`a`, `ğ`). All other weights are the teacher's, unchanged.
3. **Offline distillation (in progress).** The student is trained to match precomputed teacher
   embeddings on [alibayram/wikipedia-40-langs-with-embeddings-embeddinggemma2](https://huggingface.co/datasets/alibayram/wikipedia-40-langs-with-embeddings-embeddinggemma2):
   40-language Wikipedia, with retrieval-style targets for documents
   (`title: {title} | text: {text}`) and title queries (`task: search result | query: {title}`).
   Texts are clipped so they fit in 8,192 tokens under **both** the teacher and the Altus tokenizer,
   so student and teacher always see identical inputs.

## Model details

| | |
|---|---|
| Base model | google/embeddinggemma-2 |
| Tokenizer | magibu/Altus (BPE, 262,144 tokens) |
| Embedding dimension | 768, L2-normalized |
| Max sequence length | 8,192 tokens |
| Pooling | mean |
| Parameters | ~744M total (text model ~271M; vision and audio towers inherited from the base model) |

Tokens needed for the same Wikipedia text, Altus vs. the original tokenizer (6,000-article sample):

| Language | Altus / original |
|---|---|
| Turkish | **0.79×** (21% fewer tokens) |
| Most other languages | 1.02–1.08× |
| Russian, Bulgarian | 1.10–1.15× |

Fewer Turkish tokens means more Turkish text fits into the 8K context and inference is faster.

## Usage

Use the same prompts as EmbeddingGemma: a query prompt for queries and a document prompt for documents.

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("alibayram/embeddingmagibu2")

queries = ["Türkiye'nin başkenti neresidir?"]
documents = [
    "Ankara, Türkiye'nin başkentidir ve İç Anadolu Bölgesi'nde yer alır.",
    "İstanbul, Türkiye'nin en kalabalık şehridir.",
]

query_embeddings = model.encode(queries, prompt_name="query")
document_embeddings = model.encode(documents, prompt_name="document")
print(model.similarity(query_embeddings, document_embeddings))
```

## Limitations

- Distillation is still in progress; see *Training progress* for the current quality.
- Languages whose text becomes longer under Altus (e.g. Russian, Bulgarian) may lose some quality.
- The vision and audio towers are inherited but not adapted; use this model for text.

## Citation

```bibtex
@misc{bayram2026embeddingmagibu,
  title         = {Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation},
  author        = {Bayram, M. Ali and Diri, Banu and Y{\i}ld{\i}r{\i}m, Sava{\c{s}}},
  year          = {2026},
  eprint        = {2605.29992},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL}
}
```

Developed by [Magibu AI Research](https://huggingface.co/magibu).