README.md

---
library_name: sentence-transformers
license: apache-2.0
language:
- multilingual
base_model:
- Hcompany/NeoMME-800M-Retriever
pipeline_tag: sentence-similarity
tags:
- multimodal
- document-retrieval
- dense-retrieval
---

<p align="left">
  <img src="https://github.com/tonywu71/colpali-cookbooks/blob/6ef1332da6bcb48c7ef1f19b25bfa555be7031a8/assets/neomme/neomme_logo.webp?raw=true" alt="NeoMME logo" style="max-height: 140px;">
</p>

# NeoMME-Retriever (800M): Single-Tower Multimodal-Native Multilingual Foundation Encoder 🔎

> [!IMPORTANT]
> NeoMME-Retriever (800M) variants:
>
> - [Default (`transformers`)](https://huggingface.co/Hcompany/NeoMME-800M-Retriever): Returns dense and multi-vector embeddings together with a single forward pass. Recommended for most use cases and inference.
> - [ST dense](https://huggingface.co/Hcompany/NeoMME-800M-Retriever-ST-dense) [current]: Supports independent dense fine-tuning with Sentence Transformers.
> - [ST late-interaction](https://huggingface.co/Hcompany/NeoMME-800M-Retriever-ST-late): Supports independent multi-vector fine-tuning with Sentence Transformers.

[![Hugging Face](https://img.shields.io/badge/Model_doc-FFD21E?style=for-the-badge&logo=huggingface&logoColor=000)](https://huggingface.co/docs/transformers/en/model_doc/neomme)
[![Hugging Face](https://img.shields.io/badge/Collection-FFD21E?style=for-the-badge&logo=huggingface&logoColor=000)](https://hf.co/collections/Hcompany/neomme)
[![arXiv](https://img.shields.io/badge/arXiv-2609.01657-b31b1b.svg?style=for-the-badge)](https://arxiv.org/abs/2609.01657)

NeoMME-800M-Retriever-ST-dense is a model for multimodal document retrieval. Fine-tuned from [NeoMME-800M](https://huggingface.co/Hcompany/NeoMME-800M), it encodes text queries and documents (text or page screenshots) using one shared bidirectional Transformer encoder.

This model can be used with Sentence Transformers, but can only generate dense embeddings.

<table>
<thead>
<tr style="background-color: rgba(146, 81, 247, 0.20);"><th>Specification</th><th>Value</th></tr>
</thead>
<tbody>
<tr><td>Parameters</td><td>800M</td></tr>
<tr><td>Vocabulary</td><td>131,072 tokens</td></tr>
<tr><td>Context length</td><td>16,384 tokens</td></tr>
<tr><td>Hidden size</td><td>1,792</td></tr>
<tr><td>Image patches</td><td>32 × 32 pixels, up to 2,048 pixels on the longest side (default)</td></tr>
<tr><td>Dense embeddings</td><td>1,792 dimensions (Matryoshka: [128, 256, 512, 1,024, 1,792])</td></tr>
<tr><td>Dense pooling strategy</td><td>Mean</td></tr>
</tbody>
</table>

Dense embeddings are L2-normalized and use cosine similarity. They match `NeoMMEForRetrieval.dense_embeddings`.

## Performance

All scores use the metric shown at the full trained dimensions. Higher is better. ViDoRe v3, v2, and v1 measure visual document retrieval, while BEIR-15 measures text retrieval.

<table>
<thead>
<tr><th rowspan="2">Benchmark</th><th rowspan="2">Metric</th><th colspan="2">NeoMME-260M</th><th colspan="2" style="background-color: rgba(37, 99, 235, 0.20);">NeoMME-800M</th></tr>
<tr><th>Late interaction</th><th>Dense</th><th>Late interaction</th><th style="background-color: rgba(37, 99, 235, 0.20);">Dense [current]</th></tr>
</thead>
<tbody>
<tr><td>ViDoRe v3</td><td>nDCG@10</td><td>0.5226</td><td>0.3907</td><td>0.5560</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.4391</td></tr>
<tr><td>ViDoRe v2</td><td>nDCG@5</td><td>0.5218</td><td>0.4075</td><td>0.5591</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.4475</td></tr>
<tr><td>ViDoRe v1</td><td>nDCG@5</td><td>0.8598</td><td>0.7552</td><td>0.8744</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.7993</td></tr>
<tr><td>BEIR-15</td><td>nDCG@10</td><td>0.4881</td><td>0.3055</td><td>0.5126</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.3686</td></tr>
</tbody>
</table>


## Usage

```bash
pip install -U "sentence-transformers[image]"
```

```python
from sentence_transformers import SentenceTransformer


model = SentenceTransformer("Hcompany/NeoMME-800M-Retriever-ST-dense")

queries = [
    "Quelle partie de la production pétrolière du Kazakhstan provient de champs en mer ?",
    "Which hour of the day had the highest overall electricity generation in 2019?",
]
documents = [
    "https://github.com/tonywu71/colpali-cookbooks/blob/main/examples/data/shift_kazakhstan.jpg?raw=true",
    "https://github.com/tonywu71/colpali-cookbooks/blob/main/examples/data/energy_electricity_generation.jpg?raw=true",
]

query_embeddings = model.encode_query(queries, convert_to_tensor=True)
document_embeddings = model.encode_document(documents, convert_to_tensor=True)
scores = model.similarity(query_embeddings, document_embeddings)

# Expected: scores[0, 0] > scores[0, 1] and scores[1, 1] > scores[1, 0].
print(scores)
```

The score tensor has shape `(num_queries, num_documents)` and `scores[i, j]` is the score between query `i` and document `j`. A larger value indicates a closer match.

## Training

NeoMME-800M-Retriever was fine-tuned from [NeoMME-800M](https://huggingface.co/Hcompany/NeoMME-800M) on text retrieval and document-page images. Training uses a joint late-interaction and Matryoshka dense contrastive objective.

The [NeoMME technical report](https://arxiv.org/abs/2609.01657) describes the full fine-tuning recipe.

## Limitations

With Sentence Transformers, only one of the two retrieval heads can be used at a time.

## License

Model weights are released under the Apache 2.0 license.

## Citation

```bibtex
@misc{lac2026neommesingletowermultimodalnativemultilingual,
      title={NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference},
      author={Aurélien Lac and Tony Wu},
      year={2026},
      eprint={2609.01657},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2609.01657},
}
```