README.md

---
language:
- multilingual
license: apache-2.0
base_model:
- ibm-granite/granite-embedding-311m-multilingual-r2
library_name: llama.cpp
pipeline_tag: sentence-similarity
tags:
- gguf
- llama.cpp
- embeddings
- multilingual
- retrieval
- rag
- sentence-transformers
- granite
- modernbert
---

# Granite Embedding 311M Multilingual R2 - GGUF

This repository provides a GGUF conversion of the IBM Granite embedding model:

**Base model:** `ibm-granite/granite-embedding-311m-multilingual-r2`

Original model card:
https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2

## Model Description

Granite Embedding 311M Multilingual R2 is a multilingual embedding model developed by IBM for semantic search, retrieval, RAG, clustering, and similarity tasks.

Key features:

- Multilingual support
- 311M parameters
- 768-dimensional embeddings
- Up to 32k context length
- Optimized for retrieval and semantic similarity
- Compatible with llama.cpp through GGUF conversion

## Files

| File | Precision | Recommended Use |
|------|-----------|-----------------|
| `granite-embedding-311m-multilingual-r2-F16.gguf` | F16 | Maximum quality and accuracy |

## Conversion Details

This GGUF file was generated using the official `llama.cpp` conversion tools.

### Conversion command

```bash
python convert_hf_to_gguf.py \
    granite-embedding-311m-multilingual-r2 \
    --outfile granite-embedding-311m-multilingual-r2-F16.gguf \
    --outtype f16
```

### llama.cpp version

```text
Commit: 96fbe0039337a999613a983d66e2bfcc4bb554d7
```

## Usage with llama.cpp

### Embedding generation

```bash
llama-embedding \
    -m granite-embedding-311m-multilingual-r2-F16.gguf \
    -p "Artificial intelligence is transforming software engineering."
```

### OpenAI-compatible server

```bash
llama-server \
    -m granite-embedding-311m-multilingual-r2-F16.gguf \
    --embedding
```

Example request:

```bash
curl http://localhost:8080/v1/embeddings \
    -H "Content-Type: application/json" \
    -d '{
        "input": "Hello world"
    }'
```

## Intended Uses

This model is suitable for:

- Retrieval-Augmented Generation (RAG)
- Semantic search
- Document retrieval
- Similarity search
- Clustering
- Deduplication
- Cross-lingual retrieval
- Recommendation systems

## Notes

This repository only provides a GGUF conversion of the original IBM model. All credit for the model architecture, training, and evaluation belongs to IBM Research.

Please refer to the original model card for:

- Training details
- Evaluation results
- Benchmark scores
- Limitations
- Responsible AI considerations

Original repository:

https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2

## License

This GGUF conversion is distributed under the same license as the original model:

**Apache License 2.0**

Please verify license compatibility with your intended use case.

## Acknowledgements

- IBM Research for developing the Granite Embedding model.
- The llama.cpp project for GGUF support and inference.
- The Hugging Face community for model hosting and distribution.