Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,58 @@
+---
+license: apache-2.0
+base_model: ibm-granite/granite-embedding-311m-multilingual-r2
+tags:
+  - gguf
+  - llama.cpp
+  - embeddings
+  - granite
+  - modernbert
+  - edge
+---
+
+# granite-embedding-311m-multilingual-r2-GGUF
+
+F16 GGUF conversion of [`ibm-granite/granite-embedding-311m-multilingual-r2`](https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2) for local serving with [llama.cpp](https://github.com/ggml-org/llama.cpp). Built and independently verified by ATF (Agent Taskflow) for edge-local embedding serving via `atf-serve`.
+
+All model weights are © IBM, licensed Apache-2.0 (same as the base model).
+
+## Why this exists
+
+IBM does not publish a GGUF for this model. This repo documents its own build end-to-end — source checksum, conversion command, and independent correctness verification — rather than asking you to trust an unverified re-hosted binary.
+
+## Conversion details
+
+- **Source**: `ibm-granite/granite-embedding-311m-multilingual-r2`, `model.safetensors` (bf16, 623,341,952 bytes)
+- **Tool**: `llama.cpp` built from source at commit `11924d4c17abc27383376a1ac6a24fa3e36c1c0c` (2026-08-02). This model's tokenizer (`granite-embed-multi-311m`, maps to `LLAMA_VOCAB_PRE_TYPE_GEMMA4`) is **not** recognized by llama.cpp release `b9204` or earlier — the registration landed upstream after that tag. A current build (or any release ≥ the commit that added it) is required both to *convert* and to *serve* this model; older binaries fail with `unknown pre-tokenizer type: 'granite-embed-multi-311m'` at load time, not at conversion time.
+- **Command**:
+  ```
+  python3 convert_hf_to_gguf.py <model-dir> \
+    --outfile granite-embedding-311m-multilingual-r2-f16.gguf \
+    --outtype f16
+  ```
+- **Output**: F16, 768-dim, 638,121,344 bytes.
+
+## Verification (independent, not vendor-claimed)
+
+Embedded the same test sentence through both this GGUF (via `llama-server --embedding --pooling cls`) and the original HF model (via `sentence-transformers`, loaded directly from the source safetensors), then computed cosine similarity between the two output vectors.
+
+| Check | Result |
+|---|---|
+| Output dimension | 768 (matches source `hidden_size`) |
+| Cosine similarity vs. HF reference pipeline | **0.999970** |
+| Required pooling mode | `cls` (matches source `classifier_pooling: "cls"` / `pooling_mode_cls_token: true` in `config.json`; mean pooling is **not** correct for this model) |
+
+## Usage
+
+```
+llama-server --model granite-embedding-311m-multilingual-r2-f16.gguf \
+  --embedding --pooling cls --port 8089
+```
+
+Requires a llama.cpp build that includes `granite-embed-multi-311m` tokenizer support (see Conversion details above — current upstream `master` has it; check your pinned release tag if serving fails with an `unknown pre-tokenizer type` error).
+
+```
+curl http://127.0.0.1:8089/v1/embeddings \
+  -H "Content-Type: application/json" \
+  -d '{"input": "your text here", "model": "granite-embedding-311m"}'
+```