Publish harrier-oss-v1-0.6b-b4e5762f15-7f200bf07a-2607200600.fxb

README.md

@@ -1,238 +1,241 @@
 ---
-tags:
-- mteb
-- sentence-transformers
-- transformers
+base_model: microsoft/harrier-oss-v1-0.6b
 language:
-- multilingual
-- af
-- am
-- ar
-- as
-- az
-- be
-- bg
-- bn
-- br
-- bs
-- ca
-- cs
-- cy
-- da
-- de
-- el
-- en
-- eo
-- es
-- et
-- eu
-- fa
-- fi
-- fr
-- fy
-- ga
-- gd
-- gl
-- gu
-- ha
-- he
-- hi
-- hr
-- hu
-- hy
-- id
-- is
-- it
-- ja
-- jv
-- ka
-- kk
-- km
-- kn
-- ko
-- ku
-- ky
-- la
-- lo
-- lt
-- lv
-- mg
-- mk
-- ml
-- mn
-- mr
-- ms
-- my
-- ne
-- nl
-- 'no'
-- om
-- or
-- pa
-- pl
-- ps
-- pt
-- ro
-- ru
-- sa
-- sd
-- si
-- sk
-- sl
-- so
-- sq
-- sr
-- su
-- sv
-- sw
-- ta
-- te
-- th
-- tl
-- tr
-- ug
-- uk
-- ur
-- uz
-- vi
-- xh
-- yi
-- zh
+  - multilingual
+  - af
+  - am
+  - ar
+  - as
+  - az
+  - be
+  - bg
+  - bn
+  - br
+  - bs
+  - ca
+  - cs
+  - cy
+  - da
+  - de
+  - el
+  - en
+  - eo
+  - es
+  - et
+  - eu
+  - fa
+  - fi
+  - fr
+  - fy
+  - ga
+  - gd
+  - gl
+  - gu
+  - ha
+  - he
+  - hi
+  - hr
+  - hu
+  - hy
+  - id
+  - is
+  - it
+  - ja
+  - jv
+  - ka
+  - kk
+  - km
+  - kn
+  - ko
+  - ku
+  - ky
+  - la
+  - lo
+  - lt
+  - lv
+  - mg
+  - mk
+  - ml
+  - mn
+  - mr
+  - ms
+  - my
+  - ne
+  - nl
+  - 'no'
+  - om
+  - or
+  - pa
+  - pl
+  - ps
+  - pt
+  - ro
+  - ru
+  - sa
+  - sd
+  - si
+  - sk
+  - sl
+  - so
+  - sq
+  - sr
+  - su
+  - sv
+  - sw
+  - ta
+  - te
+  - th
+  - tl
+  - tr
+  - ug
+  - uk
+  - ur
+  - uz
+  - vi
+  - xh
+  - yi
+  - zh
 license: mit
+pipeline_tag: feature-extraction
+library_name: furiosa-llm
+tags:
+  - furiosa-ai
+  - harrier-oss-v1
+  - qwen3
+  - mteb
+  - sentence-transformers
+  - transformers
 ---
+# harrier-oss-v1-0.6b
 
-## harrier-oss-v1
+This repository contains [`microsoft/harrier-oss-v1-0.6b`](https://huggingface.co/microsoft/harrier-oss-v1-0.6b)
+together with a Furiosa Executable Bundle (FXB) for running it on
+[FuriosaAI RNGD](https://furiosa.ai) with [Furiosa-LLM](https://developer.furiosa.ai/latest/en/furiosa_llm/intro.html).
+The same model also runs on other frameworks (such as Sentence Transformers and
+Transformers); for usage with those, see the upstream
+[`microsoft/harrier-oss-v1-0.6b`](https://huggingface.co/microsoft/harrier-oss-v1-0.6b) model card.
 
-harrier-oss-v1 is a family of multilingual text embedding models developed by Microsoft.
-The models use decoder-only architectures with last-token pooling and L2 normalization to produce dense text embeddings.
-They can be applied to a wide range of tasks, including but not limited to **retrieval**, **clustering**, **semantic similarity**, **classification**, **bitext mining**, and **reranking**.
-The models achieve state-of-the-art results on the [Multilingual MTEB v2](https://huggingface.co/spaces/mteb/leaderboard) benchmark as of the release date.
+## Overview
 
-| Model                                                                       | Parameters | Embedding Dimension | Max Tokens | MTEB v2 Score |
-|-----------------------------------------------------------------------------|------------|---------------------|------------|---------------|
-| [harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m) | 270M       | 640                 | 32,768     | 66.5          |
-| [harrier-oss-v1-0.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b) | 0.6B       | 1,024               | 32,768     | 69.0          |
-| [harrier-oss-v1-27b](https://huggingface.co/microsoft/harrier-oss-v1-27b)   | 27B        | 5,376               | 32,768     | **74.3**      |
+Harrier OSS v1 is a family of multilingual text-embedding models developed by
+Microsoft. The 0.6B model uses a dense, decoder-only Qwen3 architecture, but it
+is trained with Harrier's own multilingual, instruction-aware embedding recipe
+rather than the Qwen3-Embedding training recipe. It produces 1,024-dimensional
+embeddings through last-token pooling and L2 normalization. It is designed for
+retrieval, clustering, semantic similarity, classification, bitext mining, and
+reranking. Its intended use is the same as the upstream
+[`microsoft/harrier-oss-v1-0.6b`](https://huggingface.co/microsoft/harrier-oss-v1-0.6b),
+and it is released under the [MIT License](https://opensource.org/license/mit).
 
-## Training
+- **Architecture:** Qwen3 (dense), `Qwen3Model`
+- **Input / Output:** Text / Embeddings (vector)
+- **Supported Inference Engine:** Furiosa LLM
+- **Supported Hardware:** FuriosaAI RNGD
 
-All models are trained with contrastive learning objectives on a large-scale mixture of multilingual datasets covering diverse tasks.
-The 270m and 0.6b variants are additionally trained with knowledge distillation from larger embedding models.
+### Quantization
 
-## Usage
+No quantization — the model runs in its native BF16 precision.
 
-Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset.
+### Parallelism Strategy
 
-### Sentence Transformers
+On RNGD, harrier-oss-v1-0.6b runs with a **tensor-parallel size of 8 PEs**, which
+maps to a **single RNGD card** (8 PEs per card).
 
-```python
-from sentence_transformers import SentenceTransformer
+## Usage
 
-model = SentenceTransformer("microsoft/harrier-oss-v1-0.6b", model_kwargs={"dtype": "auto"})
+To run this model with Furiosa-LLM, follow the examples below after
+[installing Furiosa-LLM and its prerequisites](https://developer.furiosa.ai/latest/en/get_started/furiosa_llm.html#installing-furiosa-llm).
+You can use the model either online through the OpenAI-compatible server or
+offline through the Furiosa-LLM Python API.
 
-queries = [
-    "how much protein should a female eat",
-    "summit define",
-]
-documents = [
-    "As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
-    "Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."
-]
+### Launch the server
 
-query_embeddings = model.encode(queries, prompt_name="web_search_query")
-document_embeddings = model.encode(documents)
+Serve the model by passing its `furiosa-ai/<repo>` identifier:
 
-scores = (query_embeddings @ document_embeddings.T) * 100
-print(scores.tolist())
+```sh
+# Launch the server, listening on port 8000 by default
+furiosa-llm serve furiosa-ai/harrier-oss-v1-0.6b
 ```
 
-Have a look at [config_sentence_transformers.json](config_sentence_transformers.json) for the prompts that are pre-configured, such as `web_search_query`, `sts_query`, and `bitext_query`. You can also use a custom instruction directly via e.g. `model.encode(queries, prompt="Instruct: Retrieve semantically similar text\nQuery: ")`.
+When the server is ready, you will see:
+
+```sh
+INFO:     Started server process [27507]
+INFO:     Waiting for application startup.
+INFO:     Application startup complete.
+INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
+```
 
+### Basic Usage
+
+The server exposes an OpenAI-compatible `/v1/embeddings` endpoint. Harrier is
+instruction-aware: prepend a one-sentence task description to each query in the
+`Instruct: ...\nQuery: ...` format, and do not add the instruction to documents.
+For more details, see the
+[base model card](https://huggingface.co/microsoft/harrier-oss-v1-0.6b).
+Request embeddings with `curl`:
+
+```sh
+curl http://localhost:8000/v1/embeddings \
+    -H "Content-Type: application/json" \
+    -d '{
+    "model": "furiosa-ai/harrier-oss-v1-0.6b",
+    "input": [
+      "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: summit define",
+      "Definition of summit: the highest point of a mountain."
+    ]
+    }' \
+    | python -m json.tool
+```
 
-### Transformers
+Because the endpoint is OpenAI-compatible, you can also use the OpenAI Python
+client:
 
 ```python
-import torch
-import torch.nn.functional as F
-
-from torch import Tensor
-from transformers import AutoTokenizer, AutoModel
-
-
-def last_token_pool(last_hidden_states: Tensor, attention_mask: Tensor) -> Tensor:
-    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
-    if left_padding:
-        return last_hidden_states[:, -1]
-    else:
-        sequence_lengths = attention_mask.sum(dim=1) - 1
-        batch_size = last_hidden_states.shape[0]
-        return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), sequence_lengths]
-
-
-def get_detailed_instruct(task_description: str, query: str) -> str:
-    return f'Instruct: {task_description}\nQuery: {query}'
-
-
-# Each query must come with a one-sentence instruction that describes the task
-task = 'Given a web search query, retrieve relevant passages that answer the query'
-queries = [
-    get_detailed_instruct(task, 'how much protein should a female eat'),
-    get_detailed_instruct(task, 'summit define')
-]
-# No need to add instruction for retrieval documents
-documents = [
-    "As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
-    "Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."
-]
-input_texts = queries + documents
-
-tokenizer = AutoTokenizer.from_pretrained('microsoft/harrier-oss-v1-0.6b')
-model = AutoModel.from_pretrained('microsoft/harrier-oss-v1-0.6b', dtype='auto')
-model.eval()
-model.cuda()
-
-max_length = 32768
-# Tokenize the input texts
-batch_dict = tokenizer(input_texts, max_length=max_length, padding=True, truncation=True, return_tensors='pt')
-batch_dict = {k: v.cuda() for k, v in batch_dict.items()}
-
-outputs = model(**batch_dict)
-embeddings = last_token_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
-
-# normalize embeddings
-embeddings = F.normalize(embeddings, p=2, dim=1)
-scores = (embeddings[:2] @ embeddings[2:].T) * 100
-print(scores.tolist())
-```
+from openai import OpenAI
 
-## Supported Languages
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
 
-The models are trained on multilingual data and support a wide range of languages,
-including but not limited to: Arabic, Bulgarian, Catalan, Czech, Danish, German, Greek, English, Spanish,
-Estonian, Persian, Finnish, French, Hebrew, Hindi, Croatian, Hungarian, Indonesian, Italian, Japanese,
-Korean, Lithuanian, Latvian, Macedonian, Malay, Dutch, Norwegian, Polish, Portuguese, Romanian, Russian,
-Slovak, Slovenian, Albanian, Serbian, Swedish, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Chinese.
+query = (
+    "Instruct: Given a web search query, retrieve relevant passages that answer the query\n"
+    "Query: summit define"
+)
+document = "Definition of summit: the highest point of a mountain."
 
-## Evaluation
+response = client.embeddings.create(
+    model="furiosa-ai/harrier-oss-v1-0.6b",
+    input=[query, document],
+)
 
-Please follow the [mteb](https://github.com/embeddings-benchmark/mteb) repository on how to reproduce our scores.
-The evaluation prompts used for each task are also available at [mteb_v2_eval_prompts.json](mteb_v2_eval_prompts.json).
+for data in response.data:
+    print(f"Index {data.index}: {len(data.embedding)} dimensions")
+```
 
-## FAQ
+### Advanced Usage
 
-**1. Do I need to add instructions to the query?**
+For offline use, load the model with the `LLM` constructor (the FXB shipped in
+the repo is discovered automatically) and call `embed` to obtain L2-normalized
+dense vectors. Their dot product is therefore the cosine similarity:
 
-Yes, this is how the model is trained, otherwise you will see a performance degradation.
-The task definition should be a one-sentence instruction that describes the task.
-This is a way to customize text embeddings for different scenarios through natural language instructions.
+```python
+from furiosa_llm import LLM
 
-On the other hand, there is no need to add instructions to the document side.
+query = (
+    "Instruct: Given a web search query, retrieve relevant passages that answer the query\n"
+    "Query: summit define"
+)
+document = "Definition of summit: the highest point of a mountain."
 
-**2. Why are my reproduced results slightly different from reported in the model card?**
+with LLM("furiosa-ai/harrier-oss-v1-0.6b") as llm:
+    outputs = llm.embed([query, document])
+    embeddings = [output.outputs.embedding for output in outputs]
 
-Different versions of `transformers` and `pytorch` could cause negligible but non-zero performance differences.
+similarity = sum(a * b for a, b in zip(*embeddings, strict=True))
+print(f"Cosine similarity: {similarity:.4f}")
+```
 
-**3. What pooling strategy does this model use?**
+## Learn more
 
-The model uses **last-token pooling** — the embedding of the last non-padding token is used as the sentence representation.
-The embedding is then L2-normalized. This is handled automatically when using Sentence Transformers.
+* [Furiosa-LLM Server (`furiosa-llm serve`)](https://developer.furiosa.ai/latest/en/furiosa_llm/furiosa-llm-serve.html) — full OpenAI-compatible API reference, including the Embeddings API
+* [Furiosa-LLM](https://developer.furiosa.ai/latest/en/furiosa_llm/intro.html) — Furiosa-LLM documentation and API reference
+* [`microsoft/harrier-oss-v1-0.6b`](https://huggingface.co/microsoft/harrier-oss-v1-0.6b) — upstream model card

harrier-oss-v1-0.6b-b4e5762f15-7f200bf07a-2607200600.fxb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c90ec2ad6ac655c376e07c0e82b7bd6a41d33dba1fad49d434522350465ea16
+size 19796989