Instructions to use lightonai/LightOn-rerank-PW-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lightonai/LightOn-rerank-PW-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="lightonai/LightOn-rerank-PW-2B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("lightonai/LightOn-rerank-PW-2B")
model = AutoModelForMultimodalLM.from_pretrained("lightonai/LightOn-rerank-PW-2B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

sentence-transformers

How to use lightonai/LightOn-rerank-PW-2B with sentence-transformers:

from sentence_transformers import CrossEncoder

model = CrossEncoder("lightonai/LightOn-rerank-PW-2B")

query = "Which planet is known as the Red Planet?"
passages = [
	"Venus is often called Earth's twin because of its similar size and proximity.",
	"Mars, known for its reddish appearance, is often referred to as the Red Planet.",
	"Jupiter, the largest planet in our solar system, has a prominent red spot.",
	"Saturn, famous for its rings, is sometimes mistaken for the Red Planet."
]

scores = model.predict([(query, passage) for passage in passages])
print(scores)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lightonai/LightOn-rerank-PW-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lightonai/LightOn-rerank-PW-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightonai/LightOn-rerank-PW-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lightonai/LightOn-rerank-PW-2B

SGLang

How to use lightonai/LightOn-rerank-PW-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lightonai/LightOn-rerank-PW-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightonai/LightOn-rerank-PW-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lightonai/LightOn-rerank-PW-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightonai/LightOn-rerank-PW-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use lightonai/LightOn-rerank-PW-2B with Docker Model Runner:
```
docker model run hf.co/lightonai/LightOn-rerank-PW-2B
```

LightOn-rerank-PW-2B / README.md

ameliechatelain

Add model card

58807b1 verified about 2 months ago

11.8 kB

license: apache-2.0
language:
  - en
  - fr
base_model:
  - Qwen/Qwen3.5-2B
pipeline_tag: image-text-to-text
library_name: transformers
tags:
  - reranker
  - cross-encoder
  - multimodal
  - text-ranking
  - document-reranking
  - vidore
  - beir

📝 Blog post: coming soon

LightOn-rerank-PW-2B

Unified Text + Visual Document Reranker by LightOn

PW-0.8B | LW-0.8B | PW-2B | LW-2B | PW-4B | LW-4B

About the LightOn-rerank family

Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). LightOn-rerank models are unified cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models).

The models are built on Qwen3.5 vision-language backbones (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone.

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise) — each candidate is scored independently: the model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate, no generation — simple to serve (vLLM-compatible) and embarrassingly parallel.
LW (listwise) — generative listwise ranking: 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

LightOn-rerank-PW-2B is the pointwise flagship of the family: a single 2B model that is simultaneously competitive with dedicated text rerankers on BEIR and with vision-specialised rerankers on ViDoRe V2/V3 — with none of the usual trade-off between the two modalities, and the serving simplicity of independent per-candidate scoring.

Results

ViDoRe V3 (visual document reranking, 8 domains × EN/FR queries), NDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models — including baselines — were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results.

Model	Params	Scoring	ViDoRe V3 overall @10
LightOn-rerank-LW-4B	4.5B	listwise	0.6469
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	0.6423
LightOn-rerank-LW-2B	2.2B	listwise	0.6266
LightOn-rerank-PW-2B (this model)	2.2B	pointwise	0.5987
LightOn-rerank-PW-4B	4.5B	pointwise	0.5980
jina-reranker-m0	2.4B	pointwise	0.5939
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	0.5918
LightOn-rerank-LW-0.8B	0.85B	listwise	0.5825
LightOn-rerank-PW-0.8B	0.85B	pointwise	0.4820

ViDoRe V3 detail (NDCG, ColQwen2.5 first stage, rerank-100)

Domain	EN @5	EN @10	FR @5	FR @10
finance_en	0.5893	0.6113	0.5585	0.5716
finance_fr	0.4643	0.4934	0.4947	0.5260
computer_science	0.7239	0.7398	0.6814	0.7047
hr	0.6570	0.6665	0.5996	0.6139
energy	0.6634	0.6921	0.6735	0.7013
industrial	0.5420	0.5520	0.4808	0.4864
pharmaceuticals	0.6298	0.6385	0.5958	0.6039
physics	0.4551	0.4851	0.4568	0.4927
mean	0.5906	0.6098	0.5676	0.5876

Overall @10: 0.5987 (EN 0.6098 / FR 0.5876).

Other benchmarks (same fixed-first-stage protocol)

Benchmark	LightOn-rerank-PW-2B	Best 2B-class baseline
ViDoRe V2, mean NDCG@10 (ColQwen2.5 first stage)	0.8558	jina-reranker-m0: 0.8596; MonoQwen2-VL-v0.1: 0.8454; Qwen3-VL-Reranker-2B: 0.8428
BEIR 15 datasets, clean mean NDCG@10 (jina-embeddings-v3 first stage)	0.5227	jina-reranker-m0: 0.5677; Qwen3-VL-Reranker-2B: 0.5025
BEIR 15 datasets, clean mean NDCG@10 (BM25 first stage)	0.4968	jina-reranker-m0: 0.5397; Qwen3-VL-Reranker-2B: 0.4815

Clean mean excludes the two contaminated datasets (NQ, MSMARCO). On text it sits between the two strongest 2B baselines; on visual reranking it matches or beats every 2B-class model — jina-reranker-m0 is the only baseline that is close on both, and it trails on ViDoRe V3 (0.5939 vs 0.5987).

Model Details

Model type: multimodal cross-encoder reranker — pointwise: each candidate is scored independently as logit("Yes") − logit("No")
Base model: Qwen/Qwen3.5-2B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈2.2B (bfloat16, 4.4 GB)
Inputs: query (text) + candidate document(s) — text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA — merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 1e-4, 1 epoch (465 steps)
Data: ~120k EN text queries (NQ, TriviaQA, MSMARCO) with mined hard negatives + ~118k visual query–page pairs with ColPali-family hard negatives (dedup_120k_en_v3 + ColPali_hard_neg_v2)
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture)
Internal experiment ID: exp30

Usage — pointwise reranking

Each candidate is scored independently as logit("Yes") − logit("No") at the first generated position; sort candidates by descending score. The model was trained with a fixed system prompt and user template (RERANKER_SYSTEM_V2 / RERANKER_TEXT_V2 / RERANKER_VISION_V2) — these will be published alongside the blog post and must be used verbatim for best results.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-PW-2B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)
processor.tokenizer.padding_side = "left"  # scores are read at the last position

YES_TOKEN_ID = 9175  # "Yes"
NO_TOKEN_ID = 2665   # "No"

SYSTEM_PROMPT = ...   # RERANKER_SYSTEM_V2 — released with the blog post
USER_TEMPLATE = ...   # RERANKER_TEXT_V2, with {query} and {doc} fields

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

texts = [
    processor.apply_chat_template(
        [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_TEMPLATE.format(query=query, doc=doc)},
        ],
        tokenize=False,
        add_generation_prompt=True,
    )
    for doc in documents
]
inputs = processor(
    text=texts, return_tensors="pt", padding=True, truncation=True, max_length=2048
).to(model.device)
with torch.inference_mode():
    logits = model(**inputs).logits[:, -1]
scores = (logits[:, YES_TOKEN_ID] - logits[:, NO_TOKEN_ID]).tolist()

ranked = sorted(zip(scores, documents), reverse=True)

To score a page image instead of a text passage, replace the user message with:

{"role": "user", "content": [
    {"type": "image", "image": page_image},  # PIL.Image
    {"type": "text", "text": VISION_TEMPLATE.format(query=query)},  # RERANKER_VISION_V2
]}

and pass images=[page_image, ...] to the processor call.

Serving with vLLM

vllm serve lightonai/LightOn-rerank-PW-2B --trust-remote-code --max-model-len 16384

resp = client.chat.completions.create(
    model="lightonai/LightOn-rerank-PW-2B",
    messages=messages,          # same system + user messages as above
    max_tokens=1,
    logprobs=True,
    top_logprobs=20,
    temperature=0.0,
)
top = resp.choices[0].logprobs.content[0].top_logprobs
lp = {t.token: t.logprob for t in top}
score = lp.get("Yes", -100.0) - lp.get("No", -100.0)

Full-page document images can exceed 8k tokens — keep --max-model-len at 16384 or higher when reranking page images.

Notes & limitations

Pointwise scoring does not improve with backbone scale (the 4B pointwise model ties this one) — if you have the compute budget, LightOn-rerank-LW-2B is +0.03 NDCG@10 on ViDoRe V3 at the same size.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
Vision hard negatives were mined with ColPali-family retrievers; the model pairs best with a ColPali-family first stage (e.g. ColQwen2.5) for visual reranking.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use clean means that exclude them.

The LightOn-rerank family

Model	Backbone	Scoring	ViDoRe V3 @10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	0.4820
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	0.5825
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	0.5987
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	0.6266
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	0.5980
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	0.6469

Rule of thumb: LW models are stronger at every size (and the gap grows with size); PW models are cheaper to serve and score candidates independently. For the best quality pick LW-4B; for the best quality/cost trade-off pick LW-2B; for maximum throughput on text-heavy workloads pick a PW model.