---
license: other
license_name: openmdw-1.1
license_link: https://openmdw.ai/license/1-1/
base_model:
- nvidia/Nemotron-3-Embed-8B-BF16
tags:
- text-embeddings
- retrieval
- rag
- nvfp4
- modelopt
- vllm
library_name: sentence-transformers
pipeline_tag: sentence-similarity
---

# Experimental Nemotron-3-Embed-8B ModelOpt NVFP4 PTQ

> **Experimental community post-training-quantized export. Not an NVIDIA release, not an NVIDIA QAD checkpoint, and not validated as equivalent to `nvidia/Nemotron-3-Embed-8B-BF16`.**

This checkpoint is a local NVIDIA ModelOpt 0.45.0 NVFP4 post-training quantization (PTQ) export of [`nvidia/Nemotron-3-Embed-8B-BF16`](https://huggingface.co/nvidia/Nemotron-3-Embed-8B-BF16). It is published for reproducibility and compatibility testing on Blackwell-class GPUs.

## Provenance and limits

- Source: `nvidia/Nemotron-3-Embed-8B-BF16`
- Quantization: ModelOpt `NVFP4_DEFAULT_CFG`, group size 16
- Calibration: 256 FineWeb-Edu samples, 4,096 tokens, with the `passage: ` prefix
- Source BF16 checkpoint: about 15 GiB; this NVFP4 artifact: about 5 GiB
- This is **PTQ**, not NVIDIA quantization-aware distillation (QAD).

Held-out 24-pair English retrieval gate, comparing BF16 with the in-memory ModelOpt fake-quant graph:

| Length | Mean cosine | Minimum cosine | Top-1 agreement |
|---:|---:|---:|---:|
| 256 | 0.97864 | 0.97232 | 1.00 |
| 1,024 | 0.97775 | 0.97056 | 1.00 |
| 4,096 | 0.97770 | 0.97171 | 1.00 |

These are narrow fidelity checks, **not** MTEB, RTEB, multilingual, long-context, or production-quality results. Evaluate on your own retrieval data before use.

Detailed machine-readable provenance and validation are included in `quantization_provenance.json` and `validation.json`.

## Input and output

The model outputs 4,096-dimensional L2-normalized average-pooled embeddings when served with the configured pooling behavior.

Use asymmetric retrieval prefixes with `/v1/embeddings`:

- Queries: `query: your question`
- Indexed passages: `passage: your document text`

Do not mix vectors made by this checkpoint with vectors from the BF16 source or another embedding model in one collection.

## vLLM serving

Tested locally with vLLM 0.25.0 on an RTX PRO 6000 Blackwell MIG slice:

```bash
vllm serve . \
  --runner pooling \
  --quantization modelopt_fp4 \
  --dtype bfloat16 \
  --pooler-config '{"pooling_type":"MEAN"}' \
  --max-model-len 4096 \
  --max-num-batched-tokens 16384
```

The 4,096-token serving limit above is a tested operational profile, not the source model's maximum context capability.

## License and attribution

The source model and its configuration are subject to the [OpenMDW 1.1 license](https://openmdw.ai/license/1-1/). Users must review and comply with the source model license and all applicable terms. NVIDIA does not endorse or support this community PTQ artifact.