Sentence Similarity
MLX
Safetensors
ministral3
apple-silicon
embeddings
text-embeddings
feature-extraction
retrieval
semantic-search
rag
mteb
nemotron
quantized
4-bit precision
information-retrieval
dense-retrieval
vector-search
matryoshka
arxiv:2502.13595
Eval Results (legacy)
Instructions to use shadowrock-io/Nemotron-3-Embed-8B-Community-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use shadowrock-io/Nemotron-3-Embed-8B-Community-MLX-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download shadowrock-io/Nemotron-3-Embed-8B-Community-MLX-4bit --local-dir Nemotron-3-Embed-8B-Community-MLX-4bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
card: model-index benchmark metadata, retrieval tags, arxiv link, attribution + citation
Browse files
README.md
CHANGED
|
@@ -19,10 +19,34 @@ tags:
|
|
| 19 |
- ministral3
|
| 20 |
- quantized
|
| 21 |
- 4-bit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
language:
|
| 23 |
- en
|
| 24 |
- multilingual
|
| 25 |
library_name: mlx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
---
|
| 27 |
|
| 28 |
# Nemotron-3-Embed-8B — Community MLX 4-bit
|
|
@@ -105,6 +129,19 @@ Throughput saturates near 1,300–1,400 tokens/s at document lengths (memory-ban
|
|
| 105 |
- `mlx-embeddings`/`mlx-lm` do not load this architecture correctly (causal attention). Use the bundled module.
|
| 106 |
- Quantization was validated on macOS 27 / mlx 0.29; older mlx releases may not support the quantized layout.
|
| 107 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 108 |
## License
|
| 109 |
|
| 110 |
OpenMDW-1.1, inherited from the base model (see `LICENSE`). `NOTICE` carries the upstream Apache-2.0 attribution for the Ministral component plus our modification statement. Community build by [ShadowRock](https://www.linkedin.com/company/shadowrock); no NVIDIA affiliation or endorsement.
|
|
|
|
| 19 |
- ministral3
|
| 20 |
- quantized
|
| 21 |
- 4-bit
|
| 22 |
+
- information-retrieval
|
| 23 |
+
- dense-retrieval
|
| 24 |
+
- vector-search
|
| 25 |
+
- matryoshka
|
| 26 |
+
- arxiv:2502.13595
|
| 27 |
language:
|
| 28 |
- en
|
| 29 |
- multilingual
|
| 30 |
library_name: mlx
|
| 31 |
+
model-index:
|
| 32 |
+
- name: Nemotron-3-Embed-8B-Community-MLX-4bit
|
| 33 |
+
results:
|
| 34 |
+
- task:
|
| 35 |
+
type: Retrieval
|
| 36 |
+
dataset:
|
| 37 |
+
name: MTEB NFCorpus
|
| 38 |
+
type: mteb/nfcorpus
|
| 39 |
+
metrics:
|
| 40 |
+
- type: ndcg_at_10
|
| 41 |
+
value: 0.4199
|
| 42 |
+
- task:
|
| 43 |
+
type: Retrieval
|
| 44 |
+
dataset:
|
| 45 |
+
name: MTEB SciFact
|
| 46 |
+
type: mteb/scifact
|
| 47 |
+
metrics:
|
| 48 |
+
- type: ndcg_at_10
|
| 49 |
+
value: 0.8338
|
| 50 |
---
|
| 51 |
|
| 52 |
# Nemotron-3-Embed-8B — Community MLX 4-bit
|
|
|
|
| 129 |
- `mlx-embeddings`/`mlx-lm` do not load this architecture correctly (causal attention). Use the bundled module.
|
| 130 |
- Quantization was validated on macOS 27 / mlx 0.29; older mlx releases may not support the quantized layout.
|
| 131 |
|
| 132 |
+
## Attribution & citation
|
| 133 |
+
|
| 134 |
+
Quantization, validation harness, and card by [Matt Busi](https://huggingface.co/mattbusi) at [ShadowRock](https://www.linkedin.com/company/shadowrock). If you use this build, cite the NVIDIA base model — the embedding quality is theirs:
|
| 135 |
+
|
| 136 |
+
```bibtex
|
| 137 |
+
@misc{nvidia2026nemotron3embed,
|
| 138 |
+
title = {Nemotron-3-Embed-8B},
|
| 139 |
+
author = {NVIDIA},
|
| 140 |
+
year = {2026},
|
| 141 |
+
url = {https://huggingface.co/nvidia/Nemotron-3-Embed-8B-BF16}
|
| 142 |
+
}
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
## License
|
| 146 |
|
| 147 |
OpenMDW-1.1, inherited from the base model (see `LICENSE`). `NOTICE` carries the upstream Apache-2.0 attribution for the Ministral component plus our modification statement. Community build by [ShadowRock](https://www.linkedin.com/company/shadowrock); no NVIDIA affiliation or endorsement.
|