Nemotron-3-Embed-1B-oQ4-MLX

Overview

Nemotron-3-Embed-1B-oQ4-MLX is an MLX-formatted checkpoint optimized for Apple silicon, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

Field	Details
Format	MLX
Source / base	`nvidia/Nemotron-3-Embed-1B-BF16`
Intended task	`feature-extraction`
License	`other`

What is included

*.safetensors (1 file)
config.json
tokenizer.json
tokenizer_config.json
Additional configuration, tokenizer, processor, or shard files (14 visible artifacts total)

Quick start

MLX

Use an up-to-date MLX-compatible runtime on Apple silicon and point it at this repository:

mlx_lm.generate --model groxaxo/Nemotron-3-Embed-1B-oQ4-MLX --prompt "Write a concise technical summary."

Embedding and audio repositories may require the task-specific MLX package documented by the upstream project.

Compatibility and responsible use

Use a runtime that explicitly supports this format, architecture, and modality.
Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
Review the source model card and license before redistribution or deployment.
Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

An Apple-Silicon-native OMLX oQ4 conversion of nvidia/Nemotron-3-Embed-1B-BF16.

This derivative was produced locally on Apple Silicon with the same streaming, sensitivity-driven OMLX pipeline used for the oQ5 release. It uses a base 4-bit affine plan, group size 64, and bfloat16 working dtype. The calibration ran on the checkpoint's bidirectional encoder architecture rather than through a causal-LM wrapper. No weights were retrained.

The quantized safetensors payload is 670,214,436 bytes. OMLX promoted 26 sensitivity-selected projections to 5, 6-bit precision.

Embedding contract

bidirectional attention (is_causal: false)
average/mean pooling
L2-normalized 2,048-dimensional output
query: prefix for queries
passage: prefix for documents

BF16 fidelity smoke test

The source and this oQ4 checkpoint were evaluated using a query, a matching passage, and an unrelated passage. All outputs were 2,048-dimensional and L2-normalized.

Input	BF16 to oQ4 vector cosine
Query	0.992316
Related passage	0.993071
Unrelated passage	0.991795

Comparison	BF16	oQ4
Query to related passage	0.680575	0.693227
Query to unrelated passage	-0.024595	-0.013228

Quick start on Apple Silicon

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade mlx mlx-lm transformers huggingface_hub

hf download groxaxo/Nemotron-3-Embed-1B-oQ4-MLX --local-dir ./Nemotron-3-Embed-1B-oQ4-MLX
python ./Nemotron-3-Embed-1B-oQ4-MLX/embed_mlx.py \
  --model ./Nemotron-3-Embed-1B-oQ4-MLX \
  --query "Which city is known as the City of Sails?" \
  --passage "Auckland is widely known as the City of Sails."

The included script deliberately loads the bare Ministral3 backbone and applies non-causal attention. The regular MLX-LM generation path is causal and is not the correct embedding interface for this checkpoint.

Quantization details

Setting	Value
OMLX source revision	`a534507fb5ab760d239283868a4c790386b316ff`
Base model revision	`a5e0f804b9e90a1ca6784ecbf6e41595774fc834`
OMLX level	oQ4
Base quantization	4-bit affine
Group size	64
Working dtype	bfloat16
Calibration	128 samples × 256 tokens
Sensitivity-selected boosts	26

See QUANTIZATION_REPORT.md for artifact SHA-256 values and the full local validation result.

License

This derivative inherits the OpenMDW-1.1 license from the source model. LICENSE, NOTICE, and THIRD_PARTY_NOTICES.md are included unchanged from NVIDIA's source repository.