---
base_model: saricles/Qwen3-Coder-Next-NVFP4-GB10
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
language:
  - en
tags:
  - qwen3
  - qwen3_next
  - nvfp4
  - 4-bit
  - quantized
  - compressed-tensors
  - vllm
  - DGX-Spark
  - GB10
  - MoE
  - coding
  - mirror
---

# gdubicki/Qwen3-Coder-Next-NVFP4-GB10

**Public mirror of [`saricles/Qwen3-Coder-Next-NVFP4-GB10`](https://huggingface.co/saricles/Qwen3-Coder-Next-NVFP4-GB10).**

Weights are byte-identical to the upstream quant (`config.json` and `model.safetensors.index.json` SHA-256 verified). This mirror exists to provide a pinned, stable, ungated reference for the [`qwen3-coder-next`](https://github.com/grzegorzdubicki-ai/qwen3-coder-next) deployment project on DGX Spark (GB10). Use the upstream repo if you want to track author updates.

## Credits

- Base model: [`Qwen/Qwen3-Coder-Next`](https://huggingface.co/Qwen/Qwen3-Coder-Next) by Alibaba / Qwen team (Apache-2.0)
- NVFP4 quantization: [`saricles`](https://huggingface.co/saricles) using [LLM Compressor](https://github.com/vllm-project/llm-compressor) with `LLMCOMPRESSOR_MOE_CALIBRATE_ALL_EXPERTS=1` (all 512 experts calibrated)
- Calibration data: [`HuggingFaceH4/ultrachat_200k`](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) (64 samples × 2048 tok)
- License: **Apache-2.0** (inherited from base model; redistribution permitted)

## Model details

- Architecture: `qwen3_next` — Hybrid DeltaNet linear attention + full attention + latent MoE
- Layers: 48 total (36 DeltaNet linear-attention, 12 full-attention)
- Parameters: 79.7B total, **~3B active per token** (512 experts, 10 active + 1 shared)
- Quantization: NVFP4 via `compressed-tensors`; `lm_head`, `embed_tokens`, `linear_attn` layers, `mlp.gate`, `mlp.shared_expert_gate` kept in BF16
- Size on disk: 45.9 GB (70% reduction from ~149 GB BF16)
- KV cache: FP8 (12 full-attention layers only; DeltaNet has no KV cache)
- Max context: 262,144 tokens

## Verified performance

Measured on DGX Spark (GB10 Blackwell, SM12.1, 128 GB unified LPDDR5X) with vLLM (Marlin NVFP4, FP8 KV cache):

| Metric | Value |
|---|---:|
| Throughput (single request) | ~61 tok/s |
| KV cache concurrency @ 262K | ~5 concurrent sessions (DeltaNet has no KV cache → 31.65× vs a pure-attention 80B model) |

Same decode throughput as Nemotron-3-Nano-30B-A3B-NVFP4 (both 3B active), but with a native 262K context window.

## Usage

```bash
docker run --rm --runtime=nvidia --gpus all \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_NVFP4_GEMM_BACKEND=marlin \
  -e VLLM_TEST_FORCE_FP8_MARLIN=1 \
  -e VLLM_USE_FLASHINFER_MOE_FP4=0 \
  -e VLLM_MARLIN_USE_ATOMIC_ADD=1 \
  vllm/vllm-openai:cu130-nightly \
  gdubicki/Qwen3-Coder-Next-NVFP4-GB10 \
  --dtype auto \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.90 \
  --max-model-len 262144 \
  --attention-backend flashinfer \
  --tool-call-parser qwen3_coder \
  --enable-chunked-prefill \
  --enable-prefix-caching
```

Full deployment scripts: https://github.com/grzegorzdubicki-ai/qwen3-coder-next