Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-W4A16

Almost every public NVFP4 checkpoint is W4A4. This one is weight-only: 4-bit weights, BF16 activations, no calibration corpus anywhere in its construction — and therefore nothing you would have to reproduce to rebuild it.

20.6 GB. Runs on one GB10 with images and video intact, and keeps the MTP head so speculative decoding works without a separate drafter.

Is this for you

You have an NVIDIA DGX Spark — GB10, SM121, one coherent ~121 GiB memory pool — and you want a 27B multimodal model that leaves room for real context. You feed it screenshots, photos, or video, not just text.

If you are on a discrete GPU, nothing here is wrong, but none of it was tuned for you. The choices below assume unified memory, where "GPU memory" and "host RAM" are the same DRAM and offloading reclaims nothing.

Why weight-only

W4A16 and W4A4 store the same 4-bit weights. The difference is what else the build depends on. W4A4 quantizes activations too, and activation scales have to come from a calibration corpus — the checkpoint inherits whatever that corpus contained, and anyone rebuilding it has to reproduce the corpus to reproduce the model. Weight-only derives every scale from the weights themselves: rebuilding this takes the source checkpoint, quantize_w4a16.py from this repo, and nothing else.

Throughput does not break the tie. Decode on GB10 is bandwidth-bound: both schemes move the same 4-bit bytes, and this build dequantizes FP4 to BF16 rather than touching the FP4 tensor cores at all. Measured here: W4A4 decoded 20.9 tok/s against this build's 19.8, then gave the margin back on video prefill. Neither scheme has a throughput advantage worth choosing on.

Does W4A4 degrade image and video, as folklore says? No. We built one and tested it. Same source, ModelOpt stock configs on both sides, text-calibrated, same engine, same 24 prompts, same BF16 reference. Zero empty replies, every ground-truth string matched, and the burned-in subtitle transcriptions came back byte-identical to this build. 13 of the 24 replies differed, so the activation path was live rather than silently falling back to the weight-only kernel. That is one eval on one checkpoint with one calibration set -- it does not make W4A4 safe everywhere, but the failure it is famous for did not happen.

So weight-only is the simpler construction, not a rescue from a broken alternative. The compression is the win; the 4-bit math never was.

Run it

The launcher ships in this repo — there is nothing else to clone.

hf download SeatownSin/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-W4A16 --local-dir ./qwen38-nvfp4
cd qwen38-nvfp4 && chmod +x serve_export.sh
DST=. ./serve_export.sh

That gives you an OpenAI-compatible endpoint on :30000, with MTP speculative decoding, 262K context, and --mem-fraction-static 0.75.

If you would rather not run someone else's shell script, the whole thing is one docker run — this is what the script does, minus the readiness polling:

docker run -d --name qwen38-w4a16 --network host --ipc host --gpus all --shm-size 32g \
  -v "$PWD:/model:ro" lmsysorg/sglang:qwen38-27b \
  python3 -m sglang.launch_server --model-path /model --trust-remote-code \
  --mem-fraction-static 0.75 --attention-backend flashinfer \
  --chunked-prefill-size 8192 --disable-prefill-cuda-graph \
  --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 \
  --mamba-full-memory-ratio 4.21 --mamba-radix-cache-strategy extra_buffer_lazy \
  --context-length 262144 --speculative-algorithm EAGLE \
  --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder \
  --host 0.0.0.0 --port 30000

Do not serve this with MiaAI's start.sh. It has no MODEL_PATH: QUANT=nvfp4 is hardcoded to a different checkpoint, and only the HF cache is mounted, so it will quietly download and serve someone else's weights while appearing to serve yours. Its tuning notes are worth reading; its launch path is not.

One more thing: do not copy --mem-fraction-static 0.95 from elsewhere. On one coherent pool that starves the operating system as well as the server, and a long video prefill will take the whole box down with a global OOM — measured here, exit 137, dbus killed too.

Measured on the GB10

	Measured
Decode, single stream (600 tokens of code)	25.7 tok/s
Decode, mean over the eval set	19.8 tok/s
Speedup vs the reference	1.94x

These are measurements from the machine that built this checkpoint, not projections.

Speculative decoding is tuned. MTP depth was swept 2/3/4/5 and 3/1/4 wins on code (23.4 / 25.5 / 23.6 / 21.2 tok/s), which is also the setting serve_export.sh ships. The mechanism is visible in the data: accept length climbs monotonically with depth (2.45 → 2.81 → 2.95 → 3.08) while throughput peaks at 3, because the MTP head is trained for exactly three steps and deeper chains only add verify work that gets rejected. If your workload is prose rather than code, 2/1/3 is slightly better (18.4 vs 17.3).

You will see higher figures quoted for GB10. Most are extrapolated from other hardware rather than measured on one — worth checking which you are reading.

What is inside

Component	Precision
MLP gate/up/down, all layers	NVFP4 W4A16, block 16, e4m3 scales
Gated DeltaNet in_proj_qkv, in_proj_z, out_proj	NVFP4 W4A16
Full-attention projections	NVFP4 W4A16
All activations	BF16
Vision tower model.visual.*	BF16
MTP head mtp.*	BF16
lm_head, embed_tokens, conv1d, in_proj_a/b	BF16

MLP is about 62% of the parameters, which is why quantizing it and leaving everything sensitive alone gets most of the compression for very little of the risk. Produced with NVIDIA Model Optimizer's stock W4A16_NVFP4_CFG.

One packaging note if you are reading the files: hf_quant_config.json declares MIXED_PRECISION with a per-module map rather than a bare W4A16_NVFP4. That is deliberate. SGLang rejects the latter outright even though it has the matching kernel; the map is the only form that reaches it. ModelOpt's original config is preserved alongside as hf_quant_config.modelopt.json.

How this was verified

Smoke test: 6/6 cases passed (5 vision). Export verification: 1999 tensors, no activation scales, MTP head and vision tower present and unquantized.

Those are structural checks. They cannot tell you whether 4-bit weights cost real capability, because the realistic failure of a good quantization is output that is worse but still coherent, which nothing automatic catches. So:

24 prompts across text, images and video were run against both this checkpoint and BF16 source / SGLang, on the same engine so the weights are the only variable. No fatal flags were raised. Every ground-truth string the reference matched, this build matched too — including all 6 subtitle strings burned into 3 real video clips, read back intact.

If image or video quality matters to you, run that comparison on your inputs. eval_harness.py, make_evalset.py and serve_export.sh are all in this repo.

Does it hold up as an agent

Quantization damage tends to surface in agentic work before it surfaces in benchmarks. A model that still writes fluent prose can stop honouring a tool schema, lose the thread over a long run, or quietly stop noticing things. Three runs, on the GB10, served exactly as serve_export.sh ships it.

Codebase recon against known ground truth. Trace how a capability-enforcement setting travels through an unfamiliar Rust workspace — where it is defined, where it is validated, where it actually takes effect — and cite file:line for every link. It got all three links exactly right, then went past what was asked: the filter implementation, the pre-spawn validation entry, the test locations, the CI commands. What it did with its own uncertainty mattered more. Its "unknowns" section flagged two things it could not confirm, and both were real defects: a struct field whose doc comment claims it is applied at spawn but which has no production consumer anywhere, and a validator reachable only from tests. Both confirmed by grep afterwards. 16m13s.

A feature, unattended, end to end. Explore → plan → implement → verify with no human in the loop: add a --dry-run flag to two installers, one sh and one PowerShell, plus the documentation for it. 70m38s. Re-tested independently afterwards on both platforms — dry run writes nothing and exits 0, normal install unchanged, uninstall still clean, unrecognised arguments still fall through to a normal install. Its own review claimed eight checks passed; all eight held up when they were re-run by hand. It wrote scratch test scripts as it worked and deleted them when it was finished.

Vision inside the loop. Handed a screenshot through the same tool path, it transcribed every line exactly, including the hex string 0x7F4A that a 12B vision model misread on two separate attempts. It also described the image's white background as black, and labelled that observation certain. Transcription is trustworthy here. Scene and colour description is not — on this model or on the 12B.

Same tasks, same box, one run each:

	this build (27B dense)	Laguna S 2.1 (117B MoE, 8.5B active)	Nemotron 3 Super (120B MoE)
Feature run	clean, 70m	clean, 45m	shipped a real bug, 75m
Recon	all links, +2 real defects	all links	not run
Vision	yes	none	none
Decode on code	20.9 tok/s	44.7 tok/s	15.9 tok/s

Laguna is about twice as fast generating code and finished the feature 25 minutes sooner. It did not notice either defect on the same recon task. Decode here is measured with one harness across all three so the column compares — it counts prefill in wall clock, which is why this build reads 20.9 rather than the 25.7 measured above.

Two things that matter to a harness and are easy to get wrong: with --tool-call-parser qwen3_coder this build returns structured tool_calls, and with --reasoning-parser qwen3 the reasoning lands in reasoning_content with nothing leaking into content. Both are already set by serve_export.sh.

Read this for what it is: one run per model per task, one harness (Axon driving oh-my-axon), against a codebase whose ground truth was known in advance — which is what makes the recon result checkable, and also what keeps it from being a standard benchmark. Unlike everything else on this page, these numbers cannot be re-derived from this repo alone.

Lineage

Qwen/Qwen3.8-27B → DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 → this.

Finetune by DavidAU (COLD FUSION / GAIN + Unsloth). Base model by Qwen.

The full quantization and evaluation pipeline is in this repo: quantize_w4a16.py, verify_export.py, smoke_test.py, eval_harness.py, serve_export.sh, spec_sweep.sh. Everything asserted on this page can be re-derived with them.

Apache 2.0, unbroken from the base model.