Qwen3.5-9B-MXFP4-QAD-LR8e-6-s4000-BF16

QAD-trained BF16 master weights (the trainer's final save, pre-export) of the arm published as Qwen3.5-9B-MXFP4-QAD-LR8e-6-s4000. These are the latent weights that the fake-quantized forward trained; the served MXFP4 artifacts are deterministic exports of this state. Uploaded so the training output is fully archived — it supports re-exporting (any schema), resuming/continuing training, and analysis.

config.json carries the qatfactory_qad metadata block recording the training-time quantization config (quant_format: mxfp4, fused_runtime_scales: false), which is the durable record of how this model was trained.

This is not a quantized serving artifact. For deployment use Qwen3.5-9B-MXFP4-QAD-LR8e-6-s4000. To re-export with QATFactory (weili/mxfp4 @ 86b72a2):

python scripts/export_mxfp4_vllm.py --source <this-repo-dir> \
    --model-assets Qwen/Qwen3.5-9B --output <out-dir>

How these weights were trained

item	value
training repo	QATFactory branch weili/mxfp4 @ 86b72a2 (PR) — scripts/train_llm_qat.py
method	QAD (quantization-aware distillation): student trains with fake-quantized forward; frozen BF16 teacher = the base model itself
objective	pure KL at temperature 1.0 (distill_weight 1.0, hard_label_weight 0.0), loss on assistant tokens only
base model / teacher	Qwen/Qwen3.5-9B @ c202236235762e1c871ad0ccb60c8ee5ba337b9a
dataset	openperfectblend_100k_Qwen3.5-9B_think — ~100K ChatML conversations (OpenPerfectBlend-derived prompts with Qwen3.5-9B think-mode responses; native <think> spans in the assistant turns), prepared in the QATFactory project. Train file from togethercomputer/Qwen3.5-9B-reasonmix @ b88c109 (940,793,581 bytes, md5 0406bb3a7a482352360716a1bc5e9e04; ~84.3k train conversations by the trainer's epoch accounting). Held-out eval = a disjoint 256-conversation split (md5 af10c8c304c146a81ddc35e439d7ac4b), 818,944 scored positions — the same corpus used for the serving-KL rows below
preprocessing	model chat template (ChatML), assistant-only loss mask, max_seq_len 8192, right-truncated
this checkpoint	optimizer step 4000 of 4000 (32,000 conversations consumed ~= 0.38 epoch, no data repetition)
batch per weight update	8 sequences = 1/GPU x 8 GPUs x grad-accum 1 (<= 8 x 8192 = 65,536 tokens/update)
learning rate	peak 8.0e-6, cosine decay to 0 over 4000 steps, linear warmup 1% (40 steps) — dose-response point between 1e-6 (flat) and 1e-5 (winner)
optimizer	AdamW (adamw_torch, beta1 0.9 / beta2 0.999), weight_decay 0.0, max_grad_norm 1.0
precision / parallelism	bf16, FSDP2 full_shard on jbom 8xB200 (single node), gradient checkpointing (non-reentrant), sdpa attention
fake-quantized modules	all linear projections (q/k/v/o_proj, gate/up/down_proj, GatedDeltaNet in/out projections); embeddings, lm_head, norms and the vision tower stay BF16
quantization config	quant_format: mxfp4, fused_runtime_scales: false
seed / bookkeeping	seed 42; held-out eval every 100 steps; checkpoint every 1000 steps; step time ~1.85 s/step
in-loop held-out eval KL	0.0317 (s100) -> 0.02138 (plateau from ~s2500)
W&B	uxwnwoun (project qatfactory-qat, public)

Training mode: W4A16 (weight-only fake-quant). Weights are fake-quantized in the forward pass; activations stay BF16 during training. The exported artifact can still be served W4A4 — the tables below measure both.

Related checkpoints

• Other steps of this arm: s1000, s2000, s3000, s4000
• QAD-trained BF16 master weights (pre-export): Qwen3.5-9B-MXFP4-QAD-LR8e-6-s4000-BF16
• Sweep winner: MXFP4-QAD-LR1e-5

---

Part of a monitored QAD experiment series with full bookkeeping (pre-registered predictions, exact SHAs/configs/seeds per run). Produced with AI assistance (Claude).