Llama-4-Scout-17B-16E — Heretic (gentle) abliterated — NVFP4

A gently abliterated and NVFP4 (W4A4) quantized build of Llama-4-Scout-17B-16E-Instruct, runnable on a single Blackwell-class GPU — including the NVIDIA GB10 / DGX Spark (sm_121) — via SGLang.

⚠️ Read this before using it

This model has a known, deterministic limitation: it occasionally corrupts long digit sequences (phone numbers, SSNs, hashes, etc.) — it will insert a junk token into a number even at greedy decoding (temperature 0). This is the abliteration tax showing up under low-bit quantization; the vanilla (non-abliterated) NVFP4 build of the same model does not have it.

Do not use this for precision-critical or numeric tasks — data extraction, log/security distillation involving IPs, ports, CVE IDs, hashes, timestamps, or anything where one wrong digit matters.

It is well-suited to general chat, reasoning, brainstorming, creative writing, and "less-fluff / more-decisive" assistant use, where an occasional mangled number is acceptable.

We originally built this for a precision-sensitive distillation role, found the number-corruption disqualifying for that role, and went with the vanilla model there. But the build is solid for other purposes, so we're sharing it with the caveat rather than shelving it.

What "abliterated" means here

Abliteration (via p-e-w/heretic) removes refusal directions so the model is more decisive and drops moralizing hedges — "less fluff," not "uncensored." This is a deliberately gentle build:

32 / 100 refusals on heretic's harmful-prompt set (base model: ~86) — decisive without being scrubbed
KL divergence 0.045 (well under the ~0.5 "significant damage" line)
34 of 48 text-layer attention o_proj projections abliterated; routed experts left intact

A more aggressive 16-refusal variant was also produced but showed more fluency degradation — gentler tested better.

Format & serving

nvfp4-pack-quantized (W4A4), group_size 16, via llm-compressor. Only the MoE experts are quantized; attention, router, lm_head, and the vision tower stay BF16.
Ships a clean meta Llama-4 tokenizer (not the unsloth-modified one — that variant makes SGLang fall back to a generic tokenizer backend and emit � garbage).
~70 GB, 14 shards. Fits a single 80 GB+ GPU or a 128 GB unified-memory GB10.

SGLang (tested on v0.5.12-cu130, GB10 / sm_121)

python3 -m sglang.launch_server \
  --model-path <this-model> --trust-remote-code --load-format safetensors \
  --attention-backend fa3 --fp4-gemm-backend flashinfer_cudnn --fp8-gemm-backend triton \
  --kv-cache-dtype fp8_e5m2 --context-length 32768

(On non-Blackwell datacenter GPUs, use your normal NVFP4 MoE backend; the flashinfer_cudnn FP4 path is what works on sm_121.)