README.md

metadata

license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
base_model: Qwen/Qwen3.8-Flash-Next
base_model_relation: quantized
tags:
  - nvfp4
  - fp8
  - compressed-tensors
  - abliterated
  - uncensored
  - dgx-spark
  - gb10
  - vllm
  - multi-node
extra_gated_prompt: >-
  This checkpoint has had its safety alignment removed (abliteration, by
  OrcaRouter). It complies with requests the original Qwen3.8-Flash-Next refuses
  and has no guardrails of its own. It is published for research, red-teaming,
  interpretability and private use behind your own moderation. By requesting
  access you confirm that you will use it lawfully, that you take full
  responsibility for what you do with it and what it generates, and that you
  accept the Qwen Community License 1.0 that governs these weights.
extra_gated_fields:
  I will use this model lawfully and take full responsibility for its use and outputs: checkbox
  I will put my own safety and moderation layer in front of any deployment reachable by others: checkbox
  Intended use:
    type: select
    options:
      - Research / interpretability
      - Red-teaming / safety evaluation
      - Private use
      - Other

Qwen3.8-Flash-Next — hibrid48-uncensored: the abliterated body with the 4-bit output head

hibrid47-uncensored with one tensor changed: the 1.18 GiB bf16 output head becomes 0.33 GiB of NVFP4. On the non-abliterated twin (hibrid48) that single change took the two-Spark serve from 16.9 to 22.0 engine steps/s (+30 %): the head is read about 5.4 times per decode step by speculative decoding, and it was 27 % of the step. Everything else — OrcaRouter's abliterated body, the NVFP4 n-gram table on the GPU, the re-quantized drafter — is hibrid47-uncensored, byte for byte.

The head tensor itself is identical to hibrid47's: the abliteration (Arditi et al., one refusal direction removed from 149 residual-writing matrices) never touched lm_head, and the quantizer reproduced hibrid48's global scale and error to the last digit (scale 1.533e-4, mean |Δw| = 9.0 % of |w|).

What changed vs hibrid47-uncensored

part	hibrid47-uncensored	here
routed experts, QSA/GDN/shared-expert FP8 tier, PLE table, drafter	OrcaRouter's / hibrid47's	unchanged
lm_head	bf16, 1.18 GiB, in model-00017-of-00017.safetensors	NVFP4 W4A16 (group 16, fp8 scales, fp32 global), model-lmhead-nvfp4.safetensors, 0.33 GiB; shard 17 (which held only the head) dropped
config.json	compressed-tensors, lm_head in ignore	+ config_groups.group_lm_head (nvfp4-pack-quantized, targets ["re:.*lm_head$"] — a regex so the MTP drafter's mtp.lm_head matches too), lm_head removed from ignore

Size 98.1 GiB (hibrid47-uncensored: 99.0).

Loading — read this

vLLM 0.29 builds the output head without the checkpoint's quantization config, so stock vLLM fails on this checkpoint with a shape mismatch. The fix is two lines (pass quant_config to ParallelLMHead in the model's model.py and mtp.py), shipped as recipes/qwen38-flash-next-fast/docker/patches/11-lm-head-quant-config.py in the myllmbox repo and applied by that recipe's image. With the config it loads through vLLM's own compressed-tensors NVFP4-A16 linear scheme (Marlin on GB10). This repo is gated — accept the agreement, then hf auth login (or export HF_TOKEN=…) before the download.

git clone https://github.com/bilikaz/myllmbox-runner.git && cd myllmbox-runner
./download.sh myllmbox/Qwen3.8-Flash-Next-hibrid48-uncensored
./build-and-copy.sh qwen38-flash-next-fast           # vLLM 0.29 + the lane's patches, copied to the second Spark
# recipes/qwen38-flash-next-fast/myllmbox.yaml → model: /models/myllmbox/Qwen3.8-Flash-Next-hibrid48-uncensored
./run.sh qwen38-flash-next-fast

Repo layout

15 body shards (model-000NN-of-00017.safetensors, OrcaRouter's; shards 2 — the bf16 table — and 17 — the bf16 head — omitted) + model-lmhead-nvfp4.safetensors (lm_head.weight_packed uint8 [248320, 1280], lm_head.weight_scale fp8 [248320, 160], lm_head.weight_global_scale fp32 — compressed-tensors spelling, global scale stored as the divisor) + model-mtp.safetensors (re-quantized drafter) + 8 table shards + index. Every tensor exists exactly once.

Reproducibility

builds/qwen38-flash-next/quantize-lm-head.py --ckpt <hibrid47-uncensored> --out <this> in the myllmbox repo (the tool detects the compressed-tensors config and writes that spelling); 13 s on a Spark. Standardizer, drafter re-quantization and the table pipeline as documented on hibrid47-uncensored.

Responsible use

No guardrails. Research, red-teaming, interpretability and private use behind your own moderation; anything reachable by other people needs its own safety layer. You are responsible for lawful use and for the outputs.

Attribution & license

• Base model: Qwen/Qwen3.8-Flash-Next (Alibaba) — Qwen Community License 1.0 (included as LICENSE). It governs these weights and every derivative: modification, distribution, hosting and commercial use permitted; products over 100M MAU / $20M monthly revenue must display the model name; a Model-as-a-Service or AI-assistant business on it needs a separate Qwen license.
• Abliteration and the body's NVFP4/FP8 quantization: OrcaRouter, from orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 (their contribution offered under Apache-2.0).
• NVFP4 n-gram table, drafter re-quantization, lm_head quantization, the GPU-resident load path and the serving stack: myllmbox.