| --- |
| license: other |
| license_name: qwen-community-license-1.0 |
| license_link: LICENSE |
| base_model: Qwen/Qwen3.8-Flash-Next |
| base_model_relation: quantized |
| tags: |
| - nvfp4 |
| - fp8 |
| - compressed-tensors |
| - abliterated |
| - uncensored |
| - dgx-spark |
| - gb10 |
| - vllm |
| - multi-node |
| extra_gated_prompt: >- |
| This checkpoint has had its safety alignment removed (abliteration, by OrcaRouter). It complies with requests the |
| original Qwen3.8-Flash-Next refuses and has no guardrails of its own. It is published for research, red-teaming, |
| interpretability and private use behind your own moderation. By requesting access you confirm that you will use it |
| lawfully, that you take full responsibility for what you do with it and what it generates, and that you accept the |
| Qwen Community License 1.0 that governs these weights. |
| extra_gated_fields: |
| I will use this model lawfully and take full responsibility for its use and outputs: checkbox |
| I will put my own safety and moderation layer in front of any deployment reachable by others: checkbox |
| Intended use: |
| type: select |
| options: |
| - Research / interpretability |
| - Red-teaming / safety evaluation |
| - Private use |
| - Other |
| --- |
| |
| # Qwen3.8-Flash-Next — hibrid48-uncensored: the abliterated body with the 4-bit output head |
|
|
| **hibrid47-uncensored with one tensor changed: the 1.18 GiB bf16 output head becomes 0.33 GiB of NVFP4.** On the |
| non-abliterated twin ([hibrid48](https://huggingface.co/myllmbox/Qwen3.8-Flash-Next-hibrid48)) that single change |
| took the two-Spark serve from 16.9 to 22.0 engine steps/s (+30 %): the head is read about 5.4 times per decode step |
| by speculative decoding, and it was 27 % of the step. Everything else — OrcaRouter's abliterated body, the NVFP4 |
| n-gram table on the GPU, the re-quantized drafter — is hibrid47-uncensored, byte for byte. |
|
|
| The head tensor itself is identical to hibrid47's: the abliteration (Arditi et al., one refusal direction removed |
| from 149 residual-writing matrices) never touched `lm_head`, and the quantizer reproduced hibrid48's global scale |
| and error to the last digit (scale 1.533e-4, mean |Δw| = 9.0 % of |w|). |
|
|
| ## What changed vs hibrid47-uncensored |
|
|
| | part | hibrid47-uncensored | here | |
| |---|---|---| |
| | routed experts, QSA/GDN/shared-expert FP8 tier, PLE table, drafter | OrcaRouter's / hibrid47's | **unchanged** | |
| | `lm_head` | bf16, 1.18 GiB, in `model-00017-of-00017.safetensors` | **NVFP4 W4A16** (group 16, fp8 scales, fp32 global), `model-lmhead-nvfp4.safetensors`, 0.33 GiB; shard 17 (which held only the head) dropped | |
| | `config.json` | compressed-tensors, `lm_head` in `ignore` | + `config_groups.group_lm_head` (`nvfp4-pack-quantized`, targets `["re:.*lm_head$"]` — a regex so the MTP drafter's `mtp.lm_head` matches too), `lm_head` removed from `ignore` | |
|
|
| Size 98.1 GiB (hibrid47-uncensored: 99.0). |
|
|
| ## Loading — read this |
|
|
| vLLM 0.29 builds the output head **without** the checkpoint's quantization config, so stock vLLM fails on this |
| checkpoint with a shape mismatch. The fix is two lines (pass `quant_config` to `ParallelLMHead` in the model's |
| `model.py` and `mtp.py`), shipped as `recipes/qwen38-flash-next-fast/docker/patches/11-lm-head-quant-config.py` in |
| the myllmbox repo and applied by that recipe's image. With the config it loads through vLLM's own |
| compressed-tensors NVFP4-A16 linear scheme (Marlin on GB10). This repo is gated — accept the agreement, then |
| `hf auth login` (or `export HF_TOKEN=…`) before the download. |
|
|
| ``` |
| git clone https://github.com/bilikaz/myllmbox-runner.git && cd myllmbox-runner |
| ./download.sh myllmbox/Qwen3.8-Flash-Next-hibrid48-uncensored |
| ./build-and-copy.sh qwen38-flash-next-fast # vLLM 0.29 + the lane's patches, copied to the second Spark |
| # recipes/qwen38-flash-next-fast/myllmbox.yaml → model: /models/myllmbox/Qwen3.8-Flash-Next-hibrid48-uncensored |
| ./run.sh qwen38-flash-next-fast |
| ``` |
|
|
| ## Repo layout |
|
|
| 15 body shards (`model-000NN-of-00017.safetensors`, OrcaRouter's; shards 2 — the bf16 table — and 17 — the bf16 head — |
| omitted) + `model-lmhead-nvfp4.safetensors` (`lm_head.weight_packed` uint8 [248320, 1280], `lm_head.weight_scale` |
| fp8 [248320, 160], `lm_head.weight_global_scale` fp32 — compressed-tensors spelling, global scale stored as the |
| divisor) + `model-mtp.safetensors` (re-quantized drafter) + 8 table shards + index. Every tensor exists exactly once. |
|
|
| ## Reproducibility |
|
|
| `builds/qwen38-flash-next/quantize-lm-head.py --ckpt <hibrid47-uncensored> --out <this>` in the myllmbox repo (the |
| tool detects the compressed-tensors config and writes that spelling); 13 s on a Spark. Standardizer, drafter |
| re-quantization and the table pipeline as documented on hibrid47-uncensored. |
|
|
| ## Responsible use |
|
|
| No guardrails. Research, red-teaming, interpretability and private use behind your own moderation; anything reachable by |
| other people needs its own safety layer. You are responsible for lawful use and for the outputs. |
|
|
| ## Attribution & license |
|
|
| - Base model: Qwen/Qwen3.8-Flash-Next (Alibaba) — **Qwen Community License 1.0** (included as `LICENSE`). It governs these |
| weights and every derivative: modification, distribution, hosting and commercial use permitted; products over 100M MAU / |
| $20M monthly revenue must display the model name; a Model-as-a-Service or AI-assistant *business* on it needs a separate |
| Qwen license. |
| - Abliteration and the body's NVFP4/FP8 quantization: OrcaRouter, from orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 |
| (their contribution offered under Apache-2.0). |
| - NVFP4 n-gram table, drafter re-quantization, lm_head quantization, the GPU-resident load path and the serving stack: myllmbox. |
| |
