---
license: mit
base_model:
  - incoai/GLM-5.3-Flash-DFlash2
  - drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock
tags:
  - speculative-decoding
  - dflash2
  - glm
  - draft-model
library_name: transformers
---

# GLM-5.3-Flash-DFlash2 — `fc` recalibrated for the abliterated NVFP4 target

A drop-in replacement for [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2)
when the **target model is abliterated**. Exactly **one tensor** differs from the
original: `fc.weight`.

## Why this exists

DFlash2's drafter does not read the target's tokens — it reads the target's
**hidden states**, tapped at 5 layers and concatenated:

```
target_layer_ids = [5, 14, 24, 33, 42]      # 5 x 4096 = 20480 dims
fc.weight        = [4096, 20480]            # projects aux -> drafter width
```

`fc` is the entire coupling between target and drafter.

Abliteration rewrites `o_proj` on **layers 15–45**. Three of those five taps
(24, 33, 42) sit *inside* that range, so the drafter's input distribution shifts
and acceptance drops. Measured drift on the abliterated target:

| tap | drift ‖Δh‖/‖h‖ | |
|---|---:|---|
| L5 | 0.14% | stock anchor (below the edited range) |
| L14 | 0.27% | stock anchor |
| L24 | 0.52% | **abliterated** |
| L33 | 0.84% | **abliterated** |
| L42 | **2.03%** | **abliterated** |
| overall (20480-dim) | 1.50% | mean per-row cosine 0.9257 |

Drift is ~zero below L15 and grows monotonically with depth — exactly what the
edit predicts.

## What was done

**A closed-form ridge solve. No training, no SGD, no epochs.**

```
min_W  ‖ W·H_ablit − (fc_old·H_stock) ‖²_F  +  λ‖ W − fc_old ‖²_F
```

Make `fc_new(h_ablit)` reproduce what `fc_old(h_stock)` produced, so the drafter
sees the residual it was originally trained on. Rows are key-matched on
`(seq, pos)` so a dropped sequence cannot silently misalign pairs; the normal
equations accumulate in **fp64** because hidden states are collinear.

```json
{
  "method": "closed-form fc recalibration (streamed, key-aligned)",
  "ridge": 0.001,
  "paired_rows": 83101,
  "rel_move": 0.0670,
  "fit_old": 0.012050,
  "fit_new": 0.008629
}
```

Capture: 83,101 aligned pairs, domain-tagged `[chat, code, prose, technical,
uncensored]` (50% code, 9.7% prose).

### Ridge choice — 1e-3, chosen empirically

| ridge | weight move | fit_new (in-sample) | result |
|---:|---:|---:|---|
| **1e-3** | **6.7%** | 0.008629 | **shipped** |
| 1e-4 | 16.8% | 0.008226 | benchmarked → regression |
| 1e-5 | 44.6% | 0.007935 | benchmarked → regression |

**Lower ridge fit better in-sample and performed worse in practice.**
`fit_new` is a *training-set* residual with no held-out split, so it cannot
detect this — the 44.6% weight move at 1e-5 was the tell. **Judge these by
benchmark, never by `fit_new`.**

## Per-domain effect — and an honest limitation

| domain | n | residual before | after | reduction |
|---|---:|---:|---:|---:|
| chat | 11,123 | 1.18% | 0.86% | 27.3% |
| code | 41,853 | 1.23% | 0.89% | 27.6% |
| prose | 8,022 | 1.07% | 0.68% | **36.3%** |
| technical | 8,452 | 1.28% | 0.93% | 27.1% |
| uncensored | 13,651 | 1.18% | 0.89% | 24.4% |
| **overall** | 83,101 | 1.20% | 0.87% | 27.7% |

Prose is corrected **best** of all five domains — and prose acceptance still sits
near 0.33 against ~0.98 for structured output.

**Conclusion: the prose bottleneck is the target's token entropy, not
abliteration drift.** This recalibration will not make prose fast. It recovers
acceptance on context-sensitive, high-draftability output (code, structured,
tool calls), which is where speculative decoding pays anyway.

## Usage

Identical to the stock drafter — same architecture, same config, same
`num_speculative_tokens`. Only the weights differ.

```bash
vllm serve <abliterated-target> \
  --speculative-config '{"method":"dflash",
                         "model":"/path/to/this/repo",
                         "num_speculative_tokens":7}'
```

Measured on 2× DGX Spark (GB10), TP=2, against the abliterated NVFP4 target:
structured ~67 tok/s at 0.978 acceptance / 6.84 accepted per step.

## ⚠️ This drafter is target-specific

`fc` was fitted to the hidden states of **one specific abliterated checkpoint**
([`drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock`](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock)).

Point it at a **differently** abliterated model, a finetune, or the stock
censored model and three of the five taps carry a different distribution again.
**It will not error — acceptance just quietly drops.** For a different target,
re-run the solve rather than reusing this.

Conversely, if you run that abliterated target with the *stock* `incoai`
drafter, you are silently leaving acceptance on the table.

## Judging acceptance

Use **mean accepted length**, not acceptance ratio. The ratio rises when you
simply stop drafting the low-probability tail; throughput tracks accepted
length.

## Credits

- [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) — the original model (MIT)
- [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2) — the DFlash2 drafter this is derived from; **all architecture and training credit is theirs**, this repo changes one matrix
- [`drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock`](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock) — the abliterated target this was calibrated against
- [`LibertAIDAI/GLM-5.3-Flash-NVFP4`](https://huggingface.co/LibertAIDAI/GLM-5.3-Flash-NVFP4) — the NVFP4 parent quant
- [`dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4`](https://huggingface.co/dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4) — abliteration donor
- [`tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark`](https://github.com/tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark) — the 2-node deployment recipe, and the prior work identifying that abliteration costs drafter acceptance via hidden-state drift

License follows the base model (MIT).