Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/dflash2-figure.png filter=lfs diff=lfs merge=lfs -text
+model.safetensors filter=lfs diff=lfs merge=lfs -text

FC_SOLVE_META.json

@@ -0,0 +1,8 @@
+{
+  "method": "closed-form fc recalibration (streamed, key-aligned)",
+  "ridge": 0.001,
+  "paired_rows": 83101,
+  "rel_move": 0.06703612953424454,
+  "fit_old": 0.012050002627370349,
+  "fit_new": 0.008628943737344049
+}

README.md

@@ -0,0 +1,150 @@
+---
+license: mit
+base_model:
+  - incoai/GLM-5.3-Flash-DFlash2
+  - drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock
+tags:
+  - speculative-decoding
+  - dflash2
+  - glm
+  - draft-model
+library_name: transformers
+---
+
+# GLM-5.3-Flash-DFlash2 — `fc` recalibrated for the abliterated NVFP4 target
+
+A drop-in replacement for [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2)
+when the **target model is abliterated**. Exactly **one tensor** differs from the
+original: `fc.weight`.
+
+## Why this exists
+
+DFlash2's drafter does not read the target's tokens — it reads the target's
+**hidden states**, tapped at 5 layers and concatenated:
+
+```
+target_layer_ids = [5, 14, 24, 33, 42]      # 5 x 4096 = 20480 dims
+fc.weight        = [4096, 20480]            # projects aux -> drafter width
+```
+
+`fc` is the entire coupling between target and drafter.
+
+Abliteration rewrites `o_proj` on **layers 15–45**. Three of those five taps
+(24, 33, 42) sit *inside* that range, so the drafter's input distribution shifts
+and acceptance drops. Measured drift on the abliterated target:
+
+| tap | drift ‖Δh‖/‖h‖ | |
+|---|---:|---|
+| L5 | 0.14% | stock anchor (below the edited range) |
+| L14 | 0.27% | stock anchor |
+| L24 | 0.52% | **abliterated** |
+| L33 | 0.84% | **abliterated** |
+| L42 | **2.03%** | **abliterated** |
+| overall (20480-dim) | 1.50% | mean per-row cosine 0.9257 |
+
+Drift is ~zero below L15 and grows monotonically with depth — exactly what the
+edit predicts.
+
+## What was done
+
+**A closed-form ridge solve. No training, no SGD, no epochs.**
+
+```
+min_W  ‖ W·H_ablit − (fc_old·H_stock) ‖²_F  +  λ‖ W − fc_old ‖²_F
+```
+
+Make `fc_new(h_ablit)` reproduce what `fc_old(h_stock)` produced, so the drafter
+sees the residual it was originally trained on. Rows are key-matched on
+`(seq, pos)` so a dropped sequence cannot silently misalign pairs; the normal
+equations accumulate in **fp64** because hidden states are collinear.
+
+```json
+{
+  "method": "closed-form fc recalibration (streamed, key-aligned)",
+  "ridge": 0.001,
+  "paired_rows": 83101,
+  "rel_move": 0.0670,
+  "fit_old": 0.012050,
+  "fit_new": 0.008629
+}
+```
+
+Capture: 83,101 aligned pairs, domain-tagged `[chat, code, prose, technical,
+uncensored]` (50% code, 9.7% prose).
+
+### Ridge choice — 1e-3, chosen empirically
+
+| ridge | weight move | fit_new (in-sample) | result |
+|---:|---:|---:|---|
+| **1e-3** | **6.7%** | 0.008629 | **shipped** |
+| 1e-4 | 16.8% | 0.008226 | benchmarked → regression |
+| 1e-5 | 44.6% | 0.007935 | benchmarked → regression |
+
+**Lower ridge fit better in-sample and performed worse in practice.**
+`fit_new` is a *training-set* residual with no held-out split, so it cannot
+detect this — the 44.6% weight move at 1e-5 was the tell. **Judge these by
+benchmark, never by `fit_new`.**
+
+## Per-domain effect — and an honest limitation
+
+| domain | n | residual before | after | reduction |
+|---|---:|---:|---:|---:|
+| chat | 11,123 | 1.18% | 0.86% | 27.3% |
+| code | 41,853 | 1.23% | 0.89% | 27.6% |
+| prose | 8,022 | 1.07% | 0.68% | **36.3%** |
+| technical | 8,452 | 1.28% | 0.93% | 27.1% |
+| uncensored | 13,651 | 1.18% | 0.89% | 24.4% |
+| **overall** | 83,101 | 1.20% | 0.87% | 27.7% |
+
+Prose is corrected **best** of all five domains — and prose acceptance still sits
+near 0.33 against ~0.98 for structured output.
+
+**Conclusion: the prose bottleneck is the target's token entropy, not
+abliteration drift.** This recalibration will not make prose fast. It recovers
+acceptance on context-sensitive, high-draftability output (code, structured,
+tool calls), which is where speculative decoding pays anyway.
+
+## Usage
+
+Identical to the stock drafter — same architecture, same config, same
+`num_speculative_tokens`. Only the weights differ.
+
+```bash
+vllm serve <abliterated-target> \
+  --speculative-config '{"method":"dflash",
+                         "model":"/path/to/this/repo",
+                         "num_speculative_tokens":7}'
+```
+
+Measured on 2× DGX Spark (GB10), TP=2, against the abliterated NVFP4 target:
+structured ~67 tok/s at 0.978 acceptance / 6.84 accepted per step.
+
+## ⚠️ This drafter is target-specific
+
+`fc` was fitted to the hidden states of **one specific abliterated checkpoint**
+([`drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock`](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock)).
+
+Point it at a **differently** abliterated model, a finetune, or the stock
+censored model and three of the five taps carry a different distribution again.
+**It will not error — acceptance just quietly drops.** For a different target,
+re-run the solve rather than reusing this.
+
+Conversely, if you run that abliterated target with the *stock* `incoai`
+drafter, you are silently leaving acceptance on the table.
+
+## Judging acceptance
+
+Use **mean accepted length**, not acceptance ratio. The ratio rises when you
+simply stop drafting the low-probability tail; throughput tracks accepted
+length.
+
+## Credits
+
+- [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) — the original model (MIT)
+- [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2) — the DFlash2 drafter this is derived from; **all architecture and training credit is theirs**, this repo changes one matrix
+- [`drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock`](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-45-anchorstock) — the abliterated target this was calibrated against
+- [`LibertAIDAI/GLM-5.3-Flash-NVFP4`](https://huggingface.co/LibertAIDAI/GLM-5.3-Flash-NVFP4) — the NVFP4 parent quant
+- [`dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4`](https://huggingface.co/dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4) — abliteration donor
+- [`tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark`](https://github.com/tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark) — the 2-node deployment recipe, and the prior work identifying that abliteration costs drafter acceptance via hidden-state drift
+
+License follows the base model (MIT).

config.json

@@ -0,0 +1,61 @@
+{
+  "architectures": [
+    "DFlash2DraftModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dflash_config": {
+    "block_size": 8,
+    "conv_group_size": 16,
+    "conv_kernel_size": 2,
+    "mask_token_id": 154856,
+    "selector_rank": 256,
+    "selector_top_k": 16,
+    "target_layer_ids": [
+      5,
+      14,
+      24,
+      33,
+      42
+    ]
+  },
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "is_causal": false,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "max_position_embeddings": 1048576,
+  "max_window_layers": 5,
+  "model_type": "qwen3",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 5,
+  "num_key_value_heads": 8,
+  "num_target_layers": 45,
+  "pad_token_id": 154820,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": 2048,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.7.0",
+  "use_cache": false,
+  "use_sliding_window": true,
+  "vocab_size": 154880
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb4774debbf9416fc1ef78a62b33e26219aa5a389a8ae7fa3adc47b50e079a99
+size 2342169824