jayyun98
/

MiniCPM5-2B-SFT-Draftfit-DFlash-DPACE

@@ -1,78 +1,155 @@
 ---
 language: en
 pipeline_tag: text-generation
-base_model: openbmb/MiniCPM5-2B-DSpark
 tags:
 - speculative-decoding
 - draft-model
-- draftfit
 - minicpm5
 license: other
 ---
-# Draftfit validation set: MiniCPM5-SFT DFlash DPACE
-**[Draftfit](https://github.com/JayYun98/Draftfit) trains and adapts speculative-decoding drafts for a frozen target model.** These MiniCPM5 checkpoints are an experimental validation set for its three paths: (1) one shared feature cache driving independent DSpark, DFlash and DFlash2 family/ablation runs; (2) conversion from an existing DSpark draft with explicit anchor-alignment retraining; and (3) continual training from previous draft weights. Draftfit also records held-out accuracy and domain-level inference metrics. Each family has its own weights; this is **not one universal serving checkpoint**.
-This card covers **DFlash DPACE**, one of four trained family branches. The intended target is [openbmb/MiniCPM5-2B-SFT](https://huggingface.co/openbmb/MiniCPM5-2B-SFT) at revision `3e0690c57f6ac772180498467c82e56170cc00ac`. The starting draft was [openbmb/MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark) at revision `114a20fdbf53220712c7fbdd7dccddbf1dedebb4`; its publisher pairs it with the final MiniCPM5-2B target, so this SFT pairing is an experimental adaptation.
-This repository's `main` revision is the **DFlash DPACE v4** checkpoint measured in the domain table. The `v5` revision contains 1,000 additional continual-training steps on a fresh split.
-## Measured speed by validation domain
-| Draft paired with MiniCPM5-2B-SFT | A100 v4 overall tok/s | MLX v4 overall tok/s | PerfectBlend (35) | OpenCode (6) | APIGen-MT (6) | Nemotron Agentic (6) |
-| --- | ---: | ---: | ---: | ---: | ---: | ---: |
-| Target only | 175.60 | 50.98 | 50.76 | 51.04 | 51.47 | 51.81 |
-| Official DSpark | 354.69 | 105.38 | 107.83 | 112.19 | 107.79 | 85.86 |
-| **[Draftfit DSpark v4](https://huggingface.co/jayyun98/MiniCPM5-2B-SFT-Draftfit-DSpark)** | **347.94** | **116.15** | **118.29** | **122.28** | **116.04** | **91.99** |
-| **[Draftfit DFlash CE v4](https://huggingface.co/jayyun98/MiniCPM5-2B-SFT-Draftfit-DFlash-CE)** | **294.77** | **89.61** | **89.02** | **99.18** | **92.49** | **79.31** |
-| **[Draftfit DFlash DPACE v4](https://huggingface.co/jayyun98/MiniCPM5-2B-SFT-Draftfit-DFlash-DPACE)** | **304.85** | **89.97** | **89.84** | **103.20** | **91.62** | **76.97** |
-| **[Draftfit DFlash2 v4](https://huggingface.co/jayyun98/MiniCPM5-2B-SFT-Draftfit-DFlash2)** | **not measured** | **80.75** | **78.37** | **92.76** | **87.49** | **70.41** |
-**Method.** v4 has 53 held-out conversations (35/6/6/6 by source). A100 numbers are total generated tokens divided by the entire measured wall time, concurrency 1, BF16 target, SGLang 0.5.16, greedy, maximum 128 new tokens and three run medians. MLX numbers are medians of individual prompt token rates, one timed run per prompt after warmup on an M5 Pro, BF16 target, 4-bit drafts and verification width 3. A100 and MLX columns use different metric definitions and must not be compared numerically. MLX models ran in separate sessions; the minority domains contain only six prompts each. These are exploratory domain estimates.
-The source-hashed, prompt-free MLX domain summary is in [`evaluation-v4.json`](evaluation-v4.json).
-## Which draft should you try?
-These are **different research controls and training objectives**, not four interchangeable speed tiers. The numbers below refer to the `main` (v4) weights on the same 53-prompt set. A slower draft may still answer a useful training question, but no measured downstream quality gain is established by its architecture alone.
-| Draftfit artifact | Distinctive training/architecture feature | When to try it | What this release actually shows |
-| --- | --- | --- | --- |
-| **DSpark** | Keeps the DSpark draft structure and its anchor, Markov and confidence components; supports same-method adaptation from the official draft. | First control when adapting an existing DSpark to this SFT target or comparing later continual steps. | Closest A100 v4 speed to the official draft: 347.94 versus 354.69 tok/s. This pairing still needs exact serving parity checks. |
-| **DFlash CE** | Block-parallel DFlash trained with token cross-entropy; a direct loss baseline after the guarded DSpark-to-DFlash warm start. | Establish a simple DFlash reference before testing position-weighted objectives or another serving layout. | 294.77 A100 and 89.61 MLX tok/s. Weight transfer required anchor-alignment retraining; it did not preserve DSpark acceptance by itself. |
-| **DFlash DPACE** | Same DFlash serving architecture, but its training loss weights positions by a smoothed prefix-probability and continuation surrogate. | Research whether emphasizing tokens that can extend an accepted prefix helps your own held-out workload; compare with CE under identical serving settings. | 304.85 A100 and 89.97 MLX tok/s. This small speed difference from CE does not establish better acceptance or general speed. |
-| **DFlash2** | Adds block-local grouped convolution and a trained candidate selector to the DFlash family. | Study candidate ranking or a richer draft when your runtime supports DFlash2; benchmark against the same target and prompts. | 80.75 MLX tok/s; A100 serving was unavailable in the tested SGLang 0.5.16 runtime. No A100 speed or deployment claim. |
-Use `v5` to examine another 1,000-step **weights-only continual** stage on a fresh data split, not as an automatically better default. DSpark `v3-best` is an earlier checkpoint whose best A100 observation used a different holdout. Pick a revision only after checking exact token/state behavior and accepted length on your own target, runtime and workload; speed alone is insufficient.
-### Best A100 observations across experiments
-| Draftfit artifact | Best observed A100 overall tok/s | Matching official DSpark | Evidence |
-| --- | ---: | ---: | --- |
-| DSpark `v3-best` | 360.26 | 361.36 | Three-run median, v1–v3 held-out set |
-| DFlash CE `v5` | 297.86 | 353.20 | Three-run median, v5 held-out set |
-| DFlash DPACE `v5` | 306.28 | 353.20 | Three console values; raw run JSON missing |
-| DFlash2 | not measured on A100 | — | SGLang 0.5.16 lacked compatible serving |
-These best observations use **different held-out sets** across v3, v4 and v5. They do not rank model revisions across sets. The v5 DPACE value has console evidence only. None of these trained drafts demonstrated a reliable speed advantage over the matching official DSpark baseline.
-## What was validated
-- The v4 and v5 family weights were trained for 1,000 steps per stage with a disjoint 1,000-conversation training set per stage. All exports were rebuilt from locally backed-up checkpoints and reloaded with Draftfit on CPU. The v4 DFlash2 export SHA-256 exactly matches the previously exported GPU artifact.
-- The v4 MLX draft outputs matched the official DSpark outputs on 53/53 held-out prompts at verification width 3. Under the fast BF16 verifier, target-only output matched the official draft on only 18/53, so this **does not establish target-output equivalence**. On A100, the v4 fixed-12 comparison matched the target for 9/12 official/DSpark outputs and 7/12 DFlash CE/DPACE outputs.
-- DFlash2 has MLX execution evidence but no compatible A100 SGLang 0.5.16 serving result. DFlash CE/DPACE likewise need exact token/state diagnosis before deployment.
-## Download and use
-```sh
-hf download jayyun98/MiniCPM5-2B-SFT-Draftfit-DFlash-DPACE --revision main --local-dir ./draft
 ```
-`config.json` and `model.safetensors` are the draft export. Use the pinned SFT target and its tokenizer separately. A speculative-decoding backend must support this draft architecture and revision. The [Draftfit repository](https://github.com/JayYun98/Draftfit) documents training, export and benchmark workflows; the exact experimental MiniCPM5 configs are being prepared for source release. Do not treat a successful weight load as proof of correct speculative serving.
-## Provenance, data and terms
-Starting draft: [openbmb/MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), Apache-2.0. Target: [openbmb/MiniCPM5-2B-SFT](https://huggingface.co/openbmb/MiniCPM5-2B-SFT), Apache-2.0. The v4/v5 training mixture per stage used [PerfectBlend deduplicated](https://huggingface.co/datasets/tepirale/open-perfectblend-deduplicate-mlabonne) (700 rows), [OpenCodeInstruct](https://huggingface.co/datasets/nvidia/OpenCodeInstruct) (100), [APIGen-MT-5k](https://huggingface.co/datasets/Salesforce/APIGen-MT-5k) (100), and [Nemotron-Agentic-v1](https://huggingface.co/datasets/nvidia/Nemotron-Agentic-v1) (100). The APIGen source is marked **CC BY-NC 4.0**; its card also says some GPT-4-generated data should not be used to develop models competing with OpenAI. The selected rows' generator provenance was not resolved. Treat this release as **research/noncommercial** and review [source terms](SOURCE_TERMS.md) before redistribution or downstream use. No source conversations, generated responses, tokenizer weights, target weights, optimizer state or credentials are included here. See `release_manifest.json` for input and export hashes.
-This is an experimental research artifact. Its speed numbers are bounded to the hardware, target, prompt set, precision and verifier settings above. It is not a general speedup or production-correctness claim.

 ---
 language: en
 pipeline_tag: text-generation
+base_model:
+- openbmb/MiniCPM5-2B-DSpark
+- openbmb/MiniCPM5-2B
 tags:
 - speculative-decoding
 - draft-model
 - minicpm5
+- dspark
+- dflash
 license: other
+library_name: draftfit
 ---
+# Draftfit — MiniCPM5-2B-SFT DFlash DPACE
+[Draftfit](https://github.com/JayYun98/Draftfit) trains and adapts speculative-decoding draft models for a **frozen target LLM**.
+This checkpoint is the **DFlash DPACE branch** adapted to:
+- **Target:** [`openbmb/MiniCPM5-2B-SFT`](https://huggingface.co/openbmb/MiniCPM5-2B-SFT)
+- **Starting draft:** [`openbmb/MiniCPM5-2B-DSpark`](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark)
+> Each Draftfit family has separate weights. DSpark, DFlash CE, DFlash DPACE and DFlash2 are **not one universal checkpoint**.
+## Results
+### A100 / SGLang
+| Draft | Overall tok/s |
+| --- | ---: |
+| Target only | 175.60 |
+| Official DSpark | **354.69** |
+| Draftfit DSpark | **347.94** |
+| Draftfit DFlash CE | 294.77 |
+| **Draftfit DFlash DPACE (Mine)** | **304.85** |
+| DFlash2 | not measured |
+### MLX / M5 Pro
+| Draft | Overall tok/s |
+| --- | ---: |
+| Target only | 50.98 |
+| Official DSpark | 105.38 |
+| Draftfit DSpark | **116.15** |
+| DFlash CE | 89.61 |
+| **Draftfit DFlash DPACE (Mine)** | **89.97** |
+| DFlash2 | 80.75 |
+A100 and MLX use different measurement methods, so their absolute numbers should **not** be compared directly.
+Validation set: **53 held-out conversations**
+- PerfectBlend: 35
+- OpenCodeInstruct: 6
+- APIGen-MT: 6
+- Nemotron Agentic: 6
+## Domain results — MLX
+| Draft | PerfectBlend | OpenCode | APIGen-MT | Nemotron Agentic |
+| --- | ---: | ---: | ---: | ---: |
+| Target only | 50.76 | 51.04 | 51.47 | 51.81 |
+| Official DSpark | 107.83 | 112.19 | 107.79 | 85.86 |
+| Draftfit DSpark | **118.29** | **122.28** | **116.04** | **91.99** |
+| DFlash CE | 89.02 | 99.18 | 92.49 | 79.31 |
+| **Draftfit DFlash DPACE (Mine)** | **89.84** | **103.20** | **91.62** | **76.97** |
+| DFlash2 | 78.37 | 92.76 | 87.49 | 70.41 |
+Full prompt-free summary: [`evaluation-v4.json`](evaluation-v4.json)
+## Which draft should I use?
+| Draft | Best use |
+| --- | --- |
+| **DSpark** | Best first choice when adapting an existing DSpark draft to a new target |
+| **DFlash CE** | Simple block-parallel DFlash baseline |
+| **DFlash DPACE** | Test position-weighted training objectives against CE |
+| **DFlash2** | Research richer candidate ranking / grouped-convolution drafts |
+For this MiniCPM5-SFT experiment, **DSpark is the strongest practical baseline**.
+Draftfit DSpark reached **347.94 tok/s vs 354.69 tok/s** for the official DSpark on A100, while reaching **116.15 vs 105.38 tok/s** in the tested MLX setup.
+These measurements do **not** establish a general speed advantage.
+## Revisions
+| Revision | Description |
+| --- | --- |
+| `main` | DFlash DPACE, primary release |
+| `v5` | Additional 1,000-step continual-training stage on a fresh split |
+Best observed DSpark A100 result:
+- Draftfit DSpark [`v3-best`](https://huggingface.co/jayyun98/MiniCPM5-2B-SFT-Draftfit-DSpark/tree/v3-best): **360.26 tok/s**
+- Matching official DSpark: **361.36 tok/s**
+Different revisions used different held-out sets, so these numbers should not be used to rank revisions directly.
+## Validation notes
+The models were trained for **1,000 steps per stage** using disjoint 1,000-conversation training sets.
+Training mixture per stage:
+| Dataset | Rows |
+| --- | ---: |
+| PerfectBlend deduplicated | 700 |
+| OpenCodeInstruct | 100 |
+| APIGen-MT-5k | 100 |
+| Nemotron-Agentic-v1 | 100 |
+Important limitations:
+- DSpark/DFlash serving correctness still requires exact token/state and acceptance-length validation.
+- DFlash2 was tested on MLX but not on A100 with SGLang 0.5.16.
+- A successfully loaded draft checkpoint does **not** prove correct speculative decoding.
+- Reported speed is specific to the tested target, hardware, verifier settings and prompt set.
+## Download
+```bash
+hf download jayyun98/MiniCPM5-2B-SFT-Draftfit-DFlash-DPACE \
+  --revision main \
+  --local-dir ./draft
+```
+The repository contains:
+```text
+config.json
+model.safetensors
 ```
+Use them with the pinned MiniCPM5-2B-SFT target and tokenizer.
+Training, conversion and benchmarking workflows are documented in:
+[Draftfit](https://github.com/JayYun98/Draftfit)
+## License / data notice
+The training mixture includes **APIGen-MT-5k**, which is marked **CC BY-NC 4.0** and includes additional generator-provenance restrictions.
+Treat this release as **research / non-commercial** unless you have reviewed the source dataset terms.
+See [`SOURCE_TERMS.md`](SOURCE_TERMS.md) and `release_manifest.json`.
+---
+**Experimental research artifact.**
+The reported numbers are not a general production speedup or correctness claim.