Add production FP8-PLE model card

README.md

@@ -1,223 +1,96 @@
 ---
-license: other
-license_name: swift-open-license-1.0
-license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next/blob/main/LICENSE
-library_name: transformers
-pipeline_tag: image-text-to-text
-gated: true
+base_model: ukisai/Swift-1.5-Qwen3.8-Flash-Next-NVFP4
 tags:
-- quantized
+- qwen
+- qwen3.8
+- flash-next
+- swift
 - nvfp4
-- qwen3_8
-- moe
-- reasoning
-- efficient-thinking
-- token-efficient
-- post-training
-- agentic
-- terminal-bench
-datasets:
-- ukisai/Qwen3.8-27B-multi-turn-agent-sft
-base_model: ukisai/Swift-Qwen3.8-Flash-Next
-base_model_relation: quantized
+- fp8
+- sglang
+- pennyroyal
+- blackwell
+- local-llm
 ---
 
-<div align="center">
-  <a href="https://ukisai.com"><img src="ukisai-banner.png" alt="UkisAI" style="width:100%;max-width:100%;height:auto;display:block;margin-bottom:0.6em;" /></a>
-  <div style="display:flex;justify-content:center;gap:0.6em;margin-bottom:1em;">
-    <a href="https://ukisai.com"><strong>Website</strong></a> &nbsp;&bull;&nbsp;
-    <a href="https://ukisai.com/products/swift"><strong>Learn more</strong></a> &nbsp;&bull;&nbsp;
-    <a href="https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GGUF"><strong>GGUF</strong></a> &nbsp;&bull;&nbsp;
-    <a href="https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF"><strong>GSQ-RCO GGUF</strong></a> &nbsp;&bull;&nbsp;
-    <a href="#evaluation"><strong>Evaluation</strong></a> &nbsp;&bull;&nbsp;
-    <a href="#license-and-access"><strong>Enterprise licensing</strong></a>
-  </div>
-</div>
-
-# Swift 1.5 Qwen3.8-Flash-Next
-
-**NVFP4.** Derived directly from [Swift 1.5 Qwen3.8-Flash-Next](https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next). Native NVFP4 execution requires compatible NVIDIA Blackwell support.
-
-Swift 1.5 Qwen3.8-Flash-Next is UkisAI's reasoning-efficient derivative of [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next).
-It uses **63.4% fewer thinking tokens**, with a **1.8x speed up** while keeping the **accuracy loss <1%** vs base on xhigh.
-
-## Demo
-
-We gave base Qwen3.8-Flash-Next and Swift 1.5 Qwen3.8-Flash-Next the same prompt:
-
-> Create a 3D endless runner that has the fast, playful feel of Subway Surfers, but make the world and characters your own. I want to run through a lively place, dodge things, collect rewards, and feel the pace build the longer I survive. Make it fun to control and visually memorable. Use your judgment for the setting, mechanics, and little details that make it feel like a real game. Build it so I can launch and play it locally, then tell me how to run it.
-
-<video src="https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-NVFP4/resolve/main/swift-1.5-flash-next-demo.mp4" controls autoplay muted loop playsinline style="width:100%;height:auto;border-radius:12px;"></video>
-
-Try the game yourself here: [https://ukisai.com/swift-games/flash-next](https://ukisai.com/swift-games/flash-next)
-
-Base Qwen3.8-Flash-Next took 8 minutes 52 seconds to build its game. Swift 1.5 took 4 minutes 56 seconds.
-
-## Training approach
-
-We made Swift Flash Next efficient by figuring out which tokens were linked to pathological overthinking and penalizing them without "attacking" the reasoning length directly then regained the accuracy with RL and OPD, leading to "compressed" token usage while maintaining accuracy.
-
-Swift 1.5 produces shorter reasoning traces. In our testing, we also observe fewer overthinking errors.
-
-This release also features our previously mentioned post-training methods adapted specifically for coding and long-horizon agent
-work such as personal agents, terminal use and software engineering.
-
-Our training data is viewable here: https://huggingface.co/datasets/ukisai/Qwen3.8-27B-multi-turn-agent-sft albeit it is not used out of the box, but rather re-sampled, turned into proper RL environments etc.
-
-## Evaluation
-
-All scores compare the Qwen3.8-Flash-Next BF16 base with the Swift 1.5 BF16 checkpoint. Token columns report **thinking tokens**, except Terminal-Bench 2.1, which reports **total generated output tokens**.
-
-<style>
-.swift-table { width:100%; table-layout:fixed; border-collapse:separate; border-spacing:0; overflow:hidden; border:1px solid #27344A; border-radius:20px; background:#0D111B; font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif; font-size:14px; color:#BFBDBD; }
-.swift-table th { padding:13px 8px; text-align:center; font-weight:700; color:#AEB5C7; background:#0D111B; border-right:1px solid #27344A; border-bottom:1px solid #27344A; }
-.swift-table td { padding:14px 8px; text-align:center; color:#BFBDBD; background:#0D111B; border-right:1px solid #27344A; border-bottom:1px solid #27344A; vertical-align:middle; overflow-wrap:break-word; }
-.swift-table tr > :last-child { border-right:0; }
-.swift-table tbody tr:last-child td { border-bottom:0; }
-.swift-table .benchmark-heading { color:#B7BDCD; background:#0D111B; border-bottom:3px solid #7D45B5; }
-.swift-table .score-heading { color:#F0C5FF; background:#52239E; border-bottom:3px solid #7D45B5; }
-.swift-table .tokens-heading, .swift-table .median-heading { color:#D4E8FF; background:#304FC2; border-bottom:3px solid #5687E6; }
-.swift-table .benchmark { padding-left:18px; text-align:left; color:#FFFFFF; font-weight:600; }
-.swift-table .section { padding:12px 18px; text-align:left; color:#B489FF; background:#2A2541; font-weight:700; letter-spacing:.08em; text-transform:uppercase; border-top:1px solid #3A3159; border-bottom:1px solid #3A3159; }
-.swift-table .swift { background:#171127; }
-.swift-table thead tr:nth-child(2) .swift { color:#D3A0FF; }
-.swift-table .reduction { color:#69BFFF; background:#101B2C; font-weight:700; }
-.swift-table .detail { color:#8C94A8; font-size:12px; font-weight:500; }
-
-@media (max-width: 640px) {
-  .swift-table { display:block !important; width:100% !important; max-width:100%; overflow-x:auto !important; -webkit-overflow-scrolling:touch; table-layout:auto !important; }
-  .swift-table th, .swift-table td { min-width:100px; }
-  .swift-table th:first-child, .swift-table td:first-child { min-width:160px; }
-}
-</style>
-
-<table class="swift-table">
-  <thead>
-    <tr>
-      <th rowspan="2" class="benchmark-heading" style="width:32%;text-align:left;padding-left:18px;vertical-align:bottom;">Benchmark</th>
-      <th colspan="2" class="score-heading">Score</th>
-      <th colspan="3" class="tokens-heading">Mean tokens</th>
-      <th class="median-heading" style="width:14%;">Median tokens</th>
-    </tr>
-    <tr>
-      <th>Base</th><th class="swift">Swift 1.5</th>
-      <th>Base</th><th class="swift">Swift 1.5</th>
-      <th class="reduction">Reduction</th><th class="reduction">Reduction</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr><td class="section" colspan="7">General reasoning</td></tr>
-    <tr><td class="benchmark">GPQA-Diamond</td><td>89.80%</td><td class="swift">89.60%</td><td>17,683</td><td class="swift">7,823</td><td class="reduction">&darr; 55.8%</td><td class="reduction">&darr; 63.4%</td></tr>
-    <tr><td class="benchmark">MMLU-Pro</td><td>87.75%</td><td class="swift">87.20%</td><td>3,528</td><td class="swift">1,519</td><td class="reduction">&darr; 57.0%</td><td class="reduction">&darr; 24.0%</td></tr>
-    <tr><td class="benchmark">C-Eval</td><td>93.27%</td><td class="swift">93.60%</td><td>1,048</td><td class="swift">586</td><td class="reduction">&darr; 44.1%</td><td class="reduction">&darr; 7.1%</td></tr>
-    <tr><td class="benchmark">IFBench</td><td>73.20%</td><td class="swift">70.13%</td><td>8,310</td><td class="swift">4,411</td><td class="reduction">&darr; 46.9%</td><td class="reduction">&darr; 55.6%</td></tr>
-    <tr><td class="section" colspan="7">Mathematics</td></tr>
-    <tr><td class="benchmark">AIME 2026</td><td>98.67%</td><td class="swift">96.67%</td><td>23,015</td><td class="swift">15,806</td><td class="reduction">&darr; 31.3%</td><td class="reduction">&darr; 51.1%</td></tr>
-    <tr><td class="benchmark">HMMT (Nov 2025)</td><td>98.00%</td><td class="swift">97.33%</td><td>25,487</td><td class="swift">16,530</td><td class="reduction">&darr; 35.1%</td><td class="reduction">&darr; 54.7%</td></tr>
-    <tr><td class="section" colspan="7">Multimodal</td></tr>
-    <tr><td class="benchmark">ERQA</td><td>70.80%</td><td class="swift">69.30%</td><td>4,036</td><td class="swift">1,788</td><td class="reduction">&darr; 55.7%</td><td class="reduction">&darr; 47.7%</td></tr>
-    <tr><td class="section" colspan="7">Coding</td></tr>
-    <tr><td class="benchmark">LiveCodeBench v6</td><td>88.40%</td><td class="swift">90.39%</td><td>17,833</td><td class="swift">9,849</td><td class="reduction">&darr; 44.8%</td><td class="reduction">&darr; 52.0%</td></tr>
-    <tr><td class="section" colspan="7">Agentic coding</td></tr>
-    <tr><td class="benchmark">Terminal-Bench 2.1</td><td>67.64%</td><td class="swift">69.66%</td><td>40,591</td><td class="swift">45,428</td><td class="reduction">&uarr; 11.9%</td><td class="reduction">&darr; 17.9%</td></tr>
-  </tbody>
-</table>
-
-<details>
-<summary><strong>How to reproduce</strong></summary>
-
-<p style="font-size:13px;line-height:1.5;margin:8px 0;"><strong>Serving:</strong> BF16 · Qwen3 reasoning parser · context 262,144 · thinking xhigh · MTP disabled.<br>
-<strong>Sampling:</strong> temperature 1.0 · top_p 0.95 · top_k 20 · min_p 0 · presence_penalty 0 · repetition_penalty 1.<br>
-<strong>Benchmarks:</strong> five seeds (0–4) for the seeded question benchmarks; Terminal-Bench 2.1 uses five attempts per task. LiveCodeBench is full release v6 mean pass@1 over seeds.<br>
-<strong>Terminal-Bench 2.1:</strong> Harbor 0.20.0 / Terminus-2 2.0.0, pinned 89-task dataset, JSON parser, interleaved thinking, temperature 1, top_p 1, 131,072-token server context, 3,600-second LLM call timeout and native per-task limits. Swift used concurrency 8; its context-recovery fix was applied during the run.</p>
-
-<table style="display:table;width:100%;border-collapse:collapse;font-size:13px;line-height:1.3;margin:8px 0;">
-  <thead><tr><th style="padding:4px 8px;text-align:left;">Benchmark</th><th style="padding:4px 8px;text-align:right;">Output cap</th></tr></thead>
-  <tbody>
-    <tr><td style="padding:3px 8px;">GPQA-Diamond</td><td style="padding:3px 8px;text-align:right;">100,000</td></tr>
-    <tr><td style="padding:3px 8px;">MMLU-Pro</td><td style="padding:3px 8px;text-align:right;">100,000</td></tr>
-    <tr><td style="padding:3px 8px;">C-Eval</td><td style="padding:3px 8px;text-align:right;">16,384</td></tr>
-    <tr><td style="padding:3px 8px;">IFBench</td><td style="padding:3px 8px;text-align:right;">81,920</td></tr>
-    <tr><td style="padding:3px 8px;">AIME 2026</td><td style="padding:3px 8px;text-align:right;">250,000</td></tr>
-    <tr><td style="padding:3px 8px;">HMMT Nov 2025</td><td style="padding:3px 8px;text-align:right;">250,000</td></tr>
-    <tr><td style="padding:3px 8px;">ERQA</td><td style="padding:3px 8px;text-align:right;">100,000</td></tr>
-    <tr><td style="padding:3px 8px;">LiveCodeBench v6</td><td style="padding:3px 8px;text-align:right;">100,000</td></tr>
-  </tbody>
-</table>
-
-</details>
-
-## Efficiency across reasoning efforts
-
-GPQA-Diamond at each `reasoning_effort` setting, Swift 1.5 against the base at the same setting:
-
-<table class="swift-table" style="display:table;width:100%;table-layout:fixed;">
-  <colgroup><col style="width:16%"><col style="width:12%"><col style="width:12%"><col style="width:12%"><col style="width:12%"><col style="width:18%"><col style="width:18%"></colgroup>
-  <thead>
-    <tr>
-      <th rowspan="2" class="benchmark-heading" style="width:16%;text-align:left;padding-left:18px;vertical-align:bottom;">Reasoning effort</th>
-      <th colspan="2" class="score-heading" style="width:24%;">Score</th>
-      <th colspan="3" class="tokens-heading" style="width:42%;">Mean tokens</th>
-      <th class="median-heading" style="width:18%;">Median tokens</th>
-    </tr>
-    <tr>
-      <th>Base</th><th class="swift">Swift 1.5</th>
-      <th>Base</th><th class="swift">Swift 1.5</th>
-      <th class="reduction">Reduction</th><th class="reduction">Reduction</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr><td class="benchmark">Xhigh</td><td>89.80%</td><td class="swift">89.60%</td><td>17,683</td><td class="swift">7,823</td><td class="reduction">&darr; 55.8%</td><td class="reduction">&darr; 63.4%</td></tr>
-    <tr><td class="benchmark">Medium</td><td>86.36%</td><td class="swift">83.74%</td><td>4,157</td><td class="swift">2,483</td><td class="reduction">&darr; 40.3%</td><td class="reduction">&darr; 25.1%</td></tr>
-    <tr><td class="benchmark">Low</td><td>87.17%</td><td class="swift">84.75%</td><td>3,966</td><td class="swift">2,645</td><td class="reduction">&darr; 33.3%</td><td class="reduction">&darr; 19.7%</td></tr>
-  </tbody>
-</table>
-
-At xhigh, Swift 1.5 trails base by 0.20 percentage points while using 55.8% fewer mean and 63.4% fewer median thinking tokens. Medium and low save tokens but also lose 2.62 and 2.42 percentage points respectively.
-
-## Quantized models
-
-| Format | Repository | Runtime |
-| --- | --- | --- |
-| AWQ INT4 (W4A16) | [Swift-1.5-Qwen3.8-Flash-Next-W4A16-AWQ](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-W4A16-AWQ) | vLLM (`compressed-tensors`) |
-| AutoRound INT4 (W4A16) | [Swift-1.5-Qwen3.8-Flash-Next-W4A16-AutoRound](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-W4A16-AutoRound) | vLLM (`auto-round`) |
-| NVFP4 | [Swift-1.5-Qwen3.8-Flash-Next-NVFP4](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-NVFP4) | NVIDIA Blackwell |
-| GGUF | [Swift-1.5-Qwen3.8-Flash-Next-GGUF](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GGUF) | llama.cpp |
-| GSQ-RCO GGUF (compact 2–3 bit) | [Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF) | llama.cpp |
-
-## License and access
-
-Swift 1.5 Qwen3.8-Flash-Next is a derivative of [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)
-(Copyright (c) 2026 Qwen, [Qwen Community License 1.0](https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next/blob/main/LICENSE-QWEN)).
-UkisAI's contribution, including the adapted weights, is licensed under the
-**[Swift Open License v1.0](https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next/blob/main/LICENSE)**.
-See [NOTICE](https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next/blob/main/NOTICE) for the change notice and
-attribution details.
-
-Personal, research, educational, evaluation, and commercial use of the Swift contribution are free for individuals
-and organizations with gross annual revenue, including affiliates, of up to US$1,000,000. Above that threshold,
-commercial use requires a separate Swift Enterprise License. Contact [UkisAI](https://ukisai.com/contact) for terms.
-
-The base model's terms still apply to it. Under the Qwen Community License 1.0, organizations that run a
-Model-as-a-Service or AI Work Assistant business need a separate license from Qwen before any commercial use,
-and products above 100 million monthly active users or US$20 million monthly revenue must prominently display
-the model name. Nothing in the Swift Open License limits your rights in Qwen3.8-Flash-Next itself under the
-Qwen Community License.
-
-## Citation
-
-~~~bibtex
-@misc{swift-qwen3.8-flash-next,
-  title  = {Swift 1.5 Qwen3.8-Flash-Next},
-  author = {UkisAI},
-  year   = {2026},
-  url    = {https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next}
-}
-~~~
-
-## Acknowledgements
-
-We acknowledge the [NVIDIA Innovation Lab](https://www.nvidia.com/en-us/data-center/innovation-lab/),
-[Amazon Web Services](https://aws.amazon.com/), and
-[Google Cloud](https://cloud.google.com/) for providing compute credits and
-infrastructure support for Swift's development, training, and evaluation.
+# Swift-1.5 Qwen3.8 Flash-Next NVFP4 — FP8 PLE
+
+Production-oriented derivative of:
+
+`ukisai/Swift-1.5-Qwen3.8-Flash-Next-NVFP4`
+
+The model weights remain in their original NVFP4 layout while the large
+PLE embedding table has been converted from BF16 to FP8 E4M3.
+
+## FP8 PLE conversion
+
+Original BF16 PLE:
+
+- 95.37 GiB
+
+Converted FP8 PLE:
+
+- 47.68 GiB
+- FP8 E4M3
+- shared BF16 scale
+
+This substantially reduces the storage and memory footprint of the PLE
+component.
+
+## Validated configuration
+
+Production validation was performed with Pennyroyal / SGLang on an
+NVIDIA RTX PRO 6000 Blackwell.
+
+Configuration:
+
+- Qwen3.8 Flash-Next
+- native NEXTN
+- FR-Spec disabled
+- native context: 262144 tokens
+- FP8 KV cache
+- FP8 PLE
+- NVMe SSD-streamed PLE
+- online MXFP8 enabled at runtime
+- max running requests: 4
+
+The production deployment uses Pennyroyal's prepared NVMe PLE overlay.
+
+The NVMe overlay is **not included** here because it is a runtime-specific,
+regenerable artifact. This repository contains the portable checkpoint
+from which the overlay is created.
+
+Online MXFP8 is also a runtime optimization and is not baked into this
+checkpoint.
+
+## Validation results
+
+Selected measurements on RTX PRO 6000 Blackwell:
+
+- MMLU-Pro: 226 / 280 = 80.71%
+- historical agentic workload: 174.68 effective tok/s
+- fixed 4096 generation median: 133.07 tok/s
+- native 262144-token context: PASS
+- retrieval at ~261.7K input tokens: PASS
+- agentic tool/workflow smoke: 7/7
+
+Performance numbers are runtime- and hardware-specific.
+
+## PLE conversion validation
+
+The FP8 PLE derivative was validated against the source model before
+production deployment.
+
+## Runtime
+
+Validated with Pennyroyal / SGLang on NVIDIA Blackwell.
+
+Other runtimes may require support for the model's NVFP4 checkpoint format
+and FP8 PLE representation.
+
+## License and provenance
+
+This is a derivative of the upstream Swift/Qwen model.
+
+Please review the upstream model card and license files included in the
+repository before redistribution or deployment.