| --- |
| license: other |
| license_name: deepseek |
| license_link: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp |
| base_model: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp |
| base_model_relation: quantized |
| library_name: exllamav3 |
| tags: [exl3, trellis, deepseek_v4, moe, mixed-precision, dgx-spark] |
| --- |
| |
| # DeepSeek-V4-Flash-Vision-Exp — EXL3 MixedK |
|
|
| Sensitivity-allocated mixed-precision EXL3 quantization of |
| [deepseek-ai/DeepSeek-V4-Flash-Vision-Exp](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp): |
| a K2 base with the most quantization-sensitive MoE layers upgraded to K3. |
| ~95 GiB — fits one DGX Spark. |
|
|
| ## Strategy |
| Full K2 base (~2.0 bpw experts) with **K3 (~3.0 bpw) on layers 21, 22, 13, |
| 41, 28, 3**, selected by a per-layer K2-vs-K3 proxy-error scan. K is |
| self-describing per tensor via trellis shape, so the pack needs no metadata |
| changes — kernels dispatch per layer at load. |
|
|
| | Layer | K2->K3 gain | |
| |---|---| |
| | 21 | 0.081 | |
| | 22 | 0.081 | |
| | 13 | 0.081 | |
| | 41 | 0.081 | |
| | 28 | 0.081 | |
| | 3 | 0.081 | |
|
|
| The landscape is **near-flat** (~0.3% spread across all 43 layers), so this |
| top-6 is a noise-level ordering of near-ties, not a set of distinct outliers. |
| Whether mixed-K measurably beats uniform K2 is pending PPL/KLD. |
|
|
| ## Source, format, serving |
| Identical to the [K2 base](https://huggingface.co/vcruz305/DSV4-Flash-Vision-EXL3-K2): |
| mixed-format source (MXFP4 experts + FP8 attention) dequanted to BF16 before |
| trellis (4-bit -> 2-bit double quant on experts); MTP/DSpark drafter kept at |
| source MXFP4 verbatim. vLLM EXL3 overlay, one Spark, text-only serve. |
|
|
| ## Limitations |
| Not boot-tested; mixed-format load (EXL3 + MXFP4 drafter) unverified; quality |
| gain over K2 not yet measured (flat landscape suggests it may be small); |
| DeepSeek license (`other`). See the K2 card for full detail. |
|
|
| ## Related |
| - [K2 base](https://huggingface.co/vcruz305/DSV4-Flash-Vision-EXL3-K2) |
| - [k3-delta + evidence](https://huggingface.co/vcruz305/dsv4-flash-vision-exl3-k3-delta) |
| - [ablit MixedK](https://huggingface.co/vcruz305/DSV4-Flash-Vision-ablit-EXL3-MixedK) — uncensored variant |
|
|
