license: other
license_name: deepseek
license_link: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
base_model: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
base_model_relation: quantized
library_name: exllamav3
tags:
- exl3
- trellis
- deepseek_v4
- moe
- mixed-precision
- dgx-spark
DeepSeek-V4-Flash-Vision-Exp — EXL3 MixedK
Sensitivity-allocated mixed-precision EXL3 quantization of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: a K2 base with the most quantization-sensitive MoE layers upgraded to K3. ~95 GiB — fits one DGX Spark.
Strategy
Full K2 base (2.0 bpw experts) with **K3 (3.0 bpw) on layers 21, 22, 13,
41, 28, 3**, selected by a per-layer K2-vs-K3 proxy-error scan. K is
self-describing per tensor via trellis shape, so the pack needs no metadata
changes — kernels dispatch per layer at load.
| Layer | K2->K3 gain |
|---|---|
| 21 | 0.081 |
| 22 | 0.081 |
| 13 | 0.081 |
| 41 | 0.081 |
| 28 | 0.081 |
| 3 | 0.081 |
The landscape is near-flat (~0.3% spread across all 43 layers), so this top-6 is a noise-level ordering of near-ties, not a set of distinct outliers. Whether mixed-K measurably beats uniform K2 is pending PPL/KLD.
Source, format, serving
Identical to the K2 base: mixed-format source (MXFP4 experts + FP8 attention) dequanted to BF16 before trellis (4-bit -> 2-bit double quant on experts); MTP/DSpark drafter kept at source MXFP4 verbatim. vLLM EXL3 overlay, one Spark, text-only serve.
Limitations
Not boot-tested; mixed-format load (EXL3 + MXFP4 drafter) unverified; quality
gain over K2 not yet measured (flat landscape suggests it may be small);
DeepSeek license (other). See the K2 card for full detail.
Related
- K2 base
- k3-delta + evidence
- ablit MixedK — uncensored variant