vcruz305/DSV4.1-Flash-SAGE-EXL3-3.30bpw

DeepSeek-V4.1-Flash — SAGE EXL3 3.30 bpw

Loadable ExLlamaV3 pack of deepseek-ai/DeepSeek-V4.1-Flash.

Quantized by vcruz305 with an internal SAGE allocator: mixed EXL3 K per tensor (not a single uniform K) so bitrate goes where it protects quality, and stays lean where it does not. The method itself is not published. Kernels read K from each trellis tensor at load.

This is the TP2-class sibling of DSV4.1-Flash-EXL3-4.75bpw. Same SAGE method, tighter expert budget.

Status: compiled pack live (40/40 routed-expert layers + native/Engram join). 31 model-*-of-00031 shards on this repo.

SAGE EXL3 3.30 bpw routed-expert quantization

The routed MoE expert bank is dynamically allocated at 3.30 average bpw (K2–K8) by SAGE. Engram remains native FP8 and protected tensors remain at their native precision.

Final packed model size: 446.44 GB / 415.78 GiB (446440212472 bytes).

If total packed bytes are divided by all ~763B model parameters, the resulting model-wide effective storage average is ~4.68 bits/parameter. This is not the EXL3 expert quantization bitrate.

Pack

Item	Value
Format	EXL3 (trellis) + native tables where noted
Routed-expert bpw	3.30 SAGE mixed-K (K2–K8, no K1)
Packed size	446.44 GB / 415.78 GiB
Shards	31 compiled model-*-of-00031 (Engram last two)
Index	model.safetensors.index.json
quant_method	exl3
Routed experts	EXL3, mixed K (SAGE, TP2-class)
Engram	Native FP8 retained (same tables as the 4.75 pack)
Protected non-expert tensors	Copied (attn / shared / DSpark / vision / head), not wholesale-requantized

This is a compiled 8 GiB-class shard pack. It is not a dump of per-expert work files.

What SAGE is (and is not)

SAGE is my internal sensitivity-aware EXL3 recipe for this model. It mixes Ks instead of stamping one K on every expert. Goal is highest quality at a TP2-class weight budget, not a write-up of the search.

Not in this card: the layer list, the scores, the capture/trace, or the allocator code. If a tensor is EXL3, K is self-describing on the trellis.

This is not an identity-Hessian / uncalibrated uniform-K dump labeled as SAGE.

Source model (DeepSeek, not this pack)

V4.1-Flash is DeepSeek’s MIT CED MoE: 40 layers (20 encoder + 20 decoder), 552B backbone + ~196B Engram, 384 routed experts + 1 shared, top-6. Architecture, sampling defaults, and any benchmark rows on the upstream card are DeepSeek’s. They are not scores of this EXL3 pack. I have not claimed them as mine.

Upstream sampling (start here, then tune): temperature=1.0, top_p=0.95.

Load

ExLlamaV3, point at this repo (or a local snapshot):

from exllamav3 import Config, Model, Tokenizer

cfg = Config.from_directory("vcruz305/DSV4.1-Flash-SAGE-EXL3-3.30bpw")
model = Model.from_config(cfg)
model.load()
tokenizer = Tokenizer.from_config(cfg)

Needs a recent ExLlamaV3 with DeepSeek V4.1 / CED support. Multi-GPU via whatever your ExLlamaV3 build exposes (tp, device map). This pack is sized as a TP2-class weight ceiling (~416 GiB on disk), not a promise of a specific serve topology.

Files

• model-*-of-*.safetensors — EXL3 body + copied protected tensors (after join)
• last two shards — Engram (native FP8, hardlinked from upstream 00047 / 00048)
• model.safetensors.index.json, config.json, tokenizer.json, tokenizer_config.json

License

MIT, same as DeepSeek-V4.1-Flash. Cite DeepSeek for the base model. This repository is the EXL3 pack only.

Notes

• Quantization: vcruz305, SAGE mixed-K EXL3, TP2, 3.30 expert-bpw.
• Please do not file “missing experts/ work tree” issues. That is not this repo.