DeepSeek-V4-Flash-Abliterated-DSpark

Abliterated DeepSeek-V4-Flash with DSpark speculative decoding. Faster than MTP on 2-4x RTX Pro 6000. By Fraser Price.

The abliterated FP8 DeepSeek-V4-Flash with DeepSeek's DSpark draft module bolted on. DSpark (from DeepSpec) is block-wise speculative decoding: it drafts several tokens per step through a small Markov head, so single-stream decode is ~1.2-1.4x faster than the model's stock MTP at identical quality (speculative decoding is lossless). It runs on a Blackwell / CUDA-13.2 vLLM image that ships the dspark speculative method (voipmonitor/vllm:eldritch-enlightenment-v2226f26-b12x15cd38c-cu132-20260629), the build from local-inference-lab/rtx6kpro.

Update

The serving image has moved from the original fraserpricee/vllm:dspark-cu132-20260627 to the local-inference-lab vLLM build (voipmonitor/vllm:eldritch-enlightenment-v2226f26-b12x15cd38c-cu132-20260629) — a more performant DSpark + sparse-MLA implementation on Blackwell. All credits to them! The performance numbers below were measured on this V2 build, weights are the same.

Related models

Repo	What
DeepSeek-V4-Flash-FP8-Abliterated	abliterated base, stock MTP (no DSpark)
DeepSeek-V4-Flash-Abliterated-DSpark	this repo (abliterated + DSpark)
DeepSeek-V4-Flash-DSpark	stock DeepSeek-V4-Flash + DSpark

Quick start

Needs Docker with the NVIDIA Container Toolkit and 2-4 GPUs (see Hardware).

curl -fsSL https://huggingface.co/fraserprice/DeepSeek-V4-Flash-Abliterated-DSpark/resolve/main/run.sh -o run.sh && bash run.sh

That one command grabs the launcher, downloads the weights (~167 GB) into your Hugging Face cache, pulls the inference image, and serves an OpenAI-compatible API on http://localhost:8000/v1. No other files or arguments needed. On 4 GPUs:

GPUS=0,1,2,3 TP=4 bash run.sh

Everything is overridable by env var:

Variable	Default	Meaning
GPUS	0,1	GPU indices (comma-separated)
TP	2	Tensor-parallel size; set to the number of GPUs
DSPARK_TOKENS	5	DSpark draft tokens per step
PORT	8000	API port
MAX_MODEL_LEN	262144	Context length
GPU_MEM_UTIL	0.92	Fraction of VRAM to use
HF_REPO	fraserprice/DeepSeek-V4-Flash-Abliterated-DSpark	repo to download
MODEL_DIR	(HF cache)	serve a local dir instead of downloading
SERVED_NAME	DeepSeek-V4-Flash-Abliterated-DSpark	API model name
IMAGE	(pinned DSpark image)	inference container

Hardware

Tuned for NVIDIA RTX Pro 6000 Blackwell (96 GB, sm_120). Needs roughly 180 GB total VRAM for weights + DSpark module + KV cache at long context, split across GPUs by tensor parallelism.

Setup	Works?
2 x RTX Pro 6000 (96 GB), TP=2	✅ reference config
4 x RTX Pro 6000, TP=4	✅ more KV headroom / throughput
< ~180 GB total VRAM	❌ won't fit

The serving image targets Blackwell (sm_120) / CUDA 13.2 and adds the dspark speculative method to vLLM. See local-inference-lab/rtx6kpro for the build source.

Performance

Measured on RTX Pro 6000 Blackwell (PCIe 5.0), FP8 weights + FP8 KV cache, gpu_memory_utilization=0.92, DSpark num_speculative_tokens=5, 256 generated tokens/request. PP is prefill throughput (tok/s); TG is per-request decode throughput (tok/s); total is aggregate throughput (tok/s) across concurrent requests. MTP TG is the base model's shipped single-layer-MTP decode for reference; Decode ↑ is DSpark's gain over it.

Latency

GPUs	prompt	conc	TTFT p50	ITL p50	E2E p50
2	1,000	1	155 ms	15.6 ms	1.34 s
2	1,000	3	435 ms	23.1 ms	2.04 s
2	10,000	1	1,232 ms	15.9 ms	2.32 s
2	10,000	3	2,495 ms	23.8 ms	5.30 s
2	100,000	1	14,568 ms	16.4 ms	15.71 s
2	100,000	3	29,078 ms	26.1 ms	44.77 s
4	1,000	1	127 ms	11.9 ms	0.96 s
4	1,000	3	259 ms	16.3 ms	1.52 s
4	10,000	1	983 ms	12.0 ms	1.87 s
4	10,000	3	2,229 ms	16.9 ms	4.06 s
4	100,000	1	12,772 ms	12.5 ms	13.59 s
4	100,000	3	26,698 ms	18.9 ms	40.11 s

Throughput (tok/s)

GPUs	prompt	conc	PP	MTP TG	DSpark TG	Decode ↑	total
2	1,000	1	8,080	190.9	224.7	1.18×	1,160
2	1,000	3	8,377	138.4	152.5	1.10×	2,120
2	10,000	1	10,081	191.5	239.5	1.25×	5,469
2	10,000	3	10,050	90.3	106.6	1.18×	7,046
2	100,000	1	8,525	185.4	247.1	1.33×	7,965
2	100,000	3	8,585	60.4	67.1	1.11×	8,320
4	1,000	1	9,769	233.3	322.7	1.38×	1,605
4	1,000	3	10,578	181.9	212.8	1.17×	2,927
4	10,000	1	12,645	234.5	314.3	1.34×	6,984
4	10,000	3	12,642	115.0	148.3	1.29×	9,122
4	100,000	1	9,734	232.8	335.8	1.44×	9,194
4	100,000	3	9,507	74.7	92.7	1.24×	9,278

~1.2-1.4x faster single-stream decode than the base model's shipped MTP, at the same (lossless) quality.

Credits

• deepseek-ai/DeepSeek-V4-Flash: the base model (MIT).
• DeepSeek-AI / DeepSpec and DeepSeek-V4-Flash-DSpark: the DSpark technique and draft weights.
• huihui-ai: the abliteration the refusal directions were derived from (MIT).
• local-inference-lab/rtx6kpro: the Blackwell / CUDA-13.2 vLLM build (voipmonitor/vllm:eldritch-…) that serves this model, with the dspark speculative method and B12X sparse-MLA stack.
• voipmonitor/vllm: the vLLM image registry the image is published to.

Disclaimer

Uncensored model: the refusal subspace has been removed, so it will not decline requests the way the base model does. Released as-is, for research and lawful use only.

• Not safety-aligned. Outputs may be inaccurate, offensive, or harmful. Apply your own filtering and safeguards.
• No warranty. Provided "as is", without warranty of any kind. You use it entirely at your own risk.
• You are responsible for what you generate. The author does not endorse or accept liability for any unlawful, harmful, or unethical use.

License

MIT, inheriting from the base model and the abliteration source.

---

Built by Fraser Price, @fraserpricee. Found this useful? A follow is appreciated.