Daiqing commited on
Commit
023bf8b
·
verified ·
1 Parent(s): 16be4ce

Add SparkLab model card

Browse files

Document provenance, native MXFP4/MXFP8 execution, DSpark-5 benchmark results, and the validated DGX Spark command.

Files changed (1) hide show
  1. README.md +109 -0
README.md ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ base_model: deepseek-ai/DeepSeek-V4.1-Flash
4
+ license: mit
5
+ library_name: sparklab
6
+ tags:
7
+ - deepseek-v4.1
8
+ - mxfp4
9
+ - mxfp8
10
+ - sparklab
11
+ - ftw
12
+ - speculative-decoding
13
+ ---
14
+
15
+ # DeepSeek V4.1 Flash — SparkLab disk-ready checkpoint
16
+
17
+ This repository is a byte-preserving mirror of
18
+ [`deepseek-ai/DeepSeek-V4.1-Flash`](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash)
19
+ at revision `df42c109f1defefcbfcedbe7d905718a12266e40`, published for the
20
+ [SparkLab](https://github.com/sixteen-miles-labs/sparklab) native NVIDIA DGX Spark
21
+ research path.
22
+
23
+ The tensors retain DeepSeek's original mixed checkpoint representation: routed experts
24
+ are MXFP4, dense projections are MXFP8 with UE8M0 scales, and embeddings and the output
25
+ head are BF16. No training, tensor conversion, or additional quantization was performed.
26
+ Although the repository uses the `FTW` deployment label, DeepSeek V4.1 currently runs
27
+ through SparkLab's model-owned safetensors reader and packed expert cache rather than the
28
+ generic FTW container format.
29
+
30
+ ## Validated scope
31
+
32
+ SparkLab currently supports text-only, TP=1, batch-one eager execution with a 2,048-token
33
+ total context on one 128 GB NVIDIA GB10. The model owns a 64 GiB packed expert LRU and a
34
+ bounded 24 GiB weight cache, reading the remaining checkpoint from local NVMe.
35
+
36
+ The default target-only profile measured 0.969 decode tokens/s and 74.387 seconds warm
37
+ TTFT on a fixed 74-input/128-output greedy probe. Opt-in native greedy DSpark-5 measured
38
+ 1.053 decode tokens/s and 76.329 seconds warm TTFT, an 8.6% decode improvement. Both
39
+ DSpark trials reproduced the target-only output hash. These are narrow performance
40
+ measurements, not general quality, concurrency, long-context, or endurance certification.
41
+
42
+ The upstream vLLM recipe uses probabilistic draft sampling, block rejection, and adaptive
43
+ verification. SparkLab's current V4.1 path implements greedy drafting and exact
44
+ accepted-prefix state commits; probabilistic sampling and adaptive verification remain
45
+ out of scope.
46
+
47
+ ## Download and run
48
+
49
+ The repository is approximately 480 GiB. Download it to local NVMe:
50
+
51
+ ```bash
52
+ hf download oakmindai/DeepSeek-V4.1-Flash-FTW \
53
+ --local-dir ~/models/DeepSeek-V4.1-Flash-FTW
54
+ ```
55
+
56
+ Install SparkLab from source, then start the target-only server:
57
+
58
+ ```bash
59
+ git clone https://github.com/sixteen-miles-labs/sparklab.git
60
+ cd sparklab
61
+ ./install.sh
62
+
63
+ SPARKLAB_DSV41_EXPERT_CACHE_GB=64 sparklab serve \
64
+ --model ~/models/DeepSeek-V4.1-Flash-FTW \
65
+ --dtype bfloat16 \
66
+ --max-running-requests 1 \
67
+ --max-seq-len-override 2048 \
68
+ --num-tokens 2048 \
69
+ --attention-backend triton \
70
+ --cache-type naive \
71
+ --moe-backend fused \
72
+ --cuda-graph-max-bs 0 \
73
+ --disable-startup-prefill-warmup \
74
+ --port 8000
75
+ ```
76
+
77
+ Enable the measured DSpark profile by adding:
78
+
79
+ ```bash
80
+ --speculative-method dspark \
81
+ --speculative-tokens 5 \
82
+ --draft-sample-method greedy
83
+ ```
84
+
85
+ Send a request:
86
+
87
+ ```bash
88
+ curl http://127.0.0.1:8000/v1/chat/completions \
89
+ -H 'Content-Type: application/json' \
90
+ -d '{
91
+ "model": "DeepSeek-V4.1-Flash-FTW",
92
+ "messages": [{"role": "user", "content": "What is 17*19?"}],
93
+ "temperature": 0,
94
+ "max_tokens": 32
95
+ }'
96
+ ```
97
+
98
+ ## Provenance and license
99
+
100
+ DeepSeek AI developed and released the architecture, code, tokenizer, and weights. This
101
+ mirror preserves the upstream MIT license and source files. SparkLab supplies the native
102
+ runtime, direct packed MXFP4/MXFP8 kernels, bounded disk-backed execution, DSpark
103
+ integration, validation, and model recipe. Oakmind AI publishes this mirror.
104
+
105
+ - Upstream model: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
106
+ - Exact upstream revision: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/tree/df42c109f1defefcbfcedbe7d905718a12266e40
107
+ - SparkLab: https://github.com/sixteen-miles-labs/sparklab
108
+ - vLLM architecture recipe: https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4.1-Flash
109
+