---
base_model: upstage/Solar-Open2-250B
license: other
license_name: upstage-solar-license
license_link: LICENSE
language:
- en
- ko
- ja
pipeline_tag: text-generation
tags:
- gguf
- solar
- moe
- mixed-precision
- text-generation
- dgx-spark
---
# Solar Open2 250B Mixed-Quant GGUF
**Built with Solar.**
This repository contains the locked `MXQ-v1` candidate derived from
[`upstage/Solar-Open2-250B`](https://huggingface.co/upstage/Solar-Open2-250B).
It preserves the complete topology: 48 layers, all 320 routed experts per
layer, one shared expert, and the original `[GQA, KDA, KDA, KDA] × 12`
schedule. No expert, layer, or parameter pruning, merging, or distillation was
performed. This is an independent conversion and is not an official Upstage
release.
## Support my work
I work on making large language models practical on hardware they were never really designed to fit on — through mixed quantization, inference optimization, custom kernels, and serving experiments.
While much of the development happens on local hardware, calibration, profiling, and large-scale validation often require expensive on-demand GPUs.
Contributions help pay for that compute, storage, and testing infrastructure, so I can keep experimenting and publishing the results openly.
## Artifact
| Variant | Split | Size | Approximate BPW |
|---|---:|---:|---:|
| MXQ-v1 | 11 shards | 95,533,532,160 bytes (88.973 GiB) | 3.05 |
Download the complete 11-file split beginning with
`Solar-Open2-250B-MXQ-v1-00001-of-00011.gguf`. Exact hashes are in
`MXQ-v1-SHA256SUMS`.
The full BF16 and full-model Q8_0 reference conversions are published
separately in
[`Baekpica/Solar-Open2-250B-GGUF`](https://huggingface.co/Baekpica/Solar-Open2-250B-GGUF).
## Precision recipe
| Module family | Precision |
|---|---|
| Token embedding, LM head | Q8_0 |
| GQA q/k/v/o/g projections | Q8_0 |
| KDA matrix projections | Q8_0 |
| Shared expert gate/up/down | Q8_0 |
| Routed gate/up, layers 4–43 | IQ2_XXS + imatrix |
| Routed down, layers 4–43 | Q3_K + imatrix |
| Routed gate/up/down, layers 0–3 and 44–47 | Q4_K + imatrix |
| Router, norms, small KDA control/norm/conv tensors | F32 |
The final GGUF contains 1,083 tensors: 409 F32, 530 Q8_0, 80 IQ2_XXS,
40 Q3_K, and 24 Q4_K tensors. `quant-recipe-v1.yaml` and
`verify-v1.json` contain the machine-readable recipe and verification result.
## Calibration
The Solar chat-template calibration corpus follows the published mix from
`Baekpica/Solar-Open2-120B-A15B-REAM-148E-Healing-Mix`: instruction chat 22%,
Cascade stage-1 reasoning 16%, Cascade stage-2 reasoning 16%, Korean 16%,
other multilingual 12%, finance 6%, SWE 6%, and algorithmic code 6%.
The imatrix processed 590 × 512-token chunks. Layers 1–47 reached strict
320/320 routed-expert coverage. Layer 0 retained one explicit, pinned
exception of 29 zero-activation experts; the exact IDs and gate are recorded
in `verify-imatrix.json`. No precision was lowered to compensate for this
exception.
## Validation status
- All 11 shards passed split ordering, bounds, topology, tensor count, tensor
family, and exact tensor-type verification with zero errors.
- The artifact loaded resident across four H100 NVL GPUs and completed 32
deterministic short generations plus eight longer no-think quality checks.
- Every candidate output was non-empty with valid token IDs. Fatal/content
checks passed for Korean, English, Japanese, code, arithmetic, exact JSON,
and constrained-list prompts; no NaN or malformed structured output was
observed.
- A development ds4 CUDA runtime mapped the complete artifact on one H100 and
passed full-weight, recurrent-state snapshot/restore, replay, and cold
invalidation regressions.
- A native `sm_121a` ds4 build serves the complete artifact on one DGX Spark
GB10 through the OpenAI-compatible server at `-c 196608` with three
persistent banks. See [DGX Spark](#dgx-spark) for the measured point.
H100 timings are development and artifact-sanity evidence only. Do not
present them as DGX Spark performance. 1,048,576-token serving on Spark is
**not** claimed.
## DGX Spark
One NVIDIA DGX Spark GB10 (128 GB unified memory, driver 610.43.02, CUDA
13.3, `sm_121a`). Runtime
[`Baekpica/ds4@b2e52b9`](https://github.com/Baekpica/ds4/commit/b2e52b9048ba339327539212de1c47d009dde126)
on the `dfm` line. Artifact: this MXQ-v1 11-shard split. A long-lived VMM
weight owner (`--backend vmm --scope base --reserve-gb 16`, 453 derived
aligned artifacts) plus a restartable `ds4-server` worker:
```sh
DS4_CUDA_WEIGHT_IPC_MANIFEST="$RUN/weights.manifest" \
DS4_CUDA_WEIGHT_IPC_SCOPE=base \
DS4_SERVER_COALESCE_MAX=3 \
DS4_SERVER_COALESCE_MAX_TOKENS=4096 \
DS4_CONT_PREFILL_CHUNK=4096 \
DS4_METAL_PREFILL_CHUNK=4096 \
./ds4-server -m Solar-Open2-250B-MXQ-v1-00001-of-00011.gguf \
--cuda -c 196608 -n 1024 -t 6 \
--host 0.0.0.0 --port 8002 --model-id solar-open2-250b \
--no-spec --no-update-check --mem-floor-gb 8
```
OpenAI Chat Completions, thinking disabled, exact-cold (`cached_tokens=0`),
128 decode tokens, median of three. Loaded SM clocks 2,411–2,561 MHz.
`banks_total=3` still admitted.
| Depth | Prompt tokens | Prefill | Decode p50 | Decode API |
|---|---:|---:|---:|---:|
| 8K | 8,222 | 1,050.7 tok/s | 19.05 tok/s | 18.9 tok/s |
| 64K | 66,761 | 804.5 tok/s | 13.07 tok/s | 14.1 tok/s |
On the same host and artifact, before the `dfm` default-path series that
ends at `b2e52b9`, 8K decode was 17.5 tok/s and 64K average prefill was
710 tok/s. The series kept the production path as: skip width-1 inner
CUDA graph capture; walk FATTN KV in 32-key then GQA-pair 64-key tiles;
keep the KDA 64 KiB state tile in shared memory; tile grouped GQA decode
KV in 32 keys with packed K-FP8/V-FP4 dequant.
`test-solar-kv` on this binary: 512-token GQA2 vs one-head `rel_rms=0`;
split vs direct `rel_rms=8.45e-7`. Incremental `T(64K)−T(60K)` last-4K is
not a published figure. Native 1,048,576-token metadata is not a measured
Spark pass.
### Partial prefix reuse
Runtime
[`Baekpica/ds4@042bcea`](https://github.com/Baekpica/ds4/commit/042bcea5302bff775d0a508cec643f1f06754df9)
adds a bounded, shared Solar KDA checkpoint pool to the continuous lane.
The 32 slots are demand-mapped, exact forks share references, and long
prefills retain roughly 24 periodic checkpoints across the configured
context in addition to request boundaries. A divergent prompt restores the
nearest safe checkpoint and replays only the gap.
One controlled A/B run used the same GB10, artifact, four-bank worker,
`-c 65536`, 4,096-token prefill chunks, and greedy no-think Chat requests.
The control set `DS4_SERVER_FORK_PARTIAL=0`, which also disables checkpoint
reservation. The treatment used the default.
| Request | Prompt | Control cached | Treatment cached | Control TTFT | Treatment TTFT | Speedup |
|---|---:|---:|---:|---:|---:|---:|
| Source | 12,123 | 0 | 0 | 10,654.0 ms | 10,710.7 ms | 0.995× |
| Branch at ~6K | 6,079 | 0 | 4,096 | 5,323.7 ms | 1,868.8 ms | 2.85× |
| Branch at ~10K | 10,148 | 0 | 8,192 | 8,920.2 ms | 1,929.8 ms | 4.62× |
Both branch responses were byte-identical between control and treatment.
Treatment reported two partial-fork admits, 12,288 cached tokens, zero failed
requests, and zero admission rejects. Source checkpoint capture added 56.7 ms
(0.53%) in this single run. After the three requests, device-live memory was
102.04 GiB and system-available memory was 14.59 GiB with an 8 GiB floor.
This is a bounded worker-local cache, not an unbounded radix tree. Token
history and GQA prefix rows remain anchored to a retained bank, and the KDA
checkpoint pool is not serialized into disk-KV payloads.
## Provenance
- Source: `upstage/Solar-Open2-250B@9190fbe63a2ad8e17fc766ccceb36de7c66f004b`
- Official implementation oracle:
`upstageAI/transformers@ef5c37485091057070b36dd0620798f74988eb01`
- llama.cpp conversion base:
`ggml-org/llama.cpp@6ea215d171fd31df943bf1ac8227129f2b963160`
- Solar conversion/runtime patch SHA-256:
`7c267850de7a08af66a9bd64e9d4a3b33e085a4d2888d94a4c3d919f87283e44`
- Reproduction repository:
[`Baekpica/solar-open2-mixed-ds4`](https://github.com/Baekpica/solar-open2-mixed-ds4)
Current upstream runtimes may not yet recognize the `solar-open2` GGUF
architecture. Use a compatible Solar Open 2 llama.cpp port or the linked ds4
development branch; compatibility claims should be tied to an exact runtime
revision.
## License and attribution
The source model and this derivative artifact are distributed under the
included Upstage Solar License. The model name begins with `Solar`, and this
card prominently states “Built with Solar” as required by that license.