nitinpanj

/

Qwen3.8-27B-Splash-HQ

Like 8

Hi @napoleon-blownapart ,

Thank you for reporting this! You haven't done anything wrong—there are two reasons this occurred:

1. Initial Upload in Progress: The full 27 GB model weights are actively uploading to this repository right now (~35% complete at the moment). Initially, only the tokenizer and manifest metadata were published.
2. Q8 Runtime Format (splash-packed-q8): This model is packed in the native 8-bit splash-packed-q8 (schema version 5) format with custom Metal Q8 tiled decode kernels. Official upstream Splash 1.0 (incoai/splash) only has 4-bit schemas (splash-packed-q4, schema 3/4) hardcoded in its package validator, which causes it to reject schema 5 as an unsupported runtime package.

To run this model once the weight files finish streaming:

• Use the Q8-enabled Splash runtime fork documented here:
  👉 https://github.com/npanj/qwen3.8-27b-apple-silicon-eval

We've updated the model card on this repo with the engine requirements and real-time status. The upload should finish within ~1.5 hours!

Ok, cheers.
Two questions:

1. Will that fork also run the Q8 quant Qwen3.6 model?
2. Can I run the Q4 quant models from this fork or do I need to use the one I installed using brew install incoai/tap/splash?

Hi @napoleon-blownapart ,

Good questions!

1. Q8 on Qwen3.6: Not currently. Qwen3.6 is an MoE architecture (splash-packed-q4-moe, 256 experts routed per token, schema 4 / MDFM0001). The Q8 Metal tiled kernels and packaging format (splash-packed-q8 / MDFL0008) in this fork were built specifically for dense models (Qwen3.8's 64 dense layers). A Q8 MoE kernel would need custom tiled dequantization for the routed expert blocks, which hasn't been implemented yet.
2. Backwards compatibility with Q4: Yes, 100%. The fork is a strict superset of upstream Splash. All original 4-bit dense and MoE schemas (splash-packed-q4, splash-packed-q4-moe) and compiled Metal kernels remain fully intact. You can serve official models (like incoai/Qwen3.8-27B-Splash or incoai/Qwen3.6-35B-A3B-Splash) directly with this fork without needing to switch back to the Homebrew version.