0xSero

/

DeepSeek-V4-Flash-0731-REAP

like 35

Request: a K128 (128-experts) REAP variant for single 96GB GPUs

First of all, thank you for the excellent K160 REAP checkpoint and the very thorough structural validation + manifest. This is exactly the kind of work that makes DeepSeek-V4 usable on memory-constrained hardware.

I'd like to request a K128 (128 routed experts per MoE scope) variant of the same pipeline.

Why: My setup is a single NVIDIA RTX PRO 6000 Blackwell (96 GB GDDR7). The K160 checkpoint's tensor payload is 100.4 GiB (107.8 GB), which is just above what a 96 GB card can hold — roughly 4 GiB over, so it can't run on single-GPU hardware. A K128 variant would cut the routed experts from 160 → 128 (another ~20% of the remaining experts), and by linear scaling of the expert tensor volume the payload should land around **84-85 GiB**, which fits comfortably in 96 GB with room left for a short/medium KV cache (plus FP8 KV cache and small context windows).

This would unlock single-GPU (96 GB) inference of DeepSeek-V4-Flash-0731 on workstation-class Blackwell hardware — a big deal for people who can't afford 2× GPU setups or a DGX Spark.

A few questions / notes if you do decide to build it:

1. Would the existing REAP observation data (from the previous checkpoint revision) transfer as cleanly to a K128 cut as it did for K160? The router-alignment cosine values you published (worst same-index 0.9923) suggest yes, but a K128 cut removes more experts, so the nearest-retained-expert matching might need care.
2. For the three hash-routed layers: with more experts removed, the cosine-similarity remap to retained rows should still work, but it'd be great to see the same alignment stats published for K128.
3. If you're able to also keep the 3 DSpark MTP blocks pruned consistently (as in K160), that would keep speculative decoding working on the smaller variant.

Quality trade-off is understood and acceptable for my use case (coding + agentic tool calls, not production). Even a "structural validation + smoke test only" release like the K160 would be very welcome.

Thanks again for the great work — and for anyone else reading: a K128 build would make this model runnable on single 96 GB Blackwell cards (RTX PRO 6000, and likely L20/RTX 6000 Ada-class via offload).

Best regards

Have you ever found a REAP to be more usable than the best-available smaller model that fits?