aquaman164 commited on
Commit
8ebb9dd
·
verified ·
1 Parent(s): 024fb62

Pin OneComp serving plugin to immutable tag nex-serving-v1 (was mutable main); content identical, repro-stable

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -31,7 +31,7 @@ serves on a single workstation with **2×96 GB GPUs**.
31
 
32
  > ⚠️ **This checkpoint uses a custom mixed-bit MoE format and does NOT load with
33
  > stock vLLM / Transformers / llama.cpp.** Serving requires the mixed-bit MoE plugin
34
- > ([**mmzz164/OneCompression**](https://github.com/mmzz164/OneCompression/tree/main/vllm_plugins/gptq),
35
  > MIT) — see *How to serve*. If you want a plug-and-play quant, use one of the standard
36
  > GGUF/AWQ builds instead.
37
 
@@ -110,10 +110,10 @@ This is a **custom mixed-bit MoE checkpoint**. Stock loaders cannot read it. To
110
  quantization/serving base.
111
  2. Add the **mixed-bit grouped fused-MoE serving plugin** (the part that actually loads
112
  and runs per-expert 2/3/4/8-bit MoE — not in upstream) from
113
- **[mmzz164/OneCompression](https://github.com/mmzz164/OneCompression/tree/main/vllm_plugins/gptq)**:
114
  `mixed_moe.py` (the `mixed_gptq` `FusedMoEMethodBase`), plus `grouped_moe.py` and
115
  `fused_dq_gemm.py` (the Triton kernels). See
116
- [`MIXED_BIT_MOE_SERVING.md`](https://github.com/mmzz164/OneCompression/blob/main/vllm_plugins/gptq/MIXED_BIT_MOE_SERVING.md).
117
  3. Serve with vLLM using pipeline-parallel across your two GPUs and the `mixed_gptq`
118
  quantization, e.g. (illustrative):
119
  ```bash
@@ -160,7 +160,7 @@ model nex-agi/Nex-N2-Pro; only the weight precision was changed.
160
  - **Fujitsu Research** for [OneComp / OneCompression](https://github.com/FujitsuResearch/OneCompression)
161
  (MIT), including the AutoBit allocator and QEP.
162
  - Mixed-bit MoE serving kernels:
163
- [mmzz164/OneCompression](https://github.com/mmzz164/OneCompression/tree/main/vllm_plugins/gptq) (MIT).
164
 
165
  ## Citation
166
 
 
31
 
32
  > ⚠️ **This checkpoint uses a custom mixed-bit MoE format and does NOT load with
33
  > stock vLLM / Transformers / llama.cpp.** Serving requires the mixed-bit MoE plugin
34
+ > ([**mmzz164/OneCompression**](https://github.com/mmzz164/OneCompression/tree/nex-serving-v1/vllm_plugins/gptq),
35
  > MIT) — see *How to serve*. If you want a plug-and-play quant, use one of the standard
36
  > GGUF/AWQ builds instead.
37
 
 
110
  quantization/serving base.
111
  2. Add the **mixed-bit grouped fused-MoE serving plugin** (the part that actually loads
112
  and runs per-expert 2/3/4/8-bit MoE — not in upstream) from
113
+ **[mmzz164/OneCompression](https://github.com/mmzz164/OneCompression/tree/nex-serving-v1/vllm_plugins/gptq)**:
114
  `mixed_moe.py` (the `mixed_gptq` `FusedMoEMethodBase`), plus `grouped_moe.py` and
115
  `fused_dq_gemm.py` (the Triton kernels). See
116
+ [`MIXED_BIT_MOE_SERVING.md`](https://github.com/mmzz164/OneCompression/blob/nex-serving-v1/vllm_plugins/gptq/MIXED_BIT_MOE_SERVING.md).
117
  3. Serve with vLLM using pipeline-parallel across your two GPUs and the `mixed_gptq`
118
  quantization, e.g. (illustrative):
119
  ```bash
 
160
  - **Fujitsu Research** for [OneComp / OneCompression](https://github.com/FujitsuResearch/OneCompression)
161
  (MIT), including the AutoBit allocator and QEP.
162
  - Mixed-bit MoE serving kernels:
163
+ [mmzz164/OneCompression](https://github.com/mmzz164/OneCompression/tree/nex-serving-v1/vllm_plugins/gptq) (MIT).
164
 
165
  ## Citation
166