jarrelscy commited on
Commit
f07e0bd
·
verified ·
1 Parent(s): 5f4d3e3

README: pin neutral serving branch serving/arvq-v4-v5 (same commit d7ada6d)

Browse files
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -52,11 +52,13 @@ regression; treat general-English quality accordingly.
52
  ## Serving (vLLM)
53
 
54
  Serve with the SM120 fork `github.com/jarrelscy/vllm-glm52-sm120`, branch
55
- `experiment/arvq-mcbook16`, commit `d7ada6d` or later. That build loads the
56
- v4 cold-expert format `rvq256_256x8_expert_fp16block` used by all PV-replaced
57
- layers here (fitted FP16 block scales serialized directly, applied in FP32
58
- after the FP4 MMA; experimental ABI — rebuild the ARVQ libraries together
59
- with the loader via arvq/build.sh). The config.json `arvq` marker declares
 
 
60
  format version 4 to match. Earlier branches cannot load this repo:
61
  `experiment/arvq-fp16-block-scales` (00f57ca51) registers uint8 E4M3 scales
62
  and predates the v4 format string; the `glm52-sm120` main branch and the v3
 
52
  ## Serving (vLLM)
53
 
54
  Serve with the SM120 fork `github.com/jarrelscy/vllm-glm52-sm120`, branch
55
+ `serving/arvq-v4-v5`, commit `d7ada6d` or later (the same commit is the tip
56
+ of `experiment/arvq-mcbook16`; one serving line loads v3, v4 and v5
57
+ checkpoints). That build loads the v4 cold-expert format
58
+ `rvq256_256x8_expert_fp16block` used by all PV-replaced layers here (fitted
59
+ FP16 block scales serialized directly, applied in FP32 after the FP4 MMA;
60
+ experimental ABI — rebuild the ARVQ libraries together with the loader via
61
+ arvq/build.sh). The config.json `arvq` marker declares
62
  format version 4 to match. Earlier branches cannot load this repo:
63
  `experiment/arvq-fp16-block-scales` (00f57ca51) registers uint8 E4M3 scales
64
  and predates the v4 format string; the `glm52-sm120` main branch and the v3