Hakureirm/rwkv7-sglang-w4rtn-1.5b

RWKV-7 G1 1.5B — int4 RTN for rwkv-sglang

Hand-written weight-only int4 (calibration-free RTN, symmetric, group 64) quantization of BlinkDL's RWKV-7 "Goose" G1 1.5B, for the rwkv-sglang serving overlay. Same on-disk format and same serving kernel as the GPTQ sibling — the two run at the same speed; they differ only in accuracy (RTN is the calibration-free baseline, GPTQ recovers part of the loss).

• Accuracy (perplexity-style): lambada −4.95pt vs fp16. The GPTQ sibling (rwkv7-sglang-w4gptq-1.5b) recovers +1.6pt over this checkpoint and is the recommended int4 variant at every size.
• Speed: identical to the GPTQ sibling (same kernel, same format): faster than fp16 at every batch size ≤ 32 on an RTX 3090 (1.03–1.56× decode), JIT kernels Turing→Blackwell.
• VRAM: checkpoint 1.2 GB vs 2.9 GB fp16 (~2.4×).

⚠️ Accuracy warning — multi-step reasoning (read before use)

Perplexity-style metrics understate int4's damage to multi-step reasoning at this model size. Even the better-calibrated GPTQ sibling collapses on MATH500 (avg@64, 32,000 rollouts): 14.98% vs fp16's 40.60% (−25.6pt), losing the thread mid-derivation and running to the token cap. RTN is strictly weaker than GPTQ on every metric where both were measured at this size, so expect at least that much damage here.

Recommendation: treat this checkpoint as a memory-footprint tool for non-reasoning workloads only. For reasoning-heavy use at 1.5B, use the int8 w8g64 tier instead (greedy-exact, no measurable accuracy cost) — int4 is not the lossless tier at this size.

Where this fits

This checkpoint loads through the rwkv-sglang overlay. Native RWKV-7 support is being upstreamed into SGLang — see sgl-project/sglang#30115; once that lands the overlay is no longer required.

If you want RWKV-7 without quantization or a custom runtime, the same base weights are also published in standard HuggingFace layout — plain safetensors, ordinary config.json, no trust_remote_code:

0.1B	rwkv7-0.1b-hf
0.4B	rwkv7-0.4b-hf
1.5B	rwkv7-1.5b-hf
2.9B	rwkv7-2.9b-hf
7.2B (G1)	rwkv7-g1h-7.2b-hf
Pile 168M	rwkv7-168m-pile-hf

Transformers support for the architecture itself is open as huggingface/transformers#47780.

Format & loading (important)

Not a drop-in HuggingFace checkpoint. Weights are group-wise (GROUP=64) symmetric int4 (.qweight + .scale); they load only through the rwkv-sglang overlay:

bash scripts/deploy.sh   # from github.com/Hakureirm/rwkv-sglang, built against sglang v0.5.10.post1 — newer releases need the overlay rebased
RWKV_W4=1 python -m sglang.launch_server --model-path <this-dir> --dtype float16 \
  --trust-remote-code --disable-radix-cache

LoRA/norm/embedding/head stay full precision. Base model © BlinkDL (Bo Peng), Apache-2.0.