xv0y5ncu/SmolLM3-3B-trellis-3inst-4bpw-kernel

Text Generation • 1.0B • Updated 28 days ago • 3.95k

Note Trellis (TCQ) 4 bpw - the fastest-decoding SmolLM3.

xv0y5ncu/SmolLM3-3B-GLQ-block-diagonal-3.5bpw

Text Generation • 0.9B • Updated Jun 6 • 199

Note Block-diagonal RHT, true-to-label footprint.

xv0y5ncu/SmolLM3-3B-GLQ-3.5bpw

Text Generation • 1B • Updated 12 days ago • 348

Note Earlier 3.5 bpw quant.

xv0y5ncu/SmolLM3-3B-GLQ-6bpw

Text Generation • 1B • Updated Jun 6 • 11

Note 6 bpw.

xv0y5ncu/SmolLM2-360M-Instruct-GLQ-block-diagonal-4bpw

Text Generation • 0.1B • Updated Jun 6 • 184

Note 360M, block-diagonal.

xv0y5ncu/SmolLM2-360M-Instruct-GLQ-4bpw

Text Generation • 0.2B • Updated 10 days ago • 426

Note 360M, earlier quant.

xv0y5ncu/SmolLM2-135M-Instruct-GLQ-block-diagonal-4bpw

Text Generation • 55.2M • Updated Jun 6 • 197

Note 135M, block-diagonal.

xv0y5ncu/SmolLM2-135M-Instruct-GLQ-4bpw

Text Generation • 95.7M • Updated 12 days ago • 295

Note 135M, earlier quant.

xv0y5ncu/SmolLM3-3B-trellis-3inst-5bpw

1B • Updated Aug 2 • 13

Note Trellis 5 bpw (stacked RVQ 4+1). wikitext2 PPL 9.1402 vs bf16 9.1220. Needs glq >= 0.8.0.

xv0y5ncu/SmolLM3-3B-trellis-3inst-6bpw

1B • Updated Aug 1 • 16

Note Trellis 6 bpw (stacked RVQ 4+2). wikitext2 PPL 9.1310 vs bf16 9.1220 - the closest to bf16 of the ladder. Needs glq >= 0.8.0.

xv0y5ncu/SmolLM3-3B-trellis-3inst-3bpw

Text Generation • 0.8B • Updated 25 days ago • 170

Note Trellis (3INST) 3 bpw - SmolLM3 ladder rung

xv0y5ncu/SmolLM2-360M-Instruct-GLQ-trellis-3inst-6bpw

0.2B • Updated 10 days ago • 11

Note 0.31 GiB · wikitext-2 PPL +0.16% vs bf16

xv0y5ncu/SmolLM2-360M-Instruct-GLQ-trellis-3inst-5bpw

0.1B • Updated 10 days ago • 7

Note 0.27 GiB · PPL +0.78% vs bf16

xv0y5ncu/SmolLM2-360M-Instruct-GLQ-trellis-3inst-4bpw

0.1B • Updated 10 days ago • 5

Note 0.24 GiB · PPL +2.7% vs bf16

xv0y5ncu/SmolLM2-360M-Instruct-GLQ-trellis-3inst-3bpw

0.1B • Updated 10 days ago • 5

Note 0.20 GiB · PPL +11.3% vs bf16 · fastest decode of the ladder