FrogBoss-35B-Flash / README.md
SofiTesfay2010's picture
Publish mixed STQ/FP8 checkpoint and retire old shards
e39eed9 verified
|
Raw
History Blame Contribute Delete
1.04 kB
---
library_name: pytorch
base_model: microsoft/FrogBoss-32B-2510
tags:
- quantized
- custom-code
- safetensors
---
# Mixed STQ / FP8 checkpoint
Source: `microsoft/FrogBoss-32B-2510`
Source revision: `cc930952b8751de86a2a44debaf26e45e677a291`
MLP projections use custom Sherry-style STQ:
42 bytes per 256 weights, with unit importance weights.
Attention projections, embeddings, and the output head use scaled E4M3FN
FP8 with FP32 per-row scales. Vectors and scalars use FP16.
Packed checkpoint size: 11.731 GB.
This is NOT a verified Tencent serialization format. It requires the
matching mixed-stq-fp8-v2 Colab loader and cannot be loaded through ordinary
Transformers `from_pretrained()`.
No activation calibration, recovery training, or quality evaluation has
been performed. Meaningful generation and useful accuracy are not
guaranteed.
Repository naming does not change the source model's parameter count.
Retain the accompanying Colab notebook as the custom runtime.
Review the source model's license before redistribution.