jarrelscy's picture
Publish quantization/data reproduction source and correct recipe documentation
cb95231 verified
|
Raw History Blame Contribute Delete
917 Bytes
# ARVQ-v2 methodology correction
The previous file was copied from v1 and described the wrong target, scale dtype, corpus and schedule. The current methodology is documented in [reproduce/README.md](reproduce/README.md), with exact stage order in [RUNBOOK.md](reproduce/RUNBOOK.md) and settings in [recipe.json](reproduce/recipe.json).
Key facts: same-input target; original FP8 expert teacher; FP4 per-expert books; FP16 block scales;15,007,754-token cleaned corpus;50× next-boundary loss weighting; reassignment every10; decay by25 or earlier; patience10. Validation selection is unweighted target-relative L2. Most layers stop early. All75 layer receipts remain available. Historical contradictory metadata is archived under reproduce/historical_metadata.
The bundled reference packer/decoder now handles both uint8 E4M3 and actual FP16 scales, tested on CPU. No model weights were changed by this correction.