A beggar chooses (or begs) for FP8 attention:
I'd love to see an option with the attention weights at the original FP8. Why blow them up to BF16 and then either keep them, or requant down to nvfp4?
Hi! Fair point, and It was a design choice in the end - since I couldn't convert with the specific mix from Unsloth's source, I needed to convert that in some way. Blowing it back up to BF16 was bit of a "safer" choice, and a bit of a gamble too. I've double-checked after your comment to see if I didn't miss anything, but I still have the same findings as I documented in the model card: https://huggingface.co/esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF#why-the-orig-file-exists-and-why-the-tiers-are-the-way-they-are
That said, there's a first-class conversion path between FP8 and Q8 with (allegedly) better efficiency and nearly the same accuracy. I couldn't find a way to keep FP8 exactly. I'm cooking the Q8 attention version, and I'll upload it in a few hours as a HIGHEST version. Stay tuned if you're interested :)
It's there!