inference-optimization/ctest-Qwen3-8B-speculator.dspark-fp8ablation-fp8

ctest: Qwen3-8B DSpark speculator — FP8 hidden-states ablation (fp8)

Test checkpoint — not a production release. Trained to validate the FP8-quantized hidden-states transfer backend added in vllm-project/speculators#1028 (draft, reviving #491).

• Verifier: Qwen/Qwen3-8B
• Algorithm: dspark
• Hidden-states precision during training: fp8 (new FP8 backend, per-token float8_e4m3fn quantization)
• Training data: 5K magpie + 5K ultrachat samples from inference-optimization/Dataset-Qwen3-235B-Instruct

This is one of a matched pair (...-fp8ablation-bf16 / ...-fp8ablation-fp8) trained identically except for the hidden-states transfer precision, to isolate the effect of FP8 quantization on speculator quality. See the sibling repo for the other precision.

Validation metrics (final epoch)

• confidence_loss_epoch: 0.253213
• confidence_abs_error_epoch: 0.224864
• confidence_pred_mean_epoch: 0.495482
• confidence_cumprod_bias_epoch: 0.019816
• loss_epoch: 0.584251
• ce_loss_epoch: 1.195752
• tv_loss_epoch: 0.234959
• accept_rate_epoch: 0.455243
• accept_len_epoch: 2.868270
• full_acc_epoch: 0.481714
• position_0_acc_epoch: 0.696390
• position_1_acc_epoch: 0.592052
• position_2_acc_epoch: 0.524082
• position_3_acc_epoch: 0.470779
• position_4_acc_epoch: 0.433401
• position_5_acc_epoch: 0.401091
• position_6_acc_epoch: 0.376266
• position_7_acc_epoch: 0.358048

guidellm serving eval

See acceptance.csv in this repo for the full per-subset guidellm breakdown (9 subset rows).

Full ablation writeup, all 6 checkpoints' results, exact commands, and code: see the results package referenced from PR #1028.