# Qwen3.8-27B Heretic ARA DFlash2 selective-FP8 experiment

## Result

**PASS — experimental checkpoint**

- Source: [`alphakek/Qwen3.8-27B-heretic-ara-DFlash2`](https://huggingface.co/alphakek/Qwen3.8-27B-heretic-ara-DFlash2)
- FP8 derivative: `magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8`
- Profile: five decoder MLPs in native block-FP8; sensitive and currently incompatible paths remain BF16
- FP8 format: E4M3 weights, 128 x 128 FP32 inverse scales, dynamic activation metadata
- Quantized matrices: 15 (gate, up, and down projection in each of five layers)
- Preserved tensors: 66, byte-identical to the BF16 source
- Tensor payload: 2,512,200,960 bytes (2.51 GB decimal / 2.34 GiB)
- BF16 tensor payload: 3,848,808,960 bytes
- Tensor-payload reduction: 34.73%

## Why this is mixed precision

The following remain BF16 deliberately:

- Q/K/V projections, because the current DFlash fused context-KV path reads their raw weights and bypasses quantization dispatch
- Attention output projections, because preserving only about 105 MB materially improved proposal KL and top-1 stability
- The target-hidden-state `fc` adapter
- Dynamic convolution projections and kernels
- Candidate-selector projection and codebooks
- Norms and other small parameters

The metadata contains both Transformers checkpoint names and vLLM runtime names for these exclusions.

## Structural validation

- Source tensor entries: 81
- Candidate tensor entries: 96
- FP8 scale tensors: 15
- Preserved tensors verified byte-for-byte: 66
- Aggregate relative weight RMSE: 0.0264555
- FP8 dtype, FP32 scale dtype, positive finite scales, block geometry, tensor closure, and safetensors headers: passed

## Controlled sublayer arithmetic

Fixed-seed RMSNorm-like BF16 activation vectors were evaluated through every MLP using the BF16 source and the serialized FP8 weights dequantized to BF16.

- Aggregate MLP relative output RMSE: 0.0288080
- Aggregate MLP output cosine similarity: 0.9995862
- Preserved attention-output projection cosine similarity: 1.0
- Result: PASS

## Target-conditioned DFlash2 proposal validation

Eight diverse prompts were passed through the real BF16 27B target. Hidden states from target layers 5, 19, 33, 47, and 61 were supplied identically to the BF16 and selective-FP8 drafters. Their 56 draft positions were then projected through the target's actual 248,320-token LM head.

- Mean exact proposal KL, `D_KL(P_BF16 || P_FP8)`: 0.0018930
- Median proposal KL: 0.0015949
- P95 proposal KL: 0.0030100
- Maximum proposal KL: 0.0085216
- Proposal top-1 agreement: 96.43%
- Top-16 candidate overlap: 97.66%
- Draft-hidden relative RMSE: 0.0306350
- Draft-hidden cosine similarity: 0.9995342
- Selector path-token agreement: 85.71%
- Complete seven-token selector-path agreement: 75.0%
- Result: PASS

The selector metric is intentionally harsh: an early path difference changes the predecessor used by every later selector position. In this test, six of eight complete paths matched, one matched six of seven positions, and one diverged from its first position. Target verification still protects final model correctness; this difference can affect acceptance and speed.

## Loader validation

The official `dflash` 0.1.0 `DFlash2DraftModel` loader successfully loaded the checkpoint with Transformers 5.15.0. On RTX 3090 it correctly dequantized the FP8 MLP weights to BF16 while retaining the selected BF16 modules.

## Deployment limitations

RTX 3090 cannot execute native W8A8 FP8. These tests validate serialized FP8 weight quality, full DFlash2 proposal behavior with real target hidden states, and checkpoint loading, but not native R9700 kernel throughput.

DFlash2's vLLM support is still an experimental, unmerged branch as of this validation. A final R9700 trial must measure end-to-end acceptance length and tokens per second against the BF16 drafter. The BF16 model card baseline is acceptance length 4.24 and throughput 123.2 tok/s under its documented 4x3090 test. Promote this FP8 experiment only if native throughput improves without a material acceptance regression.

The BF16 source and the previously finalized 27B FP8 target were not modified.