YSLAB-ai's picture
Publish BF16 MTP overlay recipe and qualification results
8e8ea9a verified
|
Raw
History Blame Contribute Delete
2.49 kB

Troubleshooting

Gated checkpoint download fails

First sign in on the orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 page and accept the request to share your contact information. A CLI token cannot grant access until the browser-side conditions have been accepted for that same account.

Then log the recipe into that account:

scripts/qwen38-dgx-spark login

Then rerun the recipe's download command. Do not bypass the checkpoint selection or pinned revision checks.

The disk gate rejects the download

The recipe requires at least 100 GiB free before it downloads a target. Free space on the storage used for the checkpoint, preferably NVMe, then rerun the command. The PLE is memory-mapped from that storage during serving.

CUDA graph capture errors around PLE lookup

Use the recipe defaults. The lookup must be separated from captured graph segments, so the runtime uses PIECEWISE capture. Do not switch this recipe to a full capture mode for the mapped PLE path.

KV cache configuration fails

Use the BF16 KV default. This model path requires BF16 KV, and an FP8 KV setting is not a supported substitute.

MTP load guard reports fewer than 31/31 tensors

Do not bypass the guard. Re-run preparation for the overlay so the recipe verifies and rebuilds its compact BF16 MTP file from the pinned source shards:

scripts/qwen38-dgx-spark prepare orca-uncensored-bf16-mtp

The original orca-uncensored target intentionally has no MTP weights. Use the separately named overlay target when passing --mtp.

Deep MTP reports a QSA capacity or block-size error

Use the recipe command rather than constructing a raw vLLM invocation. The recipe automatically selects the required 48-token block alignment for depths five and higher. The qualified setting remains MTP=2.

Replacing a running recipe container

The serve command unloads an existing recipe container before it starts a replacement. If the qualified Orcarouter MTP attempt ended unexpectedly, run the guarded replacement:

scripts/qwen38-dgx-spark serve orca-uncensored-bf16-mtp --mtp 2 --replace

For another manifest target, replace the target alias and use only options supported for that target. Do not start an additional model container manually.

For supported target layouts, see Compatibility. Benchmark scope and limitations are in Benchmarks.