madeby561

/

GLM-5.2-MXFP8-NVFP4-NF3-Hybrid

Like 70

A user deploying a derivative of this checkpoint on 4x RTX PRO 6000 Blackwell reported an infinite worker-init hang on a host with P2P only within GPU pairs (0<->1 and 2<->3), not across all four ranks. The detailed reproduction and stack are here:

https://huggingface.co/jacklarmer/GLM-5.2-Abliterated-MXFP8-NVFP4-NF3-Hybrid/discussions/1

Observed stack:

_broadcast_gather_object  sparkinfer/comm/pcie/pcie_oneshot.py:152
_allocate_shared_buffer   sparkinfer/comm/pcie/pcie_oneshot.py:477
__init__                  vllm/distributed/device_communicators/custom_all_reduce.py:492

The pinned July 26 v20 launcher forces VLLM_ENABLE_PCIE_ALLREDUCE=1, so a Compose-level 0 does not disable the path. I also inspected the public July 30 r13 image and do not think it is safe to recommend as the fix yet:

voipmonitor/vllm:gilded-gnosis-v20-vllm69ba80b-sia2ea608-fi801d57a-cu132-20260730-r13
sha256:02796036c96a52fda0919aa260c45c70bc97d8e662a6ae5e614b5f987c20851b

Its image config still contains VLLM_ENABLE_PCIE_ALLREDUCE=1. Its GLM chain is serve-glm52-hybrid-v19.sh -> serve-glm52-v19.sh -> glm52-pcie-runtime-env.sh, and configure_glm52_pcie_runtime_env() again exports:

VLLM_ENABLE_PCIE_ALLREDUCE=1
VLLM_PCIE_ALLREDUCE_BACKEND=b12x

The vLLM default changing to false therefore does not help this launcher path. The DS4 flash/pro launchers also still hardcode the opt-in.

The reporter's working fallback was to prevent _b12x_pcie_allreduce_requested() from selecting the one-shot path and use DCP_BACKEND=ag_rs; a separately qualified partial-mesh host likewise required disabling PCIe custom allreduce and B12X DCP A2A. The checkpoint itself is healthy.

Suggested runtime fixes:

1. Respect an explicit VLLM_ENABLE_PCIE_ALLREDUCE=0 in every launcher/helper.
2. Preflight cudaDeviceCanAccessPeer over every participating rank pair rather than relying on NUMA placement alone.
3. Fail with a clear topology error or bounded timeout instead of waiting forever in shared-buffer exchange.
4. Apply the same correction to the DS4 launchers.
5. Add nvidia-smi topo -p2p r to the serving preflight.

I have added a warning to the derivative card and will not point users at r13 until a corrected image is qualified. Please let me know if there is a newer tag or intended override path I missed.