# LingBot SDNQ runtime adapter

`lingbot_sdnq_runtime` is part of this model repository. It loads ordinary
SDNQ-packed Linear weights and also reconstructs LingBot's 3-D MoE `w1`, `w2`,
and `w3` tensors from packed UINT4 storage.

On the official RunPod Torch 2.8 / CUDA 12.8 image, `torch._grouped_mm` only
accepts compute capability 9.0 and the upstream `sglang-kernel` wheel targets a
newer Torch ABI. The adapter therefore supplies Torch implementations of only
SGLang's token-alignment and top-k sum-reduction helpers. Expert GEMMs and
activation stay on the pinned SGLang Triton fused-MoE implementation.

For MoE inference install the upstream-pinned runtime without dependency
resolution so it does not replace the tested Torch build:

```bash
pip install --no-deps sglang==0.5.13.post1 apache-tvm-ffi==0.1.9 tilelang==0.1.8 triton==3.6.0
```

The adapter intentionally does not cache dequantized bf16 expert weights;
reported residency therefore remains the SDNQ packed representation.