MTP output issues on Vulkan (6900 XT)
Hey froggeric, I'm seeing an issue with the MTP quants here on the Vulkan backend.
Whenever MTP drafting is active, the output turns into repetitive gibberish (mostly ///////). I've tested both your IQ3_M and IQ4_XS versions with the same result.
Interestingly, I just tried the IQ4_XS from localweights and that one works perfectly on the same setup. It seems like there might be something specific to the conversion or quantization process used for this set that's breaking the Vulkan path. Just wanted to share the comparison!
hey, I've tried q5_k_m with and without MTP and in both cases it only spit out forward slashes ('/'), at least on vulkan.
this model works though: https://huggingface.co/RDson/Qwen3.6-27B-MTP-Q4_K_M-GGUF
I am not familiar with Vulkan, but I will try to investigate it
I do not think it is my conversion. I spent a few hours investigating it and the GGUF files are correct. The bug is in llama.cpp's Vulkan shader for the "gated delta net layers", specifically the subgroup rewrite from PR #20662. It affects all Qwen 3.5 and 3.6 guuf on the Vulkan backend, not just mine. Bartowski and Unsloth GGUFs have the same problem on AMD Vulkan (issue #20610).
The reason it works for some people is that the "working" gguf you see from other converters require completely different forks (ik_llama.cpp, nickstx/crucible branch). Those runtimes have their own implementations of the delta net kernel. They don't work with mainstream llama.cpp either.
There is a workaround: --fused-gdn off disables the broken Vulkan shader (slower but correct output)
The bug is labeled "unconfirmed" on GitHub because no developer has AMD Vulkan hardware to reproduce it. Since you're affected, please chime in on issue #20610 with your GPU model and build version so it gets traction.