spills reasoning into code
with qwen's official parameter settings for code/precise tasks, the model starts spilling its chain of thought in code. i have noticed this happens in comments. it starts to describe what it is about to do next, in a comment, (// in cpp) and then it will fall into a long conversation, quite funny at time where it is attempting to talk itself out of the loop. only experienced it with this intel-provdided model.
is there a quick fix or tuning of parameters that helps?
Would you please provide some prompts (and parameter settings for code/precise tasks you mentioned)that can reproduce this issue?
it is a frequent occurrence. serving on vllm, fp8 kv cache, nothing really special. as conversation drags, it occurs often. i feel it should be easy to reproduce? i am on nightly vllm and flashinfer, if that helps.
Sorry for the late reply. Although we have been tracking this issue in our repo, we still haven’t had the bandwidth to reproduce it.
One quick question: it looks like you have tried different quantization versions. May I ask if you have also tried Lorbus/Qwen3.6-27B-int4-AutoRound? If so, does it work well for you? This could help us narrow down the issue.
I'm sorry, I didnt try that model, and now I only use the 3.8
No worries. I’ll run some tests in the near future.
I did a quick test with two tasks, quicksort and mixed-precision LLM quantization with and without fp8 kv cache, and I haven't been able to reproduce the issue you mentioned so far. It would be helpful if you could provide some additional details about the issue and the exact setup that triggers it.
The generated results and the serving command are included in issue #2142.