Loop
I noticed in two or three different sessions that around the 100th step, the model would start making the same request over and over again and couldn't break out of the loop. I switched to unslosh, and the problem went away.
Thank you for your feedback,
We’d really appreciate it if you could share which model(s) you tested, along with your llama.cpp build and sampling parameters, ideally the full server command.
It would also be very helpful to know what coding harness you were using, and any details you can share about the prompt or task where the loop happened. That will help us investigate and try to reproduce the issue.
Thanks again!
I'm experiencing the same looping issue. Around the 100th step, the model starts repeating the same request and gets stuck in an infinite loop.
Here are my details:
Model: byteshape/Qwen3.6-35B-A3B-MTP-GGUF:IQ2_S
Interface: llama.cpp built-in web UI via browser (port 8080)
Full server command:
llama-server -hf byteshape/Qwen3.6-35B-A3B-MTP-GGUF:IQ2_S --mmproj-auto --port 8080 -c 32768 --spec-type draft-mtp --spec-draft-n-max 4 --temp 1 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 1.5 --repeat-penalty 1
The loop consistently appears after approximately 100 steps during agentic/coding tasks. Notably, --repeat-penalty 1 means repeat penalty is effectively disabled — not sure if that's contributing to the issue. Happy to provide more details if needed.
@Kamil21322 , many thanks for providing more details.
I did some testing, and I think adjusting the sampling parameters may help with this. Please give the settings below a try and let me know how it goes.
A few suggested changes:
- --min-p 0 → --min-p 0.05
- --presence-penalty 1.5 → --presence-penalty 0.5
- --repeat-penalty 1 → --repeat-penalty 1.05
Also, the 2.25-bit model is an extreme quantization case and loses more than 10% of baseline quality. If possible, please try one of the larger quants as well, since the difference should be meaningful.
I also increased the context window and tested the model up to 128K tokens. Here is the command I used:
llama-server -m Qwen3.6-35B-A3B-IQ2_S-2.25bpw.gguf --mmproj-auto --port 8080 \
-c 131072--spec-type draft-mtp --spec-draft-n-max 4 \
--temp 1 --top-k 20 --top-p 0.95 \
--min-p 0.05 --presence-penalty 0.5 --repeat-penalty 1.05
The issue was resolved with the command you provided.
Many thanks for trying it and updating us. I’ll try to put some comments about it on the model card and our blog/website.
Really appreciate your help.
Any fix this please ?
Let me try a different approach. Let me search for the extension loading logic in the loader.js file, and also check if there's any auto-discovery logic that scans the .pi/extensions
directory.
$ grep -n "extensionPaths\|\.pi/extensions" ~/.nvm/versions/node/v24.14.0/lib/node_modules/@earendil-works/pi-coding-agent/dist/core/sdk.js 2>/dev/null | head -5 && echo "--- done ---"
--- done ---
Took 0.0s
Let me try a different approach. Let me search for the extension loading logic in the loader.js file, and also check if there's any auto-discovery logic that scans the .pi/extensions
directory.
$ grep -n "extensionPaths\|\.pi/extensions" ~/.nvm/versions/node/v24.14.0/lib/node_modules/@earendil-works/pi-coding-agent/dist/core/sdk.js 2>/dev/null | head -5 && echo "--- done ---"
--- done ---
Took 0.0s
Let me try a different approach. Let me search for the extension loading logic in the loader.js file, and also check if there's any auto-discovery logic that scans the .pi/extensions
directory.
$ grep -n "extensionPaths\|\.pi/extensions" ~/.nvm/versions/node/v24.14.0/lib/node_modules/@earendil-works/pi-coding-agent/dist/core/sdk.js 2>/dev/null | head -5 && echo "--- done ---"
--- done ---
Took 0.0s
I am using the below
/app/llama.cpp/build/bin/llama-server \
-m /app/models/byteshape/mtp/Qwen3.6-35B-A3B-IQ3_S-3.06bpw.gguf \
--fit on \
-ngl 28 \
--flash-attn on \
--parallel 1 \
--host 192.168.18.201 --port 8080 --api-key home-key \
--ctx-size 90000 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.05 \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--spec-type draft-mtp \
--spec-draft-n-max 4 \
-b 1024 -ub 512 \
--no-mmap \
--mlock \
--threads 12 \
--jinja \
--repeat-penalty 1.05 --presence-penalty 0.5 \
--reasoning on --reasoning-budget 8192 \
--alias qwen --no-context-shift \
--timeout 1800 \
--no-mmproj
I would like to make this work if possible with the non-MTP version since this gives me around 95-130 tok/s but same looping issue is present here as well.. GPU is RTX 5070 .
/app/llama.cpp/build/bin/llama-server \
-m /app/models/byteshape/non-mtp/Qwen3.6-35B-A3B-IQ2_S-2.17bpw.gguf \
--host 192.168.18.201 --port 8080 --api-key home-key \
--fit on \
--flash-attn on \
--parallel 1 \
--ctx-size 131072 \
-ngl 999 \
--temp 1 --top-p 0.95 --top-k 20 --min-p 0.05 \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
-b 3072 -ub 1536 \
--no-mmap \
--mlock \
--threads 12 \
--repeat-penalty 1.05 --presence-penalty 0.5 \
--reasoning on --reasoning-budget 8192 \
--alias qwen \
--timeout 1800 \
--no-mmproj
Hi @swiftarrow , thank you for the detailed report and for sharing your commands.
I looked into this a bit more. It's hard to reproduce the exact outputs, but I think we have a lead. It looks like the repetition penalty's lookback window is simply too short by default to catch a tool call that repeats this far into a session.
Could you try the following and let us know if it helps?
1. Main fix — widen the repetition window (biggest expected impact):
--repeat-last-n 256 # default is only 64
2. Worth adding on top:
--repeat-penalty 1.1 # up from 1.05
…and on the non-MTP command specifically, set:
--temp 0.6 # instead of --temp 1
3. Optional, may help further:
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2
If you get a chance to test, let us know whether the looping improves.
Thanks! 🙏
Just use recommended values by Qwen and use this chat template - https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
This template should help you.
@szwedek @Ali93H Appreciate the quick response . Thanks :)
After using the latest chat template, the looping has reduced 70% . I am not able to confirm if parameters you have shared are contributing much.
exact command I am using :
/app/llama.cpp/build/bin/llama-server \
-m /app/models/byteshape/non-mtp/Qwen3.6-35B-A3B-IQ2_S-2.17bpw.gguf \
--host 192.168.18.201 --port 8080 --api-key home-key \
--fit on \
--flash-attn on \
--parallel 2 \
--ctx-size 180000 \
-ngl 999 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.01 \
-b 1024 -ub 512 \
--no-mmap \
--mlock \
--threads 8 \
--cache-reuse 256 \
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 \
--repeat-penalty 1.1 \
--repeat-last-n 256 \
--samplers "top_k;top_p;min_p;temperature;dry;typ_p;xtc" \
--alias qwen \
--timeout 1800 \
--no-mmproj \
--chat-template-kwargs '{"enable_thinking": true}' \
--jinja --chat-template-file /app/models/byteshape/chat_template.jinja
@swiftarrow --chat-template-kwargs '{"preserve_thinking": true}'
enable_thinking is enabled by default.
Also --temperature 0.6 --top-p 0.95 --top-k 20 --presence-penalty 0.0 --repeat-penalty 1.0 --min-p 0.0
If this won't help you, then you can try apex quants - i-quality or i-balanced versions.