froggeric

/

Qwen-Fixed-Chat-Templates

like 1.6k

Hi froggeric — first of all, thanks for maintaining these templates, they're excellent. I deployed v22.4 with a Qwen3.8 Flash-Next model served via SGLang 0.5.18 (which uses transformers 5.12.1 as its Jinja renderer) and hit two rendering bugs. I've patched both and verified them across 4 scenarios; sharing the diff in case you want to fold them upstream.

Bug 1 — chain-of-elif short-circuits after | trim eats leading whitespace

In the reasoning-extraction block the branch conditions are a chain of elifs. When the incoming content has its leading whitespace trimmed by | trim earlier in the template, the first condition misses and every subsequent elif is short-circuited, so the block falls through and emits nothing — the model then sees an empty reasoning wrapper and degenerates into echo/repetition.

Fix: flatten the elif chain into independent if statements, each guarded with not _think_end, so every condition is evaluated independently regardless of preceding matches:

{% if not _think_end and reasoning_content is defined and reasoning_content %}
  {% set _think_end = true %}
  {{ reasoning_content }}
{% endif %}
{% if not _think_end and (reasoning_content is not defined or not reasoning_content) %}
  {% if _prompt_end is not defined or not _prompt_end %}
    {% set _prompt_end = true %}
    {{ '<|im_start|>assistant
' }}
  {% endif %}
{% endif %}

Bug 2 — split(_think_start) with a variable argument fails in the sandbox

reasoning_content.split(_think_start)[-1].lstrip('\n') silently returns the wrong value when _think_start is a variable, while the identical call with a string literal works. This is a Jinja-sandbox quirk (the variable resolves correctly — tojson of both is identical — but split() on the variable form misbehaves).

Fix: replace it with explicit prefix checks and slicing, which are bulletproof in the sandbox:

{% if reasoning_content[:9] == ' thinking' %}
  {% set _reasoning_content = reasoning_content[9:] %}
{% elif reasoning_content[:10] == '<thinking>' %}
  {% set _reasoning_content = reasoning_content[10:] %}
{% else %}
  {% set _reasoning_content = reasoning_content %}
{% endif %}

Verification

• Original 433 lines → patched 442 lines (net +9, single hunk in the reasoning block).
• Tested 4/4 scenarios: standard thinking... single-string format, plain content without thinking, <thinking> angle-bracket format, and trailing assistant-prompt preservation.
• SGLang logs confirm the patched template loads (Loading chat template from argument).

Deployment note for SGLang users

SGLang will not pick up chat_template.jinja from the model dir unless you pass it explicitly. Without --chat-template, it silently falls back to the tokenizer_config.json embedded template and your fixes never load. Use kebab-case in the YAML config:

chat-template: /path/to/chat_template_froggeric_v22.4.jinja

(Note: chat_template with underscore is rejected as an unrecognized argument.)

Happy to open a PR or share the full patched file if useful.