Wrong jinja Template
I get this error
Failed to parse Jinja template: Parser Error: Expected closing statement token. Comma !== CloseStatement.
Would be nice if you could fix that (;
This Template works:
{{- bos_token }}
{%- if messages[0]['role'] == 'system' %}
{%- set system_message = messages[0]['content']|trim %}
{%- set messages = messages[1:] %}
{%- else %}
{%- set system_message = "" %}
{%- endif %}
{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}{{- system_message }}{{- "<|eot_id|>" }}
{%- for message in messages %}
{%- if message['role'] == 'assistant' and '' in message['content'] %}
{%- set content = message['content'].split('')|last %}
{%- else %}
{%- set content = message['content'] %}
{%- endif %}
{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + content | trim + '<|eot_id|>' }}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
{%- endif %}
Thank you!
There are couple of issues with this template replacement:
- Tool calling is gone
- Pre-defined system prompt is gone (this is a major issue, because it is highly recommended to use certain predefined system prompt with this model and add all user instructions into the prompt itself)
- Tags are similar, but not identical to the ones used in the original template, possibly leading to issues with the model
All of these issues could lead to sub-optimal performance of the model.
I manually identified the part of the template code which was incompatible with LM Studio. The issue was in the tool calling part, starting with set. I turned to Grok for help, pointed at the part of the code where the issue was and the AI helped me to fix the template as follows:
{%- set add_tool_id = true -%}
{%- set reasoning_prompt = "You are a thoughtful and systematic AI assistant built by ServiceNow Language Models (SLAM) lab. Before providing an answer, analyze the problem carefully and present your reasoning step by step. After explaining your thought process, provide the final solution in the following format: [BEGIN FINAL RESPONSE] ... [END FINAL RESPONSE]." -%}
{%- set reasoning_asst_turn_start = "Here are my reasoning steps:\n" -%}
{%- if messages[0]["role"] != "system" -%}
{%- if tools is defined and tools is not none and tools | length > 0 -%}
{{- "<|system|>\n" + reasoning_prompt + "\n\nYou are provided with function signatures within <available_tools></available_tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about the arguments. You should infer the argument values from previous user responses and the system message. Here are the available tools: <available_tools>" -}}
{%- for tool in tools -%}
{{- tool -}}
{%- endfor -%}
{{- "</available_tools>.\n\nReturn all function calls as a list of json objects within <tool_calls></tool_calls> XML tags. Each json object should contain a function name and arguments as follows: <tool_calls>[{\"name\": <function-name-1>, \"arguments\": <args-dict-1>}, {\"name\": <function-name-2>, \"arguments\": <args-dict-2>},...]</tool_calls>\n<|end|>\n" -}}
{%- else -%}
{{- "<|system|>\n" + reasoning_prompt + "\n<|end|>\n" -}}
{%- endif -%}
{%- endif -%}
{%- for message in messages -%}
{%- if message["role"] == "user" -%}
{{- "<|user|>\n" + message["content"] + "\n<|end|>\n" -}}
{%- elif message["role"] == "system" -%}
{%- if tools is defined and tools is not none and tools | length > 0 -%}
{{- "<|system|>\n" + reasoning_prompt + "\n\n" + message["content"] + "\nYou are provided with function signatures within <available_tools></available_tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about the arguments. You should infer the argument values from previous user responses and the system message. Here are the available tools: <available_tools>" -}}
{%- for tool in tools -%}
{{- tool -}}
{%- endfor -%}
{{- "</available_tools>.\n\nReturn all function calls as a list of json objects within <tool_calls></tool_calls> XML tags. Each json object should contain a function name and arguments as follows: <tool_calls>[{\"name\": <function-name-1>, \"arguments\": <args-dict-1>}, {\"name\": <function-name-2>, \"arguments\": <args-dict-2>},...]</tool_calls>\n<|end|>\n" -}}
{%- else -%}
{{- "<|system|>\n" + reasoning_prompt + "\n\n" + message["content"] + "\n<|end|>\n" -}}
{%- endif -%}
{%- elif message["role"] == "assistant" -%}
{%- if loop.last -%}
{%- set add_tool_id = false -%}
{%- endif -%}
{{- "<|assistant|>\n" -}}
{%- if message["content"] is not none -%}
{{- message["content"] -}}
{%- elif message["chosen"] is not none and message["chosen"] | length > 0 -%}
{{- message["chosen"][0] -}}
{%- endif -%}
{%- if message["tool_calls"] is not none and message["tool_calls"] | length > 0 -%}
{{- "\n<tool_calls>[" -}}
{%- for tool_call in message["tool_calls"] -%}
{{- "{\"name\": \"" + tool_call["function"]["name"] + "\", \"arguments\": " + tool_call["function"]["arguments"] | string -}}
{%- if add_tool_id -%}
{{- ", \"id\": \"" + tool_call["id"] -}}
{%- endif -%}
{{- "}" -}}
{%- if not loop.last -%}
{{- ", " -}}
{%- endif -%}
{%- endfor -%}
{{- "]</tool_calls>" -}}
{%- endif -%}
{%- if not loop.last or training_prompt -%}
{{- "\n<|end|>\n" -}}
{%- endif -%}
{%- elif message["role"] == "tool" -%}
{{- "<|tool_result|>\n" + message["content"] | string + "\n<|end|>\n" -}}
{%- endif -%}
{%- if loop.last and add_generation_prompt and message["role"] != "assistant" -%}
{{- "<|assistant|>\n" + reasoning_asst_turn_start -}}
{%- endif -%}
{%- endfor -%}
This template should retain the original functionality and it works in LM Studio (I just tested it myself).