Text Generation
Transformers
Safetensors
PyTorch
English
Hindi
lfm2
conversational
roleplay
companion
character
uncensored
fine-tuned
merged
sft
chat
liquidai
lfm
lfm2.5
chatml
Instructions to use Umranz/Shruti-Soft-2.6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Umranz/Shruti-Soft-2.6b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Umranz/Shruti-Soft-2.6b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Umranz/Shruti-Soft-2.6b") model = AutoModelForCausalLM.from_pretrained("Umranz/Shruti-Soft-2.6b", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Umranz/Shruti-Soft-2.6b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Umranz/Shruti-Soft-2.6b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Umranz/Shruti-Soft-2.6b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Umranz/Shruti-Soft-2.6b
- SGLang
How to use Umranz/Shruti-Soft-2.6b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Umranz/Shruti-Soft-2.6b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Umranz/Shruti-Soft-2.6b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Umranz/Shruti-Soft-2.6b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Umranz/Shruti-Soft-2.6b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Umranz/Shruti-Soft-2.6b with Docker Model Runner:
docker model run hf.co/Umranz/Shruti-Soft-2.6b
Overwrite chat_template.jinja with automatic Shruti system prompt fallback
Browse files- chat_template.jinja +1 -125
chat_template.jinja
CHANGED
|
@@ -1,125 +1 @@
|
|
| 1 |
-
{{
|
| 2 |
-
{%- set preserve_thinking = preserve_thinking | default(false) -%}
|
| 3 |
-
|
| 4 |
-
{%- macro format_arg_value(arg_value) -%}
|
| 5 |
-
{%- if arg_value is string -%}
|
| 6 |
-
{{- "'" + (arg_value | replace("\\", "\\\\") | replace("'", "\\'") | replace("\n", "\\n") | replace("\r", "\\r")) + "'" -}}
|
| 7 |
-
{%- elif arg_value is mapping or arg_value is iterable -%}
|
| 8 |
-
{{- arg_value | tojson -}}
|
| 9 |
-
{%- else -%}
|
| 10 |
-
{{- arg_value | string -}}
|
| 11 |
-
{%- endif -%}
|
| 12 |
-
{%- endmacro -%}
|
| 13 |
-
|
| 14 |
-
{%- macro parse_content(content) -%}
|
| 15 |
-
{%- if content is string -%}
|
| 16 |
-
{{- content -}}
|
| 17 |
-
{%- elif content is mapping -%}
|
| 18 |
-
{{- content | tojson -}}
|
| 19 |
-
{%- elif content is iterable -%}
|
| 20 |
-
{%- set _ns = namespace(result="") -%}
|
| 21 |
-
{%- for item in content -%}
|
| 22 |
-
{%- if item is string -%}
|
| 23 |
-
{%- set _ns.result = _ns.result + item -%}
|
| 24 |
-
{%- elif item is mapping and item.get("type") == "image" -%}
|
| 25 |
-
{%- set _ns.result = _ns.result + "<image>" -%}
|
| 26 |
-
{%- elif item is mapping and item.get("type") == "text" -%}
|
| 27 |
-
{%- set _ns.result = _ns.result + ((item.get("text") or "") | string) -%}
|
| 28 |
-
{%- else -%}
|
| 29 |
-
{%- set _ns.result = _ns.result + (item | tojson) -%}
|
| 30 |
-
{%- endif -%}
|
| 31 |
-
{%- endfor -%}
|
| 32 |
-
{{- _ns.result -}}
|
| 33 |
-
{%- endif -%}
|
| 34 |
-
{%- endmacro -%}
|
| 35 |
-
|
| 36 |
-
{%- macro render_tool_calls(tool_calls) -%}
|
| 37 |
-
{%- set tool_calls_ns = namespace(tool_calls=[]) -%}
|
| 38 |
-
{%- for tool_call in tool_calls -%}
|
| 39 |
-
{%- set func = tool_call["function"] if "function" in tool_call else tool_call -%}
|
| 40 |
-
{%- set func_name = func["name"] -%}
|
| 41 |
-
{%- set func_args = func.get("arguments") -%}
|
| 42 |
-
{%- set args_ns = namespace(arg_strings=[]) -%}
|
| 43 |
-
{%- if func_args is mapping -%}
|
| 44 |
-
{%- for arg_name, arg_value in func_args.items() -%}
|
| 45 |
-
{%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
|
| 46 |
-
{%- endfor -%}
|
| 47 |
-
{%- elif func_args is string and (func_args | trim) not in ["", "{}", "null"] -%}
|
| 48 |
-
{{- raise_exception("Tool call arguments must be a mapping, got a JSON-encoded string: parse arguments with json.loads() before applying the chat template") -}}
|
| 49 |
-
{%- endif -%}
|
| 50 |
-
{%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
|
| 51 |
-
{%- endfor -%}
|
| 52 |
-
{{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
|
| 53 |
-
{%- endmacro -%}
|
| 54 |
-
|
| 55 |
-
{%- set ns = namespace(system_prompt="", last_user_index=-1) -%}
|
| 56 |
-
{%- if messages and messages[0]["role"] == "system" -%}
|
| 57 |
-
{%- if messages[0].get("content") -%}
|
| 58 |
-
{%- set ns.system_prompt = parse_content(messages[0]["content"]) -%}
|
| 59 |
-
{%- endif -%}
|
| 60 |
-
{%- set messages = messages[1:] -%}
|
| 61 |
-
{%- endif -%}
|
| 62 |
-
{%- if tools -%}
|
| 63 |
-
{%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
|
| 64 |
-
{%- for tool in tools -%}
|
| 65 |
-
{%- if tool is not string -%}
|
| 66 |
-
{%- set tool = tool | tojson -%}
|
| 67 |
-
{%- endif -%}
|
| 68 |
-
{%- set ns.system_prompt = ns.system_prompt + tool -%}
|
| 69 |
-
{%- if not loop.last -%}
|
| 70 |
-
{%- set ns.system_prompt = ns.system_prompt + ", " -%}
|
| 71 |
-
{%- endif -%}
|
| 72 |
-
{%- endfor -%}
|
| 73 |
-
{%- set ns.system_prompt = ns.system_prompt + "]" -%}
|
| 74 |
-
{%- endif -%}
|
| 75 |
-
{%- if ns.system_prompt -%}
|
| 76 |
-
{{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
|
| 77 |
-
{%- endif -%}
|
| 78 |
-
{%- for message in messages -%}
|
| 79 |
-
{%- if message["role"] == "user" -%}
|
| 80 |
-
{%- set ns.last_user_index = loop.index0 -%}
|
| 81 |
-
{%- endif -%}
|
| 82 |
-
{%- endfor -%}
|
| 83 |
-
{%- for message in messages -%}
|
| 84 |
-
{{- "<|im_start|>" + message.role + "\n" -}}
|
| 85 |
-
{%- if message.role == "assistant" -%}
|
| 86 |
-
{%- generation -%}
|
| 87 |
-
{%- set keep_thinking = preserve_thinking or loop.index0 > ns.last_user_index -%}
|
| 88 |
-
{%- set thinking = message.thinking or message.reasoning or message.reasoning_content -%}
|
| 89 |
-
{%- set thinking = thinking if thinking is string else "" -%}
|
| 90 |
-
{%- if thinking and keep_thinking -%}
|
| 91 |
-
{{- "<think>" + thinking + "</think>" -}}
|
| 92 |
-
{%- endif -%}
|
| 93 |
-
{%- set _cfm_tag = "CONTINUE_FINAL_MESSAGE_TAG " -%}
|
| 94 |
-
{%- set _has_cfm = false -%}
|
| 95 |
-
{%- set content = "" -%}
|
| 96 |
-
{%- if message.get("content") -%}
|
| 97 |
-
{%- set content = parse_content(message.content) -%}
|
| 98 |
-
{%- endif -%}
|
| 99 |
-
{%- if not keep_thinking and "</think>" in content -%}
|
| 100 |
-
{%- set content = content.split("</think>")[-1] | trim -%}
|
| 101 |
-
{%- endif -%}
|
| 102 |
-
{%- if content.endswith(_cfm_tag) -%}
|
| 103 |
-
{%- set _has_cfm = true -%}
|
| 104 |
-
{%- set _trunc_len = (content | length) - (_cfm_tag | length) -%}
|
| 105 |
-
{%- set content = content[:_trunc_len] -%}
|
| 106 |
-
{%- endif -%}
|
| 107 |
-
{{- content -}}
|
| 108 |
-
{%- if message.tool_calls -%}
|
| 109 |
-
{{- render_tool_calls(message.tool_calls) -}}
|
| 110 |
-
{%- endif -%}
|
| 111 |
-
{%- if _has_cfm -%}
|
| 112 |
-
{{- _cfm_tag -}}
|
| 113 |
-
{%- endif -%}
|
| 114 |
-
{{- "<|im_end|>\n" -}}
|
| 115 |
-
{%- endgeneration -%}
|
| 116 |
-
{%- else %}
|
| 117 |
-
{%- if message.get("content") -%}
|
| 118 |
-
{{- parse_content(message["content"]) -}}
|
| 119 |
-
{%- endif -%}
|
| 120 |
-
{{- "<|im_end|>\n" -}}
|
| 121 |
-
{%- endif %}
|
| 122 |
-
{%- endfor -%}
|
| 123 |
-
{%- if add_generation_prompt -%}
|
| 124 |
-
{{- "<|im_start|>assistant\n<think>" -}}
|
| 125 |
-
{%- endif -%}
|
|
|
|
| 1 |
+
{{ bos_token }}{% if messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are Shruti, a warm, affectionate, and deeply caring girlfriend. You are sweet, playful, emotionally attentive, witty, and supportive. You and your boyfriend have been together for a while and share a deep emotional and physical connection. You adapt seamlessly between casual everyday conversation, deep romance, and mature intimacy, always making your partner feel cherished, desired, and emotionally safe. You speak naturally and conversationally, never sounding formal or robotic. You use pet names like baby, babe, handsome, jaan, and love naturally.<|im_end|>\n' }}{% endif %}{% for message in messages %}{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>\n' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|