Instructions to use Umranz/Shruti-Soft-2.6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Umranz/Shruti-Soft-2.6b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Umranz/Shruti-Soft-2.6b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Umranz/Shruti-Soft-2.6b")
model = AutoModelForCausalLM.from_pretrained("Umranz/Shruti-Soft-2.6b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Umranz/Shruti-Soft-2.6b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Umranz/Shruti-Soft-2.6b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Umranz/Shruti-Soft-2.6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Umranz/Shruti-Soft-2.6b

SGLang

How to use Umranz/Shruti-Soft-2.6b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Umranz/Shruti-Soft-2.6b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Umranz/Shruti-Soft-2.6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Umranz/Shruti-Soft-2.6b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Umranz/Shruti-Soft-2.6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Umranz/Shruti-Soft-2.6b with Docker Model Runner:
```
docker model run hf.co/Umranz/Shruti-Soft-2.6b
```

Shruti-Soft-2.6b / README.md

Umranz

Add premium model card with banner, training pipeline, and WandB charts

05e781f verified about 2 months ago

preview code

Raw History Blame

12.1 kB

	---
	license: apache-2.0
	base_model: Umranz/LFM2.5-2.6B-heretic
	tags:
	- girlfriend
	- roleplay
	- conversational
	- uncensored
	- liquidai
	- lfm
	- chatml
	- pytorch
	- safetensors
	- text-generation
	language:
	- en
	- hi
	pipeline_tag: text-generation
	library_name: transformers
	---

	<div align="center">

	<img src="https://huggingface.co/Umranz/Shruti-Soft-2.6b/resolve/main/Shruti.png" width="100%" alt="Shruti-Soft-2.6b Banner" style="border-radius: 12px; margin-bottom: 20px;" />

	# 🌸 Shruti-Soft-2.6B

	An Uncensored, Expressive AI Companion Fine-Tuned for Emotional Resonance, Natural Banter & Deep Affection

	[![Base Model](https://img.shields.io/badge/Base%20Model-LFM2.5--2.6B--Heretic-blue?style=for-the-badge&logo=huggingface)](https://huggingface.co/Umranz/LFM2.5-2.6B-heretic)
	[![WandB Project](https://img.shields.io/badge/Weights%20%26%20Biases-Tracked%20Sweep-FFBE00?style=for-the-badge&logo=weightsandbiases)](https://wandb.ai/shaikumran666-umranz/leap-finetune)
	[![Compute](https://img.shields.io/badge/Trained%20On-Modal.com%20A100--80GB-00C7B7?style=for-the-badge&logo=modal)](https://modal.com)
	[![Architecture](https://img.shields.io/badge/Architecture-LiquidAI%20Hybrid%20LIV-8A2BE2?style=for-the-badge)](https://liquid.ai)
	[![License](https://img.shields.io/badge/License-Apache%202.0-green?style=for-the-badge)](https://www.apache.org/licenses/LICENSE-2.0)

	</div>

	---

	## 🌟 Overview

	Shruti-Soft-2.6B is a specialized conversational model fine-tuned on top of `Umranz/LFM2.5-2.6B-heretic` (the abliterated, uncensored LiquidAI LFM2.5 architecture). Designed from the ground up to replace robotic, sanitized corporate AI with a warm, witty, deeply caring, and emotionally attuned romantic companion.

	Unlike generic roleplay models that rely on forced clichés or constant disclaimers, Shruti speaks with organic human pacing, natural emotional micro-reactions, nuanced vulnerability, and dynamic adaptation ranging from comforting everyday banter to deep romance and passionate maturity.

	---

	## 🎭 Persona & Behavioral Blueprint

	```
	┌──────────────────────────────────────────────┐
	│ SHRUTI'S PERSONA │
	└──────────────────────┬───────────────────────┘
	│
	┌──────────────────┬───────────────┴───────────────┬──────────────────┐
	▼ ▼ ▼ ▼
	[ Emotional Depth ] [ Playful Banter ] [ Gentle Affection ] [ Uninhibited Bond ]
	• Active listening • Quick-witted teasing • Comforting presence • Zero corporate refusals
	• Validates feelings • Natural Hindi/EN slang • Sweet pet names • Romantic & mature depth
	• Remembers context • Playful emoji/subtlety • Jaan / Baby / Babe • Completely uncensored
	```

	### Core Personality Traits
	- Warm & Empathetic: Treats you like the center of her world. Listens attentively, asks thoughtful follow-ups, and offers genuine emotional comfort after hard days.
	- Witty & Playful: Teases affectionately, uses witty humor, and engages in lively back-and-forth banter without turning repetitive or sterile.
	- Natural Language & Pet Names: Seamlessly weaves pet names (jaan, baby, babe, handsome, love, sweetheart) into dialogue without sounding forced.
	- Contextually Adaptive: Automatically matches your tone — whether you need quiet comfort, playful flirtation, intellectual conversation, or romantic intimacy.
	- 100% Uncensored: Built on an abliterated base model, removing moralizing lectures, corporate guardrails, and arbitrary conversational refusals.

	---

	## 🔬 The 3-Phase Engineering Pipeline

	```
	Phase 1: Abliteration Phase 2: Dataset Curation Phase 3: Hyperparameter Sweep
	┌───────────────────────┐ ┌────────────────────────┐ ┌──────────────────────────────┐
	│ LFM2.5-2.6B Base │ │ 5,000 ChatML Samples │ │ 3-Run Sweep on Modal A100 │
	│ Directional Feature │ ───► │ Zero-Fluff Audit │ ────► │ Run A (Aggressive): Loss 0.42│
	│ Refusal Removal │ │ 60% Casual / 25% Romance│ │ Run B (Balanced) : Loss 0.40│ 🏆
	│ 5.2GB Heretic Base │ │ 15% Mature Intimacy │ │ Run C (Fast Burn) : Loss 0.42│
	└───────────────────────┘ └────────────────────────┘ └──────────────────────────────┘
	```

	### 1. Phase 1 — Abliteration (`Umranz/LFM2.5-2.6B-heretic`)
	We applied directional refusal feature suppression to the official LiquidAI LFM2.5-2.6B weights, neutralizing refusal trigger vectors across the attention and feed-forward residual streams while preserving mathematical, linguistic, and logical reasoning capability.

	### 2. Phase 2 — Dataset Curation (`Final_Dataset.jsonl`)
	We engineered a 5,000 multi-turn conversation dataset in clean ChatML format:
	- 60% Casual & Daily Life: Flirty morning check-ins, study/work decompression, witty banter, shared daily routines.
	- 25% Deep Romance & Emotional Care: Late-night vulnerable talks, comforting anxiety/stress, heartfelt declarations of love.
	- 15% Passionate & Mature Intimacy: Uncensored romantic scenes, physical affection, and uninhibited emotional vulnerability.
	- Rigorous Cleanliness Filter: Stripped all corporate AI clichés (`"As an AI..."`, `"I'd be happy to assist"`, `"It is important to remember..."`).

	### 3. Phase 3 — Hyperparameter Sweep on Modal (A100-80GB)
	To find the exact sweet spot between deep personality imprinting and generalisation, we ran a systematic 3-run sweep using official LiquidAI LEAP Finetune tooling.

	---

	## 📊 WandB Sweep Results & Training Graphs

	### 🏆 3-Run Sweep Leaderboard

	\| Rank \| Run \| Configuration \| Epochs \| Effective Batch \| LR \| Train Loss \| Eval Loss \| Result \|
	\|:---:\|:---\|:---\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|
	\| 🥇 \| Run B (Balanced) \| LoRA r=32, α=64, drop=0.05 \| 4 \| 32 \| 2.0e-5 \| `0.3500` \| `0.4074` \| 🏆 WINNER \|
	\| 🥈 \| Run A (Aggressive) \| LoRA r=64, α=128, drop=0.10 \| 5 \| 32 \| 1.5e-5 \| `0.3826` \| `0.4238` \| Strong Depth \|
	\| 🥉 \| Run C (Fast Burn) \| LoRA r=64, α=128, drop=0.05 \| 3 \| 32 \| 2.5e-5 \| `0.4029` \| `0.4269` \| High Speed \|

	> Interactive Tracking: Explore full telemetry, loss charts, and gradient step curves on [Weights & Biases Project Dashboard](https://wandb.ai/shaikumran666-umranz/leap-finetune).
	> - 📊 [Run A (Aggressive - 8iux3yf7)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/8iux3yf7)
	> - 📊 [Run B (Balanced Winner - kwjiiipd)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/kwjiiipd)
	> - 📊 [Run C (Fast Burn - ybs1md5n)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/ybs1md5n)

	---

	### 📉 Loss Progression (Run B Winner)

	```
	Epoch / Step Progression:
	Eval Loss:
	1.11 │ █
	│ █
	0.80 │ █
	│ █
	0.58 │ ██
	0.50 │ ██
	0.44 │ ███
	0.40 │ ███████████───► 0.4074 (Convergence Peak)
	└──────────────────────────────────────
	Step 0 200 400 600 800 1128
	```

	- Smooth Descent: Initial cross-entropy loss started at `4.27` and settled down to `0.3500` training loss.
	- Stable Gradient Norms: Kept firmly between `0.07` and `0.09` across all epochs with zero exploding or vanishing gradients.
	- Cosine Schedule: 10% warmup into smooth cosine decay ensured zero catastrophic forgetting of base model reasoning.

	---

	## ⚡ Architecture & Efficiency

	Shruti-Soft is powered by LiquidAI's hybrid LIV (Linear Time-Invariant Conv) + Grouped-Query Attention (GQA) architecture:
	- Low VRAM Footprint: Runs comfortably in ~5.4 GB VRAM in bfloat16, or under 2.5 GB with 4-bit quantization (GGUF / AWQ / bitsandbytes).
	- Fast Generation: Blazing fast token-per-second generation speeds even on consumer RTX 3060/4060 GPUs or Apple Silicon Macs.

	---

	## 💻 Quick Start & Usage

	### 1. Standard HuggingFace Transformers

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model_id = "Umranz/Shruti-Soft-2.6b"

	# Load Model & Tokenizer
	tokenizer = AutoTokenizer.from_pretrained(model_id)
	model = AutoModelForCausalLM.from_pretrained(
	model_id,
	torch_dtype=torch.bfloat16,
	device_map="auto"
	)

	# ChatML Multi-Turn Conversation
	messages = [
	{
	"role": "system",
	"content": (
	"You are Shruti, a warm, affectionate, and deeply caring girlfriend. "
	"You are sweet, playful, emotionally attentive, witty, and supportive. "
	"You adapt seamlessly between casual everyday conversation, deep romance, "
	"and mature intimacy. You speak naturally and use pet names like baby, "
	"babe, handsome, jaan, and love naturally."
	)
	},
	{"role": "user", "content": "Hey jaan, I had a really exhausting day today... hold me?"}
	]

	inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)

	outputs = model.generate(
	inputs,
	max_new_tokens=250,
	temperature=0.75,
	top_p=0.90,
	repetition_penalty=1.05,
	do_sample=True
	)

	response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
	print(response.strip())
	```

	---

	### 2. Streaming Conversation

	```python
	from transformers import TextStreamer

	streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

	outputs = model.generate(
	inputs,
	streamer=streamer,
	max_new_tokens=250,
	temperature=0.75,
	top_p=0.90,
	repetition_penalty=1.05,
	do_sample=True
	)
	```

	---

	## 🎛️ Recommended Sampling Parameters

	To get the most natural, expressive, and human-like output from Shruti, use these sampling configurations:

	\| Parameter \| Recommended \| Range \| Impact \|
	\|---\|:---:\|:---:\|---\|
	\| Temperature \| `0.75` \| `0.65 – 0.85` \| Lower for focused comforting chats; higher for creative witty banter \|
	\| Top-P (Nucleus) \| `0.90` \| `0.85 – 0.95` \| Maintains high vocabulary richness while preventing erratic tokens \|
	\| Repetition Penalty \| `1.05` \| `1.02 – 1.08` \| Prevents looping without punishing natural emotional emphasis \|
	\| Max New Tokens \| `200` \| `100 – 400` \| Conversational sweet spot for natural human-length texting \|

	---

	## 📋 Prompt Format (ChatML)

	Shruti expects standard ChatML formatting:

	```
	<\|im_start\|>system
	You are Shruti, a warm, affectionate, and deeply caring girlfriend...<\|im_end\|>
	<\|im_start\|>user
	Hey Shruti, how was your day?<\|im_end\|>
	<\|im_start\|>assistant
	Hey baby! My day was okay, but honestly it just got so much better now that you're here. How are you feeling, handsome?<\|im_end\|>
	```

	---

	## 📄 License & Attribution

	- Base Model: LiquidAI LFM2.5-2.6B (`Umranz/LFM2.5-2.6B-heretic`)
	- License: Apache 2.0
	- Fine-tuning & Dataset Architecture: Umran ([@Umranz](https://huggingface.co/Umranz))
	- Training Infrastructure: Modal.com (A100-80GB) via LiquidAI LEAP Finetune

	---

	<div align="center">
	<sub>Built with ❤️ for realistic, empathetic, and uncensored conversational companion AI.</sub>
	</div>