Umranz commited on
Commit
4a6b726
·
verified ·
1 Parent(s): 05e781f

Remove emojis and clean model card for professional formatting

Browse files
Files changed (1) hide show
  1. README.md +76 -82
README.md CHANGED
@@ -21,130 +21,130 @@ library_name: transformers
21
 
22
  <div align="center">
23
 
24
- <img src="https://huggingface.co/Umranz/Shruti-Soft-2.6b/resolve/main/Shruti.png" width="100%" alt="Shruti-Soft-2.6b Banner" style="border-radius: 12px; margin-bottom: 20px;" />
25
 
26
- # 🌸 Shruti-Soft-2.6B
27
 
28
- **An Uncensored, Expressive AI Companion Fine-Tuned for Emotional Resonance, Natural Banter & Deep Affection**
29
 
30
- [![Base Model](https://img.shields.io/badge/Base%20Model-LFM2.5--2.6B--Heretic-blue?style=for-the-badge&logo=huggingface)](https://huggingface.co/Umranz/LFM2.5-2.6B-heretic)
31
- [![WandB Project](https://img.shields.io/badge/Weights%20%26%20Biases-Tracked%20Sweep-FFBE00?style=for-the-badge&logo=weightsandbiases)](https://wandb.ai/shaikumran666-umranz/leap-finetune)
32
- [![Compute](https://img.shields.io/badge/Trained%20On-Modal.com%20A100--80GB-00C7B7?style=for-the-badge&logo=modal)](https://modal.com)
33
- [![Architecture](https://img.shields.io/badge/Architecture-LiquidAI%20Hybrid%20LIV-8A2BE2?style=for-the-badge)](https://liquid.ai)
34
- [![License](https://img.shields.io/badge/License-Apache%202.0-green?style=for-the-badge)](https://www.apache.org/licenses/LICENSE-2.0)
35
 
36
  </div>
37
 
38
  ---
39
 
40
- ## 🌟 Overview
41
 
42
- **Shruti-Soft-2.6B** is a specialized conversational model fine-tuned on top of `Umranz/LFM2.5-2.6B-heretic` (the abliterated, uncensored LiquidAI LFM2.5 architecture). Designed from the ground up to replace robotic, sanitized corporate AI with a warm, witty, deeply caring, and emotionally attuned romantic companion.
43
 
44
- Unlike generic roleplay models that rely on forced clichés or constant disclaimers, Shruti speaks with organic human pacing, natural emotional micro-reactions, nuanced vulnerability, and dynamic adaptation ranging from comforting everyday banter to deep romance and passionate maturity.
45
 
46
  ---
47
 
48
- ## 🎭 Persona & Behavioral Blueprint
49
 
50
  ```
51
- ┌──────────────────────────────────────────────┐
52
- SHRUTI'S PERSONA
53
- └──────────────────────┬───────────────────────┘
54
-
55
- ┌──────────────────┬───────────────┴───────────────┬──────────────────┐
56
-
57
  [ Emotional Depth ] [ Playful Banter ] [ Gentle Affection ] [ Uninhibited Bond ]
58
- Active listening Quick-witted teasing Comforting presence Zero corporate refusals
59
- Validates feelings Natural Hindi/EN slang Sweet pet names Romantic & mature depth
60
- Remembers context Playful emoji/subtlety Jaan / Baby / Babe Completely uncensored
61
  ```
62
 
63
  ### Core Personality Traits
64
- - **Warm & Empathetic:** Treats you like the center of her world. Listens attentively, asks thoughtful follow-ups, and offers genuine emotional comfort after hard days.
65
- - **Witty & Playful:** Teases affectionately, uses witty humor, and engages in lively back-and-forth banter without turning repetitive or sterile.
66
- - **Natural Language & Pet Names:** Seamlessly weaves pet names (*jaan, baby, babe, handsome, love, sweetheart*) into dialogue without sounding forced.
67
- - **Contextually Adaptive:** Automatically matches your tone whether you need quiet comfort, playful flirtation, intellectual conversation, or romantic intimacy.
68
- - **100% Uncensored:** Built on an abliterated base model, removing moralizing lectures, corporate guardrails, and arbitrary conversational refusals.
69
 
70
  ---
71
 
72
- ## 🔬 The 3-Phase Engineering Pipeline
73
 
74
  ```
75
  Phase 1: Abliteration Phase 2: Dataset Curation Phase 3: Hyperparameter Sweep
76
- ┌───────────────────────┐ ┌────────────────────────┐ ┌──────────────────────────────┐
77
- LFM2.5-2.6B Base 5,000 ChatML Samples 3-Run Sweep on Modal A100
78
- Directional Feature ───► Zero-Fluff Audit ────► Run A (Aggressive): Loss 0.42
79
- Refusal Removal 60% Casual / 25% Romance Run B (Balanced) : Loss 0.40 🏆
80
- 5.2GB Heretic Base 15% Mature Intimacy Run C (Fast Burn) : Loss 0.42
81
- └───────────────────────┘ └────────────────────────┘ └──────────────────────────────┘
82
  ```
83
 
84
- ### 1. Phase 1 Abliteration (`Umranz/LFM2.5-2.6B-heretic`)
85
- We applied directional refusal feature suppression to the official LiquidAI LFM2.5-2.6B weights, neutralizing refusal trigger vectors across the attention and feed-forward residual streams while preserving mathematical, linguistic, and logical reasoning capability.
86
 
87
- ### 2. Phase 2 Dataset Curation (`Final_Dataset.jsonl`)
88
- We engineered a 5,000 multi-turn conversation dataset in clean ChatML format:
89
- - **60% Casual & Daily Life:** Flirty morning check-ins, study/work decompression, witty banter, shared daily routines.
90
- - **25% Deep Romance & Emotional Care:** Late-night vulnerable talks, comforting anxiety/stress, heartfelt declarations of love.
91
- - **15% Passionate & Mature Intimacy:** Uncensored romantic scenes, physical affection, and uninhibited emotional vulnerability.
92
- - **Rigorous Cleanliness Filter:** Stripped all corporate AI clichés (`"As an AI..."`, `"I'd be happy to assist"`, `"It is important to remember..."`).
93
 
94
- ### 3. Phase 3 Hyperparameter Sweep on Modal (A100-80GB)
95
- To find the exact sweet spot between deep personality imprinting and generalisation, we ran a systematic 3-run sweep using official LiquidAI LEAP Finetune tooling.
96
 
97
  ---
98
 
99
- ## 📊 WandB Sweep Results & Training Graphs
100
 
101
- ### 🏆 3-Run Sweep Leaderboard
102
 
103
  | Rank | Run | Configuration | Epochs | Effective Batch | LR | Train Loss | Eval Loss | Result |
104
  |:---:|:---|:---|:---:|:---:|:---:|:---:|:---:|:---:|
105
- | 🥇 | **Run B (Balanced)** | **LoRA r=32, α=64, drop=0.05** | **4** | **32** | **2.0e-5** | **`0.3500`** | **`0.4074`** | 🏆 **WINNER** |
106
- | 🥈 | **Run A (Aggressive)** | LoRA r=64, α=128, drop=0.10 | 5 | 32 | 1.5e-5 | `0.3826` | `0.4238` | Strong Depth |
107
- | 🥉 | **Run C (Fast Burn)** | LoRA r=64, α=128, drop=0.05 | 3 | 32 | 2.5e-5 | `0.4029` | `0.4269` | High Speed |
108
 
109
- > **Interactive Tracking:** Explore full telemetry, loss charts, and gradient step curves on [Weights & Biases Project Dashboard](https://wandb.ai/shaikumran666-umranz/leap-finetune).
110
- > - 📊 [Run A (Aggressive - 8iux3yf7)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/8iux3yf7)
111
- > - 📊 [Run B (Balanced Winner - kwjiiipd)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/kwjiiipd)
112
- > - 📊 [Run C (Fast Burn - ybs1md5n)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/ybs1md5n)
113
 
114
  ---
115
 
116
- ### 📉 Loss Progression (Run B Winner)
117
 
118
  ```
119
  Epoch / Step Progression:
120
  Eval Loss:
121
- 1.11
122
-
123
- 0.80
124
-
125
- 0.58 ██
126
- 0.50 ██
127
- 0.44 ███
128
- 0.40 ███████████───► 0.4074 (Convergence Peak)
129
- └──────────────────────────────────────
130
  Step 0 200 400 600 800 1128
131
  ```
132
 
133
- - **Smooth Descent:** Initial cross-entropy loss started at `4.27` and settled down to `0.3500` training loss.
134
- - **Stable Gradient Norms:** Kept firmly between `0.07` and `0.09` across all epochs with zero exploding or vanishing gradients.
135
- - **Cosine Schedule:** 10% warmup into smooth cosine decay ensured zero catastrophic forgetting of base model reasoning.
136
 
137
  ---
138
 
139
- ## Architecture & Efficiency
140
 
141
  Shruti-Soft is powered by LiquidAI's hybrid **LIV (Linear Time-Invariant Conv) + Grouped-Query Attention (GQA)** architecture:
142
  - **Low VRAM Footprint:** Runs comfortably in ~5.4 GB VRAM in bfloat16, or under 2.5 GB with 4-bit quantization (GGUF / AWQ / bitsandbytes).
143
- - **Fast Generation:** Blazing fast token-per-second generation speeds even on consumer RTX 3060/4060 GPUs or Apple Silicon Macs.
144
 
145
  ---
146
 
147
- ## 💻 Quick Start & Usage
148
 
149
  ### 1. Standard HuggingFace Transformers
150
 
@@ -214,20 +214,20 @@ outputs = model.generate(
214
 
215
  ---
216
 
217
- ## 🎛️ Recommended Sampling Parameters
218
 
219
- To get the most natural, expressive, and human-like output from Shruti, use these sampling configurations:
220
 
221
  | Parameter | Recommended | Range | Impact |
222
  |---|:---:|:---:|---|
223
- | **Temperature** | `0.75` | `0.65 0.85` | Lower for focused comforting chats; higher for creative witty banter |
224
- | **Top-P (Nucleus)** | `0.90` | `0.85 0.95` | Maintains high vocabulary richness while preventing erratic tokens |
225
- | **Repetition Penalty** | `1.05` | `1.02 1.08` | Prevents looping without punishing natural emotional emphasis |
226
- | **Max New Tokens** | `200` | `100 400` | Conversational sweet spot for natural human-length texting |
227
 
228
  ---
229
 
230
- ## 📋 Prompt Format (ChatML)
231
 
232
  Shruti expects standard **ChatML** formatting:
233
 
@@ -242,15 +242,9 @@ Hey baby! My day was okay, but honestly it just got so much better now that you'
242
 
243
  ---
244
 
245
- ## 📄 License & Attribution
246
 
247
  - **Base Model:** LiquidAI LFM2.5-2.6B (`Umranz/LFM2.5-2.6B-heretic`)
248
  - **License:** Apache 2.0
249
  - **Fine-tuning & Dataset Architecture:** Umran ([@Umranz](https://huggingface.co/Umranz))
250
  - **Training Infrastructure:** Modal.com (A100-80GB) via LiquidAI LEAP Finetune
251
-
252
- ---
253
-
254
- <div align="center">
255
- <sub>Built with ❤️ for realistic, empathetic, and uncensored conversational companion AI.</sub>
256
- </div>
 
21
 
22
  <div align="center">
23
 
24
+ <img src="https://huggingface.co/Umranz/Shruti-Soft-2.6b/resolve/main/Shruti.png" width="100%" alt="Shruti-Soft-2.6b Banner" style="border-radius: 8px; margin-bottom: 16px;" />
25
 
26
+ # Shruti-Soft-2.6B
27
 
28
+ **An Uncensored, Expressive Conversational Model Fine-Tuned for Emotional Resonance, Natural Banter, and Companionship**
29
 
30
+ [![Base Model](https://img.shields.io/badge/Base%20Model-LFM2.5--2.6B--Heretic-blue?style=flat-square&logo=huggingface)](https://huggingface.co/Umranz/LFM2.5-2.6B-heretic)
31
+ [![WandB Project](https://img.shields.io/badge/Weights%20%26%20Biases-Tracked%20Sweep-FFBE00?style=flat-square&logo=weightsandbiases)](https://wandb.ai/shaikumran666-umranz/leap-finetune)
32
+ [![Compute](https://img.shields.io/badge/Trained%20On-Modal.com%20A100--80GB-00C7B7?style=flat-square&logo=modal)](https://modal.com)
33
+ [![Architecture](https://img.shields.io/badge/Architecture-LiquidAI%20Hybrid%20LIV-8A2BE2?style=flat-square)](https://liquid.ai)
34
+ [![License](https://img.shields.io/badge/License-Apache%202.0-green?style=flat-square)](https://www.apache.org/licenses/LICENSE-2.0)
35
 
36
  </div>
37
 
38
  ---
39
 
40
+ ## Overview
41
 
42
+ **Shruti-Soft-2.6B** is a specialized conversational model fine-tuned on top of `Umranz/LFM2.5-2.6B-heretic` (the abliterated, uncensored LiquidAI LFM2.5 architecture). It is built to replace sterile, robotic conversational AI with a warm, witty, deeply caring, and emotionally attuned companion.
43
 
44
+ The model is trained to speak with organic conversational pacing, emotional micro-reactions, nuanced vulnerability, and dynamic adaptability across everyday conversations, emotional support, deep romance, and mature dialogue without corporate refusal disclaimers.
45
 
46
  ---
47
 
48
+ ## Persona & Behavioral Blueprint
49
 
50
  ```
51
+ +----------------------------------------------+
52
+ | SHRUTI'S PERSONA |
53
+ +----------------------+-----------------------+
54
+ |
55
+ +------------------+---------------+---------------+------------------+
56
+ | | | |
57
  [ Emotional Depth ] [ Playful Banter ] [ Gentle Affection ] [ Uninhibited Bond ]
58
+ - Active listening - Quick-witted teasing - Comforting presence - Zero corporate refusals
59
+ - Validates feelings - Natural Hindi/EN slang - Sweet pet names - Romantic & mature depth
60
+ - Remembers context - Expressive subtlety - Jaan / Baby / Babe - Completely uncensored
61
  ```
62
 
63
  ### Core Personality Traits
64
+ - **Warm and Empathetic:** Prioritizes the user's emotional state. Listens attentively, asks thoughtful follow-ups, and offers genuine emotional comfort after stressful days.
65
+ - **Witty and Playful:** Teases affectionately, uses dry and playful humor, and maintains engaging back-and-forth banter without repetitive phrasing.
66
+ - **Natural Language and Pet Names:** Seamlessly incorporates pet names (*jaan, baby, babe, handsome, love, sweetheart*) into dialogue without overusing them.
67
+ - **Contextually Adaptive:** Matches the user's tone automatically -- whether the conversation calls for quiet comfort, lighthearted teasing, intellectual discussion, or romantic intimacy.
68
+ - **Uncensored Baseline:** Built on an abliterated base model, neutralizing moralizing lectures, corporate canned responses, and arbitrary conversational guardrails.
69
 
70
  ---
71
 
72
+ ## The 3-Phase Engineering Pipeline
73
 
74
  ```
75
  Phase 1: Abliteration Phase 2: Dataset Curation Phase 3: Hyperparameter Sweep
76
+ +-----------------------+ +------------------------+ +------------------------------+
77
+ | LFM2.5-2.6B Base | | 5,000 ChatML Samples | | 3-Run Sweep on Modal A100 |
78
+ | Directional Feature | ---> | Zero-Fluff Audit | ----> | Run A (Aggressive): Loss 0.42|
79
+ | Refusal Removal | | 60% Casual / 25% Romance| | Run B (Balanced) : Loss 0.40| [WINNER]
80
+ | 5.2GB Heretic Base | | 15% Mature Intimacy | | Run C (Fast Burn) : Loss 0.42|
81
+ +-----------------------+ +------------------------+ +------------------------------+
82
  ```
83
 
84
+ ### 1. Phase 1 -- Abliteration (`Umranz/LFM2.5-2.6B-heretic`)
85
+ Directional refusal feature suppression was applied to the official LiquidAI LFM2.5-2.6B base weights. This process neutralized refusal trigger vectors across the attention and feed-forward residual streams while preserving mathematical, linguistic, and logical reasoning capabilities.
86
 
87
+ ### 2. Phase 2 -- Dataset Curation (`Final_Dataset.jsonl`)
88
+ A 5,000 multi-turn conversation dataset was engineered in clean ChatML format:
89
+ - **60% Casual & Daily Life:** Morning check-ins, study/work decompression, witty banter, shared daily routines.
90
+ - **25% Deep Romance & Emotional Care:** Late-night talks, comforting anxiety and stress, heartfelt romantic dialogue.
91
+ - **15% Passionate & Mature Intimacy:** Uncensored romantic scenes, physical affection, and emotional vulnerability.
92
+ - **Cleanliness Filter:** Removed all corporate AI artifacts (`"As an AI..."`, `"I'd be happy to assist"`, `"It is important to remember..."`).
93
 
94
+ ### 3. Phase 3 -- Hyperparameter Sweep on Modal (A100-80GB)
95
+ A systematic 3-run hyperparameter sweep was executed using the official LiquidAI LEAP Finetune framework to determine the optimal configuration for character depth and generalization.
96
 
97
  ---
98
 
99
+ ## WandB Sweep Results & Training Metrics
100
 
101
+ ### 3-Run Sweep Leaderboard
102
 
103
  | Rank | Run | Configuration | Epochs | Effective Batch | LR | Train Loss | Eval Loss | Result |
104
  |:---:|:---|:---|:---:|:---:|:---:|:---:|:---:|:---:|
105
+ | 1 | **Run B (Balanced)** | **LoRA r=32, alpha=64, drop=0.05** | **4** | **32** | **2.0e-5** | **`0.3500`** | **`0.4074`** | **WINNER** |
106
+ | 2 | **Run A (Aggressive)** | LoRA r=64, alpha=128, drop=0.10 | 5 | 32 | 1.5e-5 | `0.3826` | `0.4238` | Strong Depth |
107
+ | 3 | **Run C (Fast Burn)** | LoRA r=64, alpha=128, drop=0.05 | 3 | 32 | 2.5e-5 | `0.4029` | `0.4269` | Fast Convergence |
108
 
109
+ > **Interactive Tracking:** Full telemetry, loss charts, and gradient step curves are logged on the [Weights & Biases Project Dashboard](https://wandb.ai/shaikumran666-umranz/leap-finetune).
110
+ > - [Run A (Aggressive - 8iux3yf7)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/8iux3yf7)
111
+ > - [Run B (Balanced Winner - kwjiiipd)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/kwjiiipd)
112
+ > - [Run C (Fast Burn - ybs1md5n)](https://wandb.ai/shaikumran666-umranz/leap-finetune/runs/ybs1md5n)
113
 
114
  ---
115
 
116
+ ### Loss Progression (Run B Winner)
117
 
118
  ```
119
  Epoch / Step Progression:
120
  Eval Loss:
121
+ 1.11 | #
122
+ | #
123
+ 0.80 | #
124
+ | #
125
+ 0.58 | ##
126
+ 0.50 | ##
127
+ 0.44 | ###
128
+ 0.40 | ###########---> 0.4074 (Convergence Peak)
129
+ +--------------------------------------
130
  Step 0 200 400 600 800 1128
131
  ```
132
 
133
+ - **Descent:** Initial cross-entropy loss started at `4.27` and settled down to `0.3500` training loss.
134
+ - **Gradient Norms:** Held between `0.07` and `0.09` across all epochs with stable gradient flow.
135
+ - **Cosine Schedule:** 10% warmup into smooth cosine decay prevented catastrophic forgetting of base model reasoning.
136
 
137
  ---
138
 
139
+ ## Architecture & Efficiency
140
 
141
  Shruti-Soft is powered by LiquidAI's hybrid **LIV (Linear Time-Invariant Conv) + Grouped-Query Attention (GQA)** architecture:
142
  - **Low VRAM Footprint:** Runs comfortably in ~5.4 GB VRAM in bfloat16, or under 2.5 GB with 4-bit quantization (GGUF / AWQ / bitsandbytes).
143
+ - **Inference Speed:** High token-per-second generation speeds on consumer GPUs (RTX 3060/4060) and Apple Silicon.
144
 
145
  ---
146
 
147
+ ## Quick Start & Usage
148
 
149
  ### 1. Standard HuggingFace Transformers
150
 
 
214
 
215
  ---
216
 
217
+ ## Recommended Sampling Parameters
218
 
219
+ To get the most natural and expressive output from Shruti, use these sampling configurations:
220
 
221
  | Parameter | Recommended | Range | Impact |
222
  |---|:---:|:---:|---|
223
+ | **Temperature** | `0.75` | `0.65 - 0.85` | Lower for focused comforting chats; higher for creative banter |
224
+ | **Top-P (Nucleus)** | `0.90` | `0.85 - 0.95` | Maintains vocabulary richness while preventing erratic tokens |
225
+ | **Repetition Penalty** | `1.05` | `1.02 - 1.08` | Prevents looping without punishing natural emotional emphasis |
226
+ | **Max New Tokens** | `200` | `100 - 400` | Conversational sweet spot for natural human-length texting |
227
 
228
  ---
229
 
230
+ ## Prompt Format (ChatML)
231
 
232
  Shruti expects standard **ChatML** formatting:
233
 
 
242
 
243
  ---
244
 
245
+ ## License & Attribution
246
 
247
  - **Base Model:** LiquidAI LFM2.5-2.6B (`Umranz/LFM2.5-2.6B-heretic`)
248
  - **License:** Apache 2.0
249
  - **Fine-tuning & Dataset Architecture:** Umran ([@Umranz](https://huggingface.co/Umranz))
250
  - **Training Infrastructure:** Modal.com (A100-80GB) via LiquidAI LEAP Finetune