---
license: apache-2.0
base_model: unsloth/Qwen3.5-2B
tags:
  - qwen3.5
  - voice-assistant
  - conversational
  - sft
  - lora
  - unsloth
  - gguf
language:
  - en
pipeline_tag: text-generation
library_name: transformers
---

# Qwen3.5-2B Voice Assistant

Fine-tuned [Qwen3.5-2B](https://huggingface.co/unsloth/Qwen3.5-2B) for voice assistant / conversational use.

This is designed to be short responses without thinking.

Trained on curated, concise datasets — all assistant responses are short and natural-sounding, optimized for spoken output rather than written text.

## Training Details

| Parameter | Value |
|---|---|
| Base model | `unsloth/Qwen3.5-2B` |
| Method | LoRA (rank=16, alpha=32) |
| LoRA dropout | 0.05 |
| Learning rate | 0.0001 |
| Epochs | 3 (early stopping, patience=4) |
| Effective batch size | 64 |
| Max sequence length | 1024 |
| Scheduler | Cosine with 50 warmup steps |
| Precision | bf16 |
| Thinking mode | Disabled |
| GPU | NVIDIA L4 (22 GB) |
| Framework | [Unsloth](https://unsloth.ai) + TRL SFTTrainer |

## Datasets

All datasets are filtered for concise, voice-friendly assistant responses (20–400 chars for general data,
20–500 chars for reasoning). Responses containing markdown formatting (bold, inline code, numbered lists,
bullet points, headings) are excluded. Exact-match deduplication is applied across all sources before training.

| Dataset | Rows | Purpose |
|---|---|---|
| [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25) | 2,388 | Real human multi-turn conversations |
| [HuggingFaceTB/everyday-conversations-llama3.1-2k](https://huggingface.co/datasets/HuggingFaceTB/everyday-conversations-llama3.1-2k) | 1,910 | Greetings, small talk, basic Q&A |
| [argilla/synthetic-concise-reasoning-sft](https://huggingface.co/datasets/argilla/synthetic-concise-reasoning-sft) | 535 | Short factual reasoning answers |
| [WizardLM/WizardLM_evol_instruct_70k](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k) | 7,000 | Casual single-turn Q&A |
| **Duplicates removed** | **1,992** | |
| **Total (after dedup)** | **9,841** | |

## Filtering Pipeline (v7)

Each assistant response is checked against the following before inclusion:

- **Length:** 20–400 chars (general), 20–500 chars (reasoning)
- **No markdown:** `**bold**`, `` `inline code` ``, `[link](url)`, `# headings` all excluded
- **No lists:** numbered (`1.`) and bullet (`-`, `*`) patterns excluded at line-start and after colons
- **No list lead-ins:** phrases like *"the process involves:"*, *"as follows:"*, *"the following"* excluded
- **No AI-isms:** *"certainly!"*, *"as an AI"*, *"in conclusion"*, *"delve"* excluded
- **Post-dedup sanity check:** % of markdown patterns logged to W&B before training

## System Prompt

All training samples include this system prompt:

> You are a casual, hands-free voice assistant. Speak in short, punchy sentences as if we are having a real-time conversation. Never use bullet points, markdown, or code. If explaining a complex topic, use a simple, everyday analogy. Respond immediately without any preamble or internal monologue.

## Available Formats

| Repo | Format | Use case |
|---|---|---|
| [cowWhySo/qwen3_5_2B_voice_assistant](https://huggingface.co/cowWhySo/qwen3_5_2B_voice_assistant) | Merged 16-bit | Transformers / vLLM / SGLang |
| [cowWhySo/qwen3_5_2B_voice_assistant-lora](https://huggingface.co/cowWhySo/qwen3_5_2B_voice_assistant-lora) | LoRA adapters | Merge with base yourself |
| [cowWhySo/qwen3_5_2B_voice_assistant-GGUF](https://huggingface.co/cowWhySo/qwen3_5_2B_voice_assistant-GGUF) | GGUF (q4_k_m, q5_k_m, q8_0, f16) | llama.cpp / Ollama / LM Studio |

## Usage with llama.cpp
```bash
huggingface-cli download cowWhySo/qwen3_5_2B_voice_assistant-GGUF --include "*q4_k_m*" --local-dir .
./llama-cli -m *q4_k_m*.gguf --ctx-size 2048 --temp 0.7 --top-p 0.9
```

## Usage with Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("cowWhySo/qwen3_5_2B_voice_assistant")
tokenizer = AutoTokenizer.from_pretrained("cowWhySo/qwen3_5_2B_voice_assistant")

messages = [
    {"role": "system", "content": "You are a casual, hands-free voice assistant. Speak in short, punchy sentences as if we are having a real-time conversation. Never use bullet points, markdown, or code. If explaining a complex topic, use a simple, everyday analogy. Respond immediately without any preamble or internal monologue."},
    {"role": "user", "content": "What's the weather like today?"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

Fine-tuned with [Unsloth](https://unsloth.ai) on an NVIDIA L4 GPU.