README.md

# mmBERT-32K Feedback Detector (LoRA Adapter)

LoRA adapter for 4-class user feedback/satisfaction classification based on **mmBERT-32K-YaRN**.

## Model Description

This is the LoRA adapter version. For the merged model, see [mmbert32k-feedback-detector-merged](https://huggingface.co/llm-semantic-router/mmbert32k-feedback-detector-merged).

### Classes
- **SAT**: User is satisfied
- **NEED_CLARIFICATION**: User needs more explanation  
- **WRONG_ANSWER**: System provided incorrect information
- **WANT_DIFFERENT**: User wants alternative options

### Base Model
- **Base**: [llm-semantic-router/mmbert-32k-yarn](https://huggingface.co/llm-semantic-router/mmbert-32k-yarn)
- **Architecture**: ModernBERT with YaRN RoPE scaling
- **Context Length**: 32,768 tokens

### LoRA Configuration
- **Rank**: 8
- **Alpha**: 16
- **Dropout**: 0.1
- **Target Modules**: attn.Wqkv, attn.Wo, mlp.Wi, mlp.Wo
- **Trainable Parameters**: 1.69M (0.55% of base model)

### Performance

| Metric | Score |
|--------|-------|
| **Accuracy** | 98.46% |
| **F1 Macro** | 97.69% |

## Usage

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel

# Load base model and adapter
base_model = "llm-semantic-router/mmbert-32k-yarn"
adapter = "llm-semantic-router/mmbert32k-feedback-detector-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForSequenceClassification.from_pretrained(base_model, num_labels=4)
model = PeftModel.from_pretrained(model, adapter)

# Inference
text = "Thanks, that's exactly what I needed!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
```

## License

Apache 2.0