File size: 2,080 Bytes
f88c5b0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
language:
- en
license: apache-2.0
tags:
- rag
- faithfulness
- hallucination-detection
- lora
- microguard
datasets:
- galileo-ai/ragbench
- wandb/RAGTruth-processed
- PatronusAI/HaluBench
metrics:
- balanced_accuracy
- f1
pipeline_tag: text-classification
base_model: Qwen/Qwen2.5-0.5B-Instruct
---
# MicroGuard — Qwen-0.5B
A LoRA-adapted faithfulness classifier for RAG systems. Detects whether a generated answer is faithful to the retrieved context.
## Performance
| Metric | Value |
|--------|-------|
| Balanced Accuracy | 67.6% |
| F1 Score | 0.698 |
| Cohen's Kappa | 0.401 |
| Inference Latency | 56ms |
Evaluated on a combined test set of 15,976 examples from RAGBench, RAGTruth, and HaluBench.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base, "tarun5986/MicroGuard-Qwen-0.5B")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
# Or use the MicroGuard package
from microguard import MicroGuard
guard = MicroGuard(model="tarun5986/MicroGuard-Qwen-0.5B", base_model="Qwen/Qwen2.5-0.5B-Instruct")
result = guard.check(
context="The Eiffel Tower was built in 1889 by Gustave Eiffel.",
question="Who built the Eiffel Tower?",
answer="The Eiffel Tower was built by Gustave Eiffel in 1889."
)
print(result) # {'verdict': 'FAITHFUL', 'confidence': 74.2, 'latency_ms': 64.0}
```
## Training
- **Method**: LoRA (r=16, alpha=32, targets: q,k,v,o projections)
- **Data**: 127,932 examples from RAGBench + RAGTruth + HaluBench
- **Evaluation**: Constrained decoding via logit comparison (0% garbage outputs)
## Paper
[MicroGuard: Sub-Billion Parameter Faithfulness Classification for Real-Time RAG QA](https://github.com/tarun-ks/MicroGuard)
## Citation
```bibtex
@article{microguard2026,
title={MicroGuard: Sub-Billion Parameter Faithfulness Classification for Real-Time RAG QA},
author={Sharma, Tarun},
journal={IEEE Access},
year={2026}
}
```
|