---
base_model: BioMistral/BioMistral-7B
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:BioMistral/BioMistral-7B
- lora
- sft
- transformers
- trl
- unsloth
- medical
- indian-healthcare
- multilingual
- clinical-nlp
license: mit
language:
- en
- hi
---

# BioMistral-7B Fine-Tuned on Indian Medical Data

A domain-adapted clinical LLM fine-tuned on synthetic Indian medical Q&A records using QLoRA (4-bit quantization) with Unsloth 2x speedup. Built to power the conversational AI layer.



---

## Model Details

- **Developed by:** B. Nikita Reddy
- **Model type:** Causal LLM — BioMistral-7B + LoRA adapter (PEFT)
- **Languages:** English, Hindi
- **License:** MIT
- **Base model:** BioMistral/BioMistral-7B

---

## What It Does

- Generates structured treatment recommendations — Allopathy, Homeopathy, Home Remedy
- Supports voice input via Whisper ASR and returns spoken responses via TTS
- Feeds patient vitals and symptom data into an **XGBoost-based risk analyzer** that scores patient risk from 0–100 (Low / Moderate / High / Critical) with SHAP explainability

---

## Training Details

| Parameter | Value |
|---|---|
| Base Model | BioMistral/BioMistral-7B |
| Method | QLoRA (4-bit) + Unsloth 2x speedup |
| Dataset | Synthetic Indian medical Q&A records |
| Hardware | Kaggle T4 GPU (15.6 GB VRAM) |
| Training Time | ~9.5 hours |
| Steps / Epochs | 873 steps, 5 epochs |
| Parameters Trained | ~0.5% (LoRA only) |
| Adapter Size | 167.8 MB LoRA safetensors |
| Final Train Loss | 0.065 |
| Final Val Loss | 0.163 (target < 1.0 ✅) |

---

### Loss Curve (per checkpoint)

| Step | Train Loss | Val Loss |
|---|---|---|
| 200 | 0.129 | 0.219 |
| 400 | 0.094 | 0.188 |
| 600 | 0.076 | 0.176 |
| 800 | 0.068 | 0.163 |
| 873 | 0.065 | 0.163 |

Checkpoints saved at steps 600, 800, and 873.

---

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("BioMistral/BioMistral-7B")
model = PeftModel.from_pretrained(base_model, "YOUR_HF_USERNAME/biomistral-7b-indian-medical")
tokenizer = AutoTokenizer.from_pretrained("BioMistral/BioMistral-7B")

prompt = "Patient reports bukhar (fever) for 2 days. Suggest treatment."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Preview

![VitalKiosk Live Demo](https://huggingface.co/nikitaredy/medictron-7B/resolve/main/real-time.png)
---

## Limitations

- Not a replacement for licensed medical advice
- Optimised for Indian medical terminology — may underperform in other clinical contexts
- Hindi coverage may not extend to all regional dialects
- synthetic training samples — rare conditions may be underrepresented

---

## Contact

B. Nikita Reddy 

### Framework Versions
- PEFT 0.18.1
- Unsloth (latest at training time)
- Transformers (latest at training time)