Update model card with full 3-stage training pipeline explanation

README.md

@@ -1,21 +1,156 @@
 ---
+language:
+- en
 tags:
-- gguf
-- llama.cpp
+- medical
+- triage
+- emergency-medicine
+- esi
+- dpo
+- rlhf
 - unsloth
+- qwen3
 - vision-language-model
+base_model: Qwen/Qwen3.5-9B
+license: apache-2.0
 ---
 
-# qwen3.5-medical-ft-stage3-dpo : GGUF
+# qwen3.5-medical-ft-stage3-dpo
 
-This model was finetuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
+**Qwen3.5-9B** fine-tuned through a three-stage supervised + preference-alignment pipeline for **emergency department triage** using the [Emergency Severity Index (ESI)](https://www.acep.org/patient-care/esi/) 1–5 scale.
 
-**Example usage**:
-- For text only LLMs:    `llama-cli -hf vadimbelsky/qwen3.5-medical-ft-stage3-dpo --jinja`
-- For multimodal models: `llama-mtmd-cli -hf vadimbelsky/qwen3.5-medical-ft-stage3-dpo --jinja`
+This repository contains the **Stage 3** merged 16-bit weights — the final production model.
+
+---
+
+## Training Pipeline
+
+### Stage 1 — Domain SFT (medical Q&A)
+- **Base model**: `Qwen/Qwen3.5-9B`
+- **Dataset**: [`vadimbelsky/medical-triage-qa-50k`](https://huggingface.co/datasets/vadimbelsky/medical-triage-qa-50k) — 50 k medical triage Q&A pairs
+- **Method**: Supervised fine-tuning with LoRA (r=16) via Unsloth + TRL SFTTrainer
+- **Goal**: Inject emergency medicine domain knowledge and ESI reasoning into the base model
+
+### Stage 2 — Continued SFT (intake notes)
+- **Base model**: Stage 1 LoRA merged into base weights
+- **Dataset**: `intake_notes_10k.jsonl` — 10 k real-format SOAP intake notes with structured triage decisions
+- **Method**: Continued SFT with LoRA (r=32, alpha=64) — doubled rank to capture finer-grained triage reasoning
+- **Goal**: Align the model to the exact input/output format used in clinical practice (SOAP note → ESI level + justification + interventions)
+
+### Stage 3 — DPO Alignment (this model)
+- **Base model**: Stage 2 LoRA checkpoint
+- **Dataset**: `dpo_dataset_clean.jsonl` — preference pairs targeting over-triage correction
+- **Method**: Direct Preference Optimization ([DPO](https://arxiv.org/abs/2305.18290)) via TRL DPOTrainer
+- **Goal**: Reduce systematic over-escalation of low-acuity patients (ESI 3/4/5 → ESI 1/2) while preserving 100% recall on genuinely high-risk patients
+
+#### DPO hyperparameters
+| Parameter | Value | Notes |
+|-----------|-------|-------|
+| beta (KL penalty) | 0.3 | beta=0.1 caused reward margin to explode to 31, leading to under-triage |
+| Loss type | sigmoid | Standard DPO |
+| Learning rate | 5e-5 | Lower than SFT to avoid catastrophic forgetting |
+| Epochs | 0.15 | DPO overfits fast — loss hits zero well before epoch 1 |
+| Effective batch size | 16 | 2 x device batch x 8 gradient accumulation steps |
+| LoRA r / alpha | 8 / 8 | Conservative rank — DPO requires minimal capacity |
+| Optimizer | AdamW 8-bit | |
+| Precision | BF16 | |
+
+---
+
+## Model Task
+
+Given a **SOAP intake note**, the model produces a structured triage decision:
+
+- **ESI level** (1–5) with clinical justification
+- **Key clinical findings** driving the decision
+- **Time-to-provider target**
+- **Immediate interventions** required
+
+**System prompt**:
+```
+You are an expert emergency medicine triage nurse. Given a SOAP intake note, provide a structured triage decision including ESI level with justification, key clinical findings, time-to-provider target, and any immediate interventions required.
+```
+
+### ESI Scale Reference
+| Level | Acuity | Time-to-Provider |
+|-------|--------|-----------------|
+| ESI 1 | Immediate life threat | Immediate |
+| ESI 2 | High risk / emergent | < 10 min |
+| ESI 3 | Urgent, stable | 30–60 min |
+| ESI 4 | Less urgent | 1–2 hours |
+| ESI 5 | Non-urgent | 2–4 hours |
+
+---
+
+## Stage 3 Training Targets
+
+| Metric | Target | Pre-DPO Baseline |
+|--------|--------|-----------------|
+| Overall accuracy | > 82% | — |
+| Over-triage rate (ESI 3/4/5 escalated to 1/2) | < 10% | 22.2% |
+| Under-triage rate | < 6% | — |
+| High-risk recall (ESI 1/2) | 100% | 100% |
+| ESI 3 accuracy | > 65% | ~40% |
+
+---
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `model.safetensors-0000{1-4}-of-00004.safetensors` | Merged 16-bit weights (4-shard) |
+| `config.json` | Model configuration |
+| `tokenizer.json` / `tokenizer_config.json` | Tokenizer |
+| `processor_config.json` | Vision processor config |
+| `chat_template.jinja` | Chat template |
+
+---
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "vadimbelsky/qwen3.5-medical-ft-stage3-dpo",
+    torch_dtype="auto",
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained("vadimbelsky/qwen3.5-medical-ft-stage3-dpo")
+
+SYSTEM_PROMPT = (
+    "You are an expert emergency medicine triage nurse. "
+    "Given a SOAP intake note, provide a structured triage decision including "
+    "ESI level with justification, key clinical findings, time-to-provider target, "
+    "and any immediate interventions required."
+)
+
+soap_note = """S (Subjective): 58-year-old male with sudden onset chest pain radiating to left arm, diaphoresis, onset 30 minutes ago...
+O (Objective): BP 160/95, HR 110, RR 22, SpO2 94% on room air, Temp 37.1C..."""
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {"role": "user", "content": soap_note},
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(inputs, max_new_tokens=512)
+print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+
+---
+
+## Training Infrastructure
+
+Trained with [Unsloth](https://github.com/unslothai/unsloth) for 2x faster throughput and reduced VRAM usage.
 
-## Available Model files:
-- `merged_stage1.F16.gguf`
-- `merged_stage1.BF16-mmproj.gguf`
-This was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+
+---
+
+## Disclaimer
+
+This model is intended for **research and educational purposes only**. It is not validated for clinical use and must not be used to make real patient triage decisions without oversight from licensed medical professionals.