Instructions to use vadimbelsky/qwen3.5-medical-ft-stage3-dpo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use vadimbelsky/qwen3.5-medical-ft-stage3-dpo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="vadimbelsky/qwen3.5-medical-ft-stage3-dpo")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("vadimbelsky/qwen3.5-medical-ft-stage3-dpo")
model = AutoModelForMultimodalLM.from_pretrained("vadimbelsky/qwen3.5-medical-ft-stage3-dpo", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use vadimbelsky/qwen3.5-medical-ft-stage3-dpo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vadimbelsky/qwen3.5-medical-ft-stage3-dpo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vadimbelsky/qwen3.5-medical-ft-stage3-dpo",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/vadimbelsky/qwen3.5-medical-ft-stage3-dpo

SGLang

How to use vadimbelsky/qwen3.5-medical-ft-stage3-dpo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "vadimbelsky/qwen3.5-medical-ft-stage3-dpo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vadimbelsky/qwen3.5-medical-ft-stage3-dpo",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "vadimbelsky/qwen3.5-medical-ft-stage3-dpo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vadimbelsky/qwen3.5-medical-ft-stage3-dpo",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use vadimbelsky/qwen3.5-medical-ft-stage3-dpo with Docker Model Runner:
```
docker model run hf.co/vadimbelsky/qwen3.5-medical-ft-stage3-dpo
```

vadimbelsky commited on Mar 24

Commit

c43ba49

verified ·

1 Parent(s): b5731e9

Unsloth Model Card

Browse files

Files changed (1) hide show

README.md +11 -146

README.md CHANGED Viewed

@@ -1,156 +1,21 @@
 ---
-language:
-- en
 tags:
-- medical
-- triage
-- emergency-medicine
-- esi
-- dpo
-- rlhf
 - unsloth
-- qwen3
-- vision-language-model
-base_model: Qwen/Qwen3.5-9B
 license: apache-2.0
 ---
-# qwen3.5-medical-ft-stage3-dpo
-**Qwen3.5-9B** fine-tuned through a three-stage supervised + preference-alignment pipeline for **emergency department triage** using the [Emergency Severity Index (ESI)](https://www.acep.org/patient-care/esi/) 1–5 scale.
-This repository contains the **Stage 3** merged 16-bit weights — the final production model.
----
-## Training Pipeline
-### Stage 1 — Domain SFT (medical Q&A)
-- **Base model**: `Qwen/Qwen3.5-9B`
-- **Dataset**: [`vadimbelsky/medical-triage-qa-50k`](https://huggingface.co/datasets/vadimbelsky/medical-triage-qa-50k) — 50 k medical triage Q&A pairs
-- **Method**: Supervised fine-tuning with LoRA (r=16) via Unsloth + TRL SFTTrainer
-- **Goal**: Inject emergency medicine domain knowledge and ESI reasoning into the base model
-### Stage 2 — Continued SFT (intake notes)
-- **Base model**: Stage 1 LoRA merged into base weights
-- **Dataset**: `intake_notes_10k.jsonl` — 10 k real-format SOAP intake notes with structured triage decisions
-- **Method**: Continued SFT with LoRA (r=32, alpha=64) — doubled rank to capture finer-grained triage reasoning
-- **Goal**: Align the model to the exact input/output format used in clinical practice (SOAP note → ESI level + justification + interventions)
-### Stage 3 — DPO Alignment (this model)
-- **Base model**: Stage 2 LoRA checkpoint
-- **Dataset**: `dpo_dataset_clean.jsonl` — preference pairs targeting over-triage correction
-- **Method**: Direct Preference Optimization ([DPO](https://arxiv.org/abs/2305.18290)) via TRL DPOTrainer
-- **Goal**: Reduce systematic over-escalation of low-acuity patients (ESI 3/4/5 → ESI 1/2) while preserving 100% recall on genuinely high-risk patients
-#### DPO hyperparameters
-| Parameter | Value | Notes |
-|-----------|-------|-------|
-| beta (KL penalty) | 0.3 | beta=0.1 caused reward margin to explode to 31, leading to under-triage |
-| Loss type | sigmoid | Standard DPO |
-| Learning rate | 5e-5 | Lower than SFT to avoid catastrophic forgetting |
-| Epochs | 0.15 | DPO overfits fast — loss hits zero well before epoch 1 |
-| Effective batch size | 16 | 2 x device batch x 8 gradient accumulation steps |
-| LoRA r / alpha | 8 / 8 | Conservative rank — DPO requires minimal capacity |
-| Optimizer | AdamW 8-bit | |
-| Precision | BF16 | |
----
-## Model Task
-Given a **SOAP intake note**, the model produces a structured triage decision:
-- **ESI level** (1–5) with clinical justification
-- **Key clinical findings** driving the decision
-- **Time-to-provider target**
-- **Immediate interventions** required
-**System prompt**:
-```
-You are an expert emergency medicine triage nurse. Given a SOAP intake note, provide a structured triage decision including ESI level with justification, key clinical findings, time-to-provider target, and any immediate interventions required.
-```
-### ESI Scale Reference
-| Level | Acuity | Time-to-Provider |
-|-------|--------|-----------------|
-| ESI 1 | Immediate life threat | Immediate |
-| ESI 2 | High risk / emergent | < 10 min |
-| ESI 3 | Urgent, stable | 30–60 min |
-| ESI 4 | Less urgent | 1–2 hours |
-| ESI 5 | Non-urgent | 2–4 hours |
----
-## Stage 3 Training Targets
-| Metric | Target | Pre-DPO Baseline |
-|--------|--------|-----------------|
-| Overall accuracy | > 82% | — |
-| Over-triage rate (ESI 3/4/5 escalated to 1/2) | < 10% | 22.2% |
-| Under-triage rate | < 6% | — |
-| High-risk recall (ESI 1/2) | 100% | 100% |
-| ESI 3 accuracy | > 65% | ~40% |
----
-## Files
-| File | Description |
-|------|-------------|
-| `model.safetensors-0000{1-4}-of-00004.safetensors` | Merged 16-bit weights (4-shard) |
-| `config.json` | Model configuration |
-| `tokenizer.json` / `tokenizer_config.json` | Tokenizer |
-| `processor_config.json` | Vision processor config |
-| `chat_template.jinja` | Chat template |
----
-## Usage
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained(
-    "vadimbelsky/qwen3.5-medical-ft-stage3-dpo",
-    torch_dtype="auto",
-    device_map="auto",
-)
-tokenizer = AutoTokenizer.from_pretrained("vadimbelsky/qwen3.5-medical-ft-stage3-dpo")
-SYSTEM_PROMPT = (
-    "You are an expert emergency medicine triage nurse. "
-    "Given a SOAP intake note, provide a structured triage decision including "
-    "ESI level with justification, key clinical findings, time-to-provider target, "
-    "and any immediate interventions required."
-)
-soap_note = """S (Subjective): 58-year-old male with sudden onset chest pain radiating to left arm, diaphoresis, onset 30 minutes ago...
-O (Objective): BP 160/95, HR 110, RR 22, SpO2 94% on room air, Temp 37.1C..."""
-messages = [
-    {"role": "system", "content": SYSTEM_PROMPT},
-    {"role": "user", "content": soap_note},
-]
-inputs = tokenizer.apply_chat_template(
-    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
-).to(model.device)
-outputs = model.generate(inputs, max_new_tokens=512)
-print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
-```
----
-## Training Infrastructure
-Trained with [Unsloth](https://github.com/unslothai/unsloth) for 2x faster throughput and reduced VRAM usage.
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
----
-## Disclaimer
-This model is intended for **research and educational purposes only**. It is not validated for clinical use and must not be used to make real patient triage decisions without oversight from licensed medical professionals.

 ---
+base_model: vadimbelsky/qwen3.5-medical-ft-stage3-dpo
 tags:
+- text-generation-inference
+- transformers
 - unsloth
+- qwen3_5
 license: apache-2.0
+language:
+- en
 ---
+# Uploaded finetuned  model
+- **Developed by:** vadimbelsky
+- **License:** apache-2.0
+- **Finetuned from model :** vadimbelsky/qwen3.5-medical-ft-stage3-dpo
+This qwen3_5 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)