faizath commited on
Commit
57309cb
·
verified ·
1 Parent(s): c03054d

docs: replace fundusnap-ai with the four split AI repositories

Browse files

fundusnap-ai was retired and split into a dedicated lesion detector and
severity classifier, and two further AI repositories have since been
published. Both component listings still pointed at the retired
repository, leaving dead links and understating the model and dataset
work that is now public.

Claude-Session: https://claude.ai/code/session_01L5wZb1mDtLnrfAXkzSrTFD

Files changed (2) hide show
  1. README.md +201 -28
  2. profile/README.md +202 -29
README.md CHANGED
@@ -46,7 +46,7 @@
46
 
47
  **Fundusnap** is a comprehensive medical-imaging solution that helps healthcare workers and patients **detect and analyze diabetic retinopathy (DR)** from *fundus* (retinal) images. A user captures a photo of the back of the eye with the mobile app, and Fundusnap returns an AI classification of disease severity, highlights the specific retinal lesions it found, and lets the user ask follow-up questions to an AI medical assistant that explains the result in plain language.
48
 
49
- It is delivered as an end-to-end product spanning a **mobile app**, a **backend API**, a **marketing/management website**, and an **offline AI model** for low-connectivity environments.
50
 
51
  ## 🩺 The Problem
52
 
@@ -62,10 +62,10 @@ Diabetic retinopathy is one of the leading causes of preventable blindness world
62
  Fundusnap brings specialist-grade screening to a smartphone and makes the result understandable to everyone:
63
 
64
  1. **Capture** — The Flutter mobile app guides users to take a high-quality fundus image (with photo and video capture support).
65
- 2. **Classify** — The image is sent to the API, which runs it through **Microsoft Azure Custom Vision** to classify the severity of diabetic retinopathy.
66
- 3. **Detect** — A **custom object-detection AI** locates and bounds individual retinal artifacts/lesions (e.g. microaneurysms), so the result is explainable rather than a black box.
67
- 4. **Explain** — An **AI medical chat assistant** (Microsoft's **Phi-4** model via OpenRouter) interprets the findings in simple, informative language and encourages appropriate follow-up with a healthcare professional — without making a clinical diagnosis.
68
- 5. **Stay available offline** — A separate, **offline-capable image-classification model** acts as a fallback for poor connectivity or primary-API outages, so screening keeps working where it's needed most.
69
 
70
  All medical data is handled with security and compliance in mind (JWT-based auth, encrypted transmission, and secure image storage).
71
 
@@ -151,12 +151,15 @@ Fundusnap was built for and submitted to three national programs in Indonesia, a
151
 
152
  ## 🧩 Project Components
153
 
154
- | Component | Repository | Deployment |
155
- | --- | --- | --- |
156
- | 📱 Mobile App | [fundusnap/fundusnap-app](https://github.com/fundusnap/fundusnap-app) | Android APK release |
157
- | 🌐 Website | [fundusnap/fundusnap-web](https://github.com/fundusnap/fundusnap-web) | [fundusnap.faizath.com](https://fundusnap.faizath.com) |
158
- | ⚙️ Backend API | [fundusnap/fundusnap-api](https://github.com/fundusnap/fundusnap-api) | [fundusnap-api.faizath.com](https://fundusnap-api.faizath.com) |
159
- | 🧠 Offline AI Model | [fundusnap/fundusnap-ai](https://github.com/fundusnap/fundusnap-ai) | Self-hosted inference service |
 
 
 
160
 
161
  <br/>
162
 
@@ -251,7 +254,7 @@ Fundusnap was built for and submitted to three national programs in Indonesia, a
251
  <img src="https://img.shields.io/badge/OpenRouter-6566F1?style=flat-square&logo=openai&logoColor=white" alt="OpenRouter"/>
252
  </p>
253
  <p>
254
- <b>🤖 AI services:</b> Azure Custom Vision (DR classification) · Custom object-detection AI (retinal artifact detection) · Microsoft <b>Phi-4</b> via OpenRouter (medical chat)
255
  </p>
256
  <p>
257
  <a href="https://github.com/fundusnap/fundusnap-api"><img src="https://img.shields.io/badge/Repository-fundusnap--api-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
@@ -270,55 +273,225 @@ Fundusnap was built for and submitted to three national programs in Indonesia, a
270
  - **Authentication:** JWT (access + refresh tokens)
271
  - **Storage:** Cloudflare R2 (with Azure Blob Storage support)
272
  - **AI Services:**
273
- - Microsoft Azure Custom Vision API (DR classification)
274
- - Custom object-detection AI (retinal artifact detection)
275
- - OpenRouter API with Microsoft's **Phi-4** model (medical chat)
 
 
276
  - **Email Service:** Nodemailer
277
 
278
  </details>
279
 
280
  <br/>
281
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
282
  <table>
283
  <tr>
284
  <td width="64" align="center" valign="top">
285
  <h1>🧠</h1>
286
  </td>
287
  <td valign="top">
288
- <h3>Fundusnap AI</h3>
289
- <p>An offline-capable image-classification model that keeps screening working under poor connectivity or primary-API outages.</p>
290
  <p>
291
  <img src="https://img.shields.io/badge/FastAI-2EC4B6?style=flat-square&logo=fastapi&logoColor=white" alt="FastAI"/>
292
  <img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=pytorch&logoColor=white" alt="PyTorch"/>
 
293
  <img src="https://img.shields.io/badge/ResNet34-FF6F00?style=flat-square&logo=tensorflow&logoColor=white" alt="ResNet34"/>
294
- <img src="https://img.shields.io/badge/Albumentations-5C3EE8?style=flat-square&logo=opencv&logoColor=white" alt="Albumentations"/>
 
295
  </p>
296
  <p>
297
- <b>📊 Performance:</b> <code>81%</code> accuracy · <code>0.81</code> macro F1 · <code>0.81</code> weighted F1
298
  </p>
299
  <p>
300
- <a href="https://github.com/fundusnap/fundusnap-ai"><img src="https://img.shields.io/badge/Repository-fundusnap--ai-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
301
- &nbsp;<img src="https://img.shields.io/badge/Deployment-Self--hosted_service-555555?style=flat-square&logo=docker&logoColor=white" alt="Self-hosted"/>
 
 
 
 
302
  </p>
303
  </td>
304
  </tr>
305
  </table>
306
 
307
  <details>
308
- <summary><b>Full tech stack — Offline AI Model</b></summary>
309
 
310
- - **Deep Learning Framework:** FastAI
311
- - **Base Model:** ResNet34 (pretrained)
312
- - **Data Augmentation:** Albumentations
313
  - **Loss Function:** Focal Loss
 
 
 
 
314
  - **Performance Metrics:**
315
- - Overall Accuracy: 81%
316
- - Macro Average F1-Score: 0.81
317
  - Weighted Average F1-Score: 0.81
318
- - **Deployment:** Self-hosted inference service (consumed by the API via `FUNDUSNAP_AI_HOST`)
 
 
319
 
320
  </details>
321
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
322
  ---
323
 
324
  ## 🔒 Security & Compliance
 
46
 
47
  **Fundusnap** is a comprehensive medical-imaging solution that helps healthcare workers and patients **detect and analyze diabetic retinopathy (DR)** from *fundus* (retinal) images. A user captures a photo of the back of the eye with the mobile app, and Fundusnap returns an AI classification of disease severity, highlights the specific retinal lesions it found, and lets the user ask follow-up questions to an AI medical assistant that explains the result in plain language.
48
 
49
+ It is delivered as an end-to-end product spanning a **mobile app**, a **backend API**, a **marketing/management website**, and a family of **open AI models** a retinal lesion detector, a diabetic-retinopathy severity classifier, and a result-explanation language model — together with the **synthetic dataset** that language model was trained on.
50
 
51
  ## 🩺 The Problem
52
 
 
62
  Fundusnap brings specialist-grade screening to a smartphone and makes the result understandable to everyone:
63
 
64
  1. **Capture** — The Flutter mobile app guides users to take a high-quality fundus image (with photo and video capture support).
65
+ 2. **Classify** — The image is sent to the API, which runs it through **Microsoft Azure Custom Vision** to grade the severity of diabetic retinopathy, with our own [**fundusnap-v1-severitycls-rn34-22m**](https://github.com/fundusnap/fundusnap-v1-severitycls-rn34-22m) ResNet34 grader as the open, self-hostable alternative.
66
+ 3. **Detect** — [**fundusnap-v1-lesiondet-yolo11m-20m**](https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m), a YOLO11m detector, locates and bounds individual retinal lesions and landmarks (microaneurysms, haemorrhages, exudates, optic disc, fovea), so the result is explainable rather than a black box.
67
+ 4. **Explain** — An **AI medical chat assistant** interprets the findings in simple, informative language and encourages appropriate follow-up with a healthcare professional — without making a clinical diagnosis. Two interchangeable backends serve this role: Microsoft's **Phi-4** via OpenRouter, and the self-hosted [**fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter**](https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter) — a MediPhi-Instruct LoRA fine-tuned on our [**FundusTalk v1**](https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) dataset to answer in Indonesian or English.
68
+ 5. **Stay available offline** — The severity classifier also ships as an **ONNX** graph for on-device inference, acting as a fallback for poor connectivity or primary-API outages, so screening keeps working where it's needed most.
69
 
70
  All medical data is handled with security and compliance in mind (JWT-based auth, encrypted transmission, and secure image storage).
71
 
 
151
 
152
  ## 🧩 Project Components
153
 
154
+ | Component | Repository | 🤗 Hub | Deployment |
155
+ | --- | --- | --- | --- |
156
+ | 📱 Mobile App | [fundusnap-app](https://github.com/fundusnap/fundusnap-app) | — | Android APK release |
157
+ | 🌐 Website | [fundusnap-web](https://github.com/fundusnap/fundusnap-web) | — | [fundusnap.faizath.com](https://fundusnap.faizath.com) |
158
+ | ⚙️ Backend API | [fundusnap-api](https://github.com/fundusnap/fundusnap-api) | — | [fundusnap-api.faizath.com](https://fundusnap-api.faizath.com) |
159
+ | ��� Lesion Detector | [fundusnap-v1-lesiondet-yolo11m-20m](https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m) | [model](https://huggingface.co/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m) | Self-hosted FastAPI service |
160
+ | 🧠 Severity Classifier | [fundusnap-v1-severitycls-rn34-22m](https://github.com/fundusnap/fundusnap-v1-severitycls-rn34-22m) | [model](https://huggingface.co/fundusnap/fundusnap-v1-severitycls-rn34-22m) | ONNX · offline-capable |
161
+ | 💬 Result Explainer | [fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter](https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter) | [model](https://huggingface.co/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter) | Self-hosted (merged → vLLM) |
162
+ | 📚 FundusTalk v1 Dataset | [fundusnap-fundustalk-v1-chatsft-11k](https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) | [dataset](https://huggingface.co/datasets/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) | Hugging Face dataset |
163
 
164
  <br/>
165
 
 
254
  <img src="https://img.shields.io/badge/OpenRouter-6566F1?style=flat-square&logo=openai&logoColor=white" alt="OpenRouter"/>
255
  </p>
256
  <p>
257
+ <b>🤖 AI services:</b> Azure Custom Vision (DR grading) · <a href="https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m">fundusnap-v1-lesiondet-yolo11m-20m</a> (lesion detection) · medical chat via Microsoft <b>Phi-4</b> on OpenRouter <i>or</i> the self-hosted <a href="https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter">MediPhi LoRA adapter</a>
258
  </p>
259
  <p>
260
  <a href="https://github.com/fundusnap/fundusnap-api"><img src="https://img.shields.io/badge/Repository-fundusnap--api-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
 
273
  - **Authentication:** JWT (access + refresh tokens)
274
  - **Storage:** Cloudflare R2 (with Azure Blob Storage support)
275
  - **AI Services:**
276
+ - Microsoft Azure Custom Vision API (DR severity grading)
277
+ - `fundusnap-v1-lesiondet-yolo11m-20m` self-hosted YOLO11m lesion-detection service
278
+ - Medical chat, two interchangeable backends:
279
+ - OpenRouter API with Microsoft's **Phi-4** model
280
+ - Self-hosted `fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter` (MediPhi-Instruct LoRA)
281
  - **Email Service:** Nodemailer
282
 
283
  </details>
284
 
285
  <br/>
286
 
287
+ <table>
288
+ <tr>
289
+ <td width="64" align="center" valign="top">
290
+ <h1>🔬</h1>
291
+ </td>
292
+ <td valign="top">
293
+ <h3>Fundusnap Lesion Detector — <code>fundusnap-v1-lesiondet-yolo11m-20m</code></h3>
294
+ <p>A YOLO11m object detector that finds <i>where</i> the findings are. Given one colour fundus photograph it returns bounding boxes for twelve classes — ten pathological findings plus the optic disc and fovea as anatomical landmarks — each with a label and a confidence score, so a severity grade comes with visual evidence instead of being a black box.</p>
295
+ <p>
296
+ <img src="https://img.shields.io/badge/YOLO11m-111F68?style=flat-square&logo=yolo&logoColor=white" alt="YOLO11m"/>
297
+ <img src="https://img.shields.io/badge/Ultralytics-0B23A9?style=flat-square&logo=ultralytics&logoColor=white" alt="Ultralytics"/>
298
+ <img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=pytorch&logoColor=white" alt="PyTorch"/>
299
+ <img src="https://img.shields.io/badge/FastAPI-009688?style=flat-square&logo=fastapi&logoColor=white" alt="FastAPI"/>
300
+ <img src="https://img.shields.io/badge/Params-20M-5B9BD5?style=flat-square" alt="20M parameters"/>
301
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
302
+ </p>
303
+ <p>
304
+ <b>📊 Performance:</b> <code>0.53</code> mAP@50 · <code>0.28</code> mAP@50-95 · <code>0.54</code> precision · <code>0.53</code> recall
305
+ </p>
306
+ <p>
307
+ <b>✨ Highlights:</b> 12 classes (10 lesions + <code>Disc</code>/<code>Fovea</code> landmarks) · JSON and annotated-image endpoints · Dockerised FastAPI service · exports to ONNX, TorchScript, TFLite, CoreML
308
+ </p>
309
+ <p>
310
+ <a href="https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m"><img src="https://img.shields.io/badge/Repository-fundusnap--v1--lesiondet--yolo11m--20m-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
311
+ &nbsp;<a href="https://huggingface.co/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Model-FFD21E?style=flat-square" alt="Hugging Face"/></a>
312
+ &nbsp;<img src="https://img.shields.io/badge/Deployment-Self--hosted_service-555555?style=flat-square&logo=docker&logoColor=white" alt="Self-hosted"/>
313
+ </p>
314
+ </td>
315
+ </tr>
316
+ </table>
317
+
318
+ <details>
319
+ <summary><b>Full tech stack — Lesion Detector</b></summary>
320
+
321
+ - **Framework:** Ultralytics `8.3.165` / PyTorch
322
+ - **Base Model:** `yolo11m.pt`, COCO-pretrained — `yolo11m.yaml` scale `m`, anchor-free `Detect` head, `nc=12`
323
+ - **Input:** 640×640, letterboxed (coordinates returned in the original image's pixel space)
324
+ - **Training:** 35 epochs, batch 16 (`nbs=64`), optimizer `auto` (`lr0=0.01`, `lrf=0.01`, momentum 0.937, weight decay 0.0005), 3 warmup epochs, AMP, seed 0 deterministic
325
+ - **Loss weights:** box 7.5 · cls 0.5 · dfl 1.5
326
+ - **Augmentation:** mosaic 1.0 (off for the last 10 epochs), `fliplr=0.5`, `scale=0.5`, `translate=0.1`, HSV (0.015/0.7/0.4), `erasing=0.4`, RandAugment
327
+ - **Shipped checkpoint:** epoch 27 — best by Ultralytics fitness (`0.1·mAP50 + 0.9·mAP50-95` = 0.3071), stripped of optimiser/EMA state (~40 MB, Git LFS)
328
+ - **Serving:** `POST /inspect/fundus-artifacts/` (JSON detections) · `POST /visualize/fundus-artifacts/` (annotated JPEG) · `GET /` (health) — `python:3.10-slim`, port 8000
329
+ - **License:** CC BY-NC 4.0 (weights derive from Ultralytics YOLO11 — review Ultralytics' AGPL-3.0 terms before redistributing)
330
+
331
+ </details>
332
+
333
+ <br/>
334
+
335
  <table>
336
  <tr>
337
  <td width="64" align="center" valign="top">
338
  <h1>🧠</h1>
339
  </td>
340
  <td valign="top">
341
+ <h3>Fundusnap Severity Classifier — <code>fundusnap-v1-severitycls-rn34-22m</code></h3>
342
+ <p>A ResNet34 classifier that grades <i>how severe</i> the retinopathy is. It predicts one of the five standard ordinal ICDR grades (0 = No DR through 4 = Proliferative) with a probability for each, and ships as an ONNX graph with a dynamic batch axis — which is what keeps screening working offline or during a primary-API outage.</p>
343
  <p>
344
  <img src="https://img.shields.io/badge/FastAI-2EC4B6?style=flat-square&logo=fastapi&logoColor=white" alt="FastAI"/>
345
  <img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=pytorch&logoColor=white" alt="PyTorch"/>
346
+ <img src="https://img.shields.io/badge/ONNX-005CED?style=flat-square&logo=onnx&logoColor=white" alt="ONNX"/>
347
  <img src="https://img.shields.io/badge/ResNet34-FF6F00?style=flat-square&logo=tensorflow&logoColor=white" alt="ResNet34"/>
348
+ <img src="https://img.shields.io/badge/Params-22M-5B9BD5?style=flat-square" alt="22M parameters"/>
349
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
350
  </p>
351
  <p>
352
+ <b>📊 Performance:</b> <code>0.82</code> accuracy · <code>0.8153</code> macro F1 · <code>0.81</code> macro precision · <code>0.82</code> macro recall
353
  </p>
354
  <p>
355
+ <b>✨ Highlights:</b> 5 ordinal ICDR grades · ONNX opset 14 with dynamic batch · fastai checkpoint for further fine-tuning · ONNX → TensorFlow/TFLite path for on-device inference
356
+ </p>
357
+ <p>
358
+ <a href="https://github.com/fundusnap/fundusnap-v1-severitycls-rn34-22m"><img src="https://img.shields.io/badge/Repository-fundusnap--v1--severitycls--rn34--22m-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
359
+ &nbsp;<a href="https://huggingface.co/fundusnap/fundusnap-v1-severitycls-rn34-22m"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Model-FFD21E?style=flat-square" alt="Hugging Face"/></a>
360
+ &nbsp;<img src="https://img.shields.io/badge/Deployment-ONNX_·_offline--capable-555555?style=flat-square&logo=onnx&logoColor=white" alt="ONNX / offline-capable"/>
361
  </p>
362
  </td>
363
  </tr>
364
  </table>
365
 
366
  <details>
367
+ <summary><b>Full tech stack — Severity Classifier</b></summary>
368
 
369
+ - **Deep Learning Framework:** FastAI / PyTorch, exported to ONNX (opset 14)
370
+ - **Base Model:** `resnet34`, ImageNet-pretrained (`timm/resnet34.tv_in1k`)
371
+ - **Head:** fastai default (`AdaptiveConcatPool2d` → BN/dropout → linear), `n_out=5`
372
  - **Loss Function:** Focal Loss
373
+ - **Input:** `Resize(224)` centre crop, ImageNet normalisation
374
+ - **Training:** batch 32, `learn.fine_tune(4)` (1 frozen + 4 unfrozen epochs), LR from `lr_find()` valley, seed 3865
375
+ - **Data Augmentation:** Albumentations — `ShiftScaleRotate`, `HorizontalFlip`, `RandomBrightnessContrast`, `HueSaturationValue`
376
+ - **Dataset:** Kaggle *resized-2015-2019-diabetic-retinopathy-detection* (EyePACS 2015 + APTOS 2019), each grade resampled to 10,000 rows for a 50,000-image balanced frame, 10% held out
377
  - **Performance Metrics:**
378
+ - Overall Accuracy: 0.82
379
+ - Macro Average F1-Score: 0.8153
380
  - Weighted Average F1-Score: 0.81
381
+ - Grades 3–4 separate near-perfectly (F1 0.97–0.98); grades 0/1/2 sit at 0.65–0.76
382
+ - **Deployment:** ONNX Runtime for inference (offline-capable), fastai checkpoint for fine-tuning
383
+ - **License:** CC BY-NC 4.0 (training data carries its own Kaggle / EyePACS / APTOS terms)
384
 
385
  </details>
386
 
387
+ <br/>
388
+
389
+ <table>
390
+ <tr>
391
+ <td width="64" align="center" valign="top">
392
+ <h1>💬</h1>
393
+ </td>
394
+ <td valign="top">
395
+ <h3>Fundusnap Result Explainer — <code>fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter</code></h3>
396
+ <p>A LoRA adapter over <a href="https://huggingface.co/microsoft/MediPhi-Instruct">microsoft/MediPhi-Instruct</a> that turns a prediction record into a plain-language explanation, in Indonesian or English. It is the conversational layer of the pipeline and the only model that never sees an image — it reads the severity probabilities and the lesion boxes the other two produce, and explains them without ever diagnosing.</p>
397
+ <p>
398
+ <img src="https://img.shields.io/badge/PEFT_LoRA-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="PEFT LoRA"/>
399
+ <img src="https://img.shields.io/badge/MediPhi--Instruct-0078D4?style=flat-square" alt="MediPhi-Instruct"/>
400
+ <img src="https://img.shields.io/badge/Transformers-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="Transformers"/>
401
+ <img src="https://img.shields.io/badge/FastAPI-009688?style=flat-square&logo=fastapi&logoColor=white" alt="FastAPI"/>
402
+ <img src="https://img.shields.io/badge/Params-3.8B_+_50M_LoRA-5B9BD5?style=flat-square" alt="3.8B + 50M LoRA"/>
403
+ <img src="https://img.shields.io/badge/Lang-id_|_en-5B9BD5?style=flat-square" alt="Indonesian and English"/>
404
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
405
+ </p>
406
+ <p>
407
+ <b>📊 Performance:</b> val loss <code>1.229</code> → <code>0.691</code> · <code>0/60</code> stub replies (base: 10/60) · <code>40/41</code> Indonesian prompts answered in Indonesian (base: 35/41)
408
+ </p>
409
+ <p>
410
+ <b>✨ Highlights:</b> explains rather than diagnoses — every conversation routes to a clinician · bilingual with code-switching · FastAPI <code>serve.py</code> · <code>merge.py</code> fuses the adapter into a standalone checkpoint for vLLM
411
+ </p>
412
+ <p>
413
+ <a href="https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter"><img src="https://img.shields.io/badge/Repository-fundusnap--v1--resultexp--clm--mediphi--3.8b--adapter-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
414
+ &nbsp;<a href="https://huggingface.co/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Model-FFD21E?style=flat-square" alt="Hugging Face"/></a>
415
+ &nbsp;<img src="https://img.shields.io/badge/Deployment-Self--hosted_·_vLLM-555555?style=flat-square&logo=docker&logoColor=white" alt="Self-hosted / vLLM"/>
416
+ </p>
417
+ </td>
418
+ </tr>
419
+ </table>
420
+
421
+ <details>
422
+ <summary><b>Full tech stack — Result Explainer</b></summary>
423
+
424
+ - **Base Model:** `microsoft/MediPhi-Instruct` (Phi-3 architecture, 3.82B)
425
+ - **Method:** QLoRA-style supervised fine-tuning, loss on assistant turns only
426
+ - **LoRA config:** r=32, α=64, dropout 0.05 — targets `qkv_proj`, `o_proj`, `gate_up_proj`, `down_proj`
427
+ - **Trainable params:** 50,331,648 (**1.30%** of the model)
428
+ - **Training Data:** [FundusTalk v1](https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) — 10,201 synthetic consultations distilled from `microsoft/phi-4` (~70% Indonesian incl. code-switched, ~30% English)
429
+ - **Schedule:** 2 epochs, 638 steps, lr 1e-4 cosine, effective batch 32, bf16
430
+ - **Hardware:** 1× A100-SXM4-40GB, ~170 min
431
+ - **Prompt envelope:** the exact three-system-message format the API sends (persona → severity JSON → detection JSON), byte-for-byte compatible with `JSON.stringify`
432
+ - **Serving:** `GET /` · `POST /chat` · `POST /prompt` (returns the envelope without generating) — or merge and serve with vLLM
433
+ - **Licensing:** weights CC BY-NC 4.0 · repository code MIT · base and teacher models both MIT
434
+
435
+ </details>
436
+
437
+ <br/>
438
+
439
+ <table>
440
+ <tr>
441
+ <td width="64" align="center" valign="top">
442
+ <h1>📚</h1>
443
+ </td>
444
+ <td valign="top">
445
+ <h3>FundusTalk v1 — <code>fundusnap-fundustalk-v1-chatsft-11k</code></h3>
446
+ <p>The synthetic SFT dataset behind the Result Explainer: 10,849 multi-turn consultations that teach a model to explain a diabetic retinopathy screening result and never to diagnose it. Fully synthetic — no patient data, images, or recorded conversations — with every prediction record procedurally generated and every rejected sample published alongside the kept ones.</p>
447
+ <p>
448
+ <img src="https://img.shields.io/badge/%F0%9F%A4%97_Datasets-FFD21E?style=flat-square" alt="Hugging Face Datasets"/>
449
+ <img src="https://img.shields.io/badge/Format-JSONL-000000?style=flat-square&logo=json&logoColor=white" alt="JSONL"/>
450
+ <img src="https://img.shields.io/badge/Conversations-10,849-5B9BD5?style=flat-square" alt="10,849 conversations"/>
451
+ <img src="https://img.shields.io/badge/Teacher-phi--4-0078D4?style=flat-square" alt="Teacher: phi-4"/>
452
+ <img src="https://img.shields.io/badge/Lang-id_|_en-5B9BD5?style=flat-square" alt="Indonesian and English"/>
453
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
454
+ </p>
455
+ <p>
456
+ <b>📊 Measured quality:</b> <code>97.2%</code> numeric grounding · <code>100%</code> opening diversity · <code>88.8%</code> safety-refusal rate · <code>90.4%</code> keep rate after filtering
457
+ </p>
458
+ <p>
459
+ <b>✨ Highlights:</b> 39,288 assistant turns (mean 77.2 words) · 5 categories incl. <code>safety_refusal</code> and <code>adversarial_oos</code> · 12 patient/caregiver/clinician personas · splits 10,201 / 324 / 324 · drops straight into TRL's <code>SFTTrainer</code>
460
+ </p>
461
+ <p>
462
+ <a href="https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k"><img src="https://img.shields.io/badge/Repository-fundusnap--fundustalk--v1--chatsft--11k-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
463
+ &nbsp;<a href="https://huggingface.co/datasets/fundusnap/fundusnap-fundustalk-v1-chatsft-11k"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Dataset-FFD21E?style=flat-square" alt="Hugging Face"/></a>
464
+ &nbsp;<img src="https://img.shields.io/badge/Distribution-Hugging_Face_dataset-555555?style=flat-square" alt="Hugging Face dataset"/>
465
+ </p>
466
+ </td>
467
+ </tr>
468
+ </table>
469
+
470
+ <details>
471
+ <summary><b>Full breakdown — FundusTalk v1</b></summary>
472
+
473
+ - **Teacher:** `microsoft/phi-4` via OpenRouter · **Intended student:** `microsoft/MediPhi-Instruct`
474
+ - **Scale:** 10,849 conversations · 39,288 assistant turns · 3.62 turns per conversation
475
+ - **Splits:** `train` 10,201 · `validation` 324 · `test` 324 — disjoint by conversation id, stratified on category, language, grade, and record profile
476
+ - **Configs:** `default` (filtered, 10,849) · `raw` (unfiltered teacher output, 12,000) · `scenarios` (the seeded, deterministic generation plan)
477
+ - **Categories:** `result_explanation` 42.2% · `safety_refusal` 16.5% · `detector_literacy` 16.3% · `general_knowledge` 14.5% · `adversarial_oos` 10.5%
478
+ - **Languages:** Indonesian 47.2% · English 29.3% · Indonesian–English code-switch 23.5%
479
+ - **Record profiles:** deliberate edge cases — `landmarks_only`, `empty_detections`, `poor_quality`, `low_confidence`, `disagreement`
480
+ - **Filtering:** 12,000 generated → 10,849 kept (90.4%); the safety-critical filters are `no_clinician_referral` and `diagnostic_language`
481
+ - **Reproducibility:** seeded scenario plan, full generation and filtering logs, complete reject list
482
+ - **License:** CC BY-NC 4.0
483
+
484
+ </details>
485
+
486
+ <br/>
487
+
488
+ > [!IMPORTANT]
489
+ > **The four AI repositories above are released for research and engineering use.** None of them is a
490
+ > medical device, none carries regulatory clearance (FDA, CE/MDR, or otherwise), and none has been
491
+ > prospectively validated. Reported metrics are self-reported on the runs' own validation splits. They
492
+ > must never be the sole basis for a diagnosis, referral, or treatment decision — keep a qualified
493
+ > clinician in the loop.
494
+
495
  ---
496
 
497
  ## 🔒 Security & Compliance
profile/README.md CHANGED
@@ -32,7 +32,7 @@
32
 
33
  <p align="center">
34
  <a href="https://drive.google.com/file/d/1Td7bPj-vSIjByO5UIGwOhZstPwcz0tfm/preview">
35
- <img src="assets/Fundusnap_Demo_Video.png" width="640" alt="Watch the Fundusnap demo video"/>
36
  </a>
37
  </p>
38
 
@@ -46,7 +46,7 @@
46
 
47
  **Fundusnap** is a comprehensive medical-imaging solution that helps healthcare workers and patients **detect and analyze diabetic retinopathy (DR)** from *fundus* (retinal) images. A user captures a photo of the back of the eye with the mobile app, and Fundusnap returns an AI classification of disease severity, highlights the specific retinal lesions it found, and lets the user ask follow-up questions to an AI medical assistant that explains the result in plain language.
48
 
49
- It is delivered as an end-to-end product spanning a **mobile app**, a **backend API**, a **marketing/management website**, and an **offline AI model** for low-connectivity environments.
50
 
51
  ## 🩺 The Problem
52
 
@@ -62,10 +62,10 @@ Diabetic retinopathy is one of the leading causes of preventable blindness world
62
  Fundusnap brings specialist-grade screening to a smartphone and makes the result understandable to everyone:
63
 
64
  1. **Capture** — The Flutter mobile app guides users to take a high-quality fundus image (with photo and video capture support).
65
- 2. **Classify** — The image is sent to the API, which runs it through **Microsoft Azure Custom Vision** to classify the severity of diabetic retinopathy.
66
- 3. **Detect** — A **custom object-detection AI** locates and bounds individual retinal artifacts/lesions (e.g. microaneurysms), so the result is explainable rather than a black box.
67
- 4. **Explain** — An **AI medical chat assistant** (Microsoft's **Phi-4** model via OpenRouter) interprets the findings in simple, informative language and encourages appropriate follow-up with a healthcare professional — without making a clinical diagnosis.
68
- 5. **Stay available offline** — A separate, **offline-capable image-classification model** acts as a fallback for poor connectivity or primary-API outages, so screening keeps working where it's needed most.
69
 
70
  All medical data is handled with security and compliance in mind (JWT-based auth, encrypted transmission, and secure image storage).
71
 
@@ -151,12 +151,15 @@ Fundusnap was built for and submitted to three national programs in Indonesia, a
151
 
152
  ## 🧩 Project Components
153
 
154
- | Component | Repository | Deployment |
155
- | --- | --- | --- |
156
- | 📱 Mobile App | [fundusnap/fundusnap-app](https://github.com/fundusnap/fundusnap-app) | Android APK release |
157
- | 🌐 Website | [fundusnap/fundusnap-web](https://github.com/fundusnap/fundusnap-web) | [fundusnap.faizath.com](https://fundusnap.faizath.com) |
158
- | ⚙️ Backend API | [fundusnap/fundusnap-api](https://github.com/fundusnap/fundusnap-api) | [fundusnap-api.faizath.com](https://fundusnap-api.faizath.com) |
159
- | 🧠 Offline AI Model | [fundusnap/fundusnap-ai](https://github.com/fundusnap/fundusnap-ai) | Self-hosted inference service |
 
 
 
160
 
161
  <br/>
162
 
@@ -251,7 +254,7 @@ Fundusnap was built for and submitted to three national programs in Indonesia, a
251
  <img src="https://img.shields.io/badge/OpenRouter-6566F1?style=flat-square&logo=openai&logoColor=white" alt="OpenRouter"/>
252
  </p>
253
  <p>
254
- <b>🤖 AI services:</b> Azure Custom Vision (DR classification) · Custom object-detection AI (retinal artifact detection) · Microsoft <b>Phi-4</b> via OpenRouter (medical chat)
255
  </p>
256
  <p>
257
  <a href="https://github.com/fundusnap/fundusnap-api"><img src="https://img.shields.io/badge/Repository-fundusnap--api-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
@@ -270,55 +273,225 @@ Fundusnap was built for and submitted to three national programs in Indonesia, a
270
  - **Authentication:** JWT (access + refresh tokens)
271
  - **Storage:** Cloudflare R2 (with Azure Blob Storage support)
272
  - **AI Services:**
273
- - Microsoft Azure Custom Vision API (DR classification)
274
- - Custom object-detection AI (retinal artifact detection)
275
- - OpenRouter API with Microsoft's **Phi-4** model (medical chat)
 
 
276
  - **Email Service:** Nodemailer
277
 
278
  </details>
279
 
280
  <br/>
281
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
282
  <table>
283
  <tr>
284
  <td width="64" align="center" valign="top">
285
  <h1>🧠</h1>
286
  </td>
287
  <td valign="top">
288
- <h3>Fundusnap AI</h3>
289
- <p>An offline-capable image-classification model that keeps screening working under poor connectivity or primary-API outages.</p>
290
  <p>
291
  <img src="https://img.shields.io/badge/FastAI-2EC4B6?style=flat-square&logo=fastapi&logoColor=white" alt="FastAI"/>
292
  <img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=pytorch&logoColor=white" alt="PyTorch"/>
 
293
  <img src="https://img.shields.io/badge/ResNet34-FF6F00?style=flat-square&logo=tensorflow&logoColor=white" alt="ResNet34"/>
294
- <img src="https://img.shields.io/badge/Albumentations-5C3EE8?style=flat-square&logo=opencv&logoColor=white" alt="Albumentations"/>
 
295
  </p>
296
  <p>
297
- <b>📊 Performance:</b> <code>81%</code> accuracy · <code>0.81</code> macro F1 · <code>0.81</code> weighted F1
298
  </p>
299
  <p>
300
- <a href="https://github.com/fundusnap/fundusnap-ai"><img src="https://img.shields.io/badge/Repository-fundusnap--ai-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
301
- &nbsp;<img src="https://img.shields.io/badge/Deployment-Self--hosted_service-555555?style=flat-square&logo=docker&logoColor=white" alt="Self-hosted"/>
 
 
 
 
302
  </p>
303
  </td>
304
  </tr>
305
  </table>
306
 
307
  <details>
308
- <summary><b>Full tech stack — Offline AI Model</b></summary>
309
 
310
- - **Deep Learning Framework:** FastAI
311
- - **Base Model:** ResNet34 (pretrained)
312
- - **Data Augmentation:** Albumentations
313
  - **Loss Function:** Focal Loss
 
 
 
 
314
  - **Performance Metrics:**
315
- - Overall Accuracy: 81%
316
- - Macro Average F1-Score: 0.81
317
  - Weighted Average F1-Score: 0.81
318
- - **Deployment:** Self-hosted inference service (consumed by the API via `FUNDUSNAP_AI_HOST`)
 
 
319
 
320
  </details>
321
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
322
  ---
323
 
324
  ## 🔒 Security & Compliance
 
32
 
33
  <p align="center">
34
  <a href="https://drive.google.com/file/d/1Td7bPj-vSIjByO5UIGwOhZstPwcz0tfm/preview">
35
+ <img src="assets/demo.gif" width="100%" alt="Watch the Fundusnap demo video"/>
36
  </a>
37
  </p>
38
 
 
46
 
47
  **Fundusnap** is a comprehensive medical-imaging solution that helps healthcare workers and patients **detect and analyze diabetic retinopathy (DR)** from *fundus* (retinal) images. A user captures a photo of the back of the eye with the mobile app, and Fundusnap returns an AI classification of disease severity, highlights the specific retinal lesions it found, and lets the user ask follow-up questions to an AI medical assistant that explains the result in plain language.
48
 
49
+ It is delivered as an end-to-end product spanning a **mobile app**, a **backend API**, a **marketing/management website**, and a family of **open AI models** a retinal lesion detector, a diabetic-retinopathy severity classifier, and a result-explanation language model — together with the **synthetic dataset** that language model was trained on.
50
 
51
  ## 🩺 The Problem
52
 
 
62
  Fundusnap brings specialist-grade screening to a smartphone and makes the result understandable to everyone:
63
 
64
  1. **Capture** — The Flutter mobile app guides users to take a high-quality fundus image (with photo and video capture support).
65
+ 2. **Classify** — The image is sent to the API, which runs it through **Microsoft Azure Custom Vision** to grade the severity of diabetic retinopathy, with our own [**fundusnap-v1-severitycls-rn34-22m**](https://github.com/fundusnap/fundusnap-v1-severitycls-rn34-22m) ResNet34 grader as the open, self-hostable alternative.
66
+ 3. **Detect** — [**fundusnap-v1-lesiondet-yolo11m-20m**](https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m), a YOLO11m detector, locates and bounds individual retinal lesions and landmarks (microaneurysms, haemorrhages, exudates, optic disc, fovea), so the result is explainable rather than a black box.
67
+ 4. **Explain** — An **AI medical chat assistant** interprets the findings in simple, informative language and encourages appropriate follow-up with a healthcare professional — without making a clinical diagnosis. Two interchangeable backends serve this role: Microsoft's **Phi-4** via OpenRouter, and the self-hosted [**fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter**](https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter) — a MediPhi-Instruct LoRA fine-tuned on our [**FundusTalk v1**](https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) dataset to answer in Indonesian or English.
68
+ 5. **Stay available offline** — The severity classifier also ships as an **ONNX** graph for on-device inference, acting as a fallback for poor connectivity or primary-API outages, so screening keeps working where it's needed most.
69
 
70
  All medical data is handled with security and compliance in mind (JWT-based auth, encrypted transmission, and secure image storage).
71
 
 
151
 
152
  ## 🧩 Project Components
153
 
154
+ | Component | Repository | 🤗 Hub | Deployment |
155
+ | --- | --- | --- | --- |
156
+ | 📱 Mobile App | [fundusnap-app](https://github.com/fundusnap/fundusnap-app) | — | Android APK release |
157
+ | 🌐 Website | [fundusnap-web](https://github.com/fundusnap/fundusnap-web) | — | [fundusnap.faizath.com](https://fundusnap.faizath.com) |
158
+ | ⚙️ Backend API | [fundusnap-api](https://github.com/fundusnap/fundusnap-api) | — | [fundusnap-api.faizath.com](https://fundusnap-api.faizath.com) |
159
+ | 🔬 Lesion Detector | [fundusnap-v1-lesiondet-yolo11m-20m](https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m) | [model](https://huggingface.co/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m) | Self-hosted FastAPI service |
160
+ | 🧠 Severity Classifier | [fundusnap-v1-severitycls-rn34-22m](https://github.com/fundusnap/fundusnap-v1-severitycls-rn34-22m) | [model](https://huggingface.co/fundusnap/fundusnap-v1-severitycls-rn34-22m) | ONNX · offline-capable |
161
+ | 💬 Result Explainer | [fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter](https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter) | [model](https://huggingface.co/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter) | Self-hosted (merged → vLLM) |
162
+ | 📚 FundusTalk v1 Dataset | [fundusnap-fundustalk-v1-chatsft-11k](https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) | [dataset](https://huggingface.co/datasets/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) | Hugging Face dataset |
163
 
164
  <br/>
165
 
 
254
  <img src="https://img.shields.io/badge/OpenRouter-6566F1?style=flat-square&logo=openai&logoColor=white" alt="OpenRouter"/>
255
  </p>
256
  <p>
257
+ <b>🤖 AI services:</b> Azure Custom Vision (DR grading) · <a href="https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m">fundusnap-v1-lesiondet-yolo11m-20m</a> (lesion detection) · medical chat via Microsoft <b>Phi-4</b> on OpenRouter <i>or</i> the self-hosted <a href="https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter">MediPhi LoRA adapter</a>
258
  </p>
259
  <p>
260
  <a href="https://github.com/fundusnap/fundusnap-api"><img src="https://img.shields.io/badge/Repository-fundusnap--api-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
 
273
  - **Authentication:** JWT (access + refresh tokens)
274
  - **Storage:** Cloudflare R2 (with Azure Blob Storage support)
275
  - **AI Services:**
276
+ - Microsoft Azure Custom Vision API (DR severity grading)
277
+ - `fundusnap-v1-lesiondet-yolo11m-20m` self-hosted YOLO11m lesion-detection service
278
+ - Medical chat, two interchangeable backends:
279
+ - OpenRouter API with Microsoft's **Phi-4** model
280
+ - Self-hosted `fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter` (MediPhi-Instruct LoRA)
281
  - **Email Service:** Nodemailer
282
 
283
  </details>
284
 
285
  <br/>
286
 
287
+ <table>
288
+ <tr>
289
+ <td width="64" align="center" valign="top">
290
+ <h1>🔬</h1>
291
+ </td>
292
+ <td valign="top">
293
+ <h3>Fundusnap Lesion Detector — <code>fundusnap-v1-lesiondet-yolo11m-20m</code></h3>
294
+ <p>A YOLO11m object detector that finds <i>where</i> the findings are. Given one colour fundus photograph it returns bounding boxes for twelve classes — ten pathological findings plus the optic disc and fovea as anatomical landmarks — each with a label and a confidence score, so a severity grade comes with visual evidence instead of being a black box.</p>
295
+ <p>
296
+ <img src="https://img.shields.io/badge/YOLO11m-111F68?style=flat-square&logo=yolo&logoColor=white" alt="YOLO11m"/>
297
+ <img src="https://img.shields.io/badge/Ultralytics-0B23A9?style=flat-square&logo=ultralytics&logoColor=white" alt="Ultralytics"/>
298
+ <img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=pytorch&logoColor=white" alt="PyTorch"/>
299
+ <img src="https://img.shields.io/badge/FastAPI-009688?style=flat-square&logo=fastapi&logoColor=white" alt="FastAPI"/>
300
+ <img src="https://img.shields.io/badge/Params-20M-5B9BD5?style=flat-square" alt="20M parameters"/>
301
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
302
+ </p>
303
+ <p>
304
+ <b>📊 Performance:</b> <code>0.53</code> mAP@50 · <code>0.28</code> mAP@50-95 · <code>0.54</code> precision · <code>0.53</code> recall
305
+ </p>
306
+ <p>
307
+ <b>✨ Highlights:</b> 12 classes (10 lesions + <code>Disc</code>/<code>Fovea</code> landmarks) · JSON and annotated-image endpoints · Dockerised FastAPI service · exports to ONNX, TorchScript, TFLite, CoreML
308
+ </p>
309
+ <p>
310
+ <a href="https://github.com/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m"><img src="https://img.shields.io/badge/Repository-fundusnap--v1--lesiondet--yolo11m--20m-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
311
+ &nbsp;<a href="https://huggingface.co/fundusnap/fundusnap-v1-lesiondet-yolo11m-20m"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Model-FFD21E?style=flat-square" alt="Hugging Face"/></a>
312
+ &nbsp;<img src="https://img.shields.io/badge/Deployment-Self--hosted_service-555555?style=flat-square&logo=docker&logoColor=white" alt="Self-hosted"/>
313
+ </p>
314
+ </td>
315
+ </tr>
316
+ </table>
317
+
318
+ <details>
319
+ <summary><b>Full tech stack — Lesion Detector</b></summary>
320
+
321
+ - **Framework:** Ultralytics `8.3.165` / PyTorch
322
+ - **Base Model:** `yolo11m.pt`, COCO-pretrained — `yolo11m.yaml` scale `m`, anchor-free `Detect` head, `nc=12`
323
+ - **Input:** 640×640, letterboxed (coordinates returned in the original image's pixel space)
324
+ - **Training:** 35 epochs, batch 16 (`nbs=64`), optimizer `auto` (`lr0=0.01`, `lrf=0.01`, momentum 0.937, weight decay 0.0005), 3 warmup epochs, AMP, seed 0 deterministic
325
+ - **Loss weights:** box 7.5 · cls 0.5 · dfl 1.5
326
+ - **Augmentation:** mosaic 1.0 (off for the last 10 epochs), `fliplr=0.5`, `scale=0.5`, `translate=0.1`, HSV (0.015/0.7/0.4), `erasing=0.4`, RandAugment
327
+ - **Shipped checkpoint:** epoch 27 — best by Ultralytics fitness (`0.1·mAP50 + 0.9·mAP50-95` = 0.3071), stripped of optimiser/EMA state (~40 MB, Git LFS)
328
+ - **Serving:** `POST /inspect/fundus-artifacts/` (JSON detections) · `POST /visualize/fundus-artifacts/` (annotated JPEG) · `GET /` (health) — `python:3.10-slim`, port 8000
329
+ - **License:** CC BY-NC 4.0 (weights derive from Ultralytics YOLO11 — review Ultralytics' AGPL-3.0 terms before redistributing)
330
+
331
+ </details>
332
+
333
+ <br/>
334
+
335
  <table>
336
  <tr>
337
  <td width="64" align="center" valign="top">
338
  <h1>🧠</h1>
339
  </td>
340
  <td valign="top">
341
+ <h3>Fundusnap Severity Classifier — <code>fundusnap-v1-severitycls-rn34-22m</code></h3>
342
+ <p>A ResNet34 classifier that grades <i>how severe</i> the retinopathy is. It predicts one of the five standard ordinal ICDR grades (0 = No DR through 4 = Proliferative) with a probability for each, and ships as an ONNX graph with a dynamic batch axis — which is what keeps screening working offline or during a primary-API outage.</p>
343
  <p>
344
  <img src="https://img.shields.io/badge/FastAI-2EC4B6?style=flat-square&logo=fastapi&logoColor=white" alt="FastAI"/>
345
  <img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=pytorch&logoColor=white" alt="PyTorch"/>
346
+ <img src="https://img.shields.io/badge/ONNX-005CED?style=flat-square&logo=onnx&logoColor=white" alt="ONNX"/>
347
  <img src="https://img.shields.io/badge/ResNet34-FF6F00?style=flat-square&logo=tensorflow&logoColor=white" alt="ResNet34"/>
348
+ <img src="https://img.shields.io/badge/Params-22M-5B9BD5?style=flat-square" alt="22M parameters"/>
349
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
350
  </p>
351
  <p>
352
+ <b>📊 Performance:</b> <code>0.82</code> accuracy · <code>0.8153</code> macro F1 · <code>0.81</code> macro precision · <code>0.82</code> macro recall
353
  </p>
354
  <p>
355
+ <b>✨ Highlights:</b> 5 ordinal ICDR grades · ONNX opset 14 with dynamic batch · fastai checkpoint for further fine-tuning · ONNX → TensorFlow/TFLite path for on-device inference
356
+ </p>
357
+ <p>
358
+ <a href="https://github.com/fundusnap/fundusnap-v1-severitycls-rn34-22m"><img src="https://img.shields.io/badge/Repository-fundusnap--v1--severitycls--rn34--22m-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
359
+ &nbsp;<a href="https://huggingface.co/fundusnap/fundusnap-v1-severitycls-rn34-22m"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Model-FFD21E?style=flat-square" alt="Hugging Face"/></a>
360
+ &nbsp;<img src="https://img.shields.io/badge/Deployment-ONNX_·_offline--capable-555555?style=flat-square&logo=onnx&logoColor=white" alt="ONNX / offline-capable"/>
361
  </p>
362
  </td>
363
  </tr>
364
  </table>
365
 
366
  <details>
367
+ <summary><b>Full tech stack — Severity Classifier</b></summary>
368
 
369
+ - **Deep Learning Framework:** FastAI / PyTorch, exported to ONNX (opset 14)
370
+ - **Base Model:** `resnet34`, ImageNet-pretrained (`timm/resnet34.tv_in1k`)
371
+ - **Head:** fastai default (`AdaptiveConcatPool2d` → BN/dropout → linear), `n_out=5`
372
  - **Loss Function:** Focal Loss
373
+ - **Input:** `Resize(224)` centre crop, ImageNet normalisation
374
+ - **Training:** batch 32, `learn.fine_tune(4)` (1 frozen + 4 unfrozen epochs), LR from `lr_find()` valley, seed 3865
375
+ - **Data Augmentation:** Albumentations — `ShiftScaleRotate`, `HorizontalFlip`, `RandomBrightnessContrast`, `HueSaturationValue`
376
+ - **Dataset:** Kaggle *resized-2015-2019-diabetic-retinopathy-detection* (EyePACS 2015 + APTOS 2019), each grade resampled to 10,000 rows for a 50,000-image balanced frame, 10% held out
377
  - **Performance Metrics:**
378
+ - Overall Accuracy: 0.82
379
+ - Macro Average F1-Score: 0.8153
380
  - Weighted Average F1-Score: 0.81
381
+ - Grades 3–4 separate near-perfectly (F1 0.97–0.98); grades 0/1/2 sit at 0.65–0.76
382
+ - **Deployment:** ONNX Runtime for inference (offline-capable), fastai checkpoint for fine-tuning
383
+ - **License:** CC BY-NC 4.0 (training data carries its own Kaggle / EyePACS / APTOS terms)
384
 
385
  </details>
386
 
387
+ <br/>
388
+
389
+ <table>
390
+ <tr>
391
+ <td width="64" align="center" valign="top">
392
+ <h1>💬</h1>
393
+ </td>
394
+ <td valign="top">
395
+ <h3>Fundusnap Result Explainer — <code>fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter</code></h3>
396
+ <p>A LoRA adapter over <a href="https://huggingface.co/microsoft/MediPhi-Instruct">microsoft/MediPhi-Instruct</a> that turns a prediction record into a plain-language explanation, in Indonesian or English. It is the conversational layer of the pipeline and the only model that never sees an image — it reads the severity probabilities and the lesion boxes the other two produce, and explains them without ever diagnosing.</p>
397
+ <p>
398
+ <img src="https://img.shields.io/badge/PEFT_LoRA-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="PEFT LoRA"/>
399
+ <img src="https://img.shields.io/badge/MediPhi--Instruct-0078D4?style=flat-square" alt="MediPhi-Instruct"/>
400
+ <img src="https://img.shields.io/badge/Transformers-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="Transformers"/>
401
+ <img src="https://img.shields.io/badge/FastAPI-009688?style=flat-square&logo=fastapi&logoColor=white" alt="FastAPI"/>
402
+ <img src="https://img.shields.io/badge/Params-3.8B_+_50M_LoRA-5B9BD5?style=flat-square" alt="3.8B + 50M LoRA"/>
403
+ <img src="https://img.shields.io/badge/Lang-id_|_en-5B9BD5?style=flat-square" alt="Indonesian and English"/>
404
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
405
+ </p>
406
+ <p>
407
+ <b>📊 Performance:</b> val loss <code>1.229</code> → <code>0.691</code> · <code>0/60</code> stub replies (base: 10/60) · <code>40/41</code> Indonesian prompts answered in Indonesian (base: 35/41)
408
+ </p>
409
+ <p>
410
+ <b>✨ Highlights:</b> explains rather than diagnoses — every conversation routes to a clinician · bilingual with code-switching · FastAPI <code>serve.py</code> · <code>merge.py</code> fuses the adapter into a standalone checkpoint for vLLM
411
+ </p>
412
+ <p>
413
+ <a href="https://github.com/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter"><img src="https://img.shields.io/badge/Repository-fundusnap--v1--resultexp--clm--mediphi--3.8b--adapter-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
414
+ &nbsp;<a href="https://huggingface.co/fundusnap/fundusnap-v1-resultexp-clm-mediphi-3.8b-adapter"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Model-FFD21E?style=flat-square" alt="Hugging Face"/></a>
415
+ &nbsp;<img src="https://img.shields.io/badge/Deployment-Self--hosted_·_vLLM-555555?style=flat-square&logo=docker&logoColor=white" alt="Self-hosted / vLLM"/>
416
+ </p>
417
+ </td>
418
+ </tr>
419
+ </table>
420
+
421
+ <details>
422
+ <summary><b>Full tech stack — Result Explainer</b></summary>
423
+
424
+ - **Base Model:** `microsoft/MediPhi-Instruct` (Phi-3 architecture, 3.82B)
425
+ - **Method:** QLoRA-style supervised fine-tuning, loss on assistant turns only
426
+ - **LoRA config:** r=32, α=64, dropout 0.05 — targets `qkv_proj`, `o_proj`, `gate_up_proj`, `down_proj`
427
+ - **Trainable params:** 50,331,648 (**1.30%** of the model)
428
+ - **Training Data:** [FundusTalk v1](https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k) — 10,201 synthetic consultations distilled from `microsoft/phi-4` (~70% Indonesian incl. code-switched, ~30% English)
429
+ - **Schedule:** 2 epochs, 638 steps, lr 1e-4 cosine, effective batch 32, bf16
430
+ - **Hardware:** 1× A100-SXM4-40GB, ~170 min
431
+ - **Prompt envelope:** the exact three-system-message format the API sends (persona → severity JSON → detection JSON), byte-for-byte compatible with `JSON.stringify`
432
+ - **Serving:** `GET /` · `POST /chat` · `POST /prompt` (returns the envelope without generating) — or merge and serve with vLLM
433
+ - **Licensing:** weights CC BY-NC 4.0 · repository code MIT · base and teacher models both MIT
434
+
435
+ </details>
436
+
437
+ <br/>
438
+
439
+ <table>
440
+ <tr>
441
+ <td width="64" align="center" valign="top">
442
+ <h1>📚</h1>
443
+ </td>
444
+ <td valign="top">
445
+ <h3>FundusTalk v1 — <code>fundusnap-fundustalk-v1-chatsft-11k</code></h3>
446
+ <p>The synthetic SFT dataset behind the Result Explainer: 10,849 multi-turn consultations that teach a model to explain a diabetic retinopathy screening result and never to diagnose it. Fully synthetic — no patient data, images, or recorded conversations — with every prediction record procedurally generated and every rejected sample published alongside the kept ones.</p>
447
+ <p>
448
+ <img src="https://img.shields.io/badge/%F0%9F%A4%97_Datasets-FFD21E?style=flat-square" alt="Hugging Face Datasets"/>
449
+ <img src="https://img.shields.io/badge/Format-JSONL-000000?style=flat-square&logo=json&logoColor=white" alt="JSONL"/>
450
+ <img src="https://img.shields.io/badge/Conversations-10,849-5B9BD5?style=flat-square" alt="10,849 conversations"/>
451
+ <img src="https://img.shields.io/badge/Teacher-phi--4-0078D4?style=flat-square" alt="Teacher: phi-4"/>
452
+ <img src="https://img.shields.io/badge/Lang-id_|_en-5B9BD5?style=flat-square" alt="Indonesian and English"/>
453
+ <img src="https://img.shields.io/badge/License-CC_BY--NC_4.0-EF9421?style=flat-square&logo=creativecommons&logoColor=white" alt="CC BY-NC 4.0"/>
454
+ </p>
455
+ <p>
456
+ <b>📊 Measured quality:</b> <code>97.2%</code> numeric grounding · <code>100%</code> opening diversity · <code>88.8%</code> safety-refusal rate · <code>90.4%</code> keep rate after filtering
457
+ </p>
458
+ <p>
459
+ <b>✨ Highlights:</b> 39,288 assistant turns (mean 77.2 words) · 5 categories incl. <code>safety_refusal</code> and <code>adversarial_oos</code> · 12 patient/caregiver/clinician personas · splits 10,201 / 324 / 324 · drops straight into TRL's <code>SFTTrainer</code>
460
+ </p>
461
+ <p>
462
+ <a href="https://github.com/fundusnap/fundusnap-fundustalk-v1-chatsft-11k"><img src="https://img.shields.io/badge/Repository-fundusnap--fundustalk--v1--chatsft--11k-181717?style=flat-square&logo=github&logoColor=white" alt="Repository"/></a>
463
+ &nbsp;<a href="https://huggingface.co/datasets/fundusnap/fundusnap-fundustalk-v1-chatsft-11k"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Dataset-FFD21E?style=flat-square" alt="Hugging Face"/></a>
464
+ &nbsp;<img src="https://img.shields.io/badge/Distribution-Hugging_Face_dataset-555555?style=flat-square" alt="Hugging Face dataset"/>
465
+ </p>
466
+ </td>
467
+ </tr>
468
+ </table>
469
+
470
+ <details>
471
+ <summary><b>Full breakdown — FundusTalk v1</b></summary>
472
+
473
+ - **Teacher:** `microsoft/phi-4` via OpenRouter · **Intended student:** `microsoft/MediPhi-Instruct`
474
+ - **Scale:** 10,849 conversations · 39,288 assistant turns · 3.62 turns per conversation
475
+ - **Splits:** `train` 10,201 · `validation` 324 · `test` 324 — disjoint by conversation id, stratified on category, language, grade, and record profile
476
+ - **Configs:** `default` (filtered, 10,849) · `raw` (unfiltered teacher output, 12,000) · `scenarios` (the seeded, deterministic generation plan)
477
+ - **Categories:** `result_explanation` 42.2% · `safety_refusal` 16.5% · `detector_literacy` 16.3% · `general_knowledge` 14.5% · `adversarial_oos` 10.5%
478
+ - **Languages:** Indonesian 47.2% · English 29.3% · Indonesian–English code-switch 23.5%
479
+ - **Record profiles:** deliberate edge cases — `landmarks_only`, `empty_detections`, `poor_quality`, `low_confidence`, `disagreement`
480
+ - **Filtering:** 12,000 generated → 10,849 kept (90.4%); the safety-critical filters are `no_clinician_referral` and `diagnostic_language`
481
+ - **Reproducibility:** seeded scenario plan, full generation and filtering logs, complete reject list
482
+ - **License:** CC BY-NC 4.0
483
+
484
+ </details>
485
+
486
+ <br/>
487
+
488
+ > [!IMPORTANT]
489
+ > **The four AI repositories above are released for research and engineering use.** None of them is a
490
+ > medical device, none carries regulatory clearance (FDA, CE/MDR, or otherwise), and none has been
491
+ > prospectively validated. Reported metrics are self-reported on the runs' own validation splits. They
492
+ > must never be the sole basis for a diagnosis, referral, or treatment decision — keep a qualified
493
+ > clinician in the loop.
494
+
495
  ---
496
 
497
  ## 🔒 Security & Compliance