model card

README.md

@@ -1,27 +1,167 @@
-# Generation LoRA adapters (Path B — optional, full regeneration)
+---
+license: gemma
+library_name: peft
+pipeline_tag: text-generation
+tags:
+  - lora
+  - peft
+  - medical
+  - health
+  - multilingual
+  - africa
+  - question-answering
+  - zindi
+  - mhqa-itu
+language:
+  - en
+  - sw
+  - lg
+  - ak
+  - am
+base_model:
+  - google/gemma-4-31B-it
+  - google/medgemma-27b-text-it
+---
 
-These are the three fine-tuned LoRA adapters whose merged models produce the
-generation base `work/submissions/sub_gen_v20.csv`. **The fast reproduction
-(`reproduce.sh`) does NOT need them** — it consumes the already-built
-gen base. They are only needed to regenerate that base from scratch (Path B: `bash run_full.sh`).
+# MHQA · ITU · Zindi Challenge — Generation Adapters
 
-Merge each adapter onto its base model (downloaded separately — see the README
-"What's in the ZIP vs downloaded" section), then sample with `stages/1_generation/gen_samples.py`:
+**`DariusTheGeek/mhqa-itu-adapters`** · the trained-weight component of the winning solution to the
+[Multilingual Health Question Answering in Low-Resource African Languages Challenge (ITU)](https://zindi.africa/competitions/multilingual-health-question-answering-in-low-resource-african-languages-challenge),
+hosted on Zindi. Data by the **HASH** consortium (Hub for AI in Maternal, Sexual & Reproductive Health).
 
-| adapter dir | base model to merge onto | role in the MBR medoid |
+This repo hosts **3 LoRA adapters** (the generation members). The full retrieval + generation + judge
+**system**, and one-command reproduction, live in the code repo:
+**[…-ITU-SOLUTION on GitHub](https://github.com/DariusTheGeek/Multilingual-Health-Question-Answering-in-Low-Resource-African-Languages-Challenge-by-ITU-SOLUTION)**.
+
+> ⚠️ **Not medical advice.** This is a research artifact built for a benchmark competition. It is **not a
+> medical device**, is **not for clinical use**, and its outputs must not be used to diagnose, treat, or make
+> health decisions. Sexual/reproductive-health content may be inaccurate, incomplete, or unsafe.
+
+---
+
+## Model summary
+
+| | |
+|---|---|
+| **Reproduces** | submission `sub_v40` · public LB **0.728509** |
+| **Metric** | `0.37·ROUGE-1 + 0.37·ROUGE-L + 0.26·LLM-judge` (micro-averaged, unicode-aware) |
+| **Languages** | English, Kiswahili, Luganda, Akan/Twi, Amharic (8 country×language subsets) |
+| **What's here** | 3 LoRA adapters (4.7 GB) — the *generation* component |
+| **Adapter type** | PEFT/LoRA on Gemma-4-31B-it and MedGemma-27B-text-it |
+
+### The three adapters
+
+| adapter | base model | role | LoRA config | trained on |
+|---|---|---|---|---|
+| **`gen7454`** | `google/gemma-4-31B-it` | primary generator | r64 / α128 / dropout 0 · all linear projections · 2 epochs | all 8 subsets |
+| **`raft_r1`** | *merged `gen7454`* | RAFT self-distillation member | r32 / α64 · lr 5e-5 · 1 epoch | Aka_Gha, Eng_Gha |
+| **`mg_2226`** | `google/medgemma-27b-text-it` | MedGemma member | r64 / α128 · lr 1e-4 · 2 epochs | Aka_Gha, Eng_Gha |
+
+> `raft_r1` was trained **on top of the merged `gen7454`** model (RAFT round-1), not on the raw base.
+
+---
+
+## Architecture & approach
+
+The competition is scored `0.37·ROUGE-1 + 0.37·ROUGE-L + 0.26·judge`, so ~74 % of the score is lexical
+overlap with a hidden gold answer. The test set splits into two regimes:
+
+- **Retrieval subsets** (~60 %: Eng_Uga, Eng_Ken, Eng_Eth, Swa_Ken, Lug_Uga) — the gold answer is almost
+  always a **verbatim reuse** of a Train+Val answer-bank entry, already in the candidate pool. The job is
+  *selecting the canonical twin*, done by **LightGBM selectors** that use a fine-tuned cross-encoder as a
+  feature (not an argmax chooser). These are trained inline and deterministic — **not** weight files.
+- **Generation subsets** (~37 %: Aka_Gha, Eng_Gha) — golds are original/local, so answers are **generated**.
+  The **3 LoRA adapters in this repo** each produce a K36 sample pool; a cross-model ROUGE-MBR **medoid**
+  picks the consensus answer.
+
+A final **LLM-judge pass** (gemma-4-31B-it) reverts off-topic picks (27 cached decisions).
+
+**What is a weight vs. deterministic code:**
+
+| component | form | where |
 |---|---|---|
-| `gen7454/` | `google/gemma-4-31B-it` | primary generator (E1, r64, ckpt-7454, all-8 subsets, 2 ep) |
-| `raft_r1/` | `google/gemma-4-31B-it` | RAFT self-distill member (E20) |
-| `mg_2226/` | `google/medgemma-27b`   | MedGemma member, fine-tuned on Aka_Gha + Eng_Gha (E24) |
+| 3 generation adapters | **trained LoRA weights** | this repo |
+| cross-encoder reranker (CE-ft) | trained *inline per fold*, discarded; frozen feature outputs shipped | dataset repo (`mhqa-itu-artifacts`) |
+| 3 LightGBM selectors | trained *inline each run*, deterministic | code (GitHub) |
+| LLM judge | 27 cached revert decisions, replayed | dataset repo |
+
+---
+
+## Intended use
+
+- **In scope:** reproducing / validating the competition result; research on retrieval-vs-generation for
+  low-resource multilingual health QA; a baseline for the 8 named country×language subsets.
+- **Out of scope:** any clinical, diagnostic, or patient-facing use; generating medical advice; deployment as
+  a health chatbot; languages/domains outside the 8 competition subsets (behaviour is untested and unsafe).
+
+## Training data
 
-Merge example (uses the bundled helper):
+All data is from the Zindi competition (HASH consortium sexual/reproductive-health Q&A across Uganda, Kenya,
+Ghana, Ethiopia). **No external or hand-authored data**; no preprocessing outside the code. Raw `Train.csv` /
+`Val.csv` are Zindi competition data and are **not redistributed** — download them from the competition page.
+The adapters are trained *on* this data but do not *contain* it.
+
+## Evaluation
+
+| submission | Public LB | ROUGE-1 | ROUGE-L | Judge |
+|---|---|---|---|---|
+| **`sub_v40`** | **0.728509** | 0.7187 | 0.6514 | 0.8522 |
+
+The exact `0.728509` requires the hidden Test gold + Zindi's hosted judge. `common/eval.py` computes the exact
+`0.37/0.37/0.26` blend given a gold file (and per-row judge scores); `reproduce.sh` proves the shipped output
+is byte-identical to the scored `sub_v40`.
+
+## Known limitations
+
+- **Coverage:** only the 8 competition subsets/languages; no guarantees elsewhere.
+- **ROUGE-optimized:** the system is tuned to a lexical-overlap metric; high ROUGE ≠ clinical correctness.
+- **Answer-bank reliance:** retrieval subsets depend heavily on verbatim reuse of Train/Val bank answers, so
+  generalization to genuinely novel questions is weak.
+- **Amharic:** ROUGE is effectively 0 for Ge'ez script under the metric; Amh_Eth passes through generation.
+- **Generation is stochastic:** the shipped pool uses temperature>0 sampling (reproducible up to sampling; a
+  greedy demo path is deterministic). Not a source of clinical reliability.
+- **Health-safety:** outputs may be inaccurate/unsafe; see the disclaimer above.
+
+## How to use
+
+The adapters are components of a pipeline — use the GitHub repo, which pulls these weights automatically:
 
 ```bash
-PY=.envs/venv-serve/bin/python
-$PY stages/1_generation/merge_adapter.py --base models/gemma-4-31B-it \
-    --adapter models/adapters/gen7454 --out models/merged_gen7454
-$PY stages/1_generation/merge_adapter.py --base models/gemma-4-31B-it \
-    --adapter models/adapters/raft_r1 --out models/merged_raft_r1
-$PY stages/1_generation/merge_adapter.py --base models/medgemma-27b \
-    --adapter models/adapters/mg_2226 --out models/merged_mg_2226
+git clone https://github.com/DariusTheGeek/Multilingual-Health-Question-Answering-in-Low-Resource-African-Languages-Challenge-by-ITU-SOLUTION.git
+cd Multilingual-Health-…-SOLUTION
+bash env/install.sh
+bash stages/0_setup/fetch_artifacts.sh              # ~250 MB data/features (Path A, CPU)
+bash reproduce.sh                                    # -> sub_v40, BYTE-IDENTICAL, minutes, no GPU
+
+# Run the actual trained model (GPU ~48 GB):
+bash stages/0_setup/fetch_artifacts.sh --adapters    # pulls these 3 adapters (4.7 GB)
+python stages/1_generation/demo_generate.py --adapter models/adapters/gen7454 --base google/gemma-4-31B-it --n 2
 ```
+
+Standalone load (PEFT):
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM
+base = AutoModelForCausalLM.from_pretrained("google/gemma-4-31B-it", torch_dtype="bfloat16", device_map="auto")
+model = PeftModel.from_pretrained(base, "DariusTheGeek/mhqa-itu-adapters", subfolder="gen7454")
+```
+
+## Dependencies
+
+Python 3.11. CPU reproduction: `lightgbm`, `pandas`, `pyarrow`, `regex`, `rouge_score`, `scikit-learn`
+(`requirements.txt`). GPU generation: `unsloth` / `vllm` + `transformers`/`peft`
+(`env/requirements-serve.txt`, `env/requirements-train.txt`).
+
+## License
+
+The adapters are LoRA derivatives of gated Google base models; base weights are **referenced, not re-hosted**.
+Use is subject to the base licenses — **`gemma-4-31B-it`: [Gemma Terms of Use](https://ai.google.dev/gemma/terms)**
+and **`medgemma-27b-text-it`: [Health AI Developer Foundations terms](https://developers.google.com/health-ai-developer-foundations/terms)**.
+Accept each on the Hub before downloading the bases. Competition data is governed by the Zindi competition rules.
+
+## Citation
+
+Solution to the ITU *Multilingual Health Question Answering in Low-Resource African Languages* challenge
+(Zindi / HASH consortium). Author: DariusTheGeek. Companion dataset repo:
+[`DariusTheGeek/mhqa-itu-artifacts`](https://huggingface.co/datasets/DariusTheGeek/mhqa-itu-artifacts).