AfriMed-PII-AfroXLMR: clinical PII de-identification for Luganda, Kiswahili, Kinyarwanda and Dholuo

AfroXLMR-base is XLM-R base adapted to African languages by multilingual adaptive fine-tuning; its adaptation covered Kiswahili and Kinyarwanda but not Luganda or Dholuo. All four checkpoints here start from a variant of those weights continued-pretrained on our four-language corpus. Model-for-model it has the highest precision and the best balanced F1 of the three backbones in three of the four languages, and the strongest recall on village and facility names; its patient-name recall is below AfriBERTa's. The tables below quantify both.

This release contains four per-language checkpoints fine-tuned from Davlan/afro-xlmr-base for token-level detection of personal information in clinical text. Each checkpoint emits character-offset spans with one of 14 categories. The model outputs positions and labels; it does not generate text.

Model details

Developed by	Crane AI Labs, under Gates Foundation grant INV-107957 (Workstream 1)
Model type	Transformer encoder with a token-classification head; BIO tagging over 14 categories (29 tags)
Base model	Davlan/afro-xlmr-base (278M parameters, MIT); Hub revision 25f27299c247a6b73a767bd82d12444138b19337
Languages	Luganda (lug), Kiswahili (swa), Kinyarwanda (kin), Dholuo (luo)
Licence	Apache-2.0 for the fine-tuned weights; the base model's MIT notice is retained
Released	2026-09

Checkpoints in this release

Language	Folder	Run	Base	Learning rate
Luganda	lug/	afroxlmr__lug__lr1e-05__seed13__cpt	continued-pretrained afro-xlmr-base	1e-5 (best epoch 1 of 10)
Kiswahili	swa/	afroxlmr__swa__lr5e-06__seed13__cpt	continued-pretrained afro-xlmr-base	5e-6 (best epoch 3 of 10)
Kinyarwanda	kin/	afroxlmr__kin__lr5e-06__seed13__cpt	continued-pretrained afro-xlmr-base	5e-6 (best epoch 5 of 10)
Dholuo	luo/	afroxlmr__luo__lr3e-05__seed13__cpt	continued-pretrained afro-xlmr-base	3e-5 (best epoch 1 of 10)

Each checkpoint is the seed selected on the development split (see Model selection). The remaining seeds are retained and are available on request; every figure below is a mean over all five.

Dholuo was included as a deliberate stress case. The other three languages are the jurisdiction-aligned core; Dholuo is Nilotic where they are Bantu, has one to two orders of magnitude less monolingual text (about 5 to 6 million tokens of licence-clean text in total), and no backbone in this series was pretrained on it. Its results therefore bound what the approach can do for a genuinely low-resource language of a different family, rather than measuring it under favourable conditions.

Categories

The 14 categories follow the project's medical-PII taxonomy (D3). Identifier categories with a fixed surface form, national ID, phone, date, record number, age, are detectable by pattern; the remaining categories require the model.

Category	Description
PERSON	Patient's name
PERSON_THIRD_PARTY	Any other person's name, relative, next of kin, clinician
ID_NATIONAL	National identity number (UG NIN, KE ID, RW NID, TZ NIDA)
ID_RECORD	Facility or patient record number
CONTACT_PHONE	Telephone number
CONTACT_OTHER	Email address, handle, or other contact string
LOCATION_VILLAGE	Village, parish, district or other administrative place
LOCATION_FACILITY	Named health facility
DATE	Calendar date
AGE	Patient age, including its unit word
PROFESSION	Occupation
TRIBE	Ethnic or tribal affiliation
RELIGION	Religious affiliation
PERSON_CLAN	Clan name

Uses

The model is a component for de-identifying clinical free text in the four languages. Its measured operating envelope is given in Evaluation; the per-category tables there describe what proportion of each kind of identifier it finds and how much non-identifier text it marks in doing so, and are the basis on which to judge fitness for a given workflow.

It performs best on clinical-register text. Its behaviour on other registers was measured on news text and is reported under Out-of-domain.

How to use

from transformers import AutoTokenizer, AutoModelForTokenClassification

repo = "CraneAILabs/afrimed-pii-afroxlmr"
lang = "lug"   # or "swa", "kin", "luo"

tok = AutoTokenizer.from_pretrained(repo, subfolder=lang)
mdl = AutoModelForTokenClassification.from_pretrained(repo, subfolder=lang)

The models were trained over a whitespace-and-punctuation word grid and expect input pre-split into words (tokenizer(words, is_split_into_words=True)), with predictions aligned back to character offsets. A reference implementation of that alignment, together with an optional rule layer for the fixed-format identifiers, is provided in deid.py in this repository. The three calls above were verified to load and run for all four language checkpoints on transformers 5.13 / torch 2.13 with no trust_remote_code and no additional files; warm inference on a 219-character record takes ~6 ms on a GB10 GPU and ~100 ms on CPU.

The system on a clinical note

The deployed demo runs the AfriBERTa checkpoints; screenshots of it on a synthetic note per language, with the system's misses annotated, are in that model's card and in figures/demo/.

Training details

Training data

The models were fine-tuned on a synthetic clinical corpus (internal designation D5, version 1): 8,000 training and 1,000 development documents per language, 36,000 in total. Documents are clinical-style records generated from language-specific templates, filled from curated East African vocabularies of given and family names, villages, districts and health facilities, and country-specific identifier formats for Uganda, Kenya, Rwanda and Tanzania. A native-speaker review of 600 documents was conducted across the four languages, and the Kinyarwanda partition was regenerated with reviewer-sourced lexical terms. Every span is labelled with one of the 14 categories above. In this corpus age spans are annotated as the bare number; the evaluation corpus annotates the number with its unit word, which accounts for the strict-match age result in the tables (the number itself is detected).

Continued-pretraining corpus. Before fine-tuning, the base was adapted by masked-language-model continued pretraining on a licence-clean monolingual corpus of the four languages: 122.0 million tokens, parity-balanced to the Luganda footprint (Luganda 38.5M, Kiswahili 38.5M capped from 258M, Kinyarwanda 38.5M capped from 362M, Dholuo 6.6M uncapped), drawn from MADLAD-400 (CC-BY), HPLT v2 (CC0), TICO-19 (CC0) and, for Dholuo, Kencorpus (CC-BY 4.0). Share-alike and non-commercial sources were excluded, an exact-text deduplication removed 14,979 records, and a mandatory contamination guard dropped any record matching an evaluation set by dataset name or by source outlet. A 3% held-out split was reserved.

No real patient data was used. Evaluation corpora, including MasakhaNER, which is licensed for non-commercial use, were used for evaluation and development only and never for training. The training corpus itself is not part of this release.

Kinyarwanda tribe and clan are out of scope, by ruling. All three backbones score 0.000 on Kinyarwanda TRIBE and PERSON_CLAN on the synthetic evaluation set. This is not a detection failure: the project's taxonomy ruling removed both categories for Kinyarwanda, Rwandan clinical records do not record Hutu/Tutsi/Twa under post-1994 law and policy, and clan has a Luganda-register basis, so the Kinyarwanda training corpus the released weights learned from contains no spans of either. The evaluation edition reported here is an earlier one that still annotates 76 tribe and 35 clan spans; the zeros record that the released Kinyarwanda checkpoint was never trained to emit those labels, which is the intended behaviour. Kinyarwanda RELIGION, which remains in scope, scores 0.86 to 0.93.

A genuine gap on this backbone. The released AfroXLMR checkpoints score 0.000 on PERSON_CLAN in every language and on all three of TRIBE, RELIGION and PERSON_CLAN in Luganda (where the other two backbones score 0.97 to 1.00), while reaching micro F1 0.92 on the same set. All 14 labels are present in the checkpoint's label space; the model did not learn to emit these. Users needing tribe, religion or clan detection should prefer the AfriBERTa or XLM-R checkpoints.

Training procedure

Each language is trained separately as a token-classification model: the continued-pretrained encoder with a randomly initialised linear head over 29 BIO tags, trained with cross-entropy over a whitespace-and-punctuation word grid (first sub-token of each word carries the label). The continued-pretrained base was retained over the public weights because, in a controlled comparison, it raised patient-name recall in every language for this backbone (Luganda +0.13, Dholuo +0.08 on the 5-seed mean). Five seeds were trained per language.

Hyperparameters

Setting	Value
Learning rate (per language)	Luganda 1e-5 (best epoch 1 of 10), Kiswahili 5e-6 (best epoch 3 of 10), Kinyarwanda 5e-6 (best epoch 5 of 10), Dholuo 3e-5 (best epoch 1 of 10)
Epochs	10-epoch budget; the checkpoint from the best development epoch is kept (1, 3, 5 and 1 for lug, swa, kin, luo)
Batch size	32 (no gradient accumulation)
Max sequence length	256
Warm-up ratio	0.1
Weight decay	0.01
Schedule	linear
Precision	bf16
Loss	cross-entropy (no focal term, no class weighting, no oversampling)
Vocabulary	unchanged (no added tokens)
Seeds	13, 42, 99, 123, 2026

Model selection

The learning rate was swept per language over {5e-6, 1e-5, 2e-5, 3e-5, 5e-5} under a 10-epoch budget, and the (learning rate, epoch) pair was chosen on a real held-out development set, the MasakhaNER validation split plus the 30% development slice of the translated-clinical corpus, by entity-strict micro-recall. On this backbone real-text recall peaks early (epoch 1 for Luganda and Dholuo) and declines with further training on synthetic data, so best-epoch selection on real text is load-bearing: a fixed five-epoch schedule left recall on the table. The test split was never used for any selection. The released checkpoint per language is the seed selected on the development set; all five seeds are reported. One released checkpoint departs from its five-seed mean in the direction that matters: the Luganda checkpoint's third-party-name recall is 0.158 against a five-seed mean of 0.482 ± 0.103 (z = −3.15); 41 of 44 other (language, category) cells agree with their means within two standard deviations. Users of the Luganda checkpoint should plan against the single-checkpoint figure for that category, not the mean. The relaxed-overlap recall column is the single released checkpoint only (n = 1). The five-seed sweep for this backbone retained score cells but not weights, and relaxed matching needs the predicted spans rather than their counts, so it could not be recomputed across seeds; the ±.00 shown beside every relaxed figure means not measured, not no seed variance. The strict columns beside it remain the five-seed means. The released checkpoint's strict per-category recall falls within ±2 SD of the five-seed mean in 41 of the 44 measured cells, so it is representative; the exception to read carefully is Luganda third-party name, where the released checkpoint scores 0.158 against a five-seed mean of 0.482.

Compute

Fine-tuning and sweeps ran on an NVIDIA A100 80GB instance (Azure, West Europe). Training time logged across the 48 sweep, ablation and final runs is 2.67 GPU-hours; with scoring the lane reported 4.3 GPU-hours. The continued pretraining of the base is additional and is reported in the accompanying technical report.

Evaluation

Evaluation data

Three evaluation sets are reported. Together they cover every category and separate three questions, performance on the clinical register, generalisation to names never seen in training, and behaviour out of domain.

1. Held-out translated-clinical test (teval). The MEDDOCAN Spanish clinical corpus (Marimon et al., 2019) machine-translated into each of the four languages with span offsets re-aligned; split 30% development / 70% test by document index. Test size per language: Luganda 161 docs / 3,351 spans; Kiswahili 163 / 3,387; Kinyarwanda 158 / 3,278; Dholuo 168 / 3,487. The source schema has no tribe, religion or clan annotations, so those categories are evaluated on set 3.
2. Name-disjoint variant (teval-novel). The same test documents with every person-name span replaced by a well-formed surrogate drawn from a Wikidata (CC0) name pool whose tokens are disjoint from the training vocabulary, verified at 0.0% overlap in all four languages against nine sources, with document and span counts preserved exactly. A reshuffle control (original names permuted across documents) separates the effect of unfamiliar names from that of merely different names.
3. Synthetic development set (d5-dev), an in-distribution ceiling, not a held-out score. 1,000 documents per language from the synthetic generator. These are the documents on which training early-stopped (best epoch selected by F1 on this set), so no training document appears in it but the released weights are, by construction, the epoch that scored best here. Its figures are therefore an upper bound for in-distribution synthetic text. It is the only set carrying gold spans for TRIBE, RELIGION and PERSON_CLAN, and is reported for those categories only, labelled as such.

Out-of-domain behaviour is measured on the MasakhaNER 1.0 test split (news text; PERSON, LOCATION, DATE), which is used for evaluation only.

Metrics

Scores are entity-level, strict: a predicted span counts only if its character offsets and category both match the gold span exactly. A relaxed score (any character overlap with matching category) is also reported where it was computed, as a diagnostic of boundary behaviour: the gap between the two is the share of identifiers the model finds but mis-delimits. All figures are micro averages unless labelled per category, and are means over five training seeds with the seed standard deviation.

We report precision, recall and F1 together, because in de-identification each answers a different question, and this follows the convention of the field's shared tasks. The 2014 i2b2/UTHealth track reported entity-level micro precision, recall and F1 with F1 as the primary metric for comparing systems; MEDDOCAN 2019 reported the same and additionally a leak score, the proportion of identifiers missed, as an explicit safety measure, observing that systems tend to favour precision over recall. Accordingly:

• Recall is the safety metric: an identifier the model does not find remains in the text.
• Precision is the utility metric: text wrongly marked as an identifier is removed unnecessarily.
• F1 is the comparison metric between systems.
• Leak = 1 − recall, reported per category. This is MEDDOCAN's safety measure and is included so the tables are read in the same terms as that benchmark.

Strict versus relaxed, and why both appear. Under the strict criterion a prediction counts only if its character offsets and its category both match the gold span exactly; this is the criterion the shared tasks use and it is the headline everywhere in this card. Under the relaxed criterion a gold span counts as found if some prediction of the same category overlaps it by at least one character. Relaxed is not an alternative score and is never compared against strict figures from elsewhere; it is a diagnostic that separates two different failures. When strict is low and relaxed is high, the model found the identifier but drew its boundary differently (an attached determiner, included punctuation, a tokeniser split); when both are low, it did not find it. A relaxed value of 1.000 therefore means every gold span of that category was touched by a correctly-typed prediction, which is common for well-formed categories such as dates and is the reason those values are not rare here. The clearest case in this series is age on the XLM-R checkpoints: strict 0.00, relaxed 0.98 to 0.99, because the model tags the number and the gold span includes the unit word.

Seed-to-seed variation in name recall spans SD 0.027-0.213 across languages; per-language differences smaller than about 0.05 are within that variation and are not interpreted.

Results, held-out translated-clinical test (model only)

These tables describe the released weights alone: no rule layer and no threshold adjustment. Each row gives the number of gold spans, so low-count categories can be weighed accordingly.

Luganda, held-out translated-clinical test, model only, 5 seeds

Category	Gold spans	Precision	Recall (strict)	Recall (relaxed)	F1	Leak (1 − R)
Patient name	324	0.342 ±.06	0.438 ±.11	0.735 n=1	0.380 ±.07	0.562
Third-party name	379	0.297 ±.04	0.482 ±.10	0.520 n=1	0.366 ±.06	0.518
National ID	126	0.915 ±.06	0.994 ±.01	1.000 n=1	0.952 ±.03	0.006
Record number	359	0.862 ±.05	0.957 ±.02	1.000 n=1	0.907 ±.04	0.043
Phone	25	0.759 ±.05	0.952 ±.06	1.000 n=1	0.844 ±.05	0.048
Other contact	154	0.798 ±.13	0.787 ±.14	0.974 n=1	0.793 ±.13	0.213
Village / area	1143	0.712 ±.04	0.653 ±.05	0.845 n=1	0.679 ±.03	0.347
Facility	124	0.203 ±.05	0.656 ±.07	0.968 n=1	0.307 ±.06	0.344
Date	377	0.908 ±.03	1.000 ±.00	1.000 n=1	0.952 ±.02	0.000
Age †	336	0.000 ±.00	0.000 ±.00	0.991 n=1	0.000 ±.00	1.000
Profession	4	0.013 ±.01, n=4	0.200 ±.19	0.000 n=1	0.032 ±.02, n=3	0.800
Micro (all gold-bearing)		0.502 ±.02	0.640 ±.01	0.863 n=1	0.562 ±.02	0.360

† Age: strict recall is 0.000 but relaxed recall is 0.991: the spans are found, with a boundary that differs from the test annotation (this backbone's training corpus annotated the bare number, the test annotates number and unit). Leak on the strict column is therefore a boundary artefact, not a miss; in the deployed system this category is rule-owned (recall 1.000), see the rule-layer section.

Not in this table: tribe, religion, clan. The translated test carries no gold spans for them (the source schema has none); they are scored on the synthetic development set under Results, tribe, religion and clan below.

Kiswahili, held-out translated-clinical test, model only, 5 seeds

Category	Gold spans	Precision	Recall (strict)	Recall (relaxed)	F1	Leak (1 − R)
Patient name	328	0.589 ±.10	0.510 ±.03	0.533 n=1	0.541 ±.04	0.490
Third-party name	374	0.424 ±.05	0.680 ±.13	0.751 n=1	0.521 ±.08	0.320
National ID	127	0.994 ±.01	0.924 ±.09	0.992 n=1	0.955 ±.05	0.076
Record number	361	0.850 ±.04	0.970 ±.00	1.000 n=1	0.905 ±.02	0.030
Phone	22	0.848 ±.09	0.991 ±.02	1.000 n=1	0.911 ±.05	0.009
Other contact	157	0.882 ±.05	0.910 ±.03	0.987 n=1	0.895 ±.04	0.090
Village / area	1152	0.814 ±.03	0.652 ±.03	0.808 n=1	0.723 ±.02	0.348
Facility	142	0.173 ±.03	0.687 ±.03	1.000 n=1	0.276 ±.03	0.313
Date	383	0.963 ±.02	0.999 ±.00	1.000 n=1	0.980 ±.01	0.001
Age †	336	0.000 ±.00	0.000 ±.00	0.985 n=1	0.000 ±.00	1.000
Profession	5	0.015 ±.00	0.480 ±.10	0.800 n=1	0.029 ±.01	0.520
Micro (all gold-bearing)		0.562 ±.03	0.675 ±.02	0.860 n=1	0.613 ±.02	0.325

† Age: strict recall is 0.000 but relaxed recall is 0.985: the spans are found, with a boundary that differs from the test annotation (this backbone's training corpus annotated the bare number, the test annotates number and unit). Leak on the strict column is therefore a boundary artefact, not a miss; in the deployed system this category is rule-owned (recall 1.000), see the rule-layer section.

Not in this table: tribe, religion, clan. The translated test carries no gold spans for them (the source schema has none); they are scored on the synthetic development set under Results, tribe, religion and clan below.

Kinyarwanda, held-out translated-clinical test, model only, 5 seeds

Category	Gold spans	Precision	Recall (strict)	Recall (relaxed)	F1	Leak (1 − R)
Patient name	318	0.317 ±.05	0.399 ±.06	0.484 n=1	0.350 ±.04	0.601
Third-party name	361	0.219 ±.04	0.496 ±.09	0.803 n=1	0.302 ±.05	0.504
National ID	124	0.929 ±.08	1.000 ±.00	1.000 n=1	0.962 ±.04	0.000
Record number	344	0.411 ±.10	0.963 ±.00	1.000 n=1	0.570 ±.10	0.037
Phone	19	0.542 ±.25	0.832 ±.14	1.000 n=1	0.640 ±.23	0.168
Other contact	159	0.722 ±.10	0.860 ±.00	1.000 n=1	0.781 ±.06	0.140
Village / area	1123	0.739 ±.05	0.636 ±.07	0.737 n=1	0.681 ±.04	0.364
Facility	117	0.065 ±.02	0.566 ±.05	1.000 n=1	0.115 ±.04	0.434
Date	385	0.790 ±.08	0.998 ±.00	1.000 n=1	0.880 ±.05	0.002
Age †	325	0.000 ±.00	0.000 ±.00	0.997 n=1	0.000 ±.00	1.000
Profession	3	0.004 ±.00	0.800 ±.16	1.000 n=1	0.007 ±.00	0.200
Micro (all gold-bearing)		0.318 ±.07	0.635 ±.02	0.838 n=1	0.421 ±.07	0.365

† Age: strict recall is 0.000 but relaxed recall is 0.997: the spans are found, with a boundary that differs from the test annotation (this backbone's training corpus annotated the bare number, the test annotates number and unit). Leak on the strict column is therefore a boundary artefact, not a miss; in the deployed system this category is rule-owned (recall 1.000), see the rule-layer section.

Not in this table: tribe, religion, clan. The translated test carries no gold spans for them (the source schema has none); they are scored on the synthetic development set under Results, tribe, religion and clan below.

Dholuo, held-out translated-clinical test, model only, 5 seeds

Category	Gold spans	Precision	Recall (strict)	Recall (relaxed)	F1	Leak (1 − R)
Patient name	338	0.478 ±.06	0.599 ±.21	0.583 n=1	0.524 ±.12	0.401
Third-party name	403	0.348 ±.05	0.574 ±.10	0.638 n=1	0.429 ±.05	0.426
National ID	131	0.981 ±.01	0.721 ±.31	1.000 n=1	0.786 ±.26	0.279
Record number	378	0.805 ±.10	0.970 ±.01	1.000 n=1	0.877 ±.07	0.030
Phone	24	0.790 ±.15	0.925 ±.02	1.000 n=1	0.843 ±.11	0.075
Other contact	169	0.887 ±.05	0.918 ±.06	1.000 n=1	0.902 ±.05	0.082
Village / area	1151	0.632 ±.06	0.603 ±.04	0.837 n=1	0.616 ±.04	0.397
Facility	118	0.142 ±.03	0.739 ±.05	1.000 n=1	0.238 ±.04	0.261
Date	426	0.956 ±.04	1.000 ±.00	1.000 n=1	0.977 ±.02	0.000
Age †	345	0.000 ±.00	0.000 ±.00	0.977 n=1	0.000 ±.00	1.000
Profession	4	0.001 ±.00	0.300 ±.10	0.250 n=1	0.002 ±.00	0.700
Micro (all gold-bearing)		0.392 ±.07	0.654 ±.02	0.861 n=1	0.487 ±.05	0.346

† Age: strict recall is 0.000 but relaxed recall is 0.977: the spans are found, with a boundary that differs from the test annotation (this backbone's training corpus annotated the bare number, the test annotates number and unit). Leak on the strict column is therefore a boundary artefact, not a miss; in the deployed system this category is rule-owned (recall 1.000), see the rule-layer section.

Not in this table: tribe, religion, clan. The translated test carries no gold spans for them (the source schema has none); they are scored on the synthetic development set under Results, tribe, religion and clan below.

Across the three released backbones

The same fine-tuning data, recipe family and evaluation were used for all three backbones in this series, so the figures below are directly comparable. Patient-name recall is the safety-critical category; micro F1 is the balanced comparison metric.

Results, names never seen in training

Language	Category	Familiar names (teval)	Unseen names (teval-novel)	Δ	Reshuffle control
Luganda	Patient name	R 0.454 / F1 0.320	R 0.478 / F1 0.329	+0.025	R 0.441 (-0.012)
Luganda	Third-party name	R 0.158 / F1 0.138	R 0.116 / F1 0.105	-0.042	R 0.119 (-0.040)
Kiswahili	Patient name	R 0.509 / F1 0.526	R 0.488 / F1 0.502	-0.021	R 0.509 (+0.000)
Kiswahili	Third-party name	R 0.618 / F1 0.476	R 0.599 / F1 0.474	-0.019	R 0.610 (-0.008)
Kinyarwanda	Patient name	R 0.475 / F1 0.456	R 0.450 / F1 0.439	-0.025	R 0.465 (-0.009)
Kinyarwanda	Third-party name	R 0.507 / F1 0.329	R 0.529 / F1 0.339	+0.022	R 0.512 (+0.006)
Dholuo	Patient name	R 0.488 / F1 0.434	R 0.408 / F1 0.389	-0.080	R 0.429 (-0.059)
Dholuo	Third-party name	R 0.528 / F1 0.384	R 0.546 / F1 0.375	+0.017	R 0.524 (-0.005)

Δ is recall on unseen names minus recall on familiar names. The reshuffle control keeps the original names but permutes them across documents, so it measures context change alone; the part of Δ not explained by the reshuffle column is the effect of name novelty itself. Model only, entity-strict.

The deployed system on the same unseen-name test. The name thresholds were fitted on the unseen-name development slice with a no-recall-regression guard. On the test slice it holds only approximately: the system gives back between 0.006 and 0.028 of patient-name recall on unseen names, against micro F1 gains of 0.19 to 0.30. Held-out unseen-name test, entity-strict, single released checkpoint.

Language	Patient-name recall, model only	Patient-name recall, with rules + calibration	Δ recall	Micro F1, model only	Micro F1, with rules + calibration
Luganda	0.478	0.451	-0.028	0.535	0.725
Kiswahili	0.488	0.479	-0.009	0.620	0.818
Kinyarwanda	0.450	0.443	-0.006	0.502	0.805
Dholuo	0.408	0.388	-0.021	0.509	0.777

Results, tribe, religion and clan (synthetic dev; in-distribution ceiling)

Language	Tribe	Religion	Clan
Luganda	R 0.000 / F1 0.000 (n=63)	R 0.000 / F1 0.000 (n=72)	R 0.000 / F1 0.000 (n=96)
Kiswahili	R 0.172 / F1 0.282 (n=64)	R 0.974 / F1 0.733 (n=76)	R 0.000 / F1 0.000 (n=29)
Kinyarwanda	R 0.000 / F1 0.000 (n=76)	R 0.897 / F1 0.912 (n=87)	R 0.000 / F1 0.000 (n=35)
Dholuo	R 0.420 / F1 0.532 (n=50)	R 0.838 / F1 0.843 (n=80)	R 0.000 / F1 0.000 (n=32)

Synthetic development set, the set training early-stopped on, so an in-distribution ceiling rather than a held-out score. Reported because it is the only set with gold for these categories, which the clinical test corpus lacks. Kinyarwanda tribe and clan are out of scope by ruling (see Training data).

All three released backbones on the same synthetic development set; AfriBERTa is a 5-seed mean, XLM-R and AfroXLMR a single checkpoint. Kinyarwanda tribe and clan are out of scope by ruling and carry no bar.

Out-of-domain

Language	Patient-name recall (news)	Micro F1 (news)
Luganda	0.546 ±.04	0.475 ±.02
Kiswahili	0.526 ±.08	0.454 ±.03
Kinyarwanda	0.480 ±.06	0.326 ±.03
Dholuo	0.358 ±.08	0.223 ±.02

MasakhaNER 1.0 test (news). Names in news text are drawn from a different distribution and register than clinical notes; the gap between this table and the clinical results is the model's domain sensitivity.

Comparison with off-the-shelf systems

Publicly available de-identification and NER systems were run zero-shot on the same held-out test split with the same scorer, so the comparison is like for like. None was trained for these languages or this task; they are the systems a practitioner could pick up today, and the comparison measures what is available without training, not what each architecture could reach if fine-tuned on this data.

GLiNER-multi deserves a specific note, because its zero-shot label prompts can be chosen and its confidence threshold can be set, and a fixed choice understates it. It is shown twice: at its published default (the 13 category names as prompts, threshold 0.3), and at the best operating point reachable by tuning the prompt wording and threshold on the development split, read once on the test split. Tuning lifts it by 0.04 to 0.12 F1, almost entirely from clinical-descriptive prompts and a lower threshold. It remains a general-domain model trained on English-centric span data with no clinical text and none of these four languages, so it is being asked to transfer across domain and language at once. On Kinyarwanda the tuned GLiNER exceeds the model-only score of every released backbone; the released system with its rule layer and calibration (next section) is well above it in every language.

System	Luganda	Kiswahili	Kinyarwanda	Dholuo
presidio_rules	0.242	0.190	0.226	0.230
piiranha_v1	0.149	0.295	0.266	0.125
openmed_pf_ml_v2	0.127	0.173	0.144	0.221
openai_privacy_filter	0.244	0.351	0.262	0.291
gliner_multi, zero-shot	0.351	0.415	0.379	0.328
gliner_multi, prompts and threshold tuned on dev	0.391	0.515	0.501	0.379
lfm25_pii_detector	0.260	0.277	0.268	0.284
masakhane_afroxlmr_ner	0.271	0.470	0.287	0.294
This model	0.562	0.613	0.421	0.487

Micro F1, entity-strict, on the same held-out translated-clinical test. The off-the-shelf systems are run zero-shot; this model is fine-tuned on synthetic data for the task. The tuned GLiNER row selects, per language, the label-prompt set and confidence threshold with the best micro F1 on the development split only, then reads the test split once (Luganda V2 at 0.25, Kiswahili V3 at 0.15, Kinyarwanda V2 at 0.1, Dholuo V2 at 0.15; V2 is a clinical-descriptive prompt set, V3 adds a generic person label; the full grid is in baselines_gliner_calibrated.json and GLINER_CALIBRATION.md).

Effect of a rule layer and threshold calibration (not part of these weights)

The deployable system built on these weights adds two components that are not part of the released model and are reported separately so the model's own performance is not conflated with them.

Why a rule layer. National ID, phone, date, record number, other contact and age have fixed surface forms. Deterministic detectors handle them, and the model is used for names and places. Measured on the held-out translated-clinical test (70% slice), rule layer alone, entity-strict: recall 1.000 in every category in all four languages; precision 1.000 for national ID, record number and phone, 0.987 to 0.994 for other contact, 0.997 to 0.998 for date, and 0.894 to 0.913 for age. The age precision is the cost of a deliberate choice: the age rule accepts both word orders (46 emyaka and emyaka 46), and in this corpus the unit-first form also appears in durations such as emyaka 3 ("three years", of treatment or follow-up), which the rule redacts and the gold does not mark. That is an over-redaction of a number that is not an identifier, accepted so that no patient age is left in the text. Record numbers that carry a facility's own prefix rather than a register name (MUL/2024/00871, KNH/24/00871) are covered by a structure rule (two to four capitals, a two- or four-digit year, four to six digits): it never fires on the translated test, where the register rule already reaches recall 1.000 on 2,078 gold spans, and the same structure matches 1,109 times there, all inside gold.

Why calibration on unseen names. The model attaches a confidence to each span. Fitting the decision threshold on the ordinary development split, where names are token-familiar and confidence is high, suppresses correct low-confidence predictions on unfamiliar names. The name thresholds are therefore fitted on the name-disjoint development slice, with an asymmetric precision floor and a no-recall-regression guard. The same procedure, unchanged, was applied to all three backbones.

Language	Model only P / R / F1	With rules + calibration P / R / F1	Over-redaction, model → system
Luganda	0.475 / 0.617 / 0.537	0.725 / 0.722 / 0.724	0.525 → 0.275
Kiswahili	0.574 / 0.685 / 0.624	0.860 / 0.790 / 0.824	0.426 → 0.140
Kinyarwanda	0.403 / 0.660 / 0.500	0.834 / 0.775 / 0.803	0.597 → 0.166
Dholuo	0.418 / 0.671 / 0.515	0.805 / 0.771 / 0.788	0.582 → 0.195

Same test split, entity-strict; supervisor re-scored. The system figures for this backbone are for the single released checkpoint (n = 1): the five-seed weights were not retained after scoring, so per-seed variance is not available for the calibrated arms. The effect has the same shape on all three backbones: the gain is predominantly precision, with recall preserved or improved.

How the calibration is done. The model attaches a confidence to every span it proposes. A single decision threshold is applied per language: spans above it are kept, spans below it are dropped. The threshold is fitted, never hand-set, on development data only, in three steps. First, a rule layer takes over the fixed-format categories (national ID, phone, date, record number, age), where deterministic detectors reach recall 1.000, so the model's threshold only governs names and places. Second, the threshold for the two name categories is fitted on the name-disjoint development slice, the same documents used for the unseen-name results, rather than on the ordinary development split. This matters because on familiar names the model is confident and any threshold looks fine; on unfamiliar names its confidence is lower, and a threshold fitted on familiar names silently discards correct low-confidence predictions. Fitting on unfamiliar names recovered most of that recall (for example, unseen-name patient-name recall on the AfriBERTa checkpoints rose from 0.343 to 0.494 in Luganda and from 0.235 to 0.524 in Dholuo) at a cost of 1 to 2 percentage points of over-redaction. Third, the fit is asymmetric: for names it maximises recall subject to a precision floor, because a missed name is a disclosure while an over-redacted token is only lost text, and a guard rejects any threshold that would lower recall relative to the uncalibrated model. The test split is not read at any point in this procedure. The identical procedure was applied to all three backbones; the shipped thresholds are in router_config.json.

Evaluation on real clinical records

These models have not been evaluated on real clinical records in any of the four languages. No annotated corpus of that kind existed when this work was done; the clinical evaluation set is a translation of a Spanish corpus, and the project's pre-registered methodology reserves deployment-grade claims for physician-labelled spans. The figures in this card are therefore development-grade measurements on a proxy. As annotated East African clinical text becomes available through the project's clinical workstream, the same evaluation harness applies unchanged, and the numbers here should be read as the estimate to be replaced by that measurement rather than as the final word.

Transfer to other languages

Nothing in the recipe is specific to these four languages beyond the training data. Extending it to another language needs three things: a synthetic clinical corpus in that language generated from the same template-and-vocabulary approach (name, place and facility lists plus the country's identifier formats), a small real-text development set for model selection and threshold fitting, and a base encoder with some pretraining coverage of the language. The last matters most. In a controlled comparison across these four languages, gains from richer training data appeared only where the backbone had pretrained on the language, and vocabulary diversification hurt where it had not. For a language none of the available encoders covers, continued pretraining on monolingual text is the first step, and the Dholuo results here are the realistic expectation for that case.

Bias, risks and limitations

Stated as measurements, on the translated-clinical test unless noted.

• Patient names are found at recall 0.40-0.60; the remainder is the principal residual risk.
• Third-party names are the weakest name category, recall 0.48-0.68. Threshold adjustment did not raise it without lowering recall elsewhere, and retraining on richer data did not move it.
• Village / area names, the largest category by span count, are found at recall 0.60-0.65. Some are mis-typed as third-party names; this is representational rather than a threshold effect.
• Names never seen in training are found at lower recall than familiar ones (table above); the gap is the memorisation effect and is the figure to plan against for real patients.
• Out-of-domain text (news) is handled at substantially lower recall (table above); thresholds fitted on clinical text do not transfer to other registers.
• Profession has 3-5 gold spans per language in the clinical test; its scores are low-powered and are reported for completeness.
• Tribe, religion and clan are evaluated only on the synthetic held-out set, which is in-distribution with training; treat those figures as upper bounds.
• Evaluation corpus. The clinical test is machine-translated from Spanish; it is a proxy for native clinical text. Physician-labelled spans in these languages did not exist at release.
• Training names. Person names in the synthetic training data are recombined from public given- and family-name vocabularies; as with any such generator, some recombinations coincide with the names of real individuals. The model emits offsets and labels only and cannot reproduce a name.

Environmental impact

Hardware: NVIDIA A100 80GB (Azure, West Europe). Usage: 4.3 GPU-hours for this lane (2.67 of training), plus continued pretraining of the adapted base. Emissions were not separately metered and are not estimated here.

Technical specifications

Architecture	AfroXLMR-base: XLM-RoBERTa base (12 layers, hidden 768, 12 heads) language-adapted to 17 African languages by multilingual adaptive fine-tuning, further continued-pretrained on our corpus, with a linear token-classification head
Parameters	278M parameters
Max sequence length	256 tokens (longer documents are windowed)
Label scheme	BIO over 14 categories = 29 tags
Input	Pre-split words over a `\w+
Precision	bf16 training; fp32 inference

Citation

@misc{craneai2026afrimedpii,
  title  = {AfriMed-PII: Clinical de-identification models for Luganda, Kiswahili, Kinyarwanda and Dholuo},
  author = {{Crane AI Labs}},
  year   = {2026},
  note   = {Gates Foundation INV-107957, Workstream 1. Technical report forthcoming.}
}

Funding

This material is based on research funded by the Gates Foundation. The findings and conclusions contained within are those of the authors and do not necessarily reflect positions or policies of the Gates Foundation.