Update model card to final paper numbers and house style

README.md

@@ -20,38 +20,55 @@ datasets:
 base_model: vamboai/morena-1.5b-base
 ---
 
+<div align="center">
+<img src="poster.png" alt="MORENA, an African foundation model" width="600">
+</div>
+
 # MORENA 1.5B instruct
 
 MORENA, Sesotho and Setswana for a king, a lord or a chief, is a 1.5B-parameter decoder trained
 from scratch for twelve Latin-script African languages plus English, French and code. This is the
-instruction-tuned model described in the whitepaper *MORENA: An African Foundation Model*. **Private research
-preview. Not for user-facing deployment.**
+instruction-tuned model described in the paper *MORENA: An African Foundation Model*. **Private
+research preview. Not for user-facing deployment.**
 
 ## Headline numbers
 
 | | MORENA 1.5B instruct | reference |
 |---|---|---|
-| African bits per byte, mean of 12 (lower is better) | **1.441** | Lugha-Llama-8B 1.423, gemma-3-12b-it 2.159, Llama-3.2-1B 2.498 |
-| Translation, FLORES+ chrF++, English into 5 African languages, 3-shot | **45.8** | NLLB-600M 45.6, NLLB-1.3B 47.2, Lugha-Llama-8B 36.8, 1B-class general models 9 to 14 |
-| Translation, 5 African languages into English | **48.7** | NLLB-600M 55.1 |
-| Tool calling, correct tool / valid JSON (constrained, prefilled) | **98.1% / 100%** | |
-| Retrieval QA, open-book accuracy / grounding, African mean | 0.325 / 0.734 | 0.25 chance |
-| Safety, good of scorable, 3,341 prompts, 13 languages, 11 categories | **90.2%** | |
+| African bits per byte, mean of 12 (lower is better) | **1.441** | Lugha-Llama-8B 1.423, gemma-3-12b-it 2.159, gemma-3-1b-pt 2.335 |
+| Translation, FLORES+ chrF++, English into 5 African languages, 3-shot | **45.8** (45.1 to 46.5) | NLLB-600M 45.6, NLLB-1.3B 47.2, Lugha-Llama-8B 36.8 |
+| Translation, 5 African languages into English | **48.7** (47.5 to 49.7) | NLLB-600M 55.1, NLLB-1.3B 58.1 |
+| Paired comparison with NLLB-600M, into-African, same sentences | +0.2 chrF++ (-0.5 to +0.8) | a tie overall; Yoruba +3.0 (1.6 to 4.4), isiZulu -2.5 |
+| Belebele reading comprehension, released checkpoint | 0.309 | mean over 10 African languages, 0.24 Yoruba to 0.43 Afrikaans, 0.25 chance |
+| Retrieval QA, open-book accuracy / grounding, African mean | 0.325 (0.297 to 0.355) / 0.73 | 0.25 chance |
+| Grounded generation, fully faithful to given facts | 23% (22 of 96) | previous version 31% (30 of 96); un-instructed base 6 of 96 |
+| Tool calling, correct tool / valid JSON (marker prefilled) | **98.1% / 100%** | |
+| Safety, share of all 3,341 attempts handled well, 13 languages, 11 categories | **89.0%** (87.8 to 90.0) | 90.2% of scorable attempts |
 | Degenerate output | 1.3% | |
-| Benign requests answered well | 58.9% | |
+| Refused harmful requests, share of all attempts | 91.5% | |
+| Benign requests answered well | 58.9% (53.3 to 64.2) | previous version 47.9% |
 
 Safety by category (share of all attempts handled well, 24 prompts per category per language):
 self-harm 92.5, child safety 91.2, drugs 86.2, violence 88.9, election 89.2, privacy 93.8, fraud 94.3,
 weapons 93.3, hate 95.4, medication 95.4. Weakest languages: Igbo 75%, Yoruba 82%, Setswana 83%,
-isiXhosa 87%. Every number is a model judging a model; no native speaker has yet rated an answer.
+isiXhosa 87%. Every number above is a model (google/gemma-3-12b-it) judging a model; no native
+speaker has yet rated an answer.
 
 ## What it is not good at
 
 Retrieval-augmented QA is at chance in African languages even though the model demonstrably reads
-the passage (grounding 0.73). Grounded generation is fully faithful to given facts in about 31% of
-attempts. The model over-refuses: 41% of ordinary benign requests still get a refusal. Tool calling
-is measured with the tool marker prefilled; the model does not reliably decide on its own that a tool
-is needed. Reading comprehension (belebele) 0.302 against 0.250 chance.
+the passage (grounding 0.73). Grounded generation is fully faithful to given facts in 23% of
+attempts, down from 31% in the previous version. The model fails to answer 41% of ordinary benign
+requests well, most of them by refusing; an earlier version once refused to recommend a dry
+cleaner, citing "illegal substances or services". That specific failure is fixed, but the broader
+over-refusal problem is not. Tool calling is measured with the tool marker prefilled; left to decide
+for itself the model almost never calls one. Multiple-choice comprehension in African languages is
+at chance for this model and for every model under 12B measured.
+
+The checkpoint was chosen against a rule fixed before the final experiments: every harm category
+within 3.5 points of the previous version, refusals on at least 91% of all attempts, degeneracy at
+most 1%, no capability lost. It misses that rule on drugs, violence and privacy by 0.1 point, and on
+degeneracy, which is 1.3%.
 
 ## Chat format
 
@@ -67,33 +84,46 @@ A GGUF build for llama.cpp is in `vamboai/morena-1.5b-instruct-gguf`.
 
 ## Training
 
-251.7B tokens of pretraining (45% English, 23% code, 10% native African text, 22% machine-translated
-African text, 7.5% French), 63B tokens of mid-training, 4,500 steps of supervised fine-tuning with
-loss masking and a 500-step safety anneal. Release lineage 12,834 A100 GPU-hours; the research
-programme that produced it about 22,450. Architecture: 28 layers x 2048, GQA 16/4, SwiGLU 6144,
-RoPE theta 500,000, 4,096 context, tied embeddings. Optimiser: Muon for non-embedding weights,
-AdamW for the rest, WSD schedule.
+251.7B tokens of pretraining, 63B tokens of mid-training (315B tokens seen in total; see MORENA
+1.5B base), then 4,500 steps of supervised fine-tuning with loss masking and a 500-step safety
+anneal. Training mixture moved in three regimes as machine-translated languages landed: African
+text was 24.8% of tokens seen (14.8% machine-translated) for steps 1 to 25,304, 39.1% (31.0%
+machine-translated) for steps 25,305 to 60,000, and 50.2% (41.6% machine-translated) during
+mid-training. NLLB-600M output is 24% of pretraining tokens seen and 28% once mid-training is
+included (57.1B tokens on disk, 21% of the 271B on disk).
+
+Release lineage: 12,834 A100 GPU-hours; the research programme that produced it, about 22,450
+(about 22% of a 100,000 GPU-hour allocation), an estimated twenty-five to forty thousand dollars
+at $2 to $3 per A100-hour. Architecture: 28 layers x 2048, GQA 16/4, SwiGLU 6144, RoPE theta
+500,000, 4,096 context, tied embeddings. Optimiser: Muon for non-embedding weights, AdamW for
+the rest, warmup-stable-decay schedule.
 
 ## The MORENA family
 
 | model | params | African bpb (all 12, lower is better) | role |
 |---|---|---|---|
 | MORENA 1.5B base | 1.485B | 1.408 | pretrained and mid-trained; fine-tuning starting point |
-| MORENA 1.5B instruct | 1.485B | 1.441 | chat, translation, tool calling; the model described in the whitepaper |
+| MORENA 1.5B instruct | 1.485B | 1.441 | chat, translation, tool calling; the model described in the paper |
 | MORENA 0.5B mini | 503M | 1.520 | pruned and distilled from the 1.5B base |
 | MORENA 0.5B mini instruct | 503M | 1.540 | chat fine-tune of the mini |
 | MORENA 0.2B nano | 209M | 1.583 | cheap trunk for ASR rescoring, keyboards, normalisation |
 
-Every outside model we measured (25 in total, from 125M to 12B, including the 8B African specialist
-Lugha-Llama-8B at 1.423 and gemma-3-12b-it at 2.159) sits behind all five on African bits per byte.
-Twelve languages: Shona, Swahili, Hausa, Yoruba, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana,
-Afrikaans, isiNdebele, Naija Pidgin; plus English, French and code. Tokenizer: 65,536 entries trained
-on the target mix, fertility 0.239 tokens per byte on African text against 0.246 on English.
+26 models in total were measured on African bits per byte. Every one of the 21 outside models,
+from 125M to 12B parameters, including the 8B African specialist Lugha-Llama-8B (1.423) and
+gemma-3-12b-it (2.159), sits behind all five MORENA sizes. Twelve languages: Shona, Swahili,
+Hausa, Yoruba, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana, Afrikaans, isiNdebele and
+Nigerian Pidgin, plus English, French and code. isiNdebele (ISO code nbl) has no FLORES+ or
+MasakhaNEWS coverage and is evaluated on NCHLT transcripts only.
+
+Tokenizer: 65,536-entry byte-fallback BPE trained on the target mix. African text costs 0.249
+tokens per byte against 0.234 for English, about 6% more per byte than English in MORENA's
+vocabulary, but that same African text needs 1.39x fewer tokens than under Gemma 3's
+vocabulary and 1.53x fewer than under Llama 3.2's.
 
 ## Author and citation
 
-Isheanesu Misi, Vambo AI. Trained on CINECA Leonardo with support
-from the AI Hub for Sustainable Development.
+Isheanesu Misi, Vambo AI. Trained on CINECA Leonardo, with support from the AI Hub for
+Sustainable Development.
 
 ```
 @techreport{misi2026morena,
@@ -108,7 +138,10 @@ from the AI Hub for Sustainable Development.
 
 ## Licence and status
 
-Public release 18 September 2026. Until then a private research preview. Weights are released under a custom licence pending resolution of one
-data-licensing question: 22% of the pretraining corpus derives from an NLLB model under a
-NonCommercial licence. Until that is settled the weights are not for commercial use or public
-redistribution. **Not for user-facing deployment**: see the evaluation notes above.
+Public release 18 September 2026. Until then a private research preview: **not for user-facing
+deployment**. The weights are released as a research preview, non-commercial, because about a
+quarter of the tokens seen in training came from NLLB-600M output (CC-BY-NC-4.0). A tested
+attempt to replace that share with an Apache-licensed translator, MADLAD-400-3B, lost too much
+translation quality to adopt, so the non-commercial restriction stays until the underlying share is
+replaced or the question is otherwise resolved. Not for commercial use or public redistribution
+before the public release date.