Initial demo: side-by-side fill-mask comparison vioBERT-v3 vs MARBERTv2

README.md

@@ -1,12 +1,37 @@
 ---
-title: VioBERT V3 Demo
-emoji: 📈
-colorFrom: gray
-colorTo: purple
+title: vioBERT-v3 Live Demo
+emoji: 🩺
+colorFrom: green
+colorTo: blue
 sdk: gradio
-sdk_version: 6.13.0
+sdk_version: 4.44.0
 app_file: app.py
-pinned: false
+pinned: true
+license: apache-2.0
+short_description: Live fill-mask — vioBERT-v3 vs MARBERTv2
+tags:
+  - arabic
+  - medical
+  - bert
+  - fill-mask
+  - clinical-nlp
+  - domain-adaptation
+  - mena
+models:
+  - Vionex-digital/vioBERT-v3
+  - UBC-NLP/MARBERTv2
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# vioBERT-v3 Live Demo
+
+Side-by-side fill-mask comparison: **vioBERT-v3** (Arabic medical BERT, MARBERTv2 + DAPT on 1.12M medical docs) vs **MARBERTv2** (base, trained on 1B Arabic tweets).
+
+## What you'll see
+
+Type an Arabic sentence with `[MASK]`. Both models predict the masked token. On medical content, vioBERT-v3 fills with anatomy / drugs / conditions; MARBERTv2 often fills with politics / sports / news terms (its training distribution).
+
+## Why it matters
+
+There are 422M Arabic speakers and previously zero strong open biomedical Arabic LM. vioBERT-v3 is the first. Apache-2.0 — fine-tune commercially without asking.
+
+📦 [Model card →](https://huggingface.co/Vionex-digital/vioBERT-v3)

app.py

@@ -0,0 +1,111 @@
+"""vioBERT-v3 vs MARBERTv2 — side-by-side fill-mask demo.
+
+Visitors see *why* domain adaptation matters for Arabic medical NLP in
+real time, on their own sentences. This is the marketing artifact —
+benchmarks on a model card don't convince like a live A/B does.
+"""
+
+import gradio as gr
+from transformers import AutoModelForMaskedLM, AutoTokenizer
+import torch
+import time
+
+
+VIOBERT_ID = "Vionex-digital/vioBERT-v3"
+BASE_ID = "UBC-NLP/MARBERTv2"
+
+print("[boot] loading models...")
+t0 = time.time()
+viobert_tok = AutoTokenizer.from_pretrained(VIOBERT_ID)
+viobert = AutoModelForMaskedLM.from_pretrained(VIOBERT_ID).eval()
+base_tok = AutoTokenizer.from_pretrained(BASE_ID)
+base = AutoModelForMaskedLM.from_pretrained(BASE_ID).eval()
+print(f"[boot] models loaded in {time.time()-t0:.1f}s")
+
+
+def fill_mask(model, tok, text, top_k=5):
+    if "[MASK]" not in text and tok.mask_token not in text:
+        return [("⚠️ add [MASK] in the sentence", 0.0)]
+    text = text.replace("[MASK]", tok.mask_token)
+    inputs = tok(text, return_tensors="pt")
+    mask_idx = (inputs["input_ids"][0] == tok.mask_token_id).nonzero(as_tuple=True)[0]
+    if len(mask_idx) == 0:
+        return [("⚠️ no [MASK] found", 0.0)]
+    with torch.no_grad():
+        logits = model(**inputs).logits[0, mask_idx[0]]
+    probs = torch.softmax(logits, dim=-1)
+    top = torch.topk(probs, top_k)
+    return [(tok.decode(t).strip(), float(s)) for t, s in zip(top.indices, top.values)]
+
+
+def compare(text):
+    if not text.strip():
+        return "👈 Enter an Arabic sentence with [MASK]", "👈 Enter an Arabic sentence with [MASK]"
+    vio = fill_mask(viobert, viobert_tok, text)
+    bas = fill_mask(base, base_tok, text)
+    def fmt(rows, label, color):
+        lines = [f"### {label}"]
+        for tok_str, score in rows:
+            bar = "█" * int(score * 30)
+            lines.append(f"`{tok_str}` &nbsp;&nbsp; **{score:.3f}** &nbsp; <span style='color:{color}'>{bar}</span>")
+        return "\n\n".join(lines)
+    return fmt(vio, "🩺 vioBERT-v3 (medical-adapted)", "#16a34a"), fmt(bas, "📰 MARBERTv2 (base, tweets)", "#64748b")
+
+
+EXAMPLES = [
+    ["المريض يعاني من [MASK] في الصدر"],
+    ["تم تشخيص الحالة على أنها [MASK] من النوع الثاني"],
+    ["الجرعة الموصى بها للأطفال هي [MASK] ملليجرام"],
+    ["تظهر صورة الأشعة وجود [MASK] في الرئة اليمنى"],
+    ["يحتاج المريض إلى [MASK] فوري في غرفة العمليات"],
+    ["ارتفاع [MASK] الدم قد يؤدي إلى السكتة الدماغية"],
+]
+
+CSS = """
+.gradio-container {max-width: 1100px !important;}
+.markdown-text {direction: rtl; text-align: right; font-size: 1.1em;}
+"""
+
+DESCRIPTION = """
+# 🩺 vioBERT-v3 vs MARBERTv2 — live fill-mask comparison
+
+**vioBERT-v3** is the first Arabic medical BERT — MARBERTv2 + 22K steps of continued pretraining
+on **Shifaa**, our 1.12M-document Arabic medical corpus.
+
+This Space lets you see the difference live. Type any Arabic sentence with `[MASK]`,
+get top-5 predictions from both models side-by-side. Medical sentences are where the
+gap is widest.
+
+Benchmarks vs MARBERTv2: −82.7% medical PPL · +15.6 pp fill-mask Top-5 ·
++0.93 pp medical NER F1 *(exceeds the +0.62 pp BioBERT got on English)*
+
+📦 [Model](https://huggingface.co/Vionex-digital/vioBERT-v3) · 
+🏥 [Vionex Digital Solutions](https://huggingface.co/Vionex-digital) ·
+📜 Apache-2.0 · free for commercial use
+"""
+
+with gr.Blocks(css=CSS, theme=gr.themes.Soft(primary_hue="green")) as demo:
+    gr.Markdown(DESCRIPTION)
+    with gr.Row():
+        text = gr.Textbox(
+            label="Arabic medical sentence with [MASK]",
+            placeholder="المريض يعاني من [MASK] في الصدر",
+            rtl=True,
+            lines=2,
+        )
+    with gr.Row():
+        btn = gr.Button("🔍 Compare predictions", variant="primary", scale=1)
+    with gr.Row():
+        vio_out = gr.Markdown(label="vioBERT-v3", elem_classes="markdown-text")
+        base_out = gr.Markdown(label="MARBERTv2", elem_classes="markdown-text")
+    gr.Examples(examples=EXAMPLES, inputs=text, label="Try a medical example")
+    btn.click(fn=compare, inputs=text, outputs=[vio_out, base_out])
+    text.submit(fn=compare, inputs=text, outputs=[vio_out, base_out])
+    gr.Markdown(
+        "---\n*Built by Vionex Digital Solutions · "
+        "Domain-adaptive pretraining for Arabic medical NLP. "
+        "If this is useful, give vioBERT a like on its [model page](https://huggingface.co/Vionex-digital/vioBERT-v3) or cite the upcoming paper.*"
+    )
+
+if __name__ == "__main__":
+    demo.launch()

requirements.txt

@@ -0,0 +1,4 @@
+transformers>=4.44.0
+torch>=2.0
+gradio>=4.44.0
+sentencepiece