---
library_name: transformers
language:
- en
- vi
base_model:
- google-bert/bert-base-multilingual-cased
pipeline_tag: question-answering
---

# 🇻🇳 mBERT fine-tuned on UIT-ViQuAD 2.0 for Vietnamese Question Answering

This repository provides a **multilingual BERT (mBERT) model fine-tuned for extractive Question Answering (QA)** on **UIT-ViQuAD 2.0**, a Vietnamese Machine Reading Comprehension (MRC) benchmark that includes both **answerable and unanswerable questions**, following the SQuAD 2.0 setting.

The model is trained using the Hugging Face `run_qa.py` pipeline with a **fixed hyperparameter configuration**, enabling a **fair and controlled comparison** with other multilingual QA models such as XLM-RoBERTa.

---

## 📌 Task Description

* **Task**: Extractive Question Answering (Machine Reading Comprehension)
* **Language**: Vietnamese
* **Input**:

  * A Vietnamese *context paragraph*
  * A *question* related to the context
* **Output**:

  * An extracted answer span from the context **or**
  * An empty string if the question is **unanswerable**

This setup strictly follows the **SQuAD 2.0 paradigm**, where the model must:

1. Predict correct answer spans
2. Detect questions with no valid answer in the given context

---

## 📚 Dataset: UIT-ViQuAD 2.0

UIT-ViQuAD 2.0 is a large-scale Vietnamese QA benchmark released for the **VLSP 2021 Machine Reading Comprehension shared task**, designed to address the lack of Vietnamese datasets containing **unanswerable questions**.

### Dataset statistics

| Split        | # Questions |
| ------------ | ----------- |
| Train        | 28,457      |
| Dev          | ~5,700      |
| Public Test  | ~3,821      |
| Private Test | 3,712       |

* Text source: Wikipedia-style Vietnamese articles
* Domains: history, geography, culture, science, etc.
* Annotation: human-annotated answer spans and unanswerable labels

---

## 🧾 Data Format

### 1️⃣ SQuAD-style format (for evaluation)

```json
{
  "data": [
    {
      "title": "...",
      "paragraphs": [
        {
          "context": "...",
          "qas": [
            {
              "id": "uit_000001",
              "question": "...",
              "answers": [
                {
                  "text": "...",
                  "answer_start": 123
                }
              ],
              "is_impossible": false
            }
          ]
        }
      ]
    }
  ]
}
```

This format is required by the **official SQuAD v2.0 evaluation script** (`evaluate-v2.0.py`) to compute **Exact Match (EM)** and **F1-score**.

---

### 2️⃣ Hugging Face QA format (for training & inference)

To train with Hugging Face `run_qa.py`, the dataset is normalized into the following flat QA format:

```json
{
  "id": "uit_000001",
  "title": "...",
  "context": "...",
  "question": "...",
  "answers": {
    "text": ["..."],
    "answer_start": [123]
  }
}
```

Unanswerable questions are represented with empty `text` and `answer_start` fields.

---

## 🔄 Data Preprocessing Pipeline

The dataset is **preprocessed in two stages** to ensure compatibility with both training and evaluation tools.

### 🔹 Stage 1: UIT-ViQuAD → SQuAD format

Purpose:

* Preserve hierarchical structure (`paragraphs`, `qas`)
* Enable evaluation using the official Stanford SQuAD v2.0 script

### 🔹 Stage 2: SQuAD format → Hugging Face QA format

Purpose:

* Enable training and inference with `run_qa.py`

Key steps:

* Flatten paragraph-level data
* Normalize answer spans
* Retain unanswerable question labels
* Validate span offsets

---

## 🧠 Model

* **Base model**: `bert-base-multilingual-cased`
* **Architecture**: Transformer encoder (mBERT)
* **Head**: Span-based QA head (start/end logits)
* **Tokenizer**: Multilingual BERT tokenizer

---

## ⚙️ Training Configuration

The model is trained using a **shared hyperparameter configuration**, identical to other baseline models in this project.

```text
Model:              bert-base-multilingual-cased
Max sequence length: 512
Document stride:     256
Train batch size:    16
Eval batch size:     8
Learning rate:       2e-5
Epochs:              3
Optimizer:           AdamW
FP16:                Enabled
Seed:                42
Version 2 QA:        Enabled (unanswerable questions)
```

Training is performed using the Hugging Face `Trainer` API via `run_qa.py`.

---

## 📊 Evaluation Results (Private Test Set)

Evaluation is conducted using the **official SQuAD v2.0 evaluation script** on the private test set.

```json
{
  "exact": 49.33,
  "f1": 60.36,
  "HasAns_exact": 41.64,
  "HasAns_f1": 57.41,
  "NoAns_exact": 67.20,
  "NoAns_f1": 67.20,
  "total": 3712
}
```

### Observations

* mBERT shows **strong performance on No-Answer detection**
* Higher **overall EM and F1** compared to XLM-R baseline
* Performance gap suggests different inductive biases between multilingual pre-trained models

---

## 🚀 Usage

### Load the model

```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer

model = AutoModelForQuestionAnswering.from_pretrained(
    "linhanhvlog123/mbert-viquad2.0-qa"
)
tokenizer = AutoTokenizer.from_pretrained(
    "linhanhvlog123/mbert-viquad2.0-qa"
)
```

### Inference example

```python
from transformers import pipeline

qa = pipeline("question-answering", model=model, tokenizer=tokenizer)

qa({
    "context": "...",
    "question": "..."
})
```

---

## 🏁 Notes

* This model serves as a **baseline multilingual QA system** for Vietnamese.
* All hyperparameters are kept fixed to ensure **fair comparison** across models.
* Further improvements may be achieved via:

  * Model-specific hyperparameter tuning
  * Larger batch sizes
  * Additional Vietnamese pretraining

---

## 📜 Citation

If you use UIT-ViQuAD 2.0 or this model, please cite the original dataset paper from **VLSP 2021**.