linhanhvlog123
/

mbert-viquad2.0-qa

@@ -1,199 +1,240 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
 library_name: transformers
+language:
+- en
+- vi
+base_model:
+- google-bert/bert-base-multilingual-cased
+pipeline_tag: question-answering
 ---
+# 🇻🇳 mBERT fine-tuned on UIT-ViQuAD 2.0 for Vietnamese Question Answering
+This repository provides a **multilingual BERT (mBERT) model fine-tuned for extractive Question Answering (QA)** on **UIT-ViQuAD 2.0**, a Vietnamese Machine Reading Comprehension (MRC) benchmark that includes both **answerable and unanswerable questions**, following the SQuAD 2.0 setting.
+The model is trained using the Hugging Face `run_qa.py` pipeline with a **fixed hyperparameter configuration**, enabling a **fair and controlled comparison** with other multilingual QA models such as XLM-RoBERTa.
+---
+## 📌 Task Description
+* **Task**: Extractive Question Answering (Machine Reading Comprehension)
+* **Language**: Vietnamese
+* **Input**:
+  * A Vietnamese *context paragraph*
+  * A *question* related to the context
+* **Output**:
+  * An extracted answer span from the context **or**
+  * An empty string if the question is **unanswerable**
+This setup strictly follows the **SQuAD 2.0 paradigm**, where the model must:
+1. Predict correct answer spans
+2. Detect questions with no valid answer in the given context
+---
+## 📚 Dataset: UIT-ViQuAD 2.0
+UIT-ViQuAD 2.0 is a large-scale Vietnamese QA benchmark released for the **VLSP 2021 Machine Reading Comprehension shared task**, designed to address the lack of Vietnamese datasets containing **unanswerable questions**.
+### Dataset statistics
+| Split        | # Questions |
+| ------------ | ----------- |
+| Train        | 28,457      |
+| Dev          | ~5,700      |
+| Public Test  | ~3,821      |
+| Private Test | 3,712       |
+* Text source: Wikipedia-style Vietnamese articles
+* Domains: history, geography, culture, science, etc.
+* Annotation: human-annotated answer spans and unanswerable labels
+---
+## 🧾 Data Format
+### 1️⃣ SQuAD-style format (for evaluation)
+```json
+{
+  "data": [
+    {
+      "title": "...",
+      "paragraphs": [
+        {
+          "context": "...",
+          "qas": [
+            {
+              "id": "uit_000001",
+              "question": "...",
+              "answers": [
+                {
+                  "text": "...",
+                  "answer_start": 123
+                }
+              ],
+              "is_impossible": false
+            }
+          ]
+        }
+      ]
+    }
+  ]
+}
+```
+This format is required by the **official SQuAD v2.0 evaluation script** (`evaluate-v2.0.py`) to compute **Exact Match (EM)** and **F1-score**.
+---
+### 2️⃣ Hugging Face QA format (for training & inference)
+To train with Hugging Face `run_qa.py`, the dataset is normalized into the following flat QA format:
+```json
+{
+  "id": "uit_000001",
+  "title": "...",
+  "context": "...",
+  "question": "...",
+  "answers": {
+    "text": ["..."],
+    "answer_start": [123]
+  }
+}
+```
+Unanswerable questions are represented with empty `text` and `answer_start` fields.
+---
+## 🔄 Data Preprocessing Pipeline
+The dataset is **preprocessed in two stages** to ensure compatibility with both training and evaluation tools.
+### 🔹 Stage 1: UIT-ViQuAD → SQuAD format
+Purpose:
+* Preserve hierarchical structure (`paragraphs`, `qas`)
+* Enable evaluation using the official Stanford SQuAD v2.0 script
+### 🔹 Stage 2: SQuAD format → Hugging Face QA format
+Purpose:
+* Enable training and inference with `run_qa.py`
+Key steps:
+* Flatten paragraph-level data
+* Normalize answer spans
+* Retain unanswerable question labels
+* Validate span offsets
+---
+## 🧠 Model
+* **Base model**: `bert-base-multilingual-cased`
+* **Architecture**: Transformer encoder (mBERT)
+* **Head**: Span-based QA head (start/end logits)
+* **Tokenizer**: Multilingual BERT tokenizer
+---
+## ⚙️ Training Configuration
+The model is trained using a **shared hyperparameter configuration**, identical to other baseline models in this project.
+```text
+Model:              bert-base-multilingual-cased
+Max sequence length: 512
+Document stride:     256
+Train batch size:    16
+Eval batch size:     8
+Learning rate:       2e-5
+Epochs:              3
+Optimizer:           AdamW
+FP16:                Enabled
+Seed:                42
+Version 2 QA:        Enabled (unanswerable questions)
+```
+Training is performed using the Hugging Face `Trainer` API via `run_qa.py`.
+---
+## 📊 Evaluation Results (Private Test Set)
+Evaluation is conducted using the **official SQuAD v2.0 evaluation script** on the private test set.
+```json
+{
+  "exact": 49.33,
+  "f1": 60.36,
+  "HasAns_exact": 41.64,
+  "HasAns_f1": 57.41,
+  "NoAns_exact": 67.20,
+  "NoAns_f1": 67.20,
+  "total": 3712
+}
+```
+### Observations
+* mBERT shows **strong performance on No-Answer detection**
+* Higher **overall EM and F1** compared to XLM-R baseline
+* Performance gap suggests different inductive biases between multilingual pre-trained models
+---
+## 🚀 Usage
+### Load the model
+```python
+from transformers import AutoModelForQuestionAnswering, AutoTokenizer
+model = AutoModelForQuestionAnswering.from_pretrained(
+    "linhanhvlog123/mbert-viquad2.0-qa"
+)
+tokenizer = AutoTokenizer.from_pretrained(
+    "linhanhvlog123/mbert-viquad2.0-qa"
+)
+```
+### Inference example
+```python
+from transformers import pipeline
+qa = pipeline("question-answering", model=model, tokenizer=tokenizer)
+qa({
+    "context": "...",
+    "question": "..."
+})
+```
+---
+## 🏁 Notes
+* This model serves as a **baseline multilingual QA system** for Vietnamese.
+* All hyperparameters are kept fixed to ensure **fair comparison** across models.
+* Further improvements may be achieved via:
+  * Model-specific hyperparameter tuning
+  * Larger batch sizes
+  * Additional Vietnamese pretraining
+---
+## 📜 Citation
+If you use UIT-ViQuAD 2.0 or this model, please cite the original dataset paper from **VLSP 2021**.