README.md

---
language:
- vi
library_name: transformers
pipeline_tag: text-classification
base_model: "bert-base-multilingual-cased"
tags:
- vietnamese
- fake-news-detection
- text-classification
- vifn
metrics:
- f1
- accuracy
---

# mbert-vifn

This model is `bert-base-multilingual-cased` fine-tuned for binary Vietnamese fake-news classification on the text-only ViFN benchmark.

## Evaluation protocol

- Dataset size: 1,406 examples.
- Fixed splits: 1,124 train / 141 development / 141 test.
- Split seed: 42, stratified by label with exact duplicate groups kept in one split.
- Fine-tuning seeds: [42, 22, 202].
- Training: 3 epoch(s), AdamW, learning rate 2e-05, weight decay 0.01, warmup ratio 0.1.
- Effective train batch size: 8.
- Maximum sequence length: 256.
- Raw Vietnamese text was tokenized directly with the released tokenizer; no external word segmentation.
- No class weighting, resampling, external metadata, images, engagement features, or test-time model selection.
- Checkpoints are selected by development Macro-F1. The representative published checkpoint is seed **42**, selected only by development Macro-F1.

## Results

Test metrics are reported as mean ± sample standard deviation over seeds [42, 22, 202].

| Metric | Mean ± std |
|---|---:|
| Test Macro-F1 | 0.8246 ± 0.0080 |
| Test accuracy | 0.8251 ± 0.0082 |
| Test macro precision | 0.8298 ± 0.0110 |
| Test macro recall | 0.8255 ± 0.0083 |
| Development Macro-F1 | 0.8481 ± 0.0150 |

### Per-seed results

|       seed |   dev_macro_f1 |   test_macro_f1 |   test_accuracy |   micro_batch_size |   gradient_accumulation_steps |
|-----------:|---------------:|----------------:|----------------:|-------------------:|------------------------------:|
|  22.000000 |       0.836353 |        0.815371 |        0.815603 |           8.000000 |                      1.000000 |
|  42.000000 |       0.865004 |        0.829573 |        0.829787 |           8.000000 |                      1.000000 |
| 202.000000 |       0.843016 |        0.828745 |        0.829787 |           8.000000 |                      1.000000 |

## Label mapping

```json
{
  "0": "0",
  "1": "1"
}
```

## Usage

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "BaoNhan/mbert-vifn"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Đây là nội dung tin tức tiếng Việt cần phân loại."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    probabilities = model(**inputs).logits.softmax(dim=-1)[0]
predicted_id = int(probabilities.argmax())
print(model.config.id2label[predicted_id], probabilities.tolist())
```

## Files

- `aggregate_metrics.json`: complete aggregate metrics and training manifest.
- `artifacts/per_seed_results.csv`: one row per fine-tuning seed.
- `artifacts/seed_*_confusion_matrix.csv`: confusion matrix for each seed.
- `artifacts/seed_*_classification_report.json`: per-class metrics.
- `artifacts/seed_*_test_predictions.csv`: IDs, gold/predicted labels and probabilities; raw text is excluded.

## Limitations

ViFN is small and domain-specific. Performance may not transfer to newly emerging misinformation, other Vietnamese writing styles, or texts requiring image/source/engagement evidence. The model predicts from linguistic content only and should not be treated as a factual verification system.

## Dataset citation

```bibtex
@article{huynh2025vifn,
  title={Utilizing Transformer Models To Detect Vietnamese Fake News on Social Media Platforms},
  author={Huynh, Anh-Tuan and Tran, Phuoc},
  journal={KSII Transactions on Internet and Information Systems},
  volume={19},
  number={2},
  pages={472--487},
  year={2025},
  doi={10.3837/TIIS.2025.02.006}
}
```