|
|
| --- |
| license: cc-by-4.0 |
| metrics: |
| - bleu4 |
| - meteor |
| - rouge-l |
| - bertscore |
| - moverscore |
| language: zh |
| datasets: |
| - lmqg/qag_zhquad |
| pipeline_tag: text2text-generation |
| tags: |
| - questions and answers generation |
| widget: |
| - text: "南安普敦的警察服务由汉普郡警察提供。南安普敦行动的主要基地是一座新的八层专用建筑，造价3000万英镑。该建筑位于南路，2011年启用，靠近 南安普敦中央 火车站。此前，南安普顿市中心的行动位于市民中心西翼，但由于设施老化，加上计划在旧警察局和地方法院建造一座新博物馆，因此必须搬迁。在Portswood、Banister Park、Hille和Shirley还有其他警察局，在南安普顿中央火车站还有一个英国交通警察局。" |
| example_title: "Questions & Answers Generation Example 1" |
| model-index: |
| - name: lmqg/mt5-base-zhquad-qag |
| results: |
| - task: |
| name: Text2text Generation |
| type: text2text-generation |
| dataset: |
| name: lmqg/qag_zhquad |
| type: default |
| args: default |
| metrics: |
| - name: QAAlignedF1Score-BERTScore (Question & Answer Generation) |
| type: qa_aligned_f1_score_bertscore_question_answer_generation |
| value: 73.57 |
| - name: QAAlignedRecall-BERTScore (Question & Answer Generation) |
| type: qa_aligned_recall_bertscore_question_answer_generation |
| value: 74.12 |
| - name: QAAlignedPrecision-BERTScore (Question & Answer Generation) |
| type: qa_aligned_precision_bertscore_question_answer_generation |
| value: 73.07 |
| - name: QAAlignedF1Score-MoverScore (Question & Answer Generation) |
| type: qa_aligned_f1_score_moverscore_question_answer_generation |
| value: 49.76 |
| - name: QAAlignedRecall-MoverScore (Question & Answer Generation) |
| type: qa_aligned_recall_moverscore_question_answer_generation |
| value: 49.92 |
| - name: QAAlignedPrecision-MoverScore (Question & Answer Generation) |
| type: qa_aligned_precision_moverscore_question_answer_generation |
| value: 49.62 |
| --- |
| |
| # Model Card of `lmqg/mt5-base-zhquad-qag` |
| This model is fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) for question & answer pair generation task on the [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation). |
| |
| |
| ### Overview |
| - **Language model:** [google/mt5-base](https://huggingface.co/google/mt5-base) |
| - **Language:** zh |
| - **Training data:** [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) (default) |
| - **Online Demo:** [https://autoqg.net/](https://autoqg.net/) |
| - **Repository:** [https://github.com/asahi417/lm-question-generation](https://github.com/asahi417/lm-question-generation) |
| - **Paper:** [https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992) |
| |
| ### Usage |
| - With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-) |
| ```python |
| from lmqg import TransformersQG |
| |
| # initialize model |
| model = TransformersQG(language="zh", model="lmqg/mt5-base-zhquad-qag") |
| |
| # model prediction |
| question_answer_pairs = model.generate_qa("南安普敦的警察服务由汉普郡警察提供。南安普敦行动的主要基地是一座新的八层专用建筑，造价3000万英镑。该建筑位于南路，2011年启用，靠近南安普敦中央火车站。此前，南安普顿市中心的行动位于市民中心西翼，但由于设施老化，加上计划在旧警察局和地方法院建造一座新博物馆，因此必须搬迁。在Portswood、Banister Park、Hille和Shirley还有其他警察局，在南安普顿中央火车站还有一个英国交通警察局。") |
|
|
| ``` |
| |
| - With `transformers` |
| ```python |
| from transformers import pipeline |
|
|
| pipe = pipeline("text2text-generation", "lmqg/mt5-base-zhquad-qag") |
| output = pipe("南安普敦的警察服务由汉普郡警察提供。南安普敦行动的主要基地是一座新的八层专用建筑，造价3000万英镑。该建筑位于南路，2011年启用，靠近 南安普敦中央 火车站。此前，南安普顿市中心的行动位于市民中心西翼，但由于设施老化，加上计划在旧警察局和地方法院建造一座新博物馆，因此必须搬迁。在Portswood、Banister Park、Hille和Shirley还有其他警察局，在南安普顿中央火车站还有一个英国交通警察局。") |
|
|
| ``` |
| |
| ## Evaluation |
| |
| |
| - ***Metric (Question & Answer Generation)***: [raw metric file](https://huggingface.co/lmqg/mt5-base-zhquad-qag/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qag_zhquad.default.json) |
| |
| | | Score | Type | Dataset | |
| |:--------------------------------|--------:|:--------|:-------------------------------------------------------------------| |
| | QAAlignedF1Score (BERTScore) | 73.57 | default | [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) | |
| | QAAlignedF1Score (MoverScore) | 49.76 | default | [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) | |
| | QAAlignedPrecision (BERTScore) | 73.07 | default | [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) | |
| | QAAlignedPrecision (MoverScore) | 49.62 | default | [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) | |
| | QAAlignedRecall (BERTScore) | 74.12 | default | [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) | |
| | QAAlignedRecall (MoverScore) | 49.92 | default | [lmqg/qag_zhquad](https://huggingface.co/datasets/lmqg/qag_zhquad) | |
| |
| |
| |
| ## Training hyperparameters |
| |
| The following hyperparameters were used during fine-tuning: |
| - dataset_path: lmqg/qag_zhquad |
| - dataset_name: default |
| - input_types: ['paragraph'] |
| - output_types: ['questions_answers'] |
| - prefix_types: None |
| - model: google/mt5-base |
| - max_length: 512 |
| - max_length_output: 256 |
| - epoch: 4 |
| - batch: 2 |
| - lr: 0.001 |
| - fp16: False |
| - random_seed: 1 |
| - gradient_accumulation_steps: 32 |
| - label_smoothing: 0.15 |
| |
| The full configuration can be found at [fine-tuning config file](https://huggingface.co/lmqg/mt5-base-zhquad-qag/raw/main/trainer_config.json). |
| |
| ## Citation |
| ``` |
| @inproceedings{ushio-etal-2022-generative, |
| title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration", |
| author = "Ushio, Asahi and |
| Alva-Manchego, Fernando and |
| Camacho-Collados, Jose", |
| booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing", |
| month = dec, |
| year = "2022", |
| address = "Abu Dhabi, U.A.E.", |
| publisher = "Association for Computational Linguistics", |
| } |
| |
| ``` |
| |
