README.md

---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: summarization
base_model: google/long-t5-tglobal-base
datasets:
- knkarthick/dialogsum
- knkarthick/samsum
tags:
- longt5
- summarization
- dialogue-summarization
metrics:
- rouge
model-index:
- name: longt5-dialogue-summarization
  results:
  - task:
      type: summarization
      name: Dialogue Summarization
    metrics:
    - type: rouge
      name: ROUGE-1
      value: 0.4738
    - type: rouge
      name: ROUGE-2
      value: 0.2316
    - type: rouge
      name: ROUGE-L
      value: 0.3913
    - type: bertscore
      name: BERTScore-F1
      value: 0.9125
---

# LongT5 — Dialogue Summarization

A [LongT5](https://huggingface.co/google/long-t5-tglobal-base) (`tglobal-base`) model fine-tuned to summarize short conversations into a sentence or two. Trained on **DialogSum + SAMSum** — two-person chats and group messenger threads.

- **Base model:** `google/long-t5-tglobal-base`
- **Task:** Abstractive dialogue summarization (English)
- **Max input / output:** 512 / 96 tokens
- **License:** Apache-2.0

> 🔧 **Reproducibility:** the pipeline lives in [`code/`](./tree/main/code) as two runnable
> notebooks — [`data_processing.ipynb`](./blob/main/code/data_processing.ipynb) (clean + EDA +
> investigation) and [`train.ipynb`](./blob/main/code/train.ipynb) (fine-tune + evaluate) —
> plus [`code/REPORT.md`](./blob/main/code/REPORT.md) for the full analysis and decision log.

## Quick start

The model was trained with a `"summarize: "` task prefix — **add it at inference too**:

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

repo = "tuanhqv123/longt5-meeting-summarization"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

dialogue = """Person1: Can I help you?
Person2: I'd like to buy a new mobile phone please."""
inputs = tok("summarize: " + dialogue, return_tensors="pt", truncation=True, max_length=512)
ids = model.generate(**inputs, max_new_tokens=96, num_beams=4, no_repeat_ngram_size=3)
print(tok.decode(ids[0], skip_special_tokens=True))
```

## Evaluation

Held-out test set (DialogSum + SAMSum), beam=4, ROUGE with stemming + BERTScore-F1:

| Metric       | Score  |
|--------------|--------|
| ROUGE-1      | 0.4738 |
| ROUGE-2      | 0.2316 |
| ROUGE-L      | 0.3913 |
| BERTScore-F1 | 0.9125 |

ROUGE measures word overlap; BERTScore measures semantic similarity, so its higher value reflects
that the summaries are usually correct in meaning even when worded differently.

### Per-source breakdown

| Source    | ROUGE-1 | ROUGE-2 | ROUGE-L | BERTScore-F1 |
|-----------|---------|---------|---------|--------------|
| SAMSum    | 0.5026  | 0.2650  | **0.4231** | **0.9156** |
| DialogSum | 0.4521  | 0.1873  | 0.3645  | 0.9100 |

SAMSum (casual messenger chats) summarizes a bit more cleanly than DialogSum (longer, more
structured two-person dialogues).

## Training

**Data:** [`knkarthick/dialogsum`](https://huggingface.co/datasets/knkarthick/dialogsum) +
[`knkarthick/samsum`](https://huggingface.co/datasets/knkarthick/samsum), cleaned and merged
(29,610 rows after cleaning). DialogSum's `#Person1#` tags are normalized to `Person1` so the
two sources share a consistent speaker style. Each dataset's original train/val/test split is kept.

| Setting              | Value                          |
|----------------------|--------------------------------|
| Base model           | `google/long-t5-tglobal-base`  |
| Task prefix          | `summarize: `                  |
| Epochs               | max 10, early stopping (patience 2) → best at epoch 6 |
| Batch size           | 16                             |
| Learning rate        | 3e-4                           |
| Warmup steps         | 200                            |
| Label smoothing      | 0.1                            |
| Precision            | BF16                           |
| Max input / output   | 512 / 96 tokens                |
| Hardware             | 1× NVIDIA RTX 4090             |

### Validation ROUGE-L per epoch

Early stopping (patience 2) picked epoch 6 — validation ROUGE-L peaked there, then didn't improve.

| Epoch | 1 | 2 | 3 | 4 | 5 | **6** | 7 | 8 |
|-------|---|---|---|---|---|---|---|---|
| ROUGE-L | 0.4119 | 0.4261 | 0.4275 | 0.4258 | 0.4335 | **0.4349** | 0.4316 | 0.4322 |

## Intended use & limitations

- **Intended:** abstractive summarization of short English conversations / chat threads.
- **Limitations:** English-only; trained on casual/everyday dialogue, so it may transfer poorly
  to technical, legal, or very long transcripts. Like all abstractive summarizers it can
  hallucinate — verify facts before relying on a summary. Inputs beyond 512 tokens are truncated.

## Citation

Built on LongT5:

```bibtex
@article{guo2021longt5,
  title={LongT5: Efficient Text-To-Text Transformer for Long Sequences},
  author={Guo, Mandy and Ainslie, Joshua and Uthus, David and Ontanon, Santiago and Ni, Jianmo and Sung, Yun-Hsuan and Yang, Yinfei},
  journal={arXiv preprint arXiv:2112.07916},
  year={2021}
}
```