---
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
- token-classification
- relation-extraction
- romanian
- span-detection
- ner
language:
- ro
library_name: transformers
pipeline_tag: token-classification
---

# Span detector (Romanian) — entity detection for end-to-end RE

[Dragoș Mitruț Vasile](https://scholar.google.com/citations?user=eD-SutAAAAAJ) · [Elena-Simona Apostol](https://scholar.google.com/citations?user=XUZcjpEAAAAJ) · [Stefan-Adrian Toma](https://scholar.google.com/citations?user=wsz8cUgAAAAJ) · [Adrian Paschke](https://scholar.google.com/citations?user=D_ZARycAAAAJ) · [Ciprian-Octavian Truică](https://scholar.google.com/citations?user=ZOKqr-QAAAAJ)

[![Paper](https://img.shields.io/badge/Paper-InProgress-blue)](https://When-Paper-Appears-it-Will-Work.com)
[![arXiv](https://img.shields.io/badge/arXiv-WIP-b31b1b)](https://arxiv.org/abs/WIP)
[![Website](https://img.shields.io/badge/Project-Website-green)](https://github.com/DS4AI-UPB/crosslingual-romanian-re)
[![GitHub](https://img.shields.io/badge/GitHub-Code-black?logo=github)](https://github.com/DS4AI-UPB/crosslingual-romanian-re)
[![License](https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey)](https://creativecommons.org/licenses/by-nc-sa/4.0/)

A token-classification model (`FacebookAI/xlm-roberta-large`, 560M) that tags the two entity spans `e1` and `e2` in a plain Romanian sentence, using a five-label BIO scheme (`O`, `B-E1`, `I-E1`, `B-E2`, `I-E2`). It is the first stage of the **end-to-end pipeline baseline**: its predicted spans are wrapped in entity markers and passed to the relation classifier ([`DS4AI-UPB/xlmr-large-ro-re`](https://huggingface.co/DS4AI-UPB/xlmr-large-ro-re)).

The SemEval-2010 Task 8 entities are common nominals, not named entities, so an off-the-shelf NER model does not transfer — this detector is trained on the task's own spans instead.

## Results (span F1-Score, held-out validation split)

| Metric | Value |
|--------|-------|
| Span F1-Score | 0.869 |
| Precision | 0.85 |
| Recall | 0.89 |

## Training

- Base: `FacebookAI/xlm-roberta-large`
- Token classification, 5 BIO labels
- 5 epochs, batch size 16, LR 2e-5, 10% warmup, weight decay 0.01
- Best checkpoint by span F1-Score on a 10% validation split
- Single NVIDIA A100 40GB, about 6 min

## Usage

```python
from transformers import AutoModelForTokenClassification, AutoTokenizer

tok = AutoTokenizer.from_pretrained("DS4AI-UPB/span-detector-ro")
model = AutoModelForTokenClassification.from_pretrained("DS4AI-UPB/span-detector-ro")
```

The full pipeline (span detection then relation classification) is in `pipeline_e2e.py` in the [code repository](https://github.com/DS4AI-UPB/crosslingual-romanian-re).

## Citation

```bibtex
@misc{vasile2026crosslingual,
  title  = {Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian},
  author = {Vasile, Drago\c{s}-Mitru\c{t} and Apostol, Elena-Simona and Toma, \c{S}tefan-Adrian and Paschke, Adrian and Truic\u{a}, Ciprian-Octavian},
  year   = {2026},
  note   = {Preprint}
}
```