inesctec/CitiLink-BERTimbau-large-metadata-pt-baseline

Baseline_BERTimbau-large-metadata-council-pt: Metadata Extraction for Municipal Meeting Minutes

Model Description

BERTimbau-large-metadata-council-pt is a baseline implementation, consisting of a fine-tuned BERT model for Named Entity Recognition (NER), to automatically extract metadata such as meeting number, date, location, participants, and time expressions from Portuguese municipal meeting minutes.
It was developed as part of a study on information extraction and indexing of administrative documents.

Key Features

• 🏛️ Domain-Specific: Trained exclusively on authentic municipal meeting minutes
• 🧠 Entity-Level Extraction: Identifies key metadata (minute ID, date, location, start/end times, participants)
• ⚙️ Transformer-based Architecture: BERTimbau backbone with fine-tuning for token classification
• 📈 Strong NER Performance: Achieves F1-score above 0.95 on the Portuguese dataset

Model Details

• Base Model: neuralmind/bert-large-portuguese-cased
• Architecture: BERT for token classification (NER)
• Parameters: ~335M
  • 24 transformer layers
  • 1024 hidden dimensions
  • 16 attention heads
  • 4096 intermediate size
• Max Sequence Length: 512 tokens
• Learning Rate: 5e-5
• Warmup: 0.1
• Batch Size: 16
• Optimizer: AdamW
• Weight Decay: 0.01
• Entity Types: minute_id, date, meeting_type, location, begin_time, end_time, participant
• Training Framework: PyTorch + Hugging Face Transformers
• Evaluation Metric: Micro F1-score, Precision and Recall (seqeval)

How It Works

The model assigns a label to each token in the input sequence, using the BIO scheme (Begin–Inside–Outside).
It can recognize and extract structured metadata from free-form text, even when expressed with stylistic variation.

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

# Load model and tokenizer
MODEL_NAME = "anonymous13542/BERTimbau-large-metadata-council-pt"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForTokenClassification.from_pretrained(MODEL_NAME)
model.eval()

# Example text
text = "ATA Nº 23 REUNIÃO ORDINÁRIA PÚBLICA DA CÂMARA MUNICIPAL DE CAMPO MAIOR, REALIZADA EM 17 DE NOVEMBRO DE 2021.\nAos dezassete dias do mês de novembro do ano de dois mil e vinte e um, no Edifício dos Paços do Concelho, nesta Vila, realizou-se, pelas nove horas e trinta minutos, a reunião Ordinária Pública da Câmara Municipal, comparecendo os Excelentíssimos Senhores Luis Fernando Martins Rosinha, Paulo Ivo Sabino Martins de Almeida, Paulo Jorge Furtado Pinheiro, Maria da Encarnação Grifo Silveirinha e Fátima do Rosário Pingo Vitorino Pereira, respetivamente, Presidente e Vereadores efetivos deste Órgão Autárquico.\n-Verificada a presença dos respectivos membros, o Senhor Presidente declarou aberta a reunião:\n-Estava presente o Chefe **************************************, Dr. *********************************** e a Assistente Técnico **************************************.\n\n-ORDEM DO DIA:\n-INICIANDO A ORDEM DO DIA, ENTREGUE A TODO O EXECUTIVO E ELABORADA NOS TERMOS DO ARTIGO 53º DA LEI Nº 75/2013, DE 12 DE SETEMBRO, A CÂMARA TRATOU OS SEGUINTES ASSUNTOS:\n-ORDEM DO DIA:\n-FINANÇAS MUNICIPAIS:\n"

# Tokenize with offset mapping
inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=512,
    return_offsets_mapping=True
)
offsets = inputs.pop("offset_mapping")[0]

# Predict
with torch.no_grad():
    outputs = model(**inputs)

predictions = torch.argmax(outputs.logits, dim=2)[0]
labels = [model.config.id2label[p.item()] for p in predictions]

# Extract entities using character spans
entities = []
current = None

for (start, end), label in zip(offsets.tolist(), labels):
    if label == "O" or start == end:
        if current:
            entities.append(current)
            current = None
        continue

    if label.startswith("B-"):
        if current:
            entities.append(current)
        current = {"label": label[2:], "start": start, "end": end}
    elif label.startswith("I-") and current and label[2:] == current["label"]:
        current["end"] = end
    else:
        if current:
            entities.append(current)
        current = None

if current:
    entities.append(current)

# Print results
print("\nDetected Entities:")
for ent in entities:
    span = text[ent["start"]:ent["end"]]
    print(f"- {ent['label']}: {span}")

Evaluation Results

Municipal Meeting Minutes Test Set

Metric	Score
F1 score	0.96
Precision	0.95
Recall	0.97

Limitations

• Domain Specificity: Best performance on administrative/governmental meeting minutes
• Language: Optimized for Portuguese; English performance may vary
• Context Window: Limited to 512 tokens

License

This project uses a custom dual-license based on AGPL v3.

See the full license terms here: LICENSE