BigZ-DuCiel/barbados-road-htr

Barbados Historic Handwriting — model weights

Weights of our solution to the Zindi Barbados Historic Handwriting Challenge: line-level transcription of 17th- and 18th-century Barbadian notarial deeds.

	public leaderboard	private leaderboard
selected submission	0.92969	0.926180

Approach

Two fine-tuned Qwen3-VL readers (8B and 32B) each propose candidate transcriptions of a line: the beam-3 top-1 and the beam-8 8-best, about 14 candidates in all. A scorer picks one, using:

• three character-level CTC judges, each averaged over three crops of the line band;
• a character 14-gram language model;
• teacher-forced cross-scores from both readers.

score = Σ z(judges) + 0.15 · z(LM) + 1.3 · mean z(cross-scores)      (z-scores within the line)

Files

file	what it is	size
weights/readers/pseudo/	LoRA adapter of the 8B reader, on Qwen/Qwen3-VL-8B-Instruct	420 MB
weights/readers/pseudo32/	LoRA adapter of the 32B reader, on Qwen/Qwen3-VL-32B-Instruct	1.16 GB
weights/judges/dino96.pt	CTC judge, ConvNeXt-small (DINOv3), band height 96 px	217 MB
weights/judges/h128.pt	CTC judge, ConvNeXt-small (ImageNet), band height 128 px	218 MB
weights/judges/dino128p.pt	CTC judge, ConvNeXt-small (DINOv3), 128 px, trained with round-1 pseudo-labels	218 MB
submission/submitted.csv	our selected submission (test set predictions)	0.1 MB

The base models Qwen/Qwen3-VL-8B-Instruct and Qwen/Qwen3-VL-32B-Instruct are not stored here; they download from the Hub on first use.

Training recipes

Readers. LoRA r 32, α 64, dropout 0.1, on every linear layer of the language model and of the vision tower. 3 epochs, lr 1.5e-4 with a cosine schedule and 5 % warm-up, effective batch 8, bf16, seed 42. Images get 512 to 1,536 visual tokens, a contrast boost of ×1.5 and light augmentation. The loss covers the answer tokens only.

reader	training lines held out	pseudo-labelled test lines
8B (pseudo)	614	round 1 (779)
32B (pseudo32)	204	round 2 (832)

CTC judges. A ConvNeXt-small backbone whose deep stages only downsample vertically, then a 4-layer Transformer (dimension 256) and a character CTC head. Each judge is trained on the central text band of the line image, with 204 training lines held out; the best checkpoint on them is kept.

Pseudo-labels are test lines whose transcription, chosen automatically by an earlier version of the pipeline, was also proposed by every reader of that version. No transcription was entered or corrected by hand. Only the competition data was used.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base = "Qwen/Qwen3-VL-8B-Instruct"
processor = AutoProcessor.from_pretrained(base, min_pixels=512 * 1024, max_pixels=1536 * 1024)
model = AutoModelForImageTextToText.from_pretrained(base, dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(model, "BigZ-DuCiel/barbados-road-htr",
                                  subfolder="weights/readers/pseudo").eval()

The readers were trained on RGB line images with the contrast boosted ×1.5 (PIL.ImageEnhance.Contrast(img).enhance(1.5)), the image placed after this prompt in a single user turn:

Transcribe the single line of 17th-century handwriting in this image. The crop may include
partial fragments of the lines above and below: ignore them and transcribe only the main, fully
visible line running across the middle. Reproduce the scribe's original spelling,
capitalisation, abbreviations and superscript markers (^) letter for letter. Never modernise or
correct spelling. Output the transcription only.

(in the code the prompt is a single line; it is wrapped here for reading).

Licences

• The LoRA adapters derive from Qwen3-VL-8B/32B-Instruct (Apache 2.0).
• The judges dino96 and dino128p use timm's convnext_small.dinov3_lvd1689m weights, under the DINOv3 License (dinov3-license).
• The judge h128 uses ImageNet convnext_small weights (Apache 2.0).
• Use each file within the terms of its upstream licence.