README.md

metadata

title: Togyzkumalak Scoresheet Reader
emoji: ♟️
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false

Togyzkumalak Move OCR

Classifies a photo of a single scoresheet cell into one of 163 classes: 162 moves (11 … 99x — start hole 1–9, end hole 1–9, optional capture x) plus empty. A full-sheet reader turns a whole scoresheet photo into PGN game records, and a Gradio demo (app.py) packages it for a HuggingFace Space.

Live demo (HuggingFace Space)

app.py is a Gradio app: upload up to 5 scoresheet photos, optionally tag each game's known result, and download the reconstructed PGNs (beam / raw / legal) as a zip. Inference runs on the exported ONNX models (torch-free), so the Space needs only the light serving deps in requirements.txt.

python scripts/export_onnx.py     # checkpoints/*.pt -> single-file .onnx (+ .classes.json)
cp checkpoints/best.onnx checkpoints/best.classes.json models/
cp checkpoints/kazan/best.onnx models/kazan.onnx
cp checkpoints/kazan/best.classes.json models/kazan.classes.json
python app.py                     # serve locally at http://127.0.0.1:7860

Deploy: create a Gradio Space and push this repo to its origin remote (the .onnx files are tracked with Git LFS via .gitattributes). The Space reads the YAML front-matter above and installs requirements.txt. No secrets or database are required — it is a stateless demo capped at 5 images per run, with requests serialized through Gradio's queue so a busy free Space degrades into a short wait rather than 429 errors.

Why the old pipeline failed (and what this one does differently)

Problem	Old pipeline	This pipeline
Glyph style	EMNIST (American digits) — but players write European/Kazakh digits: crossed 7, serif 1, cursive 9	ARDIS dataset (real European handwriting) + procedural crossbars/serifs on EMNIST glyphs
Preprocessing	Train and inference normalized differently (0.5/0.5 vs ImageNet stats)	One shared module, togyz/preprocess.py, imported by both
Framing	Small digits with wide margins in fixed 80×40 cells	Random framing from tight crops to loose cells, matching real photos
Data	48 900 fixed PNGs on disk	Infinite on-the-fly synthesis, deterministic validation set

Layout

togyz/            core library
  classes.py      canonical 163-class list (indices match legacy class_mapping.json)
  preprocess.py   THE single image->tensor preprocessing (train AND inference)
  glyphs.py       per-character glyph pools + procedural style edits
  synth.py        cell synthesizer;  python -m togyz.synth --preview preview.png
  dataset.py      on-the-fly synthetic dataset + real-crop eval dataset
  model.py        resnet18 (grayscale, 163 outputs), checkpoint helpers
train.py          training CLI (auto device: cuda/mps/cpu)
eval.py           synthetic val + per-file real-crop report
predict.py        classify images; --allowed restricts to legal moves
scripts/get_glyphs.py   downloads/prepares glyph pools (ARDIS, optional EMNIST)
data/real_crops/  10 labeled real crops (labels.csv) — evaluation only
legacy/           previous experiments, kept for reference

Quickstart (local)

source venv311/bin/activate         # or: pip install -r requirements.txt
python scripts/get_glyphs.py        # one-time: download+prepare ARDIS glyphs
python -m togyz.synth --preview preview.png   # eyeball synthetic vs data/real_crops
python train.py --epochs 1 --samples-per-epoch 4000 --batch-size 64   # smoke test
python eval.py --ckpt checkpoints/best.pt
python predict.py "data/real_crops/*.jpg"

A real training run needs a GPU — see Colab below. Defaults (--epochs 20 --samples-per-epoch 50000) are a sensible full run.

Training on Google Colab (or any GPU machine)

Open notebooks/colab_train.ipynb in Colab, or manually:

git clone <this-repo> && cd 9OCR        # or upload a zip of the project
pip install -r requirements.txt
python scripts/get_glyphs.py --emnist   # ingredients/ is gitignored, so also
                                        # rebuild the EMNIST pool from torchvision
python train.py --epochs 30
# download checkpoints/best.pt when done

predict.py/eval.py run anywhere (CPU is fine) with the downloaded checkpoint.

Inference

python predict.py cell.jpg --topk 3
python predict.py cell.jpg --allowed "12,34x,56"

At any game state at most 9 moves are legal. If an upstream game tracker passes them via --allowed, probabilities are renormalized over just those moves — a large, free accuracy boost when reading whole games.

Reading a whole scoresheet

python read_game.py "data/2026-07-06 00.00.20.jpg" --out out/sheet1 --result 0-1

Pass --result (1-0 / 0-1 / draw, from the sheet footer) when known: the beam's final pool is re-ranked to prefer reconstructions whose end state is consistent with how the game actually ended.

The summary strips between the tables record both kazan counts every 10 moves (Black's box above the strip, White's below, always two digits 00-81). A second classifier reads them (python train.py --task kazan, saved to checkpoints/kazan/best.pt) and the beam gains likelihood when a hypothesis' computed kazans match these written checkpoints.

Finds the 8 printed move tables, classifies every cell, and writes:

• game.json — per ply: all 163 class probabilities, top-5, legality info
• raw.pgn — pure classifier argmax per ply, even if illegal
• legal.pgn — strict replay, stops at the first illegal argmax
• beam.pgn — best reconstruction: beam search over all legal continuations per ply; each legal move is scored by blending its exact class probability with the probability mass of its source digit (the landing digit is derived from the position, so a misread second digit doesn't discard the right source hole). Longest fully legal sequence wins, ties broken by joint probability.
• annotated.jpg, cells/ — visual debugging

Move annotations: + capture, x tuzdyk creation. Strip + to feed the moves to the 9Q engine. Rules implementation: togyz/rules.py, validated against the 9Q C++ engine fixture (python tests/test_rules.py).

Accuracy depends heavily on photo resolution — the CLI warns when cells are under 35 px tall; photograph sheets at full camera resolution.

Improving accuracy further

1. Label more real cells — append rows to data/real_crops/labels.csv. Even ~50 real crops make the reported real-crop accuracy meaningful; a few hundred would allow fine-tuning on them.
2. Add glyph styles: put white-on-black PNG masks under glyph_data/<source>/<char>/ and register the source in togyz/glyphs.py.
3. Full-game decoding with a Togyzkumalak rules engine (choose the most probable legal move per cell) — hook already exists via --allowed.

Notes

• train_data5/ (the old pre-generated dataset) is no longer used and can be deleted; synthesis now happens on the fly.
• Old models/scripts live in legacy/ (.pkl/.pth files are gitignored).