--- title: Togyzkumalak Scoresheet Reader emoji: ♟️ colorFrom: indigo colorTo: green sdk: gradio sdk_version: 5.9.1 python_version: 3.11.9 app_file: app.py pinned: false --- # Togyzkumalak Move OCR Classifies a photo of a single scoresheet cell into one of **163 classes**: 162 moves (`11` … `99x` — start hole 1–9, end hole 1–9, optional capture `x`) plus `empty`. A full-sheet reader turns a whole scoresheet photo into PGN game records, and a **Gradio demo** (`app.py`) packages it for a HuggingFace Space. ## Live demo (HuggingFace Space) `app.py` is a Gradio app: upload up to **5** scoresheet photos of one round as a single batch, describe the tournament (`Tournament,Location,Date,Round`) and each game (`WhiteName,BlackName,Result,WhiteTime,BlackTime`, one CSV line per image) in two text fields, pick a beam width (1024 … 2^30; oversized requests are trimmed to fit memory), and download the reconstructed PGNs (beam / raw / legal, with PGN tags) as a zip. Inference runs on the exported **ONNX** models (torch-free), so the Space needs only the light serving deps in `requirements.txt`. ```bash python scripts/export_onnx.py # checkpoints/*.pt -> single-file .onnx (+ .classes.json) cp checkpoints/best.onnx checkpoints/best.classes.json models/ cp checkpoints/diagram/best.onnx models/diagram.onnx cp checkpoints/diagram/best.classes.json models/diagram.classes.json python app.py # serve locally at http://127.0.0.1:7860 ``` Deploy: create a Gradio Space and push this repo to its `origin` remote (the `.onnx` files are tracked with Git LFS via `.gitattributes`). The Space reads the YAML front-matter above and installs `requirements.txt`. No secrets or database are required — it is a stateless demo capped at 5 images per run, with requests serialized through Gradio's queue so a busy free Space degrades into a short wait rather than 429 errors. ## Why the old pipeline failed (and what this one does differently) | Problem | Old pipeline | This pipeline | |---|---|---| | Glyph style | EMNIST (American digits) — but players write **European/Kazakh** digits: crossed 7, serif 1, cursive 9 | ARDIS dataset (real European handwriting) + procedural crossbars/serifs on EMNIST glyphs | | Preprocessing | Train and inference normalized **differently** (`0.5/0.5` vs ImageNet stats) | One shared module, `togyz/preprocess.py`, imported by both | | Framing | Small digits with wide margins in fixed 80×40 cells | Random framing from tight crops to loose cells, matching real photos | | Data | 48 900 fixed PNGs on disk | Infinite on-the-fly synthesis, deterministic validation set | ## Layout ``` togyz/ core library classes.py canonical 163-class list (indices match legacy class_mapping.json) preprocess.py THE single image->tensor preprocessing (train AND inference) glyphs.py per-character glyph pools + procedural style edits synth.py cell synthesizer; python -m togyz.synth --preview preview.png dataset.py on-the-fly synthetic dataset + real-crop eval dataset model.py resnet18 (grayscale, 163 outputs), checkpoint helpers train.py training CLI (auto device: cuda/mps/cpu) eval.py synthetic val + per-file real-crop report predict.py classify images; --allowed restricts to legal moves scripts/get_glyphs.py downloads/prepares glyph pools (ARDIS, optional EMNIST) data/real_crops/ 10 labeled real crops (labels.csv) — evaluation only legacy/ previous experiments, kept for reference ``` ## Quickstart (local) ```bash source venv311/bin/activate # or: pip install -r requirements.txt python scripts/get_glyphs.py # one-time: download+prepare ARDIS glyphs python -m togyz.synth --preview preview.png # eyeball synthetic vs data/real_crops python train.py --epochs 1 --samples-per-epoch 4000 --batch-size 64 # smoke test python eval.py --ckpt checkpoints/best.pt python predict.py "data/real_crops/*.jpg" ``` A real training run needs a GPU — see Colab below. Defaults (`--epochs 20 --samples-per-epoch 50000`) are a sensible full run. ## Training on Google Colab (or any GPU machine) Open `notebooks/colab_train.ipynb` in Colab, or manually: ```bash git clone https://github.com/ansarzeinulla/9OCR.git && cd 9OCR pip install -r requirements-train.txt python scripts/get_glyphs.py --emnist # ingredients/ is gitignored, so also # rebuild the EMNIST pool from torchvision python train.py --task moves --epochs 30 python train.py --task diagram --epochs 30 python scripts/export_onnx.py # download checkpoints/best.onnx (+.classes.json) and # checkpoints/diagram/best.onnx (+.classes.json) when done ``` `predict.py`/`eval.py` run anywhere (CPU is fine) with the downloaded checkpoint. ## Inference ```bash python predict.py cell.jpg --topk 3 python predict.py cell.jpg --allowed "12,34x,56" ``` At any game state at most 9 moves are legal. If an upstream game tracker passes them via `--allowed`, probabilities are renormalized over just those moves — a large, free accuracy boost when reading whole games. ## Reading a whole scoresheet ```bash python read_game.py "data/2026-07-06 00.00.20.jpg" --out out/sheet1 --result 0-1 ``` Pass `--result` (1-0 / 0-1 / draw, from the sheet footer) when known: the beam's final pool is re-ranked to prefer reconstructions whose end state is consistent with how the game actually ended. The summary strips between the tables hold a full board diagram every 10 moves: both kazan counts (Black's box above the strip, White's below, two digits 10-81) **and** the 2×9 pit grid (upper row = Black, pits 9…1 left-to-right; lower row = White, pits 1…9; each cell is `x` = tuzdyk, `-` = 0, or a 1-2 digit count). A second, unified classifier reads all of it (`python train.py --task diagram`, saved to `checkpoints/diagram/best.pt`; classes 0-81 + `x` + `-` + `empty`, filtered per context at inference), and the beam gains likelihood when a hypothesis' computed kazans and pit counts match these written checkpoints. Finds the 8 printed move tables, classifies every cell, and writes: - `game.json` — per ply: all 163 class probabilities, top-5, legality info - `raw.pgn` — pure classifier argmax per ply, even if illegal - `legal.pgn` — strict replay, stops at the first illegal argmax - `beam.pgn` — **best reconstruction**: beam search over all legal continuations per ply; each legal move is scored by blending its exact class probability with the probability mass of its source digit (the landing digit is derived from the position, so a misread second digit doesn't discard the right source hole). Longest fully legal sequence wins, ties broken by joint probability. - `annotated.jpg`, `cells/` — visual debugging Move annotations: `+` capture, `x` tuzdyk creation. Strip `+` to feed the moves to the 9Q engine. Rules implementation: `togyz/rules.py`, validated against the 9Q C++ engine fixture (`python tests/test_rules.py`). Accuracy depends heavily on photo resolution — the CLI warns when cells are under 35 px tall; photograph sheets at full camera resolution. ## Improving accuracy further 1. **Label more real cells** — append rows to `data/real_crops/labels.csv`. Even ~50 real crops make the reported real-crop accuracy meaningful; a few hundred would allow fine-tuning on them. 2. Add glyph styles: put white-on-black PNG masks under `glyph_data///` and register the source in `togyz/glyphs.py`. 3. Full-game decoding with a Togyzkumalak rules engine (choose the most probable *legal* move per cell) — hook already exists via `--allowed`. ## Notes - `train_data5/` (the old pre-generated dataset) is no longer used and can be deleted; synthesis now happens on the fly. - Old models/scripts live in `legacy/` (`.pkl`/`.pth` files are gitignored).