burrowdweller commited on
Commit
e85e149
Β·
verified Β·
1 Parent(s): 69c9fb2

Add a public model card.

Browse files
Files changed (1) hide show
  1. README.md +14 -128
README.md CHANGED
@@ -3,144 +3,30 @@ library_name: onnx
3
  tags:
4
  - chess
5
  - chess-engine
6
- - transformer
7
- - contempt-conditioning
8
- - mcts
9
- language:
10
- - en
11
- pipeline_tag: feature-extraction
12
  ---
13
 
14
  # mini-chessformer-v1
15
 
16
- A 7.57M-parameter **Chessformer-lite** transformer for chess: 64 square tokens + Geometric Attention Bias (GAB) + from–to policy + contempt-conditioned WDL value, trained from scratch on `data/train/data.parquet` (Jan–Mar 2026 Lichess).
17
 
18
- **Goal:** match `burrowdweller/minichess-gpt-v1-final` under matched MCTS at ≀50% train FLOPs. **Composite play + cost gate: PASS.**
19
 
20
- ## Results (c=0, search-contempt off)
21
 
22
- | Gate | Value | Threshold | |
23
- |---|---:|---:|---|
24
- | Paired match score vs v1-final | **0.700** (W11/D6/L3) | β‰₯0.40 | βœ… |
25
- | Holdout MCTS mean CPL | **306.7** | 382.0 (parent+25) | βœ… (lower=better) |
26
- | Train FLOPs | **4.665e17** | 4.665e17 (0.50Γ—v1) | βœ… |
27
- | Wall (RTX 4070) | **9.80 h** | 10.5 h | βœ… |
28
- | Contempt conditioning shift (c=0β†’0.5) | Ξ”draw=βˆ’0.10, Ξ”decisive=+0.10 | Ξ΅=0.05 | βœ… |
29
 
30
- T3 contempt conditioning is alive: raising `contempt` from 0 to 0.5 measurably reduces draws and increases decisive outcomes in the predicted direction, without collapse.
31
 
32
- ## Architecture
 
 
33
 
34
- ```python
35
- ChessformerLiteConfig(
36
- d_model=256, n_layers=6, n_heads=8, d_ff=384, # narrow FFN (~1.5x)
37
- use_gab=True,
38
- gab_compress_dim=32, gab_code_dim=256, gab_n_templates=32,
39
- contempt_hidden=64,
40
- )
41
- # 7,566,471 params
42
- ```
43
 
44
- - **Encoder:** 64 square tokens (piece id embedding + learned square PE) + 1 CLS token (state projection + contempt embedding). 6 pre-norm transformer blocks.
45
- - **GAB:** Geometric Attention Bias β€” per-layer attention bias generated from square features via a templates-and-coefficients mechanism (Smolgen-style), added to attention logits.
46
- - **Policy head:** factored from–to (64Γ—64 = 4096) + 176 promotion slots β†’ dense `MOVE_SPACE = 4272` logits. Order matches `engine.interfaces`.
47
- - **Value head:** FiLM-conditioned WDL (3-class: win/draw/loss, mover POV). Contempt embedding modulates the CLS via `gamma * cls + beta`.
48
 
49
- ## Inputs (ONNX)
50
 
51
- | Name | dtype | Shape | Notes |
52
- |---|---|---|---|
53
- | `square_ids` | int64 | `[B, 64]` | piece ids 0–12 (empty=0, white P..K=1..6, black P..K=7..12) |
54
- | `state_features` | float32 | `[B, 8]` | `[stm, WK, WQ, BK, BQ, ep_file/7 or -1, halfmove_bucket, repetition]` |
55
- | `contempt` | float32 | `[B]` | scalar per batch; **0.0** for tournament/production play |
56
-
57
- ## Outputs
58
-
59
- | Name | dtype | Shape | Notes |
60
- |---|---|---|---|
61
- | `policy` | float32 | `[B, 4272]` | raw logits (MOVE_SPACE); apply `legal_mask` + softmax |
62
- | `wdl` | float32 | `[B, 3]` | raw logits (win/draw/loss, mover POV); softmax to get probs |
63
-
64
- ## ONNX parity
65
-
66
- Validated against the PyTorch forward at both c=0.0 and c=0.5:
67
-
68
- | Sweep | max |policy err| | max |wdl err| |
69
- |---|---:|---:|
70
- | c=0.0 | 1.18e-5 | 2.86e-6 |
71
- | c=0.5 | 1.26e-5 | 2.44e-6 |
72
-
73
- Both under 1e-3 β€” ONNX graph preserves T3 conditioning exactly.
74
-
75
- ## Training
76
-
77
- - **Data:** `/mnt/c/Users/jun/chessdb/data/train/data.parquet` (streaming, multi-epoch)
78
- - **Steps:** 617,523 Β· **Batch:** 256 Β· **Seq len:** 65 (1 CLS + 64 squares)
79
- - **Warmup:** 61,752 (10%) Β· **LR:** 3e-4 β†’ cosine β†’ 0 Β· **WD:** 0.01
80
- - **Seed:** 0 Β· **Device:** CUDA (RTX 4070)
81
- - **Final loss:** β‰ˆ1.98 (policy β‰ˆ1.17, value β‰ˆ0.81)
82
-
83
- FLOP/cost ledger: `runs/chessformer_lite/s2/flop_ledger.jsonl` (separate from production lineage).
84
-
85
- ## Files
86
-
87
- | File | Size | Description |
88
- |---|---:|---|
89
- | `mini-chessformer-v1.onnx` | 30.4 MB | ONNX artifact (opset 17, dynamic batch) |
90
- | `mini-chessformer-v1.pt` | 91.0 MB | PyTorch checkpoint (trainer state: model + optimizer + scheduler + step) |
91
- | `mini-chessformer-v1.json` | 1.1 KB | Export metadata (hashes, config, I/O shapes, parity) |
92
- | `REPORT.md` | β€” | S2 scale + play eval report (this card's source) |
93
- | `inference.py` | β€” | Standalone ONNX inference + board encoding + 4272-move tables + legal mask. No repo imports. |
94
- | `browser/manifest.json` | 445 B | Exact `chess-gpt-package-v1` package manifest |
95
- | `browser/entry.js` | 133 KB | Self-contained browser arena entrypoint: chess.js + MCTS + Chessformer-lite adapter |
96
- | `browser/model_final.onnx` | 30.4 MB | Browser package copy of the ONNX artifact |
97
-
98
- ## Usage (standalone)
99
-
100
- This repository ships `inference.py` β€” a **self-contained** ONNX inference module
101
- that carries its own copy of the board encoding, the 4272-move table, and the
102
- legal-move mask. It depends only on `chess`, `numpy`, and `onnxruntime`. You do
103
- not need to clone the chessdb repo or import `engine.interfaces` /
104
- `experiments.chessformer_lite.encode`.
105
-
106
- ### CLI smoke (startpos / midgame / promotion, dynamic batch)
107
-
108
- ```bash
109
- python inference.py --model mini-chessformer-v1.onnx
110
- # pass --contempt 0.5 to exercise T3 conditioning
111
- # pass --fens "<fen1>" "<fen2>" ... for custom positions
112
- ```
113
-
114
- ### Python API
115
-
116
- ```python
117
- import chess
118
- from inference import ChessformerLiteONNX
119
-
120
- eng = ChessformerLiteONNX("mini-chessformer-v1.onnx")
121
-
122
- # single-board (returns raw logits + best legal move)
123
- policy_logits, wdl_logits, best_move = eng.evaluate(board, contempt=0.0)
124
-
125
- # batched (returns raw logits, Nx4272 and Nx3)
126
- policy, wdl = eng.evaluate_batch([board1, board2, board3], contempt=0.0)
127
- ```
128
-
129
- Inputs:
130
- - `square_ids` int64 `[B, 64]` β€” piece ids 0–12 (empty=0, white P..K=1..6, black P..K=7..12)
131
- - `state_features` float32 `[B, 8]` β€” `[stm, WK, WQ, BK, BQ, ep_file/7 or -1, halfmove_bucket, repetition]`
132
- - `contempt` float32 `[B]` β€” scalar per batch; **0.0** for tournament/production
133
-
134
- Outputs:
135
- - `policy` float32 `[B, 4272]` β€” raw logits; apply `legal_mask` + softmax before MCTS
136
- - `wdl` float32 `[B, 3]` β€” raw logits (win/draw/loss, mover POV); softmax to get probs
137
-
138
-
139
- ## Citation / Provenance
140
- Trained on the `feature/chessformer-lite` branch of `junisbuilding/chessdb` (commit `3ac1ed4`). Parent baseline: `burrowdweller/minichess-gpt-v1-final`.
141
-
142
- ## Caveats / Known follow-ups
143
-
144
- - Match sample is 20 games (point estimate +147 Elo, 95% CI spans 0). Directional, not tight.
145
- - Contempt on the CLS token leaks into policy via attention β€” slight confound with the "policy independent of c in v1" design. Worth detaching c from the policy path in a follow-up.
146
- - FiLM Ξ³ initializes near 0, slowing early value conditioning. γ←1 identity init recommended.
 
3
  tags:
4
  - chess
5
  - chess-engine
6
+ - onnx
7
+ pipeline_tag: other
 
 
 
 
8
  ---
9
 
10
  # mini-chessformer-v1
11
 
12
+ A Chessformer chess engine for the browser: it reads the 64 squares, searches with MCTS, and plays one legal move.
13
 
14
+ The original Chessformer-lite release, about 7.6 million parameters.
15
 
16
+ ## What's in this repo
17
 
18
+ This is a browser chess engine package (`chess-gpt-package-v1`), not a Transformers checkpoint.
 
 
 
 
 
 
19
 
20
+ A compatible runner loads:
21
 
22
+ - `browser/manifest.json` β€” file list and hashes
23
+ - `browser/entry.js` β€” search and move picker
24
+ - the ONNX net named in that manifest
25
 
26
+ ## Use
 
 
 
 
 
 
 
 
27
 
28
+ Point a chess-gpt-compatible runner at `burrowdweller/mini-chessformer-v1`. It calls `loadPackage` β†’ `newGame` β†’ `chooseMove({ history, legalMoves })` and must return one of those legal moves.
 
 
 
29
 
30
+ The net is small on purpose. Strength comes from searching at move time.
31
 
32
+ Source: [junisbuilding/chessdb](https://github.com/junisbuilding/chessdb)