mellogood commited on
Commit
ad60a46
·
verified ·
1 Parent(s): 1c56cd2

docs: card follows the house-style sweep

Browse files
Files changed (1) hide show
  1. README.md +20 -22
README.md CHANGED
@@ -18,13 +18,12 @@ tags:
18
 
19
  > A [sup computer](https://www.supcpu.com) release — a small language model studio. [Model page](https://www.supcpu.com/models/daydream-chess-nanogpt-micro-1/) · [monorepo](https://github.com/romellogoodman/sup-computer) (frozen code: [`projects/daydream/models/daydream-chess-nanogpt-micro-1/`](https://github.com/romellogoodman/sup-computer/tree/main/projects/daydream/models/daydream-chess-nanogpt-micro-1), tag `daydream-chess-nanogpt-micro-1`) · runs in your browser at [www.supcpu.com/model-player](https://www.supcpu.com/model-player/).
20
 
21
-
22
  <div class="takeaways">
23
  <p class="takeaways-label">Key takeaways</p>
24
  <ul>
25
- <li>A <strong>0.79M-param</strong> char-level GPT trained entirely on <strong>synthetic self-play</strong> — no human corpus exists for 5×5 Gardner minichess, so all 4,135 training games came from two Fairy-Stockfish instances playing each other.</li>
26
  <li>Fixed-depth engine self-play is <strong>fully deterministic</strong> on its own — the first generation attempt produced identical games every time. Fixed by randomizing opening plies (sourced from the engine's own legal-move list) before search takes over.</li>
27
- <li><strong>100% clean completion, 39.2% legal-move rate</strong> on first try — slightly higher than the <a href="daydream-chess-nanogpt-1.md">Regular</a> tier's 35.3%, consistent with a smaller board being an easier legality problem to learn, though the corpora and vocab sizes differ too much to call it a controlled comparison.</li>
28
  <li>Smallest tier in the three-board <a href="../../projects/daydream/README.md">daydream</a> family — 5×5 is the smallest board that can hold one of every standard chess piece, which is why Micro uses Gardner's real, balance-tested arrangement rather than an invented one.</li>
29
  </ul>
30
  </div>
@@ -49,8 +48,8 @@ illegal moves render as dim near-misses instead of being discarded.
49
  |---|---|
50
  | **Version / git tag** | `daydream-chess-nanogpt-micro-1` (research run `micro-r1`) |
51
  | **Architecture** | modern char-level (RoPE, RMSNorm, bias-free) on the shared `core` engine |
52
- | **Size** | 4 layers · 4 heads · 128 embedding dim · 128 context · dropout 0.1 · **~0.79M params** |
53
- | **Tokenizer** | character-level, **15-char** vocabulary over UCI move text on a 5×5 board (files a–e, ranks 1–5, promotion letters n/q/r, space, newline) |
54
  | **Checkpoint** | `projects/daydream/models/daydream-chess-nanogpt-micro-1/` (weights not committed) |
55
  | **Built on** | the monorepo's shared [`core`](https://github.com/romellogoodman/sup-computer/tree/main/core) engine |
56
  | **Developed with** | Claude ([Claude Code](https://claude.com/claude-code)) |
@@ -71,22 +70,21 @@ on why tiers never share a vocabulary).
71
  ## Training data
72
 
73
  No human corpus exists for 5×5 chess, so this tier is entirely synthetic:
74
- **4,135 self-play games** between two Fairy-Stockfish instances under the
75
- engine's built-in `gardner` variant (bounded-depth search, not
76
- strength-reduced — see
77
- [ADR-0021](https://github.com/romellogoodman/sup-computer/blob/main/docs/adr/0021-daydream-fairy-stockfish-dependency.md)),
78
- with randomized opening plies for game-to-game diversity (fixed-depth
79
- search alone is fully deterministic and produced identical games on the
80
- first attempt — fixed by sourcing random legal openings from the engine's
81
- own `go perft 1` move list) plus a repetition-window cutoff for games that
82
- fell into shuffling loops. Corpus is vendored in-folder as `games.txt`
83
- (synthetic, seeded, code-owned — committed, same treatment as
84
- kenosha-kid's `raw.txt`).
85
 
86
  ## Training procedure
87
 
88
  - **Optimizer:** AdamW, LR 3e-4 with cosine decay to 3e-5, 100 warmup iters, β₂ 0.99, batch size 64.
89
- - **Run:** 2,500 iterations, best val loss **0.718**.
90
  - **Hardware:** Apple Silicon Mac (MPS / Metal backend), `torch.compile` disabled.
91
 
92
  ## Evaluation
@@ -97,11 +95,11 @@ kenosha-kid's `raw.txt`).
97
  | **Legal-move rate (first try)** | 121/309 (39.2%) |
98
 
99
  Micro's legal-move rate (39.2%) is somewhat higher than Regular's (35.3%,
100
- [`daydream-chess-nanogpt-1`](https://github.com/romellogoodman/sup-computer/blob/main/research-docs/model-cards/daydream-chess-nanogpt-1.md)) — consistent
101
- with a smaller board and smaller per-position legal-move count being an
102
- easier legality-learning problem, though the two aren't a strict
103
- apples-to-apples comparison (different corpora, different vocab sizes,
104
- different training run lengths).
105
 
106
  ## Limitations
107
 
 
18
 
19
  > A [sup computer](https://www.supcpu.com) release — a small language model studio. [Model page](https://www.supcpu.com/models/daydream-chess-nanogpt-micro-1/) · [monorepo](https://github.com/romellogoodman/sup-computer) (frozen code: [`projects/daydream/models/daydream-chess-nanogpt-micro-1/`](https://github.com/romellogoodman/sup-computer/tree/main/projects/daydream/models/daydream-chess-nanogpt-micro-1), tag `daydream-chess-nanogpt-micro-1`) · runs in your browser at [www.supcpu.com/model-player](https://www.supcpu.com/model-player/).
20
 
 
21
  <div class="takeaways">
22
  <p class="takeaways-label">Key takeaways</p>
23
  <ul>
24
+ <li>A 0.79M-param char-level GPT trained entirely on <strong>synthetic self-play</strong> — no human corpus exists for 5×5 Gardner minichess, so all 4,135 training games came from two Fairy-Stockfish instances playing each other.</li>
25
  <li>Fixed-depth engine self-play is <strong>fully deterministic</strong> on its own — the first generation attempt produced identical games every time. Fixed by randomizing opening plies (sourced from the engine's own legal-move list) before search takes over.</li>
26
+ <li>100% clean completion, 39.2% legal-move rate on first try — slightly higher than the <a href="daydream-chess-nanogpt-1.md">Regular</a> tier's 35.3%, consistent with a smaller board being an easier legality problem to learn, though the corpora and vocab sizes differ too much to call it a controlled comparison.</li>
27
  <li>Smallest tier in the three-board <a href="../../projects/daydream/README.md">daydream</a> family — 5×5 is the smallest board that can hold one of every standard chess piece, which is why Micro uses Gardner's real, balance-tested arrangement rather than an invented one.</li>
28
  </ul>
29
  </div>
 
48
  |---|---|
49
  | **Version / git tag** | `daydream-chess-nanogpt-micro-1` (research run `micro-r1`) |
50
  | **Architecture** | modern char-level (RoPE, RMSNorm, bias-free) on the shared `core` engine |
51
+ | **Size** | 4 layers · 4 heads · 128 embedding dim · 128 context · dropout 0.1 · ~0.79M params |
52
+ | **Tokenizer** | character-level, 15-char vocabulary over UCI move text on a 5×5 board (files a–e, ranks 1–5, promotion letters n/q/r, space, newline) |
53
  | **Checkpoint** | `projects/daydream/models/daydream-chess-nanogpt-micro-1/` (weights not committed) |
54
  | **Built on** | the monorepo's shared [`core`](https://github.com/romellogoodman/sup-computer/tree/main/core) engine |
55
  | **Developed with** | Claude ([Claude Code](https://claude.com/claude-code)) |
 
70
  ## Training data
71
 
72
  No human corpus exists for 5×5 chess, so this tier is entirely synthetic:
73
+ 4,135 self-play games between two Fairy-Stockfish instances under the
74
+ engine's built-in `gardner` variant — bounded-depth search, not
75
+ strength-reduced (see
76
+ [ADR-0021](https://github.com/romellogoodman/sup-computer/blob/main/docs/adr/0021-daydream-fairy-stockfish-dependency.md)).
77
+ Fixed-depth search alone is fully deterministic; the first attempt produced
78
+ identical games every time. The fix: randomized opening plies, sourcing
79
+ random legal openings from the engine's own `go perft 1` move list, plus a
80
+ repetition-window cutoff for games that fell into shuffling loops. Corpus
81
+ is vendored in-folder as `games.txt` — synthetic, seeded, code-owned,
82
+ committed, the same treatment as kenosha-kid's `raw.txt`.
 
83
 
84
  ## Training procedure
85
 
86
  - **Optimizer:** AdamW, LR 3e-4 with cosine decay to 3e-5, 100 warmup iters, β₂ 0.99, batch size 64.
87
+ - **Run:** 2,500 iterations, best val loss 0.718.
88
  - **Hardware:** Apple Silicon Mac (MPS / Metal backend), `torch.compile` disabled.
89
 
90
  ## Evaluation
 
95
  | **Legal-move rate (first try)** | 121/309 (39.2%) |
96
 
97
  Micro's legal-move rate (39.2%) is somewhat higher than Regular's (35.3%,
98
+ [`daydream-chess-nanogpt-1`](https://github.com/romellogoodman/sup-computer/blob/main/research-docs/model-cards/daydream-chess-nanogpt-1.md)). One reading: a
99
+ smaller board and smaller per-position legal-move count is an easier
100
+ legality-learning problem. But the two aren't a strict apples-to-apples
101
+ comparison — different corpora, different vocab sizes, different training
102
+ run lengths.
103
 
104
  ## Limitations
105