---
license: mit
language:
- en
pipeline_tag: reinforcement-learning
library_name: pytorch
tags:
- reinforcement-learning
- board-game
- mancala
- bestemshe
- resmlp
- value-head
- policy-head
- game-ai
pretty_name: Bestemshe ResMLP — Value & Policy Network Distilled from an Exact Oracle
datasets:
- ansarzeinulla/bestemshe-tablebase
metrics:
- accuracy
model-index:
- name: Bestemshe-ResMLP-395k
  results:
  - task:
      type: reinforcement-learning
      name: Play against the exact oracle (3-ply search, 2,000 games)
    dataset:
      type: ansarzeinulla/bestemshe-tablebase
      name: Bestemshe tablebase (1,000 symmetric start positions)
    metrics:
    - type: accuracy
      name: Value-preserving move rate
      value: 99.69
---

# Bestemshe ResMLP — value and policy network distilled from an exact oracle

**Authors:** Ansar Zeinulla & Murat Manassov · Nazarbayev University
**Oracle (tablebase):** [ansarzeinulla/bestemshe-tablebase](https://huggingface.co/datasets/ansarzeinulla/bestemshe-tablebase) (8.3 GiB, 1.26×10¹¹ positions)
**Code:** [github.com/ansarzeinulla/Bestemshe](https://github.com/ansarzeinulla/Bestemshe)

A dual-headed residual MLP trained by supervised distillation on positions labelled by the exact Bestemshe tablebase. It predicts, for the side to move:
- **Value:** loss / draw / win.
- **Policy:** logits over the 5 own pits; a move is a target if it preserves the game-theoretic value.

## Files

| File | Content | Size |
|---|---|---|
| `model.safetensors` + `config.json` | weights only (recommended) | 69 MB |
| `latest.pt` | full training checkpoint: weights, optimizer state, args, step | 207 MB |

Both hold the same weights: outputs are identical.

## Architecture

- **Input:** 10 pits (0–50 stones) and 2 kazans (stored as kazan/2, 0–12) through categorical embeddings of width 32.
- **Trunk:** linear 384 → 1024, then 8 residual blocks (LayerNorm → Linear → GELU → Linear), width 1024.
- **Heads:** value `1024 → 3`, policy `1024 → 5`.
- **Parameters:** 17,214,472.

## Training

- 394,852 optimizer steps at batch 32,768, i.e. 12.9 billion training samples.
- Samples came from shards of oracle-labelled positions, regenerated throughout training (continual distillation).
- Loss: cross-entropy on the value head, binary cross-entropy on the policy head.

## Results

### Against the oracle (2,000 games)

- **1,000 symmetric start positions** (equal kazans, equal stones), each played twice with colours swapped.
- **The model moves by a 3-ply minimax search over its value head**; the oracle plays a value-preserving move.
- Games are stopped at 400 plies and scored as draws. The oracle stores win/draw/loss only, so neither side is pushed to convert a win.

| Metric | Value |
|---|---|
| Value-preserving moves | 99.69% (226,232 of 226,933; 701 value-losing moves) |
| Games ending below the theoretical result | 556 of 2,000 (27.8%) = **556 of the 1,036 games the model started won or drawn (53.7%)** |
| Average model moves per game | 113 |
| Random-move baseline (other starts, 100 games) | 97.48% value-preserving moves |

Deeper search helps:
- 5-ply minimax: 21 of 51 degradable games below theory;
- 5-ply with an averaged backup: 20 of 206.

A same-start comparison of all move-selection methods is being run with the updated [`evaluation/vs_god.py`](https://github.com/ansarzeinulla/Bestemshe/blob/main/evaluation/vs_god.py).

### Value head (4,000 symmetric positions)

| Metric | Value |
|---|---|
| Accuracy (argmax WDL) | 85.7% |
| Expected calibration error (15 bins) | 0.0098 |
| Mean confidence when wrong / right | 0.673 / 0.891 |
| Draw recall | **0** (146 true draws; draw is never the argmax) |

A telling example: on the **initial position** the value head gives the first player **win 0.89 / draw 0.05 / loss 0.06**. The tablebase says it is a **loss** (the second player wins).

Errors concentrate where a position's latent neighbours carry other labels: 0% error in the most homogeneous 30% of positions, 80% in the most mixed decile. Script: [`analysis/latent_calibration.py`](https://github.com/ansarzeinulla/Bestemshe/blob/main/analysis/latent_calibration.py).

### During training

A cheap greedy 1-ply check on uniformly sampled positions, run at each checkpoint, rose to 97.0% value-preserving moves and 92.9% value accuracy at the final step. Full history: the `eval_results_*.json` files of the training run.

## Usage

```bash
pip install torch safetensors huggingface_hub
```

```python
import json, torch, torch.nn as nn, torch.nn.functional as F
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

class ResBlock(nn.Module):
    def __init__(self, w):
        super().__init__()
        self.norm, self.fc1, self.fc2 = nn.LayerNorm(w), nn.Linear(w, w), nn.Linear(w, w)
    def forward(self, x):
        return x + self.fc2(F.gelu(self.fc1(self.norm(x))))

class BestemsheNet(nn.Module):
    def __init__(self, width=1024, blocks=8, emb=32):
        super().__init__()
        self.pit_emb, self.kaz_emb = nn.Embedding(51, emb), nn.Embedding(13, emb)
        self.inp = nn.Linear(12 * emb, width)
        self.body = nn.Sequential(*[ResBlock(width) for _ in range(blocks)])
        self.value_head, self.policy_head = nn.Linear(width, 3), nn.Linear(width, 5)
    def forward(self, pits, kaz):
        x = torch.cat([self.pit_emb(pits).flatten(1), self.kaz_emb(kaz).flatten(1)], dim=1)
        h = self.body(F.gelu(self.inp(x)))
        return self.value_head(h), self.policy_head(h)

repo = "ansarzeinulla/bestemshe-resmlp"
cfg = json.load(open(hf_hub_download(repo, "config.json")))
model = BestemsheNet(cfg["width"], cfg["blocks"]).eval()
model.load_state_dict(load_file(hf_hub_download(repo, "model.safetensors")))

# side to move: pits[0:5] own, pits[5:10] opponent; kazans are even, passed as kazan // 2
pits = torch.tensor([[5] * 10]); kaz = torch.tensor([[0, 0]])
with torch.no_grad():
    value, policy = model(pits, kaz)
print(dict(zip(["loss", "draw", "win"], torch.softmax(value, 1)[0].tolist())))
```

## Citation

```bibtex
@misc{zeinulla2026bestemshe_model,
  title        = {Bestemshe ResMLP: Value and Policy Distillation from an Exact Oracle},
  author       = {Zeinulla, Ansar and Manassov, Murat},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ansarzeinulla/bestemshe-resmlp}}
}
```