---
license: mit
library_name: transformers
tags:
  - chess
  - qwen3
  - causal-lm
  - fine-tuned
  - alpha
base_model: jspaulsen/halluci-mate-v2a
---

# halluci-mate-v2b

**Alpha release.** A chess LLM fine-tuned from [`jspaulsen/halluci-mate-v2a`](https://huggingface.co/jspaulsen/halluci-mate-v2a) on a higher-quality slice of the [Lichess](https://database.lichess.org/) dataset. Uses the Qwen3-0.6B architecture and a custom UCI move tokenizer. First model in the series to score wins against Stockfish skill-5 in 100-game matches.

Source: https://github.com/jspaulsen/halluci-mate

## Model details

- **Architecture**: Qwen3 (`Qwen3ForCausalLM`), ~0.6B parameters
  - 28 layers, hidden size 1024, 16 attention heads (8 KV heads), intermediate size 3072
  - `bfloat16`, tied word embeddings, RoPE θ = 1,000,000
- **Vocabulary**: 1,974 tokens — 6 special tokens (`<PAD>`, `<UNK>`, `<EOS>`, `<WHITE>`, `<BLACK>`, `<DRAW>`) + ~1,792 geometric UCI moves + 176 promotion moves
- **Context**: 32,768 tokens
- **Base model**: `jspaulsen/halluci-mate-v2a`
- **Checkpoint**: `runs-v2a-ft/languid-sloth-169/checkpoint-4056`

## Tokenizer

The tokenizer is **custom** and is not loadable via `AutoTokenizer.from_pretrained`. It is defined in [`src/halluci_mate/chess_tokenizer.py`](https://github.com/jspaulsen/halluci-mate/blob/main/src/halluci_mate/chess_tokenizer.py) in the source repo. Install the package and use `ChessTokenizer()` directly.

Inputs are conditioned on the side-to-move winning: each game is prefixed with `<WHITE>` or `<BLACK>` (or `<DRAW>`), followed by the sequence of UCI moves.

## Usage

```python
import chess
import torch
from transformers import AutoModelForCausalLM

from halluci_mate.chess_tokenizer import ChessTokenizer
from halluci_mate.game.game import Game
from halluci_mate.inference import ChessInferenceEngine

engine = ChessInferenceEngine.from_checkpoint(
    "jspaulsen/halluci-mate-v2b",
    constrained=True,   # mask logits to legal moves
    temperature=0.0,    # greedy
)

game = Game(board=chess.Board(), condition="<WHITE>")
move = engine.predict(game)
print(move.uci())
```

Constrained decoding masks the logits to the set of legal UCI moves in the current position, which eliminates illegal-move hallucinations at the cost of potentially hiding model weaknesses. Unconstrained sampling (`constrained=False`) will occasionally produce illegal tokens — this is expected for an alpha.

## Training

- Initialized from `jspaulsen/halluci-mate-v2a` weights
- 2 epochs, 4,056 optimizer steps, effective batch size 512 (per-device 128 × 2 grad-accum × 2 GPUs)
- Optimizer: paged AdamW 8-bit, peak LR 3e-5, cosine-with-min-lr schedule, warmup ratio 0.005
- bf16 + flash_attention_2, DDP across 2 GPUs, seed 4042
- Training script: `scripts/train.py` in the source repo
- Best eval loss 1.637 at step 4,000 (epoch 1.97)

## Headline evals vs v2a and v1b

Same eval configs across all three: vs-stockfish at skill-5 with `--sf-analyze`, legal-rate over 5,000 sampled positions (seed 0), high-elo perplexity over 10,768 sequences.

| Metric | v1b | v2a | **v2b** |
|---|---|---|---|
| vs-stockfish score-rate, skill-5 | 0.104 *(500g)* | 0.065 *(100g)* | **0.135** *(100g)* |
| vs-stockfish W / L / D | 7 / 403 / 90 | 0 / 87 / 13 | **3 / 76 / 21** |
| Legal-rate (5,000 sampled positions) | 99.06% | 99.00% | 99.02% |
| High-elo perplexity (10,768 seqs) | **4.92** | 5.47 | 5.15 |
| Tactical-oversight, middle phase | 21.0% | 23.4% | **20.3%** |
| Tactical-oversight, endgame | 12.4% | 12.2% | **11.9%** |
| Blunder-rate (in-game) | 6.5% | 6.5% | 6.5% |

v2b is the first model in the series to win games against Stockfish skill-5 (3W / 21D over 100 games). Middlegame and endgame tactical oversight are both best-in-class. Perplexity recovers most of the gap v2a opened up vs v1b on the high-elo test set.

## Limitations

- Alpha quality; move strength has not been benchmarked against a rated engine
- Constrained decoding is recommended for any real use — the raw model may emit illegal move tokens
- Trained on human games, so idiosyncrasies and blunders at lower ratings are reflected in behavior
- No support for analyzing positions from arbitrary FENs beyond what `Game` constructs

## License

MIT. See the source repo for details.