File size: 3,326 Bytes
3799788 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | ---
tags:
- reinforcement-learning
- stable-baselines3
- sb3-contrib
- gomoku
- ppo
library_name: stable-baselines3
---
# Gomoku MaskablePPO Stage3 H6
This repository contains a compact release of a Gomoku (9x9, connect-5) agent trained with `MaskablePPO` from `sb3-contrib`.
## Contents
- `best_model/best_model.zip`: best checkpoint selected by evaluation callback
- `best_model/evaluations.npz`: raw evaluation callback output
- `gomoku_maskable_ppo_final.zip`: final checkpoint at the end of training
- `gomoku_rl/`: environment, opponents, and custom CNN feature extractor required to load the model
- `play.py`: local browser game against the agent
- `evaluate.py`: evaluation script
- `metrics_summary.json`: compact summary of tracked metrics
- `upload_to_hf.py`: helper for uploading this prepared folder
## Training Setup
- Board size: `9`
- Win length: `5`
- Algorithm: `MaskablePPO`
- Policy: custom CNN (`GomokuCNN`)
- Training variant: resumed from `models_stage3_h5` and continued in `models_stage3_h6`
Training command used for this stage:
```bash
python train.py --resume-from models_stage3_h5/best_model/best_model.zip --opponent heuristic --heuristic-search-depth 1 --heuristic-max-candidates 4 --heuristic-early-max-candidates 6 --vec-env subproc --n-envs 8 --total-timesteps 5000000 --models-dir models_stage3_h6 --log-dir logs_stage3_h6 --eval-opponent heuristic --eval-freq 500000 --eval-games 100
```
## Checkpoint Summary
- Best checkpoint by evaluation callback: `13350000` timesteps
- Last evaluated checkpoint: `13850000` timesteps
- Best callback mean reward: `1.5249`
- Last callback mean reward: `1.4882`
## Quick Local Benchmarks
The following checks were run locally after packaging:
### Best checkpoint
- Opponent: heuristic
- Opponent config: `depth=1`, `radius=2`, `max_candidates=4`, `early_max_candidates=6`
- Games: `50`
- Wins / Losses / Draws: `47` / `3` / `0`
- Win rate: `94%`
### Final checkpoint
- Opponent: heuristic
- Opponent config: `depth=1`, `radius=2`, `max_candidates=4`, `early_max_candidates=6`
- Games: `50`
- Wins / Losses / Draws: `38` / `10` / `2`
- Win rate: `76%`
The best checkpoint is stronger than the final checkpoint for this release, so `best_model/best_model.zip` is the recommended file.
## Install
```bash
pip install -r requirements.txt
```
## Load The Model
```python
from sb3_contrib import MaskablePPO
model = MaskablePPO.load("best_model/best_model.zip")
```
Because the policy uses a custom feature extractor, keep the `gomoku_rl/` package next to the model files or in your Python path.
## Evaluate
```bash
python evaluate.py --model-path best_model/best_model.zip --opponent heuristic --games 100 --opponent-search-depth 1 --opponent-max-candidates 4 --opponent-early-max-candidates 6
```
## Play In Browser
```bash
python play.py --model-path best_model/best_model.zip --host 127.0.0.1 --port 8000 --human-first
```
## Upload This Folder
If you cloned or copied this release locally and want to publish it under your own Hugging Face account:
```bash
python upload_to_hf.py
```
Or specify a target repository explicitly:
```bash
python upload_to_hf.py --repo-id your-name/gomoku-maskable-ppo-stage3-h6
```
|