| ---
|
| tags:
|
| - reinforcement-learning
|
| - stable-baselines3
|
| - sb3-contrib
|
| - gomoku
|
| - ppo
|
| library_name: stable-baselines3
|
| ---
|
|
|
| # Gomoku MaskablePPO Stage3 H6
|
|
|
| This repository contains a compact release of a Gomoku (9x9, connect-5) agent trained with `MaskablePPO` from `sb3-contrib`.
|
|
|
| ## Contents
|
|
|
| - `best_model/best_model.zip`: best checkpoint selected by evaluation callback
|
| - `best_model/evaluations.npz`: raw evaluation callback output
|
| - `gomoku_maskable_ppo_final.zip`: final checkpoint at the end of training
|
| - `gomoku_rl/`: environment, opponents, and custom CNN feature extractor required to load the model
|
| - `play.py`: local browser game against the agent
|
| - `evaluate.py`: evaluation script
|
| - `metrics_summary.json`: compact summary of tracked metrics
|
| - `upload_to_hf.py`: helper for uploading this prepared folder
|
|
|
| ## Training Setup
|
|
|
| - Board size: `9`
|
| - Win length: `5`
|
| - Algorithm: `MaskablePPO`
|
| - Policy: custom CNN (`GomokuCNN`)
|
| - Training variant: resumed from `models_stage3_h5` and continued in `models_stage3_h6`
|
|
|
| Training command used for this stage:
|
|
|
| ```bash
|
| python train.py --resume-from models_stage3_h5/best_model/best_model.zip --opponent heuristic --heuristic-search-depth 1 --heuristic-max-candidates 4 --heuristic-early-max-candidates 6 --vec-env subproc --n-envs 8 --total-timesteps 5000000 --models-dir models_stage3_h6 --log-dir logs_stage3_h6 --eval-opponent heuristic --eval-freq 500000 --eval-games 100
|
| ```
|
|
|
| ## Checkpoint Summary
|
|
|
| - Best checkpoint by evaluation callback: `13350000` timesteps
|
| - Last evaluated checkpoint: `13850000` timesteps
|
| - Best callback mean reward: `1.5249`
|
| - Last callback mean reward: `1.4882`
|
|
|
| ## Quick Local Benchmarks
|
|
|
| The following checks were run locally after packaging:
|
|
|
| ### Best checkpoint
|
|
|
| - Opponent: heuristic
|
| - Opponent config: `depth=1`, `radius=2`, `max_candidates=4`, `early_max_candidates=6`
|
| - Games: `50`
|
| - Wins / Losses / Draws: `47` / `3` / `0`
|
| - Win rate: `94%`
|
|
|
| ### Final checkpoint
|
|
|
| - Opponent: heuristic
|
| - Opponent config: `depth=1`, `radius=2`, `max_candidates=4`, `early_max_candidates=6`
|
| - Games: `50`
|
| - Wins / Losses / Draws: `38` / `10` / `2`
|
| - Win rate: `76%`
|
|
|
| The best checkpoint is stronger than the final checkpoint for this release, so `best_model/best_model.zip` is the recommended file.
|
|
|
| ## Install
|
|
|
| ```bash
|
| pip install -r requirements.txt
|
| ```
|
|
|
| ## Load The Model
|
|
|
| ```python
|
| from sb3_contrib import MaskablePPO
|
|
|
| model = MaskablePPO.load("best_model/best_model.zip")
|
| ```
|
|
|
| Because the policy uses a custom feature extractor, keep the `gomoku_rl/` package next to the model files or in your Python path.
|
|
|
| ## Evaluate
|
|
|
| ```bash
|
| python evaluate.py --model-path best_model/best_model.zip --opponent heuristic --games 100 --opponent-search-depth 1 --opponent-max-candidates 4 --opponent-early-max-candidates 6
|
| ```
|
|
|
| ## Play In Browser
|
|
|
| ```bash
|
| python play.py --model-path best_model/best_model.zip --host 127.0.0.1 --port 8000 --human-first
|
| ```
|
|
|
| ## Upload This Folder
|
|
|
| If you cloned or copied this release locally and want to publish it under your own Hugging Face account:
|
|
|
| ```bash
|
| python upload_to_hf.py
|
| ```
|
|
|
| Or specify a target repository explicitly:
|
|
|
| ```bash
|
| python upload_to_hf.py --repo-id your-name/gomoku-maskable-ppo-stage3-h6
|
| ```
|
|
|
