exp02c_ppo_2x4_long: checkpoint, config, evaluation files, model card

README.md

@@ -0,0 +1,150 @@
+---
+license: mit
+library_name: stable-baselines3
+pipeline_tag: reinforcement-learning
+tags:
+- 4d-snake
+- 4d-snake-2x4
+- snake
+- reinforcement-learning
+- deep-reinforcement-learning
+- stable-baselines3
+- sb3-contrib
+- maskable-ppo
+model-index:
+- name: 4d-snake-exp02c-ppo-2x4-long
+  results:
+  - task:
+      type: reinforcement-learning
+      name: reinforcement-learning
+    dataset:
+      name: 4d-snake-2x4
+      type: 4d-snake-2x4
+    metrics:
+    - type: completion_rate
+      name: completion rate (deterministic, 100 episodes x 3 seeds)
+      value: 0.933 +/- 0.005
+      verified: false
+    - type: mean_reward
+      name: mean episode return (deterministic)
+      value: 24.09
+      verified: false
+    - type: fill
+      name: mean final fill (deterministic)
+      value: 0.991
+      verified: false
+    - type: steps_to_complete
+      name: mean steps to complete (deterministic, won episodes)
+      value: 35.7
+      verified: false
+---
+
+# 4d-snake-exp02c-ppo-2x4-long
+
+A `512x512` MLP policy for **4-dimensional snake on the 2^4 board** (16 cells, 8 moves): MaskablePPO trained from scratch, no curriculum, 20,000,000 environment steps. From a length-1 start it completes the board in 93.3 % of deterministic episodes, evaluated with the protocol of [docs/evaluation.md](https://github.com/BurnyCoder/4d-snake-rl/blob/main/docs/evaluation.md) (100 episodes x 3 seeds, masked `evaluate_policy`).
+
+## Results (`eval/summary.json`)
+
+| mode | completion +- std | mean fill | steps to complete | won within 4C |
+|---|---|---|---|---|
+| deterministic (argmax) | 0.933 +- 0.005 | 0.991 | 35.7 | 0.933 |
+| sampling | 0.863 +- 0.025 | 0.983 | 36.0 | 0.863 |
+
+## How to use
+
+The observation is this repository's `4*C + 2` float vector and the action space its `2*ndim` masked moves ([docs/game_rules.md](https://github.com/BurnyCoder/4d-snake-rl/blob/main/docs/game_rules.md)), so the checkpoint runs inside `snake4d`'s environment:
+
+```bash
+git clone https://github.com/BurnyCoder/4d-snake-rl.git && cd 4d-snake-rl && uv sync
+hf download BurnyCoder/4d-snake-exp02c-ppo-2x4-long best_model.zip --local-dir weights
+uv run snake4d evaluate --set model_path=weights/best_model.zip --set size=2 --set ndim=4
+```
+
+```python
+# https://sb3-contrib.readthedocs.io/en/master/modules/ppo_mask.html
+from sb3_contrib import MaskablePPO
+from sb3_contrib.common.maskable.utils import get_action_masks
+from snake4d.config import Config
+from snake4d.vec_env import make_env
+
+cfg = Config(size=2, ndim=4)
+model = MaskablePPO.load("weights/best_model.zip", device="cpu")
+env = make_env(cfg, 1, 0)  # one board; observation shape (1, 4*C + 2)
+obs = env.reset()
+masks = get_action_masks(env)  # the legal moves, one row per board
+action, _ = model.predict(obs, action_masks=masks, deterministic=True)
+```
+
+## Training
+
+- Phase `train`; experiment file `experiments/exp02c_ppo_2x4_long.env`; write-up: https://github.com/BurnyCoder/4d-snake-rl/blob/main/reports/experiments/exp02_ppo_2x4.md.
+- Resolved configuration (`config.json`):
+
+```json
+{
+  "size": 2,
+  "ndim": 4,
+  "idle_mult": 4,
+  "r_food": 1.0,
+  "r_death": -1.0,
+  "r_win": 10.0,
+  "r_step": -0.001,
+  "shaping_coef": 0.0,
+  "n_envs": 1024,
+  "total_timesteps": 20000000,
+  "n_steps": 32,
+  "batch_size": 4096,
+  "n_epochs": 4,
+  "gamma": 0.99,
+  "gae_lambda": 0.95,
+  "lr_start": 0.0003,
+  "lr_end": 1e-05,
+  "clip_start": 0.2,
+  "clip_end": 0.05,
+  "ent_coef": 0.01,
+  "vf_coef": 0.5,
+  "max_grad_norm": 0.5,
+  "target_kl": 0.03,
+  "net_width": 512,
+  "device": "auto",
+  "torch_threads": 8,
+  "seed": 0,
+  "curriculum": 0,
+  "curriculum_window": 8,
+  "curriculum_delta": 0,
+  "curriculum_rho": 0.2,
+  "curriculum_min_eps": 200,
+  "p_true_start": 0.2,
+  "eval_episodes": 100,
+  "eval_every": 655360,
+  "ckpt_every": 5242880,
+  "eval_seeds": "0,1,2",
+  "bench_steps": 200000,
+  "runs_dir": "runs",
+  "run_name": "exp02c_ppo_2x4_long",
+  "model_path": "",
+  "policy": "route"
+}
+```
+
+![exp02c_ppo_2x4_long_curves.png](figures/exp02c_ppo_2x4_long_curves.png)
+
+![exp02c_ppo_2x4_long_fill_hist.png](figures/exp02c_ppo_2x4_long_fill_hist.png)
+
+## Provenance
+
+- Code: https://github.com/BurnyCoder/4d-snake-rl at commit `09536a78f09add329dfb0f8b018736de1a69bee3`.
+- Library versions (`versions.json`): torch 2.14.0+cu130, gymnasium 1.3.0, stable-baselines3 2.9.0, sb3-contrib 2.9.0, numpy 2.5.2, pygame-ce 2.5.8, cuda_device NVIDIA GeForce RTX 5070 Laptop GPU.
+- `eval/summary.json` and `eval/eval_episodes.csv` are the files the repository's reports quote; every evaluated network is compared in [reports/networks.md](https://github.com/BurnyCoder/4d-snake-rl/blob/main/reports/networks.md).
+- Collection: https://huggingface.co/collections/BurnyCoder/4d-snake-rl-all-evaluated-networks-6a9d0a0a66c7efcd101b7741
+
+## Files
+
+- `best_model.zip`: the evaluated checkpoint in Stable-Baselines3's save format (policy weights and optimizer state, https://stable-baselines3.readthedocs.io/en/master/guide/save_format.html).
+- `config.json`, `versions.json`: the run's resolved configuration and environment.
+- `eval/`: evaluation summary and one row per evaluation episode.
+- `train/progress.csv`: the SB3 training log; `figures/`: the learning curves.
+
+## Licence
+
+MIT, like the repository.

best_model.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bc258bc33a50f1da2601ee9f2a02d371b982c4bcc9c86bf9a7367f119d1f87a
+size 7582974

config.json

@@ -0,0 +1,44 @@
+{
+  "size": 2,
+  "ndim": 4,
+  "idle_mult": 4,
+  "r_food": 1.0,
+  "r_death": -1.0,
+  "r_win": 10.0,
+  "r_step": -0.001,
+  "shaping_coef": 0.0,
+  "n_envs": 1024,
+  "total_timesteps": 20000000,
+  "n_steps": 32,
+  "batch_size": 4096,
+  "n_epochs": 4,
+  "gamma": 0.99,
+  "gae_lambda": 0.95,
+  "lr_start": 0.0003,
+  "lr_end": 1e-05,
+  "clip_start": 0.2,
+  "clip_end": 0.05,
+  "ent_coef": 0.01,
+  "vf_coef": 0.5,
+  "max_grad_norm": 0.5,
+  "target_kl": 0.03,
+  "net_width": 512,
+  "device": "auto",
+  "torch_threads": 8,
+  "seed": 0,
+  "curriculum": 0,
+  "curriculum_window": 8,
+  "curriculum_delta": 0,
+  "curriculum_rho": 0.2,
+  "curriculum_min_eps": 200,
+  "p_true_start": 0.2,
+  "eval_episodes": 100,
+  "eval_every": 655360,
+  "ckpt_every": 5242880,
+  "eval_seeds": "0,1,2",
+  "bench_steps": 200000,
+  "runs_dir": "runs",
+  "run_name": "exp02c_ppo_2x4_long",
+  "model_path": "",
+  "policy": "route"
+}

eval/eval_episodes.csv

@@ -0,0 +1,601 @@
+seed,deterministic,success,fill,length,return
+0,1,0,0.8125,23,10.977998733520508
+0,1,1,1.0,25,24.975997924804688
+0,1,0,0.6875,26,8.97499942779541
+0,1,1,1.0,28,24.97299575805664
+0,1,1,1.0,28,24.97299575805664
+0,1,1,1.0,28,24.972997665405273
+0,1,1,1.0,28,24.972999572753906
+0,1,1,1.0,28,24.972997665405273
+0,1,1,1.0,28,24.97299575805664
+0,1,1,1.0,29,24.971996307373047
+0,1,1,1.0,29,24.971996307373047
+0,1,1,1.0,29,24.97199821472168
+0,1,1,1.0,29,24.971996307373047
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,30,24.970996856689453
+0,1,0,0.9375,30,12.970995903015137
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,30,24.970998764038086
+0,1,1,1.0,30,24.970996856689453
+0,1,1,1.0,31,24.96999740600586
+0,1,1,1.0,31,24.969995498657227
+0,1,1,1.0,31,24.96999740600586
+0,1,1,1.0,31,24.96999740600586
+0,1,1,1.0,31,24.96999740600586
+0,1,1,1.0,31,24.96999740600586
+0,1,0,0.75,31,9.96999740600586
+0,1,1,1.0,32,24.968997955322266
+0,1,1,1.0,32,24.968997955322266
+0,1,1,1.0,32,24.968997955322266
+0,1,1,1.0,32,24.968997955322266
+0,1,1,1.0,32,24.968994140625
+0,1,1,1.0,32,24.968997955322266
+0,1,1,1.0,32,24.968997955322266
+0,1,1,1.0,33,24.967994689941406
+0,1,1,1.0,33,24.967998504638672
+0,1,1,1.0,34,24.966997146606445
+0,1,1,1.0,34,24.966997146606445
+0,1,1,1.0,34,24.966995239257812
+0,1,1,1.0,34,24.966997146606445
+0,1,1,1.0,34,24.966995239257812
+0,1,1,1.0,34,24.966995239257812
+0,1,1,1.0,34,24.966995239257812
+0,1,1,1.0,35,24.96599578857422
+0,1,1,1.0,35,24.96599578857422
+0,1,1,1.0,35,24.96599578857422
+0,1,1,1.0,35,24.96599578857422
+0,1,1,1.0,35,24.965999603271484
+0,1,1,1.0,35,24.96599578857422
+0,1,1,1.0,35,24.96599769592285
+0,1,1,1.0,36,24.964996337890625
+0,1,1,1.0,36,24.964996337890625
+0,1,1,1.0,36,24.964996337890625
+0,1,1,1.0,36,24.964996337890625
+0,1,1,1.0,36,24.964994430541992
+0,1,1,1.0,36,24.964996337890625
+0,1,1,1.0,36,24.964994430541992
+0,1,1,1.0,36,24.964996337890625
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,0,0.8125,37,10.963997840881348
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,37,24.96399688720703
+0,1,1,1.0,38,24.962997436523438
+0,1,1,1.0,38,24.962995529174805
+0,1,1,1.0,38,24.962997436523438
+0,1,1,1.0,38,24.962997436523438
+0,1,1,1.0,38,24.962997436523438
+0,1,1,1.0,38,24.962997436523438
+0,1,1,1.0,38,24.962995529174805
+0,1,0,0.9375,38,12.962997436523438
+0,1,1,1.0,39,24.961994171142578
+0,1,1,1.0,39,24.96199607849121
+0,1,1,1.0,39,24.961997985839844
+0,1,1,1.0,39,24.96199607849121
+0,1,1,1.0,40,24.960994720458984
+0,1,1,1.0,40,24.960994720458984
+0,1,1,1.0,40,24.960994720458984
+0,1,1,1.0,41,24.959997177124023
+0,1,1,1.0,41,24.95999526977539
+0,1,1,1.0,42,24.958999633789062
+0,1,1,1.0,42,24.958995819091797
+0,1,1,1.0,43,24.957996368408203
+0,1,1,1.0,43,24.957996368408203
+0,1,1,1.0,43,24.957996368408203
+0,1,1,1.0,44,24.956993103027344
+0,1,1,1.0,45,24.955995559692383
+0,1,1,1.0,45,24.95599365234375
+0,1,1,1.0,46,24.954994201660156
+0,1,1,1.0,46,24.954994201660156
+0,1,1,1.0,49,24.951995849609375
+0,1,1,1.0,51,24.949993133544922
+1,1,0,0.8125,24,10.976998329162598
+1,1,1,1.0,25,24.975997924804688
+1,1,0,0.6875,26,8.97499942779541
+1,1,1,1.0,27,24.973995208740234
+1,1,1,1.0,27,24.973997116088867
+1,1,1,1.0,27,24.9739990234375
+1,1,1,1.0,28,24.972997665405273
+1,1,0,0.9375,29,12.971997261047363
+1,1,1,1.0,29,24.971996307373047
+1,1,1,1.0,30,24.970996856689453
+1,1,1,1.0,30,24.970996856689453
+1,1,1,1.0,30,24.970996856689453
+1,1,1,1.0,30,24.970996856689453
+1,1,1,1.0,31,24.96999740600586
+1,1,1,1.0,31,24.969993591308594
+1,1,1,1.0,31,24.96999740600586
+1,1,1,1.0,32,24.968996047973633
+1,1,1,1.0,32,24.968996047973633
+1,1,1,1.0,32,24.968997955322266
+1,1,0,0.75,32,9.96899700164795
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,33,24.96799659729004
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,33,24.967998504638672
+1,1,1,1.0,33,24.967998504638672
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,33,24.967998504638672
+1,1,1,1.0,33,24.967994689941406
+1,1,1,1.0,34,24.966997146606445
+1,1,1,1.0,34,24.966995239257812
+1,1,1,1.0,34,24.966995239257812
+1,1,1,1.0,34,24.966995239257812
+1,1,1,1.0,34,24.966995239257812
+1,1,1,1.0,34,24.966997146606445
+1,1,1,1.0,34,24.966997146606445
+1,1,1,1.0,34,24.966997146606445
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.965993881225586
+1,1,1,1.0,35,24.96599578857422
+1,1,0,0.9375,35,12.965994834899902
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.965993881225586
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,35,24.96599578857422
+1,1,1,1.0,36,24.964996337890625
+1,1,0,0.9375,36,12.964997291564941
+1,1,1,1.0,36,24.964996337890625
+1,1,1,1.0,36,24.964996337890625
+1,1,1,1.0,36,24.964998245239258
+1,1,1,1.0,36,24.964996337890625
+1,1,1,1.0,36,24.964996337890625
+1,1,1,1.0,36,24.964996337890625
+1,1,1,1.0,37,24.9639949798584
+1,1,1,1.0,37,24.96399688720703
+1,1,1,1.0,37,24.96399688720703
+1,1,1,1.0,37,24.963993072509766
+1,1,1,1.0,37,24.9639949798584
+1,1,1,1.0,37,24.96399688720703
+1,1,1,1.0,37,24.96399688720703
+1,1,1,1.0,38,24.962995529174805
+1,1,1,1.0,38,24.962993621826172
+1,1,1,1.0,38,24.962995529174805
+1,1,1,1.0,38,24.962993621826172
+1,1,1,1.0,38,24.962995529174805
+1,1,1,1.0,38,24.962997436523438
+1,1,1,1.0,38,24.962997436523438
+1,1,1,1.0,38,24.962997436523438
+1,1,1,1.0,39,24.961994171142578
+1,1,1,1.0,39,24.961997985839844
+1,1,1,1.0,39,24.961994171142578
+1,1,0,0.9375,39,12.961995124816895
+1,1,1,1.0,39,24.96199607849121
+1,1,1,1.0,39,24.961994171142578
+1,1,1,1.0,39,24.961994171142578
+1,1,1,1.0,40,24.960996627807617
+1,1,1,1.0,40,24.96099853515625
+1,1,1,1.0,40,24.960996627807617
+1,1,1,1.0,40,24.960996627807617
+1,1,1,1.0,40,24.960994720458984
+1,1,1,1.0,40,24.960994720458984
+1,1,1,1.0,41,24.95999526977539
+1,1,1,1.0,41,24.959997177124023
+1,1,1,1.0,42,24.958995819091797
+1,1,1,1.0,42,24.958995819091797
+1,1,1,1.0,43,24.957998275756836
+1,1,1,1.0,43,24.95799446105957
+1,1,1,1.0,43,24.95799446105957
+1,1,1,1.0,43,24.95799446105957
+1,1,1,1.0,44,24.956993103027344
+1,1,1,1.0,46,24.954994201660156
+1,1,1,1.0,47,24.953994750976562
+1,1,1,1.0,47,24.953994750976562
+1,1,1,1.0,49,24.951995849609375
+2,1,1,1.0,22,24.97899627685547
+2,1,1,1.0,25,24.975996017456055
+2,1,1,1.0,27,24.9739990234375
+2,1,1,1.0,27,24.973997116088867
+2,1,1,1.0,27,24.973995208740234
+2,1,1,1.0,27,24.973995208740234
+2,1,1,1.0,29,24.971996307373047
+2,1,1,1.0,29,24.971996307373047
+2,1,0,0.875,29,11.971996307373047
+2,1,0,0.9375,30,12.97099781036377
+2,1,1,1.0,30,24.970996856689453
+2,1,0,0.8125,30,10.970996856689453
+2,1,1,1.0,30,24.970996856689453
+2,1,1,1.0,31,24.96999740600586
+2,1,0,0.9375,31,12.969995498657227
+2,1,1,1.0,31,24.969993591308594
+2,1,1,1.0,31,24.969995498657227
+2,1,0,0.9375,31,12.96999740600586
+2,1,1,1.0,31,24.96999740600586
+2,1,1,1.0,32,24.968997955322266
+2,1,1,1.0,32,24.968997955322266
+2,1,1,1.0,32,24.968997955322266
+2,1,1,1.0,32,24.968997955322266
+2,1,1,1.0,32,24.968994140625
+2,1,1,1.0,32,24.968996047973633
+2,1,1,1.0,32,24.968994140625
+2,1,1,1.0,33,24.967994689941406
+2,1,1,1.0,33,24.967994689941406
+2,1,1,1.0,33,24.96799659729004
+2,1,1,1.0,33,24.96799659729004
+2,1,1,1.0,33,24.967994689941406
+2,1,1,1.0,33,24.96799659729004
+2,1,0,0.9375,33,12.967996597290039
+2,1,1,1.0,34,24.966997146606445
+2,1,1,1.0,34,24.966995239257812
+2,1,1,1.0,34,24.966995239257812
+2,1,1,1.0,34,24.966997146606445
+2,1,1,1.0,34,24.966995239257812
+2,1,0,0.9375,34,12.966995239257812
+2,1,1,1.0,34,24.966995239257812
+2,1,1,1.0,34,24.966995239257812
+2,1,1,1.0,34,24.966995239257812
+2,1,1,1.0,34,24.966997146606445
+2,1,1,1.0,34,24.966995239257812
+2,1,1,1.0,34,24.966997146606445
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599578857422
+2,1,1,1.0,35,24.96599769592285
+2,1,1,1.0,35,24.96599769592285
+2,1,1,1.0,35,24.965999603271484
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,36,24.964996337890625
+2,1,1,1.0,37,24.96399688720703
+2,1,1,1.0,37,24.96399688720703
+2,1,1,1.0,37,24.9639949798584
+2,1,1,1.0,37,24.96399688720703
+2,1,1,1.0,37,24.96399688720703
+2,1,1,1.0,37,24.96399688720703
+2,1,1,1.0,37,24.9639949798584
+2,1,1,1.0,37,24.96399688720703
+2,1,1,1.0,38,24.962993621826172
+2,1,1,1.0,38,24.962993621826172
+2,1,1,1.0,38,24.962997436523438
+2,1,1,1.0,38,24.962993621826172
+2,1,1,1.0,38,24.962995529174805
+2,1,1,1.0,39,24.961997985839844
+2,1,1,1.0,40,24.960994720458984
+2,1,1,1.0,40,24.960996627807617
+2,1,1,1.0,40,24.960996627807617
+2,1,1,1.0,40,24.960994720458984
+2,1,1,1.0,40,24.96099853515625
+2,1,1,1.0,40,24.960994720458984
+2,1,1,1.0,41,24.95999526977539
+2,1,1,1.0,41,24.959993362426758
+2,1,1,1.0,41,24.959993362426758
+2,1,1,1.0,41,24.95999526977539
+2,1,1,1.0,41,24.95999526977539
+2,1,1,1.0,41,24.95999526977539
+2,1,1,1.0,42,24.958995819091797
+2,1,1,1.0,42,24.958995819091797
+2,1,1,1.0,42,24.958995819091797
+2,1,1,1.0,42,24.958993911743164
+2,1,1,1.0,42,24.958995819091797
+2,1,1,1.0,43,24.957992553710938
+2,1,1,1.0,44,24.95699691772461
+2,1,1,1.0,44,24.95699691772461
+2,1,1,1.0,46,24.954998016357422
+2,1,1,1.0,48,24.95299530029297
+0,0,1,1.0,23,24.977998733520508
+0,0,1,1.0,24,24.97699737548828
+0,0,1,1.0,25,24.975997924804688
+0,0,1,1.0,25,24.975997924804688
+0,0,1,1.0,27,24.973997116088867
+0,0,1,1.0,27,24.973997116088867
+0,0,0,0.75,28,9.97299861907959
+0,0,1,1.0,28,24.97299575805664
+0,0,0,0.8125,28,10.97299861907959
+0,0,0,0.9375,29,12.971997261047363
+0,0,1,1.0,29,24.971996307373047
+0,0,1,1.0,29,24.97199821472168
+0,0,0,0.8125,29,10.97199821472168
+0,0,1,1.0,29,24.971996307373047
+0,0,0,0.9375,30,12.970995903015137
+0,0,1,1.0,30,24.970996856689453
+0,0,1,1.0,30,24.970998764038086
+0,0,1,1.0,30,24.970996856689453
+0,0,1,1.0,30,24.970996856689453
+0,0,1,1.0,31,24.969993591308594
+0,0,1,1.0,31,24.96999740600586
+0,0,1,1.0,31,24.96999740600586
+0,0,1,1.0,32,24.968997955322266
+0,0,1,1.0,32,24.968996047973633
+0,0,1,1.0,32,24.968997955322266
+0,0,1,1.0,32,24.968994140625
+0,0,1,1.0,32,24.968997955322266
+0,0,1,1.0,32,24.968997955322266
+0,0,1,1.0,32,24.968997955322266
+0,0,1,1.0,32,24.968997955322266
+0,0,1,1.0,33,24.967994689941406
+0,0,1,1.0,33,24.96799659729004
+0,0,1,1.0,33,24.967994689941406
+0,0,1,1.0,33,24.96799659729004
+0,0,1,1.0,33,24.96799659729004
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,34,24.966995239257812
+0,0,1,1.0,35,24.96599578857422
+0,0,0,0.6875,35,8.965998649597168
+0,0,0,0.875,35,11.965996742248535
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,35,24.96599578857422
+0,0,0,0.9375,35,12.965994834899902
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,35,24.96599578857422
+0,0,1,1.0,36,24.964996337890625
+0,0,1,1.0,36,24.964998245239258
+0,0,1,1.0,36,24.964996337890625
+0,0,1,1.0,36,24.964996337890625
+0,0,1,1.0,36,24.964996337890625
+0,0,1,1.0,36,24.964996337890625
+0,0,0,0.9375,36,12.964997291564941
+0,0,1,1.0,37,24.96399688720703
+0,0,1,1.0,37,24.9639949798584
+0,0,1,1.0,37,24.9639949798584
+0,0,1,1.0,37,24.963993072509766
+0,0,1,1.0,37,24.96399688720703
+0,0,1,1.0,37,24.96399688720703
+0,0,0,0.9375,37,12.963995933532715
+0,0,1,1.0,37,24.96399688720703
+0,0,1,1.0,37,24.96399688720703
+0,0,1,1.0,37,24.96399688720703
+0,0,1,1.0,38,24.962993621826172
+0,0,1,1.0,38,24.962997436523438
+0,0,1,1.0,38,24.962995529174805
+0,0,0,0.9375,39,12.961997032165527
+0,0,1,1.0,39,24.961994171142578
+0,0,1,1.0,40,24.96099853515625
+0,0,1,1.0,40,24.960994720458984
+0,0,1,1.0,40,24.960994720458984
+0,0,1,1.0,40,24.960994720458984
+0,0,1,1.0,40,24.960994720458984
+0,0,1,1.0,40,24.960994720458984
+0,0,1,1.0,40,24.960994720458984
+0,0,1,1.0,41,24.95999526977539
+0,0,1,1.0,41,24.959993362426758
+0,0,1,1.0,41,24.95999526977539
+0,0,1,1.0,41,24.95999526977539
+0,0,1,1.0,41,24.95999526977539
+0,0,1,1.0,41,24.959993362426758
+0,0,1,1.0,42,24.958995819091797
+0,0,1,1.0,42,24.95899200439453
+0,0,1,1.0,42,24.95899200439453
+0,0,1,1.0,43,24.957996368408203
+0,0,1,1.0,43,24.957996368408203
+0,0,1,1.0,43,24.957996368408203
+0,0,1,1.0,44,24.95699691772461
+0,0,1,1.0,45,24.95599365234375
+0,0,1,1.0,45,24.95599365234375
+0,0,1,1.0,48,24.95299530029297
+1,0,0,0.875,23,11.977997779846191
+1,0,0,0.9375,26,12.974997520446777
+1,0,0,0.6875,26,8.97499942779541
+1,0,0,0.6875,27,8.9739990234375
+1,0,1,1.0,28,24.972997665405273
+1,0,0,0.75,28,9.972999572753906
+1,0,1,1.0,28,24.97299575805664
+1,0,1,1.0,29,24.971996307373047
+1,0,1,1.0,29,24.97199821472168
+1,0,1,1.0,29,24.97199821472168
+1,0,1,1.0,29,24.971996307373047
+1,0,1,1.0,30,24.970996856689453
+1,0,1,1.0,30,24.970996856689453
+1,0,0,0.8125,30,10.970996856689453
+1,0,0,0.9375,30,12.970996856689453
+1,0,1,1.0,30,24.970996856689453
+1,0,1,1.0,30,24.970996856689453
+1,0,1,1.0,30,24.970993041992188
+1,0,1,1.0,30,24.970996856689453
+1,0,1,1.0,30,24.970996856689453
+1,0,0,0.6875,31,8.969996452331543
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,31,24.969995498657227
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,31,24.96999740600586
+1,0,1,1.0,32,24.968997955322266
+1,0,0,0.9375,32,12.96899700164795
+1,0,0,0.9375,32,12.96899700164795
+1,0,1,1.0,32,24.968997955322266
+1,0,1,1.0,32,24.968996047973633
+1,0,1,1.0,33,24.967994689941406
+1,0,1,1.0,33,24.967994689941406
+1,0,0,0.875,33,11.967997550964355
+1,0,1,1.0,33,24.967998504638672
+1,0,1,1.0,33,24.96799659729004
+1,0,1,1.0,33,24.967994689941406
+1,0,0,0.875,34,11.966997146606445
+1,0,1,1.0,34,24.966997146606445
+1,0,1,1.0,34,24.966997146606445
+1,0,1,1.0,34,24.966995239257812
+1,0,1,1.0,34,24.966997146606445
+1,0,1,1.0,34,24.966995239257812
+1,0,1,1.0,34,24.966997146606445
+1,0,1,1.0,34,24.966999053955078
+1,0,1,1.0,35,24.96599578857422
+1,0,1,1.0,35,24.96599578857422
+1,0,1,1.0,36,24.964994430541992
+1,0,1,1.0,36,24.964996337890625
+1,0,1,1.0,36,24.964996337890625
+1,0,1,1.0,36,24.964996337890625
+1,0,1,1.0,36,24.964998245239258
+1,0,1,1.0,36,24.964998245239258
+1,0,1,1.0,36,24.964994430541992
+1,0,1,1.0,36,24.964996337890625
+1,0,1,1.0,36,24.964994430541992
+1,0,0,0.9375,37,12.963996887207031
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.963993072509766
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.96399688720703
+1,0,1,1.0,37,24.9639949798584
+1,0,1,1.0,38,24.962993621826172
+1,0,1,1.0,38,24.962995529174805
+1,0,1,1.0,38,24.962997436523438
+1,0,1,1.0,38,24.962997436523438
+1,0,1,1.0,38,24.962993621826172
+1,0,1,1.0,38,24.962995529174805
+1,0,1,1.0,38,24.962997436523438
+1,0,1,1.0,39,24.96199607849121
+1,0,1,1.0,39,24.96199607849121
+1,0,1,1.0,39,24.961997985839844
+1,0,1,1.0,39,24.96199607849121
+1,0,1,1.0,40,24.960996627807617
+1,0,1,1.0,40,24.960994720458984
+1,0,1,1.0,41,24.95999526977539
+1,0,1,1.0,41,24.959997177124023
+1,0,1,1.0,41,24.959997177124023
+1,0,1,1.0,41,24.959997177124023
+1,0,1,1.0,42,24.958995819091797
+1,0,1,1.0,42,24.958995819091797
+1,0,1,1.0,42,24.958995819091797
+1,0,1,1.0,42,24.958993911743164
+1,0,1,1.0,43,24.957996368408203
+1,0,1,1.0,43,24.95799446105957
+1,0,1,1.0,43,24.957996368408203
+1,0,1,1.0,44,24.95699691772461
+1,0,1,1.0,44,24.95699691772461
+1,0,1,1.0,44,24.956993103027344
+1,0,1,1.0,45,24.95599365234375
+1,0,1,1.0,45,24.95599365234375
+1,0,1,1.0,49,24.951995849609375
+1,0,1,1.0,56,24.944992065429688
+2,0,0,0.75,22,9.978997230529785
+2,0,1,1.0,27,24.9739990234375
+2,0,1,1.0,27,24.973997116088867
+2,0,0,0.9375,28,12.972997665405273
+2,0,0,0.8125,28,10.972997665405273
+2,0,1,1.0,28,24.97299575805664
+2,0,1,1.0,28,24.97299575805664
+2,0,0,0.9375,28,12.972997665405273
+2,0,1,1.0,28,24.972997665405273
+2,0,1,1.0,29,24.971996307373047
+2,0,0,0.9375,30,12.970996856689453
+2,0,0,0.8125,30,10.970996856689453
+2,0,0,0.9375,30,12.970995903015137
+2,0,0,0.8125,31,10.969996452331543
+2,0,1,1.0,31,24.96999740600586
+2,0,1,1.0,31,24.969995498657227
+2,0,1,1.0,31,24.96999740600586
+2,0,1,1.0,31,24.96999740600586
+2,0,1,1.0,31,24.96999740600586
+2,0,1,1.0,32,24.968997955322266
+2,0,1,1.0,32,24.968997955322266
+2,0,1,1.0,32,24.968996047973633
+2,0,0,0.9375,32,12.968996047973633
+2,0,1,1.0,32,24.968994140625
+2,0,0,0.9375,32,12.96899700164795
+2,0,1,1.0,33,24.967994689941406
+2,0,0,0.9375,33,12.967995643615723
+2,0,1,1.0,33,24.96799659729004
+2,0,1,1.0,33,24.967994689941406
+2,0,1,1.0,34,24.966999053955078
+2,0,1,1.0,34,24.966995239257812
+2,0,1,1.0,34,24.966995239257812
+2,0,0,0.9375,34,12.966997146606445
+2,0,1,1.0,34,24.966995239257812
+2,0,1,1.0,34,24.966995239257812
+2,0,1,1.0,34,24.966995239257812
+2,0,1,1.0,35,24.96599769592285
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.96599769592285
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.96599578857422
+2,0,0,0.9375,35,12.965995788574219
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,35,24.965999603271484
+2,0,1,1.0,35,24.96599578857422
+2,0,1,1.0,36,24.964996337890625
+2,0,1,1.0,36,24.964996337890625
+2,0,1,1.0,36,24.964996337890625
+2,0,1,1.0,36,24.964994430541992
+2,0,1,1.0,36,24.964994430541992
+2,0,1,1.0,36,24.964996337890625
+2,0,1,1.0,36,24.964996337890625
+2,0,1,1.0,36,24.96499252319336
+2,0,1,1.0,37,24.9639949798584
+2,0,1,1.0,37,24.9639949798584
+2,0,1,1.0,37,24.96399688720703
+2,0,1,1.0,37,24.9639949798584
+2,0,1,1.0,37,24.9639949798584
+2,0,1,1.0,37,24.9639949798584
+2,0,1,1.0,37,24.96399688720703
+2,0,1,1.0,37,24.9639949798584
+2,0,0,0.9375,38,12.962996482849121
+2,0,1,1.0,38,24.962993621826172
+2,0,1,1.0,38,24.962993621826172
+2,0,1,1.0,38,24.962997436523438
+2,0,1,1.0,38,24.962997436523438
+2,0,1,1.0,38,24.962997436523438
+2,0,1,1.0,38,24.962995529174805
+2,0,1,1.0,38,24.962995529174805
+2,0,1,1.0,39,24.96199607849121
+2,0,1,1.0,39,24.96199607849121
+2,0,0,0.9375,39,12.961996078491211
+2,0,1,1.0,39,24.96199607849121
+2,0,1,1.0,40,24.960994720458984
+2,0,1,1.0,40,24.960994720458984
+2,0,1,1.0,40,24.96099853515625
+2,0,1,1.0,40,24.960994720458984
+2,0,1,1.0,40,24.960994720458984
+2,0,1,1.0,40,24.960994720458984
+2,0,1,1.0,40,24.960994720458984
+2,0,1,1.0,40,24.960994720458984
+2,0,0,0.9375,41,12.959994316101074
+2,0,0,0.9375,41,12.959997177124023
+2,0,1,1.0,41,24.95999526977539
+2,0,1,1.0,41,24.95999526977539
+2,0,1,1.0,41,24.959993362426758
+2,0,1,1.0,41,24.95999526977539
+2,0,1,1.0,42,24.958995819091797
+2,0,1,1.0,42,24.958995819091797
+2,0,1,1.0,42,24.958995819091797
+2,0,1,1.0,42,24.958995819091797
+2,0,1,1.0,43,24.957996368408203
+2,0,1,1.0,43,24.957996368408203
+2,0,1,1.0,45,24.95599365234375
+2,0,1,1.0,45,24.955997467041016
+2,0,1,1.0,48,24.95299530029297

eval/summary.json

@@ -0,0 +1,129 @@
+{
+  "board": "2^4",
+  "n_cells": 16,
+  "seeds": [
+    0,
+    1,
+    2
+  ],
+  "per_run": {
+    "deterministic=True/seed=0": {
+      "episodes": 100,
+      "success_rate": 0.94,
+      "fill_mean": 0.989375,
+      "return_mean": 24.13592656135559,
+      "length_mean": 35.07,
+      "steps_to_complete_mean": 35.340425531914896,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.33,
+        "4C": 0.94,
+        "C2/2": 0.94
+      }
+    },
+    "deterministic=True/seed=1": {
+      "episodes": 100,
+      "success_rate": 0.93,
+      "fill_mean": 0.99,
+      "return_mean": 24.03526602745056,
+      "length_mean": 35.73,
+      "steps_to_complete_mean": 36.043010752688176,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.16,
+        "4C": 0.93,
+        "C2/2": 0.93
+      }
+    },
+    "deterministic=True/seed=2": {
+      "episodes": 100,
+      "success_rate": 0.93,
+      "fill_mean": 0.99375,
+      "return_mean": 24.09568606376648,
+      "length_mean": 35.31,
+      "steps_to_complete_mean": 35.623655913978496,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.21,
+        "4C": 0.93,
+        "C2/2": 0.93
+      }
+    },
+    "deterministic=False/seed=0": {
+      "episodes": 100,
+      "success_rate": 0.89,
+      "fill_mean": 0.985625,
+      "return_mean": 23.525866088867186,
+      "length_mean": 35.13,
+      "steps_to_complete_mean": 35.41573033707865,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.25,
+        "4C": 0.89,
+        "C2/2": 0.89
+      }
+    },
+    "deterministic=False/seed=1": {
+      "episodes": 100,
+      "success_rate": 0.87,
+      "fill_mean": 0.979375,
+      "return_mean": 23.205646409988404,
+      "length_mean": 35.35,
+      "steps_to_complete_mean": 36.160919540229884,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.24,
+        "4C": 0.87,
+        "C2/2": 0.87
+      }
+    },
+    "deterministic=False/seed=2": {
+      "episodes": 100,
+      "success_rate": 0.83,
+      "fill_mean": 0.98375,
+      "return_mean": 22.835316019058226,
+      "length_mean": 35.68,
+      "steps_to_complete_mean": 36.33734939759036,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.15,
+        "4C": 0.83,
+        "C2/2": 0.83
+      }
+    }
+  },
+  "modes": {
+    "deterministic=True": {
+      "success_rate_mean": 0.9333333333333335,
+      "success_rate_std": 0.004714045207910268,
+      "episodes": 300,
+      "success_rate": 0.9333333333333333,
+      "fill_mean": 0.9910416666666667,
+      "return_mean": 24.088959550857545,
+      "length_mean": 35.37,
+      "steps_to_complete_mean": 35.667857142857144,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.23333333333333334,
+        "4C": 0.9333333333333333,
+        "C2/2": 0.9333333333333333
+      }
+    },
+    "deterministic=False": {
+      "success_rate_mean": 0.8633333333333333,
+      "success_rate_std": 0.024944382578492966,
+      "episodes": 300,
+      "success_rate": 0.8633333333333333,
+      "fill_mean": 0.9829166666666667,
+      "return_mean": 23.188942839304605,
+      "length_mean": 35.38666666666666,
+      "steps_to_complete_mean": 35.96138996138996,
+      "win_within": {
+        "C": 0.0,
+        "2C": 0.21333333333333335,
+        "4C": 0.8633333333333333,
+        "C2/2": 0.8633333333333333
+      }
+    }
+  }
+}

versions.json

@@ -0,0 +1,10 @@
+{
+  "torch": "2.14.0+cu130",
+  "gymnasium": "1.3.0",
+  "stable-baselines3": "2.9.0",
+  "sb3-contrib": "2.9.0",
+  "numpy": "2.5.2",
+  "pygame-ce": "2.5.8",
+  "git_commit": "09536a78f09add329dfb0f8b018736de1a69bee3",
+  "cuda_device": "NVIDIA GeForce RTX 5070 Laptop GPU"
+}