phclab commited on
Commit
62aa906
·
verified ·
1 Parent(s): b65974d

MushroomBody_Othello v1

Browse files
README.md ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ tags:
4
+ - othello
5
+ - reversi
6
+ - spiking-neural-network
7
+ - connectome
8
+ - drosophila
9
+ - rsnn
10
+ ---
11
+
12
+ # MushroomBody_Othello v1
13
+
14
+ Synaptic weights for the Othello game of **[BeatTheFly](https://phclab.github.io/BeatTheFly/othello/)** -- *A Smart Fruit Fly is playing Othello against you*:
15
+ a spiking network wired as the real *Drosophila* mushroom-body connectome that plays Othello.
16
+
17
+ **The anatomical connectome gives you wiring, not synaptic strengths. Ours are trained.**
18
+
19
+ Synaptic weights trained with PHCSSM parallel-scan mode, deployment in sequential RSNN mode ([PHCSSM](https://arxiv.org/abs/2604.01295)).
20
+
21
+ made by Po-Han Chiang @ NYCU
22
+
23
+ ## Architecture
24
+ * **Wiring:** MaleCNS v1.0 mushroom body -- 4,510 neurons (4,064 Kenyon cells, 97 MBONs,
25
+ 340 DANs, 2 APL, 2 DPM, 5 MB-C1) and 1,027,152 neuron-to-neuron connections. The
26
+ connectivity mask is fixed to the connectome; 763,700 connections carry a nonzero weight and
27
+ 0 weights lie off the connectome.
28
+ * **Dale's law:** one sign per presynaptic neuron from neurotransmitter annotations (excitatory
29
+ 4,114, inhibitory 52, modulatory 344); 0 weights violate it.
30
+ * **Inputs:** each ply provides the move token (one of 68 tokens: the 64 squares, pass, and start,
31
+ end and padding markers) and the board after it (138 binary features: the discs of the side to
32
+ move, the opponent's discs, which side is to move, and a disc-count bucket). Two linear encoders with their
33
+ own LayerNorm drive the Kenyon cells; the move token also drives the dopaminergic neurons.
34
+ * **Rules:** a pass is a move only when the side to move has no legal square; the network takes one timestep
35
+ per move, forced passes included.
36
+ * **Neurons:** leaky integrate-and-fire with per-neuron leak, threshold and reset; synaptic delay of one step.
37
+ * **Fast weight:** dopamine-gated associative memory on the Kenyon-cell -> MBON synapses, read back into
38
+ the MBON voltages.
39
+ * **Readout:** linear map from the membrane voltage of all neurons to the move vocabulary.
40
+ * **Deployment:** sequential RSNN mode, one timestep per ply, with the neuron state and fast weight
41
+ carried across the whole game.
42
+
43
+ ## Data sources
44
+ Data source: [Egaroucid Free Training Data](https://www.egaroucid.nyanyan.dev/en/technology/train-data/) (Takuto Yamana).
45
+
46
+ ## Evaluation
47
+ Agreement with the engine's move on 10,570 held-out Egaroucid games (legal moves only): 50.1% overall (moves 1–15 43.4%, 16–30 38.6%, 31–45 39.6%, 46+ 61.2%).
48
+
49
+ **Strength:** Beats simple greedy and corner-heuristic bots about 80–86% of the time; scores about one third against the Edax engine at level 1 (32–1–67 in 100 games). Matches: 100 games per opponent, colours swapped; the fly uses no search, one network timestep per move.
50
+
51
+ ## Files
52
+ * `manifest.json` -- every tensor (file, dtype, shape, bytes), the model scalars, the token map
53
+ (`scalars.tokens`: pad 0, start 1, end 2, pass 3, square `a1` = 4 ... `h8` = 67,
54
+ row-major from `a1`) and a connectome audit.
55
+ * `info.json` -- neuron metadata used by the page (cell classes, hemispheres, soma coordinates).
56
+ * `selfcheck_<precision>.json` -- reference moves and logits that the page replays when it loads.
57
+ * `fp16/`, `fp32/` -- raw little-endian arrays.
58
+
59
+ Two precisions are listed in the manifest: `fp16w32` (default, 7.6 MB: float16 for the
60
+ four large dense matrices, float32 for the recurrent weights and all small tensors) and
61
+ `fp16` (6.1 MB, recurrent weights in float16 as well).
62
+
63
+ The recurrent weight matrix W[dst, src] is stored in CSC order by source neuron (`W_colptr`,
64
+ `W_rowidx`, `W_vals`): each step multiplies W by a sparse binary spike vector, so the engine visits only
65
+ the columns of the neurons that spiked. Dense matrices are stored in the orientation they are read:
66
+ `enc_tok_T` [vocab, H] (a move token selects one row), `enc_brd_T` [138, H] (sum of the active rows),
67
+ `dec_w` [vocab, H] and `v2d_T` [vocab, n_dan].
68
+
69
+ | name | file | dtype | shape |
70
+ |---|---|---|---|
71
+ | `enc_tok_T` | `fp16/enc_tok_T.bin` | float16 | 68x4510 |
72
+ | `enc_tok_b` | `fp32/enc_tok_b.bin` | float32 | 4510 |
73
+ | `ln_tok_w` | `fp32/ln_tok_w.bin` | float32 | 4510 |
74
+ | `ln_tok_b` | `fp32/ln_tok_b.bin` | float32 | 4510 |
75
+ | `enc_brd_T` | `fp16/enc_brd_T.bin` | float16 | 138x4510 |
76
+ | `enc_brd_b` | `fp32/enc_brd_b.bin` | float32 | 4510 |
77
+ | `ln_brd_w` | `fp32/ln_brd_w.bin` | float32 | 4510 |
78
+ | `ln_brd_b` | `fp32/ln_brd_b.bin` | float32 | 4510 |
79
+ | `v2d_T` | `fp16/v2d_T.bin` | float16 | 68x340 |
80
+ | `v2d_b` | `fp32/v2d_b.bin` | float32 | 340 |
81
+ | `dec_w` | `fp16/dec_w.bin` | float16 | 68x4510 |
82
+ | `dec_b` | `fp32/dec_b.bin` | float32 | 68 |
83
+ | `Wg` | `fp32/Wg.bin` | float32 | 97x340 |
84
+ | `W_dan_val` | `fp32/W_dan_val.bin` | float32 | 97x340 |
85
+ | `alpha_exc` | `fp32/alpha_exc.bin` | float32 | 4510 |
86
+ | `alpha_inh` | `fp32/alpha_inh.bin` | float32 | 4510 |
87
+ | `v_th` | `fp32/v_th.bin` | float32 | 4510 |
88
+ | `reset_weight` | `fp32/reset_weight.bin` | float32 | 4510 |
89
+ | `kc_idx` | `fp32/kc_idx.bin` | int32 | 4064 |
90
+ | `mbon_idx` | `fp32/mbon_idx.bin` | int32 | 97 |
91
+ | `dan_idx` | `fp32/dan_idx.bin` | int32 | 340 |
92
+ | `W_colptr` | `fp32/W_colptr.bin` | uint32 | 4511 |
93
+ | `W_rowidx` | `fp32/W_rowidx.bin` | uint16 | 763700 |
94
+ | `W_vals` | `fp32/W_vals.bin` | float32 | 763700 |
95
+
96
+ **Numerical check:** legal top-1 1948/1949 vs the fp32 reference (32 fly-vs-fly reference games); 0 of 8,934,310 spike bits differ from the reference on the same weights.
97
+
98
+ ## Limitations
99
+ There is no search and no evaluation function: each move is a single timestep of the network,
100
+ restricted to the legal squares.
101
+
102
+ ## License and attribution
103
+ Weights: [CC-BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/). They are derived from the MaleCNS v1.0 connectome (Janelia FlyEM and
104
+ collaborators, https://male-cns.janelia.org/, [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) and trained with PHCSSM (https://arxiv.org/abs/2604.01295);
105
+ please credit both. Training data: Egaroucid Free Training Data by Takuto Yamana (https://www.egaroucid.nyanyan.dev/en/technology/train-data/).
106
+
107
+ ## Citation
108
+ PHCSSM: https://arxiv.org/abs/2604.01295
fp16/W_vals.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa63068d979fcd2c3ad0c02419f12c3f86c5a164c1e8b56912c429f963952500
3
+ size 1527400
fp16/dec_w.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e8b730cc35a84f63e749f90c0a67217962404680a8ebe5c5d59d15849492521
3
+ size 613360
fp16/enc_brd_T.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7469dbd7a139d7b24466e991f8fa962b8e4df19fe43455e8c11fc9c5282ca0b9
3
+ size 1244760
fp16/enc_tok_T.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4271837aed86394ade998ea92fae7a9daf2f830d866f64ac20d8eb144097c994
3
+ size 613360
fp16/v2d_T.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de5ba7a1c1fc0a3fd5e7c7db754190d708c52b34ad5d52a1e31ee4afcb6da4fa
3
+ size 46240
fp32/W_colptr.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34eb6503e566f5701953a16ab9127aab6fee0fcee4010152ffb8a1f52eada6e5
3
+ size 18044
fp32/W_dan_val.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f495b5626a57996a17ad1d7371bc0fac52c40d6466cbfee64010323df31529f8
3
+ size 131920
fp32/W_rowidx.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0109ecf26017c782c2d873537d367b5132d1c324f79290490e5dcc5c4607ddd1
3
+ size 1527400
fp32/W_vals.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae099b772f46fffae6e38304d0523a0cdda083888df2950ece466de8ab67984e
3
+ size 3054800
fp32/Wg.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3fd94f7ce600cba7d50fdc96fdbed52fc9f51433a62bbc13c70ce7074b158bf
3
+ size 131920
fp32/alpha_exc.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f38b945d3bdf2a25405317539df7a83e06b3d0edcfff8e6642e5af9b7d61b298
3
+ size 18040
fp32/alpha_inh.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01c5b87b05f2dfe2d4c92e2914d4272db3978011c3dc7698ddd22931bc78a0e2
3
+ size 18040
fp32/dan_idx.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39b0b6129a52827c06620845d067c1674760c072c28312640fd156a8e85e92d0
3
+ size 1360
fp32/dec_b.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10ad4135688649ae2454ee764d5b1f9f7881347cb1f431b22705962dab2beb68
3
+ size 272
fp32/enc_brd_b.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2b7ce22cfe49ff387a34a16490e4f87a391e2bd44b27c990bf038cbd4ba0256
3
+ size 18040
fp32/enc_tok_b.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f4494c570ee20d07e0fc4bf7b52c0d6d4e8223c5ef6d8524d5d0fed6974e41e
3
+ size 18040
fp32/kc_idx.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09480588592e17b59d916c13af103878e3670f0a8d31b47de77036bddf38f919
3
+ size 16256
fp32/ln_brd_b.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2106b102eb01a359237d944c04d603855902203f80d3c8e3bfb5d4cd194b7fb5
3
+ size 18040
fp32/ln_brd_w.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:573127c7b80aada101321475c66a6f657ea8721b16ebaf1d912987685a181c85
3
+ size 18040
fp32/ln_tok_b.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2106b102eb01a359237d944c04d603855902203f80d3c8e3bfb5d4cd194b7fb5
3
+ size 18040
fp32/ln_tok_w.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:209687b2c49371cebe8403f59b83f14abe40c5b60ace7c4950092ac72d62cdec
3
+ size 18040
fp32/mbon_idx.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8bfefedcc547942e5ebd91b7fe0aaf834d4e23a4945db8cb23e443f2b41f174
3
+ size 388
fp32/reset_weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77173d4abbac21a1e4e7e9a9e1f08149c997bb4a0e5947c167d1d964800a7b30
3
+ size 18040
fp32/v2d_b.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4dc8936ec9bee1b27ab2a451c3b505fcdc88ddcbc5e11ac4bcefec227f7bf28
3
+ size 1360
fp32/v_th.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:661b13ff23f89070821cfce59123d41b1a921a51ce06dc4938dbe8ca2aa73e5c
3
+ size 18040
info.json ADDED
The diff for this file is too large to render. See raw diff
 
manifest.json ADDED
@@ -0,0 +1,442 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "MushroomBody_Othello",
3
+ "version": "v1",
4
+ "label": "MushroomBody_Othello v1",
5
+ "scalars": {
6
+ "H": 4510,
7
+ "vocab": 68,
8
+ "d_board": 138,
9
+ "n_kc": 4064,
10
+ "n_mbon": 97,
11
+ "n_dan": 340,
12
+ "e_cl": -0.5,
13
+ "drive_alpha": 1.0,
14
+ "dan_teach": 1.0,
15
+ "lam": 1.0,
16
+ "use_tok": 1.0,
17
+ "use_board": 1.0,
18
+ "nnz": 763700,
19
+ "tokens": {
20
+ "pad": 0,
21
+ "bos": 1,
22
+ "eos": 2,
23
+ "pass": 3,
24
+ "sq0": 4
25
+ }
26
+ },
27
+ "variants": {
28
+ "fp16w32": {
29
+ "enc_tok_T": {
30
+ "file": "fp16/enc_tok_T.bin",
31
+ "dtype": "float16",
32
+ "shape": [
33
+ 68,
34
+ 4510
35
+ ],
36
+ "bytes": 613360
37
+ },
38
+ "enc_tok_b": {
39
+ "file": "fp32/enc_tok_b.bin",
40
+ "dtype": "float32",
41
+ "shape": [
42
+ 4510
43
+ ],
44
+ "bytes": 18040
45
+ },
46
+ "ln_tok_w": {
47
+ "file": "fp32/ln_tok_w.bin",
48
+ "dtype": "float32",
49
+ "shape": [
50
+ 4510
51
+ ],
52
+ "bytes": 18040
53
+ },
54
+ "ln_tok_b": {
55
+ "file": "fp32/ln_tok_b.bin",
56
+ "dtype": "float32",
57
+ "shape": [
58
+ 4510
59
+ ],
60
+ "bytes": 18040
61
+ },
62
+ "enc_brd_T": {
63
+ "file": "fp16/enc_brd_T.bin",
64
+ "dtype": "float16",
65
+ "shape": [
66
+ 138,
67
+ 4510
68
+ ],
69
+ "bytes": 1244760
70
+ },
71
+ "enc_brd_b": {
72
+ "file": "fp32/enc_brd_b.bin",
73
+ "dtype": "float32",
74
+ "shape": [
75
+ 4510
76
+ ],
77
+ "bytes": 18040
78
+ },
79
+ "ln_brd_w": {
80
+ "file": "fp32/ln_brd_w.bin",
81
+ "dtype": "float32",
82
+ "shape": [
83
+ 4510
84
+ ],
85
+ "bytes": 18040
86
+ },
87
+ "ln_brd_b": {
88
+ "file": "fp32/ln_brd_b.bin",
89
+ "dtype": "float32",
90
+ "shape": [
91
+ 4510
92
+ ],
93
+ "bytes": 18040
94
+ },
95
+ "v2d_T": {
96
+ "file": "fp16/v2d_T.bin",
97
+ "dtype": "float16",
98
+ "shape": [
99
+ 68,
100
+ 340
101
+ ],
102
+ "bytes": 46240
103
+ },
104
+ "v2d_b": {
105
+ "file": "fp32/v2d_b.bin",
106
+ "dtype": "float32",
107
+ "shape": [
108
+ 340
109
+ ],
110
+ "bytes": 1360
111
+ },
112
+ "dec_w": {
113
+ "file": "fp16/dec_w.bin",
114
+ "dtype": "float16",
115
+ "shape": [
116
+ 68,
117
+ 4510
118
+ ],
119
+ "bytes": 613360
120
+ },
121
+ "dec_b": {
122
+ "file": "fp32/dec_b.bin",
123
+ "dtype": "float32",
124
+ "shape": [
125
+ 68
126
+ ],
127
+ "bytes": 272
128
+ },
129
+ "Wg": {
130
+ "file": "fp32/Wg.bin",
131
+ "dtype": "float32",
132
+ "shape": [
133
+ 97,
134
+ 340
135
+ ],
136
+ "bytes": 131920
137
+ },
138
+ "W_dan_val": {
139
+ "file": "fp32/W_dan_val.bin",
140
+ "dtype": "float32",
141
+ "shape": [
142
+ 97,
143
+ 340
144
+ ],
145
+ "bytes": 131920
146
+ },
147
+ "alpha_exc": {
148
+ "file": "fp32/alpha_exc.bin",
149
+ "dtype": "float32",
150
+ "shape": [
151
+ 4510
152
+ ],
153
+ "bytes": 18040
154
+ },
155
+ "alpha_inh": {
156
+ "file": "fp32/alpha_inh.bin",
157
+ "dtype": "float32",
158
+ "shape": [
159
+ 4510
160
+ ],
161
+ "bytes": 18040
162
+ },
163
+ "v_th": {
164
+ "file": "fp32/v_th.bin",
165
+ "dtype": "float32",
166
+ "shape": [
167
+ 4510
168
+ ],
169
+ "bytes": 18040
170
+ },
171
+ "reset_weight": {
172
+ "file": "fp32/reset_weight.bin",
173
+ "dtype": "float32",
174
+ "shape": [
175
+ 4510
176
+ ],
177
+ "bytes": 18040
178
+ },
179
+ "kc_idx": {
180
+ "file": "fp32/kc_idx.bin",
181
+ "dtype": "int32",
182
+ "shape": [
183
+ 4064
184
+ ],
185
+ "bytes": 16256
186
+ },
187
+ "mbon_idx": {
188
+ "file": "fp32/mbon_idx.bin",
189
+ "dtype": "int32",
190
+ "shape": [
191
+ 97
192
+ ],
193
+ "bytes": 388
194
+ },
195
+ "dan_idx": {
196
+ "file": "fp32/dan_idx.bin",
197
+ "dtype": "int32",
198
+ "shape": [
199
+ 340
200
+ ],
201
+ "bytes": 1360
202
+ },
203
+ "W_colptr": {
204
+ "file": "fp32/W_colptr.bin",
205
+ "dtype": "uint32",
206
+ "shape": [
207
+ 4511
208
+ ],
209
+ "bytes": 18044
210
+ },
211
+ "W_rowidx": {
212
+ "file": "fp32/W_rowidx.bin",
213
+ "dtype": "uint16",
214
+ "shape": [
215
+ 763700
216
+ ],
217
+ "bytes": 1527400
218
+ },
219
+ "W_vals": {
220
+ "file": "fp32/W_vals.bin",
221
+ "dtype": "float32",
222
+ "shape": [
223
+ 763700
224
+ ],
225
+ "bytes": 3054800
226
+ }
227
+ },
228
+ "fp16": {
229
+ "enc_tok_T": {
230
+ "file": "fp16/enc_tok_T.bin",
231
+ "dtype": "float16",
232
+ "shape": [
233
+ 68,
234
+ 4510
235
+ ],
236
+ "bytes": 613360
237
+ },
238
+ "enc_tok_b": {
239
+ "file": "fp32/enc_tok_b.bin",
240
+ "dtype": "float32",
241
+ "shape": [
242
+ 4510
243
+ ],
244
+ "bytes": 18040
245
+ },
246
+ "ln_tok_w": {
247
+ "file": "fp32/ln_tok_w.bin",
248
+ "dtype": "float32",
249
+ "shape": [
250
+ 4510
251
+ ],
252
+ "bytes": 18040
253
+ },
254
+ "ln_tok_b": {
255
+ "file": "fp32/ln_tok_b.bin",
256
+ "dtype": "float32",
257
+ "shape": [
258
+ 4510
259
+ ],
260
+ "bytes": 18040
261
+ },
262
+ "enc_brd_T": {
263
+ "file": "fp16/enc_brd_T.bin",
264
+ "dtype": "float16",
265
+ "shape": [
266
+ 138,
267
+ 4510
268
+ ],
269
+ "bytes": 1244760
270
+ },
271
+ "enc_brd_b": {
272
+ "file": "fp32/enc_brd_b.bin",
273
+ "dtype": "float32",
274
+ "shape": [
275
+ 4510
276
+ ],
277
+ "bytes": 18040
278
+ },
279
+ "ln_brd_w": {
280
+ "file": "fp32/ln_brd_w.bin",
281
+ "dtype": "float32",
282
+ "shape": [
283
+ 4510
284
+ ],
285
+ "bytes": 18040
286
+ },
287
+ "ln_brd_b": {
288
+ "file": "fp32/ln_brd_b.bin",
289
+ "dtype": "float32",
290
+ "shape": [
291
+ 4510
292
+ ],
293
+ "bytes": 18040
294
+ },
295
+ "v2d_T": {
296
+ "file": "fp16/v2d_T.bin",
297
+ "dtype": "float16",
298
+ "shape": [
299
+ 68,
300
+ 340
301
+ ],
302
+ "bytes": 46240
303
+ },
304
+ "v2d_b": {
305
+ "file": "fp32/v2d_b.bin",
306
+ "dtype": "float32",
307
+ "shape": [
308
+ 340
309
+ ],
310
+ "bytes": 1360
311
+ },
312
+ "dec_w": {
313
+ "file": "fp16/dec_w.bin",
314
+ "dtype": "float16",
315
+ "shape": [
316
+ 68,
317
+ 4510
318
+ ],
319
+ "bytes": 613360
320
+ },
321
+ "dec_b": {
322
+ "file": "fp32/dec_b.bin",
323
+ "dtype": "float32",
324
+ "shape": [
325
+ 68
326
+ ],
327
+ "bytes": 272
328
+ },
329
+ "Wg": {
330
+ "file": "fp32/Wg.bin",
331
+ "dtype": "float32",
332
+ "shape": [
333
+ 97,
334
+ 340
335
+ ],
336
+ "bytes": 131920
337
+ },
338
+ "W_dan_val": {
339
+ "file": "fp32/W_dan_val.bin",
340
+ "dtype": "float32",
341
+ "shape": [
342
+ 97,
343
+ 340
344
+ ],
345
+ "bytes": 131920
346
+ },
347
+ "alpha_exc": {
348
+ "file": "fp32/alpha_exc.bin",
349
+ "dtype": "float32",
350
+ "shape": [
351
+ 4510
352
+ ],
353
+ "bytes": 18040
354
+ },
355
+ "alpha_inh": {
356
+ "file": "fp32/alpha_inh.bin",
357
+ "dtype": "float32",
358
+ "shape": [
359
+ 4510
360
+ ],
361
+ "bytes": 18040
362
+ },
363
+ "v_th": {
364
+ "file": "fp32/v_th.bin",
365
+ "dtype": "float32",
366
+ "shape": [
367
+ 4510
368
+ ],
369
+ "bytes": 18040
370
+ },
371
+ "reset_weight": {
372
+ "file": "fp32/reset_weight.bin",
373
+ "dtype": "float32",
374
+ "shape": [
375
+ 4510
376
+ ],
377
+ "bytes": 18040
378
+ },
379
+ "kc_idx": {
380
+ "file": "fp32/kc_idx.bin",
381
+ "dtype": "int32",
382
+ "shape": [
383
+ 4064
384
+ ],
385
+ "bytes": 16256
386
+ },
387
+ "mbon_idx": {
388
+ "file": "fp32/mbon_idx.bin",
389
+ "dtype": "int32",
390
+ "shape": [
391
+ 97
392
+ ],
393
+ "bytes": 388
394
+ },
395
+ "dan_idx": {
396
+ "file": "fp32/dan_idx.bin",
397
+ "dtype": "int32",
398
+ "shape": [
399
+ 340
400
+ ],
401
+ "bytes": 1360
402
+ },
403
+ "W_colptr": {
404
+ "file": "fp32/W_colptr.bin",
405
+ "dtype": "uint32",
406
+ "shape": [
407
+ 4511
408
+ ],
409
+ "bytes": 18044
410
+ },
411
+ "W_rowidx": {
412
+ "file": "fp32/W_rowidx.bin",
413
+ "dtype": "uint16",
414
+ "shape": [
415
+ 763700
416
+ ],
417
+ "bytes": 1527400
418
+ },
419
+ "W_vals": {
420
+ "file": "fp16/W_vals.bin",
421
+ "dtype": "float16",
422
+ "shape": [
423
+ 763700
424
+ ],
425
+ "bytes": 1527400
426
+ }
427
+ }
428
+ },
429
+ "connectome_audit": {
430
+ "connections": 1027152,
431
+ "nonzero_weights": 763700,
432
+ "off_connectome": 0,
433
+ "wrong_sign": 0,
434
+ "excitatory": 4114,
435
+ "inhibitory": 52,
436
+ "modulatory": 344
437
+ },
438
+ "validation": {
439
+ "fp16w32": "legal top-1 1948/1949 vs the fp32 reference (32 fly-vs-fly reference games); 0 of 8,934,310 spike bits differ from the reference on the same weights",
440
+ "fp16": "legal top-1 1948/1949 vs the fp32 reference (32 fly-vs-fly reference games); 0 of 8,934,310 spike bits differ from the reference on the same weights"
441
+ }
442
+ }
selfcheck_fp16.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"moves":["f5","f6","f7","g5","d3","c3","h4","h5","h6","e3","b2","d6","f3","f2","d7"],"plies":[{"ids":[23,30,41,48],"top1":41,"logits":[2.982203245162964,3.3776278495788574,3.705583333969116,3.489532470703125]},{"ids":[33,47,49],"top1":33,"logits":[1.4626246690750122,0.9479949474334717,1.3738499879837036]},{"ids":[23,30,48,57],"top1":48,"logits":[-0.6156036853790283,0.003581687808036804,2.000441312789917,0.3151703476905823]},{"ids":[24,33,38,42,47,58],"top1":47,"logits":[1.048073649406433,1.026148796081543,0.3870837092399597,1.0306525230407715,1.1200436353683472,-0.5315322875976562]},{"ids":[22,23,30,33,35,43,48,50],"top1":48,"logits":[2.0728812217712402,2.4268853664398193,2.4837071895599365,4.41459846496582,2.8621933460235596,3.001667022705078,4.763359069824219,3.9951071739196777]},{"ids":[14,22,24,38,56,58,65],"top1":24,"logits":[1.9678561687469482,3.485398530960083,4.897215843200684,3.239199638366699,3.9225809574127197,-0.1889781802892685,2.783430814743042]},{"ids":[13,21,30,33,35,43,48,50],"top1":30,"logits":[-0.7280473113059998,2.8208277225494385,6.629842281341553,6.347822666168213,3.0546302795410156,3.5111067295074463,6.4175238609313965,4.972999095916748]},{"ids":[14,15,24,33,38,43,47,58,65],"top1":33,"logits":[4.341893196105957,4.808849334716797,6.563321590423584,7.073779106140137,5.1223931312561035,5.341828346252441,5.79020881652832,0.8835207223892212,3.3021535873413086]},{"ids":[13,21,30,33,48,50,51],"top1":51,"logits":[-0.06976185739040375,3.1782138347625732,6.7355780601501465,6.6420722007751465,7.25180721282959,6.520836353302002,8.826648712158203]},{"ids":[14,15,24,33,38,47,56,58,65],"top1":33,"logits":[4.281285762786865,4.552337169647217,6.377337455749512,7.0822367668151855,5.062496185302734,5.809814929962158,4.704594135284424,2.304887056350708,4.298536777496338]},{"ids":[13,15,25,33],"top1":25,"logits":[-0.3842831552028656,3.59270977973938,6.247959136962891,5.600085258483887]},{"ids":[21,30,38,46,47,48,56,65],"top1":30,"logits":[5.5438385009765625,7.593215465545654,5.560888767242432,5.532253742218018,5.632824420928955,6.682928085327148,4.527334213256836,3.869112014770508]},{"ids":[16,25,33,38,54],"top1":33,"logits":[5.41631555557251,5.982419490814209,6.603641510009766,4.791820049285889,4.818845748901367]},{"ids":[4,14,15,16,17,18,33,48,56,58,65],"top1":48,"logits":[2.386032819747925,6.494311332702637,4.902135372161865,6.628354072570801,4.746975421905518,1.7066220045089722,7.969486236572266,8.690145492553711,5.83508825302124,0.764798641204834,3.5442051887512207]},{"ids":[9,16,33,38,46,50,54,55],"top1":16,"logits":[4.850318431854248,7.803335189819336,7.371245861053467,4.313182830810547,5.849177837371826,7.013761043548584,4.7900190353393555,4.716724872589111]},{"ids":[14,18,21,26,30,33,38,46,48,56,65],"top1":33,"logits":[7.197224140167236,2.637444257736206,4.9765305519104,6.963602066040039,6.910638809204102,7.738068580627441,4.773805618286133,6.0320234298706055,7.60657262802124,5.5844011306762695,3.967656373977661]}]}
selfcheck_fp16w32.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"moves":["f5","f6","f7","g5","d3","c3","h4","h5","h6","e3","b2","d6","f3","f2","d7"],"plies":[{"ids":[23,30,41,48],"top1":41,"logits":[2.982203245162964,3.3776278495788574,3.705583333969116,3.489532470703125]},{"ids":[33,47,49],"top1":33,"logits":[1.4626374244689941,0.9479923248291016,1.3737902641296387]},{"ids":[23,30,48,57],"top1":48,"logits":[-0.6156495213508606,0.00360884889960289,2.0003933906555176,0.31513506174087524]},{"ids":[24,33,38,42,47,58],"top1":47,"logits":[1.048072099685669,1.0261688232421875,0.3870932459831238,1.030618667602539,1.120038628578186,-0.5316087007522583]},{"ids":[22,23,30,33,35,43,48,50],"top1":48,"logits":[2.0728824138641357,2.4268088340759277,2.483745574951172,4.414621353149414,2.8621606826782227,3.0016136169433594,4.763289928436279,3.9948065280914307]},{"ids":[14,22,24,38,56,58,65],"top1":24,"logits":[1.967716097831726,3.4854066371917725,4.8972086906433105,3.2392094135284424,3.922577381134033,-0.18905344605445862,2.7834980487823486]},{"ids":[13,21,30,33,35,43,48,50],"top1":30,"logits":[-0.728205144405365,2.8208260536193848,6.6298909187316895,6.347850322723389,3.0545971393585205,3.5110509395599365,6.4174604415893555,4.9727020263671875]},{"ids":[14,15,24,33,38,43,47,58,65],"top1":33,"logits":[4.341757297515869,4.808933734893799,6.563319206237793,7.07381534576416,5.122416019439697,5.341762065887451,5.790209770202637,0.8834315538406372,3.302215814590454]},{"ids":[13,21,30,33,48,50,51],"top1":51,"logits":[-0.06988537311553955,3.178204298019409,6.73562479019165,6.642114162445068,7.251765251159668,6.520535945892334,8.826519012451172]},{"ids":[14,15,24,33,38,47,56,58,65],"top1":33,"logits":[4.2811665534973145,4.552444934844971,6.37734842300415,7.082282543182373,5.06251335144043,5.809835910797119,4.704633712768555,2.3047640323638916,4.298587799072266]},{"ids":[13,15,25,33],"top1":25,"logits":[-0.38441911339759827,3.592806100845337,6.248020172119141,5.600136756896973]},{"ids":[21,30,38,46,47,48,56,65],"top1":30,"logits":[5.543796062469482,7.593250751495361,5.560909748077393,5.5322394371032715,5.632843971252441,6.682876110076904,4.527373313903809,3.869170904159546]},{"ids":[16,25,33,38,54],"top1":33,"logits":[5.4163498878479,5.98247766494751,6.603697299957275,4.791839599609375,4.81886625289917]},{"ids":[4,14,15,16,17,18,33,48,56,58,65],"top1":48,"logits":[2.3859903812408447,6.494185447692871,4.902190208435059,6.628374099731445,4.74716854095459,1.706672191619873,7.969523906707764,8.690080642700195,5.83510684967041,0.7646984457969666,3.5442638397216797]},{"ids":[9,16,33,38,46,50,54,55],"top1":16,"logits":[4.850325584411621,7.803353786468506,7.3712873458862305,4.313199520111084,5.849151134490967,7.013440132141113,4.790039539337158,4.716854572296143]},{"ids":[14,18,21,26,30,33,38,46,48,56,65],"top1":33,"logits":[7.197088718414307,2.6375179290771484,4.9764933586120605,6.963746070861816,6.91066837310791,7.738114356994629,4.77380895614624,6.031983375549316,7.606504917144775,5.5844244956970215,3.9677133560180664]}]}