ehartford commited on
Commit
913e2e4
·
verified ·
1 Parent(s): a74af1f

Model card: training on this graph with connectome-kernels

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -150,6 +150,23 @@ x = torch.zeros(1, cns.num_neurons); x[0, 0] = 1.0
150
  incoming = cns(x) # one propagation step with raw counts
151
  ```
152
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
153
  ## Provenance
154
 
155
  Built by the script [`data/fly/export_malecns_hf.py`](https://github.com/QuixiAI/FlyGPT/blob/main/data/fly/export_malecns_hf.py) from these files in
 
150
  incoming = cns(x) # one propagation step with raw counts
151
  ```
152
 
153
+ ## Training on this graph
154
+
155
+ The tensors above are the input format of [QuixiAI/connectome-kernels](https://github.com/QuixiAI/connectome-kernels),
156
+ fused CUDA kernels for training a leaky-tanh recurrence with one trainable weight per edge on a fixed sparse graph
157
+ (about 13× faster than `torch.sparse` on a 5k-neuron, 524k-edge subgraph, with identical gradients). What the edge
158
+ values are initialized to, whether they carry a neurotransmitter sign, and how they are normalized are all decisions
159
+ made at that layer, not in this repository.
160
+
161
+ ```python
162
+ from connectome_kernels import SparseGraph, sparse_recurrence
163
+
164
+ sub = cns.subgraph(cns.status_mask("Traced") & cns.subset_mask("central_brain"), min_synapses=3)
165
+ graph = SparseGraph(sub["edge_src"].long().cuda(), sub["edge_dst"].long().cuda(), sub["neuron_id"].numel(), input_nodes)
166
+ edge_values = torch.nn.Parameter(torch.randn(graph.E, device="cuda")) # yours to initialize; sub["synapse_count"] is available
167
+ out = sparse_recurrence(edge_values, leak, bias, drives, state0, graph, microsteps=2) # [T, B, N], differentiable
168
+ ```
169
+
170
  ## Provenance
171
 
172
  Built by the script [`data/fly/export_malecns_hf.py`](https://github.com/QuixiAI/FlyGPT/blob/main/data/fly/export_malecns_hf.py) from these files in