Hugging Face's logo

Spaces:
living-models
/
botanic1-report
Running

App Files Community
1
jean-livingmodels commited on
Commit
21b304c
·
verified ·
1 Parent(s): 928caec

Remove report-repository file paths from provenance notes

Browse files
Files changed (7) hide show
  1. causal-variants.html +10 -33
  2. data/ablation.json +1 -1
  3. data/causal.json +0 -0
  4. data/finetune.json +1 -1
  5. data/leaderboard.json +1 -1
  6. data/scaling.json +0 -0
  7. data/tf.json +1 -1
causal-variants.html CHANGED
@@ -8,34 +8,16 @@
8
  <link rel="preconnect" href="https://fonts.googleapis.com">
9
  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
10
  <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Manrope:wght@500;600;700;800&family=Spectral:ital,wght@0,400;0,500;0,600;1,400&family=IBM+Plex+Mono:wght@400;500;600&display=swap">
11
- <link rel="stylesheet" href="assets/site.css?v=20260908-use-case">
12
- <link rel="stylesheet" href="assets/causal.css?v=20260908-browser">
13
  <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
14
  </head>
15
  <body>
16
- <header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org"><svg class="living-models-mark" width="18" height="20" fill="currentColor" aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 85.90 91.50"><rect x="55.1000" y="2.1000" width="3.2000" height="53.2000" /><rect x="55.1000" y="61.3000" width="3.2000" height="28.1000" /><rect x="68.9000" y="9.2000" width="3.2000" height="15.5000" /><rect x="68.9000" y="30.6000" width="3.2000" height="51.7000" /><rect x="82.7000" y="24.7000" width="3.2000" height="42.2000" /><rect x="41.4000" y="0.0000" width="3.2000" height="11.6000" /><rect x="41.4000" y="17.5000" width="3.2000" height="74.0000" /><rect x="27.6000" y="2.1000" width="3.2000" height="70.7000" /><rect x="27.6000" y="78.7000" width="3.2000" height="10.6000" /><rect x="13.8000" y="9.2000" width="3.2000" height="25.0000" /><rect x="13.8000" y="40.2000" width="3.2000" height="42.2000" /><rect x="0.0000" y="24.7000" width="3.2000" height="42.2000" /></svg>Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html" aria-current="page">Causal variants</a><a href="use-case.html">Use case</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1">Models</a></nav></div></header>
17
  <div class="page">
18
 
19
  <main class="prose">
20
  <h1>Zero-shot causal variant prioritisation</h1>
21
- <p class="lede">545 loci across 14 species, each with an experimentally validated causal SNP. The task: recover that SNP among thousands of neighbouring population variants, using sequence alone.</p>
22
-
23
- <figure class="locus-browser" aria-labelledby="locus-title">
24
- <div class="locus-heading">
25
- <div><h2 id="locus-title">ABI5 locus</h2><p><i>Arabidopsis thaliana</i> · TAIR10 · Botanic1-XL</p></div>
26
- <div class="locus-tools" role="group" aria-label="Genome browser navigation">
27
- <button type="button" id="locus-out" aria-label="Zoom out">−</button>
28
- <button type="button" id="locus-in" aria-label="Zoom in">+</button>
29
- <button type="button" id="locus-full">100 kbp locus</button>
30
- <button type="button" id="locus-causal">Causal SNP</button>
31
- </div>
32
- </div>
33
- <p class="locus-region" id="locus-region">Loading locus…</p>
34
- <div class="locus-scroll"><div id="locus-plot"></div></div>
35
- <div class="locus-readout" id="locus-readout" aria-live="polite" aria-atomic="true"></div>
36
- <figcaption><span id="locus-help">Hover or tap a SNP to inspect. Drag to pan; + / − to zoom. Arrow keys select SNPs.</span><span>Measured SNP scores · 4,096 bp context · <a href="https://plants.ensembl.org/Arabidopsis_thaliana/Gene/Summary?g=AT2G36270" target="_blank" rel="noopener">Araport11 gene models ↗</a></span></figcaption>
37
- <noscript><p class="locus-fallback">Enable JavaScript to explore the ABI5 locus. <a href="https://www.biorxiv.org/content/10.64898/2026.09.04.749355v1">Read the benchmark in the report ↗</a></p></noscript>
38
- </figure>
39
 
40
  <p>Our newly introduced causal-variant benchmark uses experimentally validated functional variants as a source of ground truth, instead of relying on indirect proxies such as allele frequency or variant class. Every language model is scored at min(4,096, trained context) (PlantCaduceus-l32 and GPN are fixed at 512 bp) for each LLR, putting the SNV in the middle of the window. All gLMs are evaluated on the full dataset from the reference genome sequences only, while bioinformatics baselines require either precise annotations for Ensembl VEP or MSA support for conservation scores. As a result, PhyloP and PhastCons, which are built on PlantRegMap, are only scored on a subset of the studies.</p>
41
  <p>Although it should not necessarily be ranked first, we expect a validated causal variant to rank near the top of its locus, within the top 1% (about 60 variants) or even 0.1% (about 6 variants), which is why we choose to assess the performance on this task using the recall at these thresholds. The recalls are tie-aware: if the candidate is within a tie block in a study and the considered top fraction has to cut within the block, then the contribution of that study to the recall is neither 0 nor 1 but the proportion of the tie block that can fit in the shortlist.</p>
@@ -64,7 +46,7 @@
64
  <div class="tablewrap"><table id="sweep"></table></div>
65
 
66
  <h2 id="limits">Simplifications</h2>
67
- <p>This benchmark restricts the search to SNPs in a 100 kbp locus. It asks whether the documented causal SNP ranks near the top by absolute LLR. Surrounding unlabelled variants may also be functional or phenotypically relevant, so the benchmark measures recovery of the documented causal SNP rather than identifying every functional variant in the locus.</p>
68
 
69
  <h2 id="sec-studies">The 545 studies</h2>
70
  <p>Below are 20 example studies from the benchmark. The full set of 545 studies, candidate variants, and curation audit will be published later.</p>
@@ -79,12 +61,11 @@
79
 
80
  <footer class="site-footer">
81
  <p>Living Models, Paris. Botanic1 is released for research use under the Living Models research licence.</p>
82
- <p>Per-study ranks from <code>figures/data/causal_ranks_per_study.tsv</code>; cohort and tie-aware recall follow <code>figures/scripts/gwas_causal_common.py</code>.</p>
83
  </footer>
84
  </div>
85
  <div class="tip" id="tip" role="tooltip"></div>
86
  <script src="assets/hero.js"></script>
87
- <script src="assets/causal-browser.js?v=20260908-browser" defer></script>
88
 
89
  <script>
90
  (async function () {
@@ -196,17 +177,13 @@
196
  }
197
 
198
  // ---- study browser ----
199
- // Preview loci with substantial candidate sets, independently of model ranks.
200
  // Filter and sort this subset only; the recall curves still use the full cohort.
201
- const previewModel = MODELS.find(m => m.key === 'XL315');
202
- const eligiblePreview = S.map((s, i) => i).filter(i => previewModel.n_variants[i] >= 1000);
203
- const previewBySpecies = [...new Set(eligiblePreview.map(i => S[i].species))].sort().map(species => eligiblePreview.filter(i => S[i].species === species));
204
- const abi5 = eligiblePreview.find(i => S[i].id === 'ABI5');
205
- const previewIdx = abi5 === undefined ? [] : [abi5];
206
- const previewTarget = Math.min(20, eligiblePreview.length);
207
- for (let offset = 0; previewIdx.length < previewTarget; offset++) {
208
  for (const group of previewBySpecies) {
209
- if (group[offset] !== undefined && !previewIdx.includes(group[offset]) && previewIdx.length < previewTarget) previewIdx.push(group[offset]);
210
  }
211
  }
212
  const RANK_KEYS = ['XL315', 'L', 'PlantCAD2', 'PhyloP'];
 
8
  <link rel="preconnect" href="https://fonts.googleapis.com">
9
  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
10
  <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Manrope:wght@500;600;700;800&family=Spectral:ital,wght@0,400;0,500;0,600;1,400&family=IBM+Plex+Mono:wght@400;500;600&display=swap">
11
+ <link rel="stylesheet" href="assets/site.css?v=20260908-swatch">
 
12
  <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
13
  </head>
14
  <body>
15
+ <header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org"><svg class="living-models-mark" width="18" height="20" fill="currentColor" aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 85.90 91.50"><rect x="55.1000" y="2.1000" width="3.2000" height="53.2000" /><rect x="55.1000" y="61.3000" width="3.2000" height="28.1000" /><rect x="68.9000" y="9.2000" width="3.2000" height="15.5000" /><rect x="68.9000" y="30.6000" width="3.2000" height="51.7000" /><rect x="82.7000" y="24.7000" width="3.2000" height="42.2000" /><rect x="41.4000" y="0.0000" width="3.2000" height="11.6000" /><rect x="41.4000" y="17.5000" width="3.2000" height="74.0000" /><rect x="27.6000" y="2.1000" width="3.2000" height="70.7000" /><rect x="27.6000" y="78.7000" width="3.2000" height="10.6000" /><rect x="13.8000" y="9.2000" width="3.2000" height="25.0000" /><rect x="13.8000" y="40.2000" width="3.2000" height="42.2000" /><rect x="0.0000" y="24.7000" width="3.2000" height="42.2000" /></svg>Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html" aria-current="page">Causal variants</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1">Models</a></nav></div></header>
16
  <div class="page">
17
 
18
  <main class="prose">
19
  <h1>Zero-shot causal variant prioritisation</h1>
20
+ <p class="lede">A new and harder zero-shot causal-variant discovery benchmark based on experimentally validated variants, totalling 545 loci across 14 species. Each task ranks one documented causal SNP among population SNPs in its surrounding 100 kbp locus.</p>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  <p>Our newly introduced causal-variant benchmark uses experimentally validated functional variants as a source of ground truth, instead of relying on indirect proxies such as allele frequency or variant class. Every language model is scored at min(4,096, trained context) (PlantCaduceus-l32 and GPN are fixed at 512 bp) for each LLR, putting the SNV in the middle of the window. All gLMs are evaluated on the full dataset from the reference genome sequences only, while bioinformatics baselines require either precise annotations for Ensembl VEP or MSA support for conservation scores. As a result, PhyloP and PhastCons, which are built on PlantRegMap, are only scored on a subset of the studies.</p>
23
  <p>Although it should not necessarily be ranked first, we expect a validated causal variant to rank near the top of its locus, within the top 1% (about 60 variants) or even 0.1% (about 6 variants), which is why we choose to assess the performance on this task using the recall at these thresholds. The recalls are tie-aware: if the candidate is within a tie block in a study and the considered top fraction has to cut within the block, then the contribution of that study to the recall is neither 0 nor 1 but the proportion of the tie block that can fit in the shortlist.</p>
 
46
  <div class="tablewrap"><table id="sweep"></table></div>
47
 
48
  <h2 id="limits">Simplifications</h2>
49
+ <p>This new benchmark also uses some simplifications: the search is limited to a 100 kbp causal region and to a single basic type of mutation, SNPs. Besides, the task relies on the assumption that the documented causal SNP should be ranked among the candidate variants assigned the lowest likelihood by the model, an assumption that could be challenged since the surrounding unlabelled variants may include other functional or phenotypically relevant mutations. This limitation affects the interpretation of the absolute benchmark scores, but not the relative comparison between methods, since all methods are evaluated against the same set of unlabelled variants.</p>
50
 
51
  <h2 id="sec-studies">The 545 studies</h2>
52
  <p>Below are 20 example studies from the benchmark. The full set of 545 studies, candidate variants, and curation audit will be published later.</p>
 
61
 
62
  <footer class="site-footer">
63
  <p>Living Models, Paris. Botanic1 is released for research use under the Living Models research licence.</p>
64
+ <p>Per-study ranks, cohort and tie-aware recall follow the technical report's causal-variant analysis.</p>
65
  </footer>
66
  </div>
67
  <div class="tip" id="tip" role="tooltip"></div>
68
  <script src="assets/hero.js"></script>
 
69
 
70
  <script>
71
  (async function () {
 
177
  }
178
 
179
  // ---- study browser ----
180
+ // Fixed preview, sampled across species independently of model ranks.
181
  // Filter and sort this subset only; the recall curves still use the full cohort.
182
+ const previewBySpecies = [...new Set(S.map(s => s.species))].sort().map(species => S.map((s, i) => s.species === species ? i : -1).filter(i => i >= 0));
183
+ const previewIdx = [];
184
+ for (let offset = 0; previewIdx.length < Math.min(20, S.length); offset++) {
 
 
 
 
185
  for (const group of previewBySpecies) {
186
+ if (group[offset] !== undefined && previewIdx.length < 20) previewIdx.push(group[offset]);
187
  }
188
  }
189
  const RANK_KEYS = ['XL315', 'L', 'PlantCAD2', 'PhyloP'];
data/ablation.json CHANGED
@@ -1 +1 @@
1
- {"effects":[{"label":"Func-balanced vs uniform sampling","contexts":[{"context":"48_crop, gscoff","delta":[1.648,1.796]},{"context":"48_crop, gscon","delta":[1.418,1.423]},{"context":"48_phylo, gscoff","delta":[1.128,2.012]},{"context":"330, gscoff","delta":[0.601,0.822]},{"context":"330, gscon","delta":[1.529,0.949]}],"mean":1.333},{"label":"Redundancy filter 0.3 vs 0.9","contexts":[{"context":"330, functional_balanced","delta":[1.174,-0.295]}],"mean":0.44},{"label":"sqrt-length vs uniform weighting","contexts":[{"context":"48_phylo, uniform","delta":[0.456,0.356]},{"context":"48_phylo, functional_balanced","delta":[0.606,0.171]}],"mean":0.397},{"label":"320 species vs 48 crop","contexts":[{"context":"uniform, gscoff","delta":[1.047,0.767]},{"context":"uniform, gscon","delta":[-0.004,0.615]},{"context":"functional_balanced, gscoff","delta":[0.0,-0.207]},{"context":"functional_balanced, gscon","delta":[0.107,0.141]},{"context":"functional_heavy, gscoff","delta":[-0.752,-0.134]},{"context":"functional_heavy, gscon","delta":[0.265,0.235]}],"mean":0.173},{"label":"GSC weighting on vs off","contexts":[{"context":"48_crop, uniform","delta":[0.46,0.159]},{"context":"48_crop, functional_balanced","delta":[0.231,-0.215]},{"context":"48_crop, functional_heavy","delta":[-0.613,-0.095]},{"context":"330, uniform","delta":[-0.591,0.007]},{"context":"330, functional_balanced","delta":[0.337,0.133]},{"context":"330, functional_heavy","delta":[0.405,0.274]},{"context":"1700, uniform","delta":[0.332,-0.775]}],"mean":0.004},{"label":"N-content filter 0.20 vs 0","contexts":[{"context":"48_phylo, functional_balanced","delta":[0.483,-0.606]}],"mean":-0.061}],"cells":{"1700_uniform_gscoff":{"mean":0.7022,"s42":0.6996,"s43":0.7049},"1700_uniform_gscon":{"mean":0.7,"s42":0.7029,"s43":0.6972},"330_functional_balanced_gscoff":{"mean":0.7081,"s42":0.7068,"s43":0.7095},"330_functional_balanced_gscon":{"mean":0.7105,"s42":0.7102,"s43":0.7108},"330_functional_balanced_minhash_t0p2":{"mean":0.7101,"s42":0.7064,"s43":0.7138},"330_functional_balanced_minhash_t0p3":{"mean":0.7037,"s42":0.7046,"s43":0.7029},"330_functional_balanced_minhash_t0p9":{"mean":0.6994,"s42":0.6929,"s43":0.7058},"330_functional_heavy_gscoff":{"mean":0.7097,"s42":0.708,"s43":0.7113},"330_functional_heavy_gscon":{"mean":0.713,"s42":0.7121,"s43":0.714},"330_uniform_gscoff":{"mean":0.701,"s42":0.7008,"s43":0.7013},"330_uniform_gscon":{"mean":0.6981,"s42":0.6949,"s43":0.7013},"48_crop_functional_balanced_gscoff":{"mean":0.7092,"s42":0.7068,"s43":0.7115},"48_crop_functional_balanced_gscon":{"mean":0.7093,"s42":0.7091,"s43":0.7094},"48_crop_functional_heavy_gscoff":{"mean":0.7141,"s42":0.7156,"s43":0.7126},"48_crop_functional_heavy_gscon":{"mean":0.7105,"s42":0.7094,"s43":0.7117},"48_crop_uniform_gscoff":{"mean":0.692,"s42":0.6903,"s43":0.6936},"48_crop_uniform_gscon":{"mean":0.6951,"s42":0.6949,"s43":0.6952},"48_phylo_functional_balanced_gscoff":{"mean":0.7052,"s42":0.7023,"s43":0.7081},"48_phylo_functional_balanced_nfilter_n0p00":{"mean":0.703,"s42":0.6983,"s43":0.7077},"48_phylo_functional_balanced_nfilter_n0p05":{"mean":0.7038,"s42":0.698,"s43":0.7095},"48_phylo_functional_balanced_nfilter_n0p20":{"mean":0.7024,"s42":0.7032,"s43":0.7016},"48_phylo_functional_balanced_nfilter_off":{"mean":0.7041,"s42":0.6987,"s43":0.7095},"48_phylo_functional_balanced_winstrat_sqrt_length":{"mean":0.7033,"s42":0.7035,"s43":0.7031},"48_phylo_functional_balanced_winstrat_uniform":{"mean":0.6994,"s42":0.6974,"s43":0.7014},"48_phylo_functional_heavy_gscoff":{"mean":0.7065,"s42":0.7038,"s43":0.7092},"48_phylo_uniform_gscoff":{"mean":0.6895,"s42":0.691,"s43":0.688},"48_phylo_uniform_winstrat_sqrt_length":{"mean":0.6961,"s42":0.6957,"s43":0.6966},"48_phylo_uniform_winstrat_uniform":{"mean":0.6921,"s42":0.6911,"s43":0.6931},"brassicales_48_with_at":{"mean":0.695,"s42":0.6945,"s43":0.6955},"brassicales_48_without_at":{"mean":0.6954,"s42":0.6915,"s43":0.6994},"global_48_with_at":{"mean":0.6907,"s42":0.6913,"s43":0.6902},"global_48_without_at":{"mean":0.6908,"s42":0.6881,"s43":0.6934}},"note":"figures/data/phase2_two_seed.json; effect = adopted setting minus alternative in S_bal^train percentage points, two seeds, iso-step, iso-token, iso-effective-epoch."}
 
1
+ {"effects":[{"label":"Func-balanced vs uniform sampling","contexts":[{"context":"48_crop, gscoff","delta":[1.648,1.796]},{"context":"48_crop, gscon","delta":[1.418,1.423]},{"context":"48_phylo, gscoff","delta":[1.128,2.012]},{"context":"330, gscoff","delta":[0.601,0.822]},{"context":"330, gscon","delta":[1.529,0.949]}],"mean":1.333},{"label":"Redundancy filter 0.3 vs 0.9","contexts":[{"context":"330, functional_balanced","delta":[1.174,-0.295]}],"mean":0.44},{"label":"sqrt-length vs uniform weighting","contexts":[{"context":"48_phylo, uniform","delta":[0.456,0.356]},{"context":"48_phylo, functional_balanced","delta":[0.606,0.171]}],"mean":0.397},{"label":"320 species vs 48 crop","contexts":[{"context":"uniform, gscoff","delta":[1.047,0.767]},{"context":"uniform, gscon","delta":[-0.004,0.615]},{"context":"functional_balanced, gscoff","delta":[0.0,-0.207]},{"context":"functional_balanced, gscon","delta":[0.107,0.141]},{"context":"functional_heavy, gscoff","delta":[-0.752,-0.134]},{"context":"functional_heavy, gscon","delta":[0.265,0.235]}],"mean":0.173},{"label":"GSC weighting on vs off","contexts":[{"context":"48_crop, uniform","delta":[0.46,0.159]},{"context":"48_crop, functional_balanced","delta":[0.231,-0.215]},{"context":"48_crop, functional_heavy","delta":[-0.613,-0.095]},{"context":"330, uniform","delta":[-0.591,0.007]},{"context":"330, functional_balanced","delta":[0.337,0.133]},{"context":"330, functional_heavy","delta":[0.405,0.274]},{"context":"1700, uniform","delta":[0.332,-0.775]}],"mean":0.004},{"label":"N-content filter 0.20 vs 0","contexts":[{"context":"48_phylo, functional_balanced","delta":[0.483,-0.606]}],"mean":-0.061}],"cells":{"1700_uniform_gscoff":{"mean":0.7022,"s42":0.6996,"s43":0.7049},"1700_uniform_gscon":{"mean":0.7,"s42":0.7029,"s43":0.6972},"330_functional_balanced_gscoff":{"mean":0.7081,"s42":0.7068,"s43":0.7095},"330_functional_balanced_gscon":{"mean":0.7105,"s42":0.7102,"s43":0.7108},"330_functional_balanced_minhash_t0p2":{"mean":0.7101,"s42":0.7064,"s43":0.7138},"330_functional_balanced_minhash_t0p3":{"mean":0.7037,"s42":0.7046,"s43":0.7029},"330_functional_balanced_minhash_t0p9":{"mean":0.6994,"s42":0.6929,"s43":0.7058},"330_functional_heavy_gscoff":{"mean":0.7097,"s42":0.708,"s43":0.7113},"330_functional_heavy_gscon":{"mean":0.713,"s42":0.7121,"s43":0.714},"330_uniform_gscoff":{"mean":0.701,"s42":0.7008,"s43":0.7013},"330_uniform_gscon":{"mean":0.6981,"s42":0.6949,"s43":0.7013},"48_crop_functional_balanced_gscoff":{"mean":0.7092,"s42":0.7068,"s43":0.7115},"48_crop_functional_balanced_gscon":{"mean":0.7093,"s42":0.7091,"s43":0.7094},"48_crop_functional_heavy_gscoff":{"mean":0.7141,"s42":0.7156,"s43":0.7126},"48_crop_functional_heavy_gscon":{"mean":0.7105,"s42":0.7094,"s43":0.7117},"48_crop_uniform_gscoff":{"mean":0.692,"s42":0.6903,"s43":0.6936},"48_crop_uniform_gscon":{"mean":0.6951,"s42":0.6949,"s43":0.6952},"48_phylo_functional_balanced_gscoff":{"mean":0.7052,"s42":0.7023,"s43":0.7081},"48_phylo_functional_balanced_nfilter_n0p00":{"mean":0.703,"s42":0.6983,"s43":0.7077},"48_phylo_functional_balanced_nfilter_n0p05":{"mean":0.7038,"s42":0.698,"s43":0.7095},"48_phylo_functional_balanced_nfilter_n0p20":{"mean":0.7024,"s42":0.7032,"s43":0.7016},"48_phylo_functional_balanced_nfilter_off":{"mean":0.7041,"s42":0.6987,"s43":0.7095},"48_phylo_functional_balanced_winstrat_sqrt_length":{"mean":0.7033,"s42":0.7035,"s43":0.7031},"48_phylo_functional_balanced_winstrat_uniform":{"mean":0.6994,"s42":0.6974,"s43":0.7014},"48_phylo_functional_heavy_gscoff":{"mean":0.7065,"s42":0.7038,"s43":0.7092},"48_phylo_uniform_gscoff":{"mean":0.6895,"s42":0.691,"s43":0.688},"48_phylo_uniform_winstrat_sqrt_length":{"mean":0.6961,"s42":0.6957,"s43":0.6966},"48_phylo_uniform_winstrat_uniform":{"mean":0.6921,"s42":0.6911,"s43":0.6931},"brassicales_48_with_at":{"mean":0.695,"s42":0.6945,"s43":0.6955},"brassicales_48_without_at":{"mean":0.6954,"s42":0.6915,"s43":0.6994},"global_48_with_at":{"mean":0.6907,"s42":0.6913,"s43":0.6902},"global_48_without_at":{"mean":0.6908,"s42":0.6881,"s43":0.6934}},"note":"Effect = adopted setting minus alternative in S_bal^train percentage points, two seeds, iso-step, iso-token, iso-effective-epoch."}
data/causal.json CHANGED
The diff for this file is too large to render. See raw diff
 
data/finetune.json CHANGED
@@ -1 +1 @@
1
- {"cells":[{"model":"AgroNT","task":"lncrna","method":"Full fine-tuning","auroc_species":{"s0":0.789,"s1":0.806,"s2":0.83,"s3":0.82,"s4":0.826,"s5":0.84},"auroc_mean":0.819,"auroc_mean_per_species":0.819},{"model":"AgroNT","task":"lncrna","method":"IA3","auroc_species":{"s1":0.777,"s2":0.749,"s3":0.756,"s4":0.776,"s5":0.869},"auroc_mean":0.785,"auroc_mean_per_species":0.783},{"model":"AgroNT","task":"lncrna","method":"LoRA","auroc_species":{"s0":0.794,"s1":0.805,"s2":0.82,"s3":0.763,"s4":0.812,"s5":0.862},"auroc_mean":0.809,"auroc_mean_per_species":0.8},{"model":"AgroNT","task":"poly_a","method":"Full fine-tuning","auroc_species":{"s0":0.911,"s1":0.967,"s2":0.898,"s3":0.954,"s4":0.948,"s5":0.968},"auroc_mean":0.941,"auroc_mean_per_species":0.941},{"model":"AgroNT","task":"poly_a","method":"IA3","auroc_species":{"s1":0.969,"s2":0.907,"s3":0.483,"s4":0.949,"s5":0.971},"auroc_mean":0.856,"auroc_mean_per_species":0.935},{"model":"AgroNT","task":"poly_a","method":"LoRA","auroc_species":{"s0":0.918,"s1":0.946,"s2":0.901,"s3":0.954,"s4":0.938,"s5":0.898},"auroc_mean":0.926,"auroc_mean_per_species":0.946},{"model":"AgroNT","task":"promoter","method":"Full fine-tuning","pearson_tobacco":0.864,"pearson_maize":0.831,"r2_tobacco":0.744,"r2_maize":0.648},{"model":"AgroNT","task":"promoter","method":"IA3","pearson_tobacco":0.843,"pearson_maize":0.816,"r2_tobacco":0.68,"r2_maize":0.573},{"model":"AgroNT","task":"promoter","method":"LoRA","pearson_tobacco":0.844,"pearson_maize":0.825,"r2_tobacco":0.654,"r2_maize":0.668},{"model":"AgroNT","task":"terminator","method":"Full fine-tuning","pearson_merged":0.834,"pearson_tobacco":0.865,"pearson_maize":0.803,"r2_merged":0.693,"r2_tobacco":0.742,"r2_maize":0.643},{"model":"AgroNT","task":"terminator","method":"IA3","pearson_merged":0.808,"pearson_tobacco":0.846,"pearson_maize":0.77,"r2_merged":0.583,"r2_tobacco":0.585,"r2_maize":0.58},{"model":"AgroNT","task":"terminator","method":"LoRA","pearson_merged":0.809,"pearson_tobacco":0.848,"pearson_maize":0.769,"r2_merged":0.626,"r2_tobacco":0.71,"r2_maize":0.542},{"model":"Botanic1-S","task":"lncrna","method":"Full fine-tuning","auroc_species":{"s0":0.831,"s1":0.854,"s2":0.85,"s3":0.814,"s4":0.833,"s5":0.906},"auroc_mean":0.848,"auroc_mean_per_species":0.842},{"model":"Botanic1-S","task":"lncrna","method":"IA3","auroc_species":{"s0":0.843,"s1":0.821,"s2":0.817,"s3":0.814,"s4":0.79,"s5":0.882},"auroc_mean":0.828,"auroc_mean_per_species":0.827},{"model":"Botanic1-S","task":"lncrna","method":"LoRA","auroc_species":{"s0":0.503,"s1":0.599,"s2":0.832,"s3":0.801,"s4":0.499,"s5":0.877},"auroc_mean":0.685,"auroc_mean_per_species":0.783},{"model":"Botanic1-S","task":"poly_a","method":"Full fine-tuning","auroc_species":{"s0":0.926,"s1":0.979,"s2":0.924,"s3":0.968,"s4":0.962,"s5":0.977},"auroc_mean":0.956,"auroc_mean_per_species":0.962},{"model":"Botanic1-S","task":"poly_a","method":"IA3","auroc_species":{"s0":0.939,"s1":0.981,"s2":0.933,"s3":0.962,"s4":0.956,"s5":0.98},"auroc_mean":0.958,"auroc_mean_per_species":0.959},{"model":"Botanic1-S","task":"poly_a","method":"LoRA","auroc_species":{"s0":0.934,"s1":0.981,"s2":0.914,"s3":0.967,"s4":0.963,"s5":0.98},"auroc_mean":0.956,"auroc_mean_per_species":0.961},{"model":"Botanic1-S","task":"promoter","method":"Full fine-tuning","pearson_tobacco":0.896,"pearson_maize":0.874,"r2_tobacco":0.8,"r2_maize":0.76},{"model":"Botanic1-S","task":"promoter","method":"IA3","pearson_tobacco":0.882,"pearson_maize":0.862,"r2_tobacco":0.663,"r2_maize":0.72},{"model":"Botanic1-S","task":"promoter","method":"LoRA","pearson_tobacco":0.9,"pearson_maize":0.872,"r2_tobacco":0.783,"r2_maize":0.689},{"model":"Botanic1-S","task":"terminator","method":"Full fine-tuning","pearson_merged":0.861,"pearson_tobacco":0.882,"pearson_maize":0.839,"r2_merged":0.736,"r2_tobacco":0.773,"r2_maize":0.698},{"model":"Botanic1-S","task":"terminator","method":"IA3","pearson_merged":0.839,"pearson_tobacco":0.883,"pearson_maize":0.794,"r2_merged":0.588,"r2_tobacco":0.702,"r2_maize":0.474},{"model":"Botanic1-S","task":"terminator","method":"LoRA","pearson_merged":0.856,"pearson_tobacco":0.889,"pearson_maize":0.822,"r2_merged":0.717,"r2_tobacco":0.783,"r2_maize":0.65},{"model":"PlantCAD2-L","task":"lncrna","method":"Full fine-tuning","auroc_species":{"s0":0.51,"s1":0.559,"s2":0.467,"s3":0.547,"s4":0.627,"s5":0.464},"auroc_mean":0.529,"auroc_mean_per_species":0.704},{"model":"PlantCAD2-L","task":"lncrna","method":"IA3","auroc_species":{"s0":0.834,"s1":0.779,"s2":0.628,"s3":0.483,"s4":0.811,"s5":0.467},"auroc_mean":0.667,"auroc_mean_per_species":0.795},{"model":"PlantCAD2-L","task":"lncrna","method":"LoRA","auroc_species":{"s0":0.525,"s2":0.429,"s3":0.514,"s4":0.538,"s5":0.854},"auroc_mean":0.572,"auroc_mean_per_species":0.769},{"model":"PlantCAD2-L","task":"poly_a","method":"Full fine-tuning","auroc_species":{"s0":0.914,"s1":0.97,"s2":0.917,"s3":0.95,"s4":0.499,"s5":0.969},"auroc_mean":0.87,"auroc_mean_per_species":0.943},{"model":"PlantCAD2-L","task":"poly_a","method":"IA3","auroc_species":{"s0":0.801,"s1":0.968,"s2":0.932,"s3":0.952,"s4":0.955,"s5":0.877},"auroc_mean":0.914,"auroc_mean_per_species":0.878},{"model":"PlantCAD2-L","task":"poly_a","method":"LoRA","auroc_species":{"s0":0.926,"s1":0.976,"s2":0.934,"s3":0.957,"s4":0.5,"s5":0.973},"auroc_mean":0.878,"auroc_mean_per_species":0.952},{"model":"PlantCAD2-L","task":"promoter","method":"Full fine-tuning","pearson_tobacco":0.853,"pearson_maize":0.831,"r2_tobacco":0.717,"r2_maize":0.684},{"model":"PlantCAD2-L","task":"promoter","method":"IA3","pearson_tobacco":0.837,"pearson_maize":0.833,"r2_tobacco":0.651,"r2_maize":0.687},{"model":"PlantCAD2-L","task":"promoter","method":"LoRA","pearson_tobacco":0.864,"pearson_maize":0.839,"r2_tobacco":0.74,"r2_maize":0.691},{"model":"PlantCAD2-L","task":"terminator","method":"Full fine-tuning","pearson_merged":0.79,"pearson_tobacco":0.82,"pearson_maize":0.76,"r2_merged":0.624,"r2_tobacco":0.671,"r2_maize":0.576},{"model":"PlantCAD2-L","task":"terminator","method":"IA3","pearson_merged":0.812,"pearson_tobacco":0.845,"pearson_maize":0.779,"r2_merged":0.624,"r2_tobacco":0.648,"r2_maize":0.6},{"model":"PlantCAD2-L","task":"terminator","method":"LoRA","pearson_merged":0.78,"pearson_tobacco":0.828,"pearson_maize":0.731,"r2_merged":0.59,"r2_tobacco":0.679,"r2_maize":0.501}],"baselines":[{"model":"gorjifard_densenet","task":"terminator","subtask":"Maize","metric":"pearson","value":0.82},{"model":"gorjifard_densenet","task":"terminator","subtask":"Maize","metric":"r2","value":0.67},{"model":"gorjifard_densenet","task":"terminator","subtask":"Tobacco","metric":"pearson","value":0.87},{"model":"gorjifard_densenet","task":"terminator","subtask":"Tobacco","metric":"r2","value":0.76},{"model":"gorjifard_densenet","task":"terminator","subtask":"merged","metric":"pearson","value":0.85},{"model":"gorjifard_densenet","task":"terminator","subtask":"merged","metric":"r2","value":0.72},{"model":"jores_cnn","task":"promoter","subtask":"Maize","metric":"pearson","value":0.82},{"model":"jores_cnn","task":"promoter","subtask":"Maize","metric":"r2","value":0.67},{"model":"jores_cnn","task":"promoter","subtask":"Tobacco","metric":"pearson","value":0.85},{"model":"jores_cnn","task":"promoter","subtask":"Tobacco","metric":"r2","value":0.71}],"note":"figures/data/botanic05_finetune_results.csv (grid_best = global tuning) and _per_species.csv; classification cells are the mean over six species."}
 
1
+ {"cells":[{"model":"AgroNT","task":"lncrna","method":"Full fine-tuning","auroc_species":{"s0":0.789,"s1":0.806,"s2":0.83,"s3":0.82,"s4":0.826,"s5":0.84},"auroc_mean":0.819,"auroc_mean_per_species":0.819},{"model":"AgroNT","task":"lncrna","method":"IA3","auroc_species":{"s1":0.777,"s2":0.749,"s3":0.756,"s4":0.776,"s5":0.869},"auroc_mean":0.785,"auroc_mean_per_species":0.783},{"model":"AgroNT","task":"lncrna","method":"LoRA","auroc_species":{"s0":0.794,"s1":0.805,"s2":0.82,"s3":0.763,"s4":0.812,"s5":0.862},"auroc_mean":0.809,"auroc_mean_per_species":0.8},{"model":"AgroNT","task":"poly_a","method":"Full fine-tuning","auroc_species":{"s0":0.911,"s1":0.967,"s2":0.898,"s3":0.954,"s4":0.948,"s5":0.968},"auroc_mean":0.941,"auroc_mean_per_species":0.941},{"model":"AgroNT","task":"poly_a","method":"IA3","auroc_species":{"s1":0.969,"s2":0.907,"s3":0.483,"s4":0.949,"s5":0.971},"auroc_mean":0.856,"auroc_mean_per_species":0.935},{"model":"AgroNT","task":"poly_a","method":"LoRA","auroc_species":{"s0":0.918,"s1":0.946,"s2":0.901,"s3":0.954,"s4":0.938,"s5":0.898},"auroc_mean":0.926,"auroc_mean_per_species":0.946},{"model":"AgroNT","task":"promoter","method":"Full fine-tuning","pearson_tobacco":0.864,"pearson_maize":0.831,"r2_tobacco":0.744,"r2_maize":0.648},{"model":"AgroNT","task":"promoter","method":"IA3","pearson_tobacco":0.843,"pearson_maize":0.816,"r2_tobacco":0.68,"r2_maize":0.573},{"model":"AgroNT","task":"promoter","method":"LoRA","pearson_tobacco":0.844,"pearson_maize":0.825,"r2_tobacco":0.654,"r2_maize":0.668},{"model":"AgroNT","task":"terminator","method":"Full fine-tuning","pearson_merged":0.834,"pearson_tobacco":0.865,"pearson_maize":0.803,"r2_merged":0.693,"r2_tobacco":0.742,"r2_maize":0.643},{"model":"AgroNT","task":"terminator","method":"IA3","pearson_merged":0.808,"pearson_tobacco":0.846,"pearson_maize":0.77,"r2_merged":0.583,"r2_tobacco":0.585,"r2_maize":0.58},{"model":"AgroNT","task":"terminator","method":"LoRA","pearson_merged":0.809,"pearson_tobacco":0.848,"pearson_maize":0.769,"r2_merged":0.626,"r2_tobacco":0.71,"r2_maize":0.542},{"model":"Botanic1-S","task":"lncrna","method":"Full fine-tuning","auroc_species":{"s0":0.831,"s1":0.854,"s2":0.85,"s3":0.814,"s4":0.833,"s5":0.906},"auroc_mean":0.848,"auroc_mean_per_species":0.842},{"model":"Botanic1-S","task":"lncrna","method":"IA3","auroc_species":{"s0":0.843,"s1":0.821,"s2":0.817,"s3":0.814,"s4":0.79,"s5":0.882},"auroc_mean":0.828,"auroc_mean_per_species":0.827},{"model":"Botanic1-S","task":"lncrna","method":"LoRA","auroc_species":{"s0":0.503,"s1":0.599,"s2":0.832,"s3":0.801,"s4":0.499,"s5":0.877},"auroc_mean":0.685,"auroc_mean_per_species":0.783},{"model":"Botanic1-S","task":"poly_a","method":"Full fine-tuning","auroc_species":{"s0":0.926,"s1":0.979,"s2":0.924,"s3":0.968,"s4":0.962,"s5":0.977},"auroc_mean":0.956,"auroc_mean_per_species":0.962},{"model":"Botanic1-S","task":"poly_a","method":"IA3","auroc_species":{"s0":0.939,"s1":0.981,"s2":0.933,"s3":0.962,"s4":0.956,"s5":0.98},"auroc_mean":0.958,"auroc_mean_per_species":0.959},{"model":"Botanic1-S","task":"poly_a","method":"LoRA","auroc_species":{"s0":0.934,"s1":0.981,"s2":0.914,"s3":0.967,"s4":0.963,"s5":0.98},"auroc_mean":0.956,"auroc_mean_per_species":0.961},{"model":"Botanic1-S","task":"promoter","method":"Full fine-tuning","pearson_tobacco":0.896,"pearson_maize":0.874,"r2_tobacco":0.8,"r2_maize":0.76},{"model":"Botanic1-S","task":"promoter","method":"IA3","pearson_tobacco":0.882,"pearson_maize":0.862,"r2_tobacco":0.663,"r2_maize":0.72},{"model":"Botanic1-S","task":"promoter","method":"LoRA","pearson_tobacco":0.9,"pearson_maize":0.872,"r2_tobacco":0.783,"r2_maize":0.689},{"model":"Botanic1-S","task":"terminator","method":"Full fine-tuning","pearson_merged":0.861,"pearson_tobacco":0.882,"pearson_maize":0.839,"r2_merged":0.736,"r2_tobacco":0.773,"r2_maize":0.698},{"model":"Botanic1-S","task":"terminator","method":"IA3","pearson_merged":0.839,"pearson_tobacco":0.883,"pearson_maize":0.794,"r2_merged":0.588,"r2_tobacco":0.702,"r2_maize":0.474},{"model":"Botanic1-S","task":"terminator","method":"LoRA","pearson_merged":0.856,"pearson_tobacco":0.889,"pearson_maize":0.822,"r2_merged":0.717,"r2_tobacco":0.783,"r2_maize":0.65},{"model":"PlantCAD2-L","task":"lncrna","method":"Full fine-tuning","auroc_species":{"s0":0.51,"s1":0.559,"s2":0.467,"s3":0.547,"s4":0.627,"s5":0.464},"auroc_mean":0.529,"auroc_mean_per_species":0.704},{"model":"PlantCAD2-L","task":"lncrna","method":"IA3","auroc_species":{"s0":0.834,"s1":0.779,"s2":0.628,"s3":0.483,"s4":0.811,"s5":0.467},"auroc_mean":0.667,"auroc_mean_per_species":0.795},{"model":"PlantCAD2-L","task":"lncrna","method":"LoRA","auroc_species":{"s0":0.525,"s2":0.429,"s3":0.514,"s4":0.538,"s5":0.854},"auroc_mean":0.572,"auroc_mean_per_species":0.769},{"model":"PlantCAD2-L","task":"poly_a","method":"Full fine-tuning","auroc_species":{"s0":0.914,"s1":0.97,"s2":0.917,"s3":0.95,"s4":0.499,"s5":0.969},"auroc_mean":0.87,"auroc_mean_per_species":0.943},{"model":"PlantCAD2-L","task":"poly_a","method":"IA3","auroc_species":{"s0":0.801,"s1":0.968,"s2":0.932,"s3":0.952,"s4":0.955,"s5":0.877},"auroc_mean":0.914,"auroc_mean_per_species":0.878},{"model":"PlantCAD2-L","task":"poly_a","method":"LoRA","auroc_species":{"s0":0.926,"s1":0.976,"s2":0.934,"s3":0.957,"s4":0.5,"s5":0.973},"auroc_mean":0.878,"auroc_mean_per_species":0.952},{"model":"PlantCAD2-L","task":"promoter","method":"Full fine-tuning","pearson_tobacco":0.853,"pearson_maize":0.831,"r2_tobacco":0.717,"r2_maize":0.684},{"model":"PlantCAD2-L","task":"promoter","method":"IA3","pearson_tobacco":0.837,"pearson_maize":0.833,"r2_tobacco":0.651,"r2_maize":0.687},{"model":"PlantCAD2-L","task":"promoter","method":"LoRA","pearson_tobacco":0.864,"pearson_maize":0.839,"r2_tobacco":0.74,"r2_maize":0.691},{"model":"PlantCAD2-L","task":"terminator","method":"Full fine-tuning","pearson_merged":0.79,"pearson_tobacco":0.82,"pearson_maize":0.76,"r2_merged":0.624,"r2_tobacco":0.671,"r2_maize":0.576},{"model":"PlantCAD2-L","task":"terminator","method":"IA3","pearson_merged":0.812,"pearson_tobacco":0.845,"pearson_maize":0.779,"r2_merged":0.624,"r2_tobacco":0.648,"r2_maize":0.6},{"model":"PlantCAD2-L","task":"terminator","method":"LoRA","pearson_merged":0.78,"pearson_tobacco":0.828,"pearson_maize":0.731,"r2_merged":0.59,"r2_tobacco":0.679,"r2_maize":0.501}],"baselines":[{"model":"gorjifard_densenet","task":"terminator","subtask":"Maize","metric":"pearson","value":0.82},{"model":"gorjifard_densenet","task":"terminator","subtask":"Maize","metric":"r2","value":0.67},{"model":"gorjifard_densenet","task":"terminator","subtask":"Tobacco","metric":"pearson","value":0.87},{"model":"gorjifard_densenet","task":"terminator","subtask":"Tobacco","metric":"r2","value":0.76},{"model":"gorjifard_densenet","task":"terminator","subtask":"merged","metric":"pearson","value":0.85},{"model":"gorjifard_densenet","task":"terminator","subtask":"merged","metric":"r2","value":0.72},{"model":"jores_cnn","task":"promoter","subtask":"Maize","metric":"pearson","value":0.82},{"model":"jores_cnn","task":"promoter","subtask":"Maize","metric":"r2","value":0.67},{"model":"jores_cnn","task":"promoter","subtask":"Tobacco","metric":"pearson","value":0.85},{"model":"jores_cnn","task":"promoter","subtask":"Tobacco","metric":"r2","value":0.71}],"note":"Fine-tuning results (grid_best = global tuning) and per-species results; classification cells are the mean over six species."}
data/leaderboard.json CHANGED
@@ -1 +1 @@
1
- {"families":[{"key":"chromatin","name":"Chromatin accessibility","metric":"macro-AUPRC","n_species":7},{"key":"grc","name":"Genomic region","metric":"balanced acc.","n_species":4},{"key":"conservation","name":"Conservation","metric":"AUPRC","n_species":2},{"key":"splicing","name":"Splicing","metric":"strat. AUPRC","n_species":2},{"key":"tis","name":"TIS","metric":"AUPRC","n_species":1},{"key":"tts","name":"TTS","metric":"AUPRC","n_species":1},{"key":"proseq","name":"PRO-seq","metric":"AUPRC","n_species":1},{"key":"llr","name":"LLR mutation effect","metric":"AUROC","n_species":3},{"key":"causal","name":"Causal variant discovery","metric":"recall-AUC","n_species":1}],"models":[{"id":"botanic1-XL","name":"Botanic1-XL","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":3180000000.0,"params_str":"3.18B","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7695,"families":{"chromatin":0.4764,"grc":0.527,"conservation":0.8126,"splicing":0.98,"tis":0.9087,"tts":0.8874,"proseq":0.8612,"llr":0.7206,"causal":0.7516},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7516},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4319},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.4093},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4575},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4599},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4361},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.575},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5648},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.767},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8581},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6547},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4847},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4815},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4872},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7527},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7291},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6801},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8612},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9782},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9817},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9087},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8874}]},{"id":"botanic1-L","name":"Botanic1-L","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":2109999999.9999998,"params_str":"2.11B","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7679,"families":{"chromatin":0.4813,"grc":0.5233,"conservation":0.813,"splicing":0.9818,"tis":0.9116,"tts":0.8711,"proseq":0.8685,"llr":0.7176,"causal":0.743},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.743},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4311},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.4005},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4688},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4739},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4441},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5764},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5745},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7695},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8565},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6568},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4701},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4779},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4886},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7463},{"label":"LLR \u00b7 maize","fam":"llr","value":0.722},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6845},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8685},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9799},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9837},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9116},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8711}]},{"id":"botanic1-M","name":"Botanic1-M","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":688000000.0,"params_str":"688M","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7652,"families":{"chromatin":0.4731,"grc":0.5236,"conservation":0.8068,"splicing":0.9836,"tis":0.9088,"tts":0.8808,"proseq":0.8609,"llr":0.7126,"causal":0.7366},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7366},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4208},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.4042},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4817},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4566},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4261},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5601},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5624},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.762},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8516},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6537},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4757},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4848},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4803},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7217},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7337},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6825},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8609},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.982},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9853},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9088},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8808}]},{"id":"botanic1-S","name":"Botanic1-S","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":318000000.0,"params_str":"318M","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7578,"families":{"chromatin":0.4666,"grc":0.5243,"conservation":0.7939,"splicing":0.9819,"tis":0.9052,"tts":0.8754,"proseq":0.8588,"llr":0.7115,"causal":0.7024},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7024},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4173},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3954},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4576},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4484},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.419},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.561},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5676},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7479},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8399},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6385},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4679},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4921},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4985},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7241},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7227},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6875},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8588},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9792},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9846},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9052},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8754}]},{"id":"PlantCAD2-L","name":"PlantCAD2-L","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":694000000.0,"params_str":"694M","logo":"cornell-seal.png","in_fig1":true,"tokens_seen":4030000000000.001,"bp_equivalent":"4.03T","s_bal":0.7556,"families":{"chromatin":0.4834,"grc":0.5251,"conservation":0.7955,"splicing":0.9775,"tis":0.8893,"tts":0.8817,"proseq":0.8669,"llr":0.7038,"causal":0.677},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.677},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4286},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.41},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4759},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4806},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4362},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5745},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5782},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7422},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8489},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6636},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4504},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4941},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4925},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7012},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7263},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6839},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8669},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9758},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9793},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8893},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8817}]},{"id":"PlantCaduceus","name":"PlantCaduceus-l32","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":225000000.0,"params_str":"225M","logo":"cornell-seal.png","in_fig1":true,"tokens_seen":251700000000.0,"bp_equivalent":"251.7B","s_bal":0.7521,"families":{"chromatin":0.4573,"grc":0.5236,"conservation":0.7937,"splicing":0.9781,"tis":0.8965,"tts":0.863,"proseq":0.8177,"llr":0.7132,"causal":0.7256},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7256},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3992},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.38},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4406},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4466},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4217},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5527},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5605},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7548},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8326},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6746},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4677},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4867},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4655},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7431},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7169},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6795},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8177},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9715},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9846},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8965},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.863}]},{"id":"PlantCAD2-S","name":"PlantCAD2-S","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":88000000.0,"params_str":"88M","logo":"cornell-seal.png","in_fig1":false,"tokens_seen":4030000000000.001,"bp_equivalent":"4.03T","s_bal":0.742,"families":{"chromatin":0.4732,"grc":0.5085,"conservation":0.7698,"splicing":0.9796,"tis":0.891,"tts":0.8632,"proseq":0.8574,"llr":0.6895,"causal":0.6458},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.6458},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4271},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.4073},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4766},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4544},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4198},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5591},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5681},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7188},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8208},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6351},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4491},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.485},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4649},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6902},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7091},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6694},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8574},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9783},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9809},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.891},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8632}]},{"id":"PlantCAD2-M","name":"PlantCAD2-M","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":311000000.0,"params_str":"311M","logo":"cornell-seal.png","in_fig1":false,"tokens_seen":4030000000000.001,"bp_equivalent":"4.03T","s_bal":0.7331,"families":{"chromatin":0.4201,"grc":0.4885,"conservation":0.7508,"splicing":0.9763,"tis":0.8955,"tts":0.8571,"proseq":0.8387,"llr":0.6933,"causal":0.6777},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.6777},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3582},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3565},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3777},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4084},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4006},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5163},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5232},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7035},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.798},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6312},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4092},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4693},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4441},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.686},{"label":"LLR \u00b7 maize","fam":"llr","value":0.707},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6869},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8387},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.969},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9836},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8955},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8571}]},{"id":"GPN","name":"GPN","org":"UC Berkeley","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":66000000.0,"params_str":"66M","logo":"berkeley.png","in_fig1":true,"tokens_seen":157300000000.0,"bp_equivalent":"157.3B","s_bal":0.7314,"families":{"chromatin":0.3796,"grc":0.4975,"conservation":0.7286,"splicing":0.9786,"tis":0.9014,"tts":0.8877,"proseq":0.8456,"llr":0.7093,"causal":0.6539},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.6539},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.422},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3129},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.2724},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.3632},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3608},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.4753},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.4506},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6958},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7614},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.675},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3945},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4526},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4679},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7153},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7352},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6773},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8456},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9773},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9799},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9014},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8877}]},{"id":"CARBON-3B","name":"Carbon-3B","org":"Hugging Face","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":3450000000.0,"params_str":"3.45B","logo":"huggingface.png","in_fig1":true,"tokens_seen":1050000000000.0,"bp_equivalent":"6.30T","s_bal":0.731,"families":{"chromatin":0.425,"grc":0.4637,"conservation":0.7635,"splicing":0.9513,"tis":0.9012,"tts":0.869,"proseq":0.8298,"llr":0.6936,"causal":0.6816},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.6816},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.4005},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3468},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3689},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4253},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4016},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5154},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5162},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7159},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8112},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5949},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3851},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4322},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4425},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6833},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7283},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6692},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8298},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9476},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.955},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9012},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.869}]},{"id":"CARBON-8B","name":"Carbon-8B","org":"Hugging Face","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":8260000000.0,"params_str":"8.26B","logo":"huggingface.png","in_fig1":true,"tokens_seen":1050000000000.0,"bp_equivalent":"6.30T","s_bal":0.7272,"families":{"chromatin":0.416,"grc":0.4441,"conservation":0.7583,"splicing":0.9323,"tis":0.8962,"tts":0.869,"proseq":0.8141,"llr":0.6941,"causal":0.7207},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7207},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3898},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.328},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3419},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4235},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3864},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5162},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5266},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7247},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7918},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5892},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3335},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4266},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4271},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6979},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7163},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6682},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8141},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9253},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9394},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8962},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.869}]},{"id":"NTv3-pre","name":"NTv3-650M-pre","org":"InstaDeep","group":"mlm","ours":false,"objective":"masked","domain":"generalist","params":652000000.0,"params_str":"652M","logo":"instadeep.png","in_fig1":true,"tokens_seen":null,"bp_equivalent":null,"s_bal":0.7245,"families":{"chromatin":0.4192,"grc":0.4839,"conservation":0.7252,"splicing":0.9506,"tis":0.8662,"tts":0.8286,"proseq":0.8151,"llr":0.6922,"causal":0.7393},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7393},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3757},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3464},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3601},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4172},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3892},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5235},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5222},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7046},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7458},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5926},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4512},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4586},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4332},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6953},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7167},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6646},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8151},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.946},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9552},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8662},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8286}]},{"id":"CARBON-500M","name":"Carbon-500M","org":"Hugging Face","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":512000000.0,"params_str":"512M","logo":"huggingface.png","in_fig1":true,"tokens_seen":600000000000.0,"bp_equivalent":"3.60T","s_bal":0.7087,"families":{"chromatin":0.3941,"grc":0.4228,"conservation":0.7342,"splicing":0.9262,"tis":0.8947,"tts":0.8514,"proseq":0.7872,"llr":0.6657,"causal":0.7025},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7025},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3734},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3061},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3174},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.3968},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3834},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.4961},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.4858},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6963},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7721},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5716},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3024},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4167},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4005},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6926},{"label":"LLR \u00b7 maize","fam":"llr","value":0.6536},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6509},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.7872},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9198},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9326},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8947},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8514}]},{"id":"PlantBiMoE","name":"PlantBiMoE","org":"Huazhong Agric. Univ.","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":116000000.0,"params_str":"116M","logo":"huazhong.png","in_fig1":true,"tokens_seen":241300000000.0,"bp_equivalent":"241.3B","s_bal":0.7036,"families":{"chromatin":0.4388,"grc":0.4813,"conservation":0.7163,"splicing":0.9619,"tis":0.8796,"tts":0.7914,"proseq":0.83,"llr":0.6669,"causal":0.5664},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.5664},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3926},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3636},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3901},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4323},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4157},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5312},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5461},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6858},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7468},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5992},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4269},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4515},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4478},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6846},{"label":"LLR \u00b7 maize","fam":"llr","value":0.6612},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6548},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.83},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9609},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9629},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8796},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.7914}]},{"id":"Evo2-7b","name":"Evo 2-7B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":7000000000.0,"params_str":"7B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":2400000000000.0,"bp_equivalent":"2.40T","s_bal":0.695,"families":{"chromatin":0.239,"grc":0.3974,"conservation":0.7373,"splicing":0.9354,"tis":0.8937,"tts":0.8847,"proseq":0.7492,"llr":0.7236,"causal":0.6946},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.6946},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.1793},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.2159},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.1882},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.1935},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.2273},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.3539},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.3146},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7043},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7703},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.4949},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3423},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.3677},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.3846},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7546},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7165},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6996},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.7492},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9385},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9324},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8937},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8847}]},{"id":"NTv3","name":"NTv3-650M-post","org":"InstaDeep","group":"mlm","ours":false,"objective":"masked","domain":"generalist","params":650000000.0,"params_str":"650M","logo":"instadeep.png","in_fig1":true,"tokens_seen":12104000000000.0,"bp_equivalent":"12.10T","s_bal":0.6903,"families":{"chromatin":0.4674,"grc":0.4565,"conservation":0.7456,"splicing":0.9784,"tis":0.8951,"tts":0.8813,"proseq":0.8129,"llr":0.5024,"causal":0.4735},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.4735},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.5007},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.4003},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.4182},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4405},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.431},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5555},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5254},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6904},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.8008},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.6207},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3585},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4298},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4171},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.5596},{"label":"LLR \u00b7 maize","fam":"llr","value":0.5362},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.4113},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8129},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9778},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.979},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8951},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8813}]},{"id":"Botanic0-L","name":"Botanic0-L","org":"Living Models","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":991000000.0,"params_str":"991M","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":226100000000.0,"bp_equivalent":"1.36T","s_bal":0.681,"families":{"chromatin":0.4273,"grc":0.4579,"conservation":0.6831,"splicing":0.9396,"tis":0.8471,"tts":0.7142,"proseq":0.8551,"llr":0.6488,"causal":0.5565},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.5565},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3602},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3531},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3816},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4309},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4046},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5258},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5349},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6659},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7003},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5716},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3995},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4248},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4355},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6717},{"label":"LLR \u00b7 maize","fam":"llr","value":0.6659},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6087},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8551},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.919},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9601},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.8471},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.7142}]},{"id":"Evo2-20b","name":"Evo 2-20B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":20000000000.0,"params_str":"20B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":9300000000000.0,"bp_equivalent":"9.30T","s_bal":0.6776,"families":{"chromatin":0.2323,"grc":0.4103,"conservation":0.7355,"splicing":0.8658,"tis":0.852,"tts":0.8313,"proseq":0.7148,"llr":0.7178,"causal":0.7388},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7388},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.167},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.222},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.1924},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.1827},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.1968},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.3514},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.3138},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.7028},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7682},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.513},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3742},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.3831},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.3712},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7361},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7194},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6979},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.7148},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.8955},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.8361},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.852},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8313}]},{"id":"Evo2-1b","name":"Evo 2-1B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":1000000000.0,"params_str":"1B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":1000000000000.0,"bp_equivalent":"1.00T","s_bal":0.6695,"families":{"chromatin":0.1807,"grc":0.3438,"conservation":0.7137,"splicing":0.9181,"tis":0.9057,"tts":0.8722,"proseq":0.6946,"llr":0.6915,"causal":0.7055},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7055},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.1386},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.1613},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.1145},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.134},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.1563},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.3113},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.2486},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6761},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7513},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.4592},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.2741},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.3445},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.2974},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7078},{"label":"LLR \u00b7 maize","fam":"llr","value":0.6976},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6691},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.6946},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.9129},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.9233},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.9057},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.8722}]},{"id":"PlantGFM","name":"PlantGFM","org":"Huazhong Agric. Univ.","group":"ar","ours":false,"objective":"autoregressive","domain":"plant","params":220000000.0,"params_str":"220M","logo":"huazhong.png","in_fig1":true,"tokens_seen":430600000000.0,"bp_equivalent":"430.6B","s_bal":0.6497,"families":{"chromatin":0.4219,"grc":0.4497,"conservation":0.6758,"splicing":0.8167,"tis":0.7695,"tts":0.712,"proseq":0.8115,"llr":0.6004,"causal":0.5898},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.5898},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3767},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3326},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3826},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4136},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.4047},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5155},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5277},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6582},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.6935},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.55},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.4097},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4303},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.4088},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.689},{"label":"LLR \u00b7 maize","fam":"llr","value":0.5714},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.5407},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8115},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.8507},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.7828},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.7695},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.712}]},{"id":"Botanic0-M","name":"Botanic0-M","org":"Living Models","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":260000000.0,"params_str":"260M","logo":"living-models-icon.png","in_fig1":false,"tokens_seen":226100000000.0,"bp_equivalent":"1.36T","s_bal":0.6435,"families":{"chromatin":0.415,"grc":0.4231,"conservation":0.6702,"splicing":0.8461,"tis":0.7865,"tts":0.6049,"proseq":0.8212,"llr":0.6279,"causal":0.5965},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.5965},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3544},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3337},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3879},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.4153},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3842},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5002},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5294},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6465},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.694},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.535},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.3565},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.4059},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.3949},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6569},{"label":"LLR \u00b7 maize","fam":"llr","value":0.6165},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6104},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8212},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.7782},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.914},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.7865},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.6049}]},{"id":"Evo2-40b","name":"Evo 2-40B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":40000000000.0,"params_str":"40B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":9300000000000.0,"bp_equivalent":"9.30T","s_bal":0.6287,"families":{"chromatin":0.2384,"grc":0.4084,"conservation":0.6922,"splicing":0.7181,"tis":0.7202,"tts":0.7187,"proseq":0.7012,"llr":0.7185,"causal":0.7426},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.7426},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.1718},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.2274},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.1964},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.1939},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.2141},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.3483},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.3166},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.6638},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.7205},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.5197},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.351},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.3828},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.38},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.7412},{"label":"LLR \u00b7 maize","fam":"llr","value":0.7168},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.6975},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.7012},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.776},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.6602},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.7202},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.7187}]},{"id":"AgroNT","name":"AgroNT","org":"InstaDeep","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":1000000000.0,"params_str":"1.0B","logo":"instadeep.png","in_fig1":true,"tokens_seen":472500000000.0,"bp_equivalent":"2.83T","s_bal":0.5748,"families":{"chromatin":0.3946,"grc":0.3559,"conservation":0.6056,"splicing":0.7815,"tis":0.6564,"tts":0.3745,"proseq":0.8075,"llr":0.6258,"causal":0.5713},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.5713},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3169},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.3235},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.3387},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.3808},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3846},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.5057},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.5119},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.5973},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.6139},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.4521},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.2893},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.3599},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.3224},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6716},{"label":"LLR \u00b7 maize","fam":"llr","value":0.6207},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.5851},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8075},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.7227},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.8402},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.6564},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.3745}]},{"id":"Botanic0-S","name":"Botanic0-S","org":"Living Models","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":114000000.0,"params_str":"114M","logo":"living-models-icon.png","in_fig1":false,"tokens_seen":226100000000.0,"bp_equivalent":"1.36T","s_bal":0.5478,"families":{"chromatin":0.3609,"grc":0.3371,"conservation":0.5885,"splicing":0.7347,"tis":0.6091,"tts":0.3663,"proseq":0.8109,"llr":0.5961,"causal":0.527},"metrics":[{"label":"GWAS \u00b7 recall_auc","fam":"causal","value":0.527},{"label":"Chromatin \u00b7 arabidopsis","fam":"chromatin","value":0.3004},{"label":"Chromatin \u00b7 brachypodium","fam":"chromatin","value":0.2841},{"label":"Chromatin \u00b7 maize","fam":"chromatin","value":0.2944},{"label":"Chromatin \u00b7 rice MH63","fam":"chromatin","value":0.3456},{"label":"Chromatin \u00b7 rice ZS97","fam":"chromatin","value":0.3693},{"label":"Chromatin \u00b7 setaria","fam":"chromatin","value":0.4469},{"label":"Chromatin \u00b7 sorghum","fam":"chromatin","value":0.4853},{"label":"Conservation \u00b7 maize","fam":"conservation","value":0.5798},{"label":"Conservation \u00b7 sorghum","fam":"conservation","value":0.5973},{"label":"GRC2 \u00b7 arabidopsis","fam":"grc","value":0.4469},{"label":"GRC2 \u00b7 maize","fam":"grc","value":0.2677},{"label":"GRC2 \u00b7 rice","fam":"grc","value":0.3317},{"label":"GRC2 \u00b7 tomato","fam":"grc","value":0.3022},{"label":"LLR \u00b7 arabidopsis","fam":"llr","value":0.6338},{"label":"LLR \u00b7 maize","fam":"llr","value":0.5615},{"label":"LLR \u00b7 tomato","fam":"llr","value":0.5929},{"label":"PRO-seq \u00b7 cassava","fam":"proseq","value":0.8109},{"label":"Splice acceptor \u00b7 arabidopsis","fam":"splicing","value":0.6956},{"label":"Splice donor \u00b7 arabidopsis","fam":"splicing","value":0.7739},{"label":"TIS \u00b7 arabidopsis","fam":"tis","value":0.6091},{"label":"TTS \u00b7 arabidopsis","fam":"tts","value":0.3663}]}],"note":"S_bal^test: species averaged within each of nine task families, then the nine families averaged. 22 metrics from the frozen-model suite (figures/data/botanic1_scorecard_28.csv, MLflow experiment 218)."}
 
1
+ {"families":[{"key":"chromatin","name":"Chromatin accessibility","metric":"macro-AUPRC","n_species":7},{"key":"grc","name":"Genomic region","metric":"balanced acc.","n_species":4},{"key":"conservation","name":"Conservation","metric":"AUPRC","n_species":2},{"key":"splicing","name":"Splicing","metric":"strat. AUPRC","n_species":2},{"key":"tis","name":"TIS","metric":"AUPRC","n_species":1},{"key":"tts","name":"TTS","metric":"AUPRC","n_species":1},{"key":"proseq","name":"PRO-seq","metric":"AUPRC","n_species":1},{"key":"llr","name":"LLR mutation effect","metric":"AUROC","n_species":3},{"key":"causal","name":"Causal variant discovery","metric":"recall-AUC","n_species":1}],"models":[{"id":"botanic1-XL","name":"Botanic1-XL","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":3180000000.0,"params_str":"3.18B","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7695,"families":{"chromatin":0.4764,"grc":0.527,"conservation":0.8126,"splicing":0.98,"tis":0.9087,"tts":0.8874,"proseq":0.8612,"llr":0.7206,"causal":0.7516},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7516},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4319},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.4093},{"label":"Chromatin · maize","fam":"chromatin","value":0.4575},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4599},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4361},{"label":"Chromatin · setaria","fam":"chromatin","value":0.575},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5648},{"label":"Conservation · maize","fam":"conservation","value":0.767},{"label":"Conservation · sorghum","fam":"conservation","value":0.8581},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6547},{"label":"GRC2 · maize","fam":"grc","value":0.4847},{"label":"GRC2 · rice","fam":"grc","value":0.4815},{"label":"GRC2 · tomato","fam":"grc","value":0.4872},{"label":"LLR · arabidopsis","fam":"llr","value":0.7527},{"label":"LLR · maize","fam":"llr","value":0.7291},{"label":"LLR · tomato","fam":"llr","value":0.6801},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8612},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9782},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9817},{"label":"TIS · arabidopsis","fam":"tis","value":0.9087},{"label":"TTS · arabidopsis","fam":"tts","value":0.8874}]},{"id":"botanic1-L","name":"Botanic1-L","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":2109999999.9999998,"params_str":"2.11B","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7679,"families":{"chromatin":0.4813,"grc":0.5233,"conservation":0.813,"splicing":0.9818,"tis":0.9116,"tts":0.8711,"proseq":0.8685,"llr":0.7176,"causal":0.743},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.743},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4311},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.4005},{"label":"Chromatin · maize","fam":"chromatin","value":0.4688},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4739},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4441},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5764},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5745},{"label":"Conservation · maize","fam":"conservation","value":0.7695},{"label":"Conservation · sorghum","fam":"conservation","value":0.8565},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6568},{"label":"GRC2 · maize","fam":"grc","value":0.4701},{"label":"GRC2 · rice","fam":"grc","value":0.4779},{"label":"GRC2 · tomato","fam":"grc","value":0.4886},{"label":"LLR · arabidopsis","fam":"llr","value":0.7463},{"label":"LLR · maize","fam":"llr","value":0.722},{"label":"LLR · tomato","fam":"llr","value":0.6845},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8685},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9799},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9837},{"label":"TIS · arabidopsis","fam":"tis","value":0.9116},{"label":"TTS · arabidopsis","fam":"tts","value":0.8711}]},{"id":"botanic1-M","name":"Botanic1-M","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":688000000.0,"params_str":"688M","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7652,"families":{"chromatin":0.4731,"grc":0.5236,"conservation":0.8068,"splicing":0.9836,"tis":0.9088,"tts":0.8808,"proseq":0.8609,"llr":0.7126,"causal":0.7366},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7366},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4208},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.4042},{"label":"Chromatin · maize","fam":"chromatin","value":0.4817},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4566},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4261},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5601},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5624},{"label":"Conservation · maize","fam":"conservation","value":0.762},{"label":"Conservation · sorghum","fam":"conservation","value":0.8516},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6537},{"label":"GRC2 · maize","fam":"grc","value":0.4757},{"label":"GRC2 · rice","fam":"grc","value":0.4848},{"label":"GRC2 · tomato","fam":"grc","value":0.4803},{"label":"LLR · arabidopsis","fam":"llr","value":0.7217},{"label":"LLR · maize","fam":"llr","value":0.7337},{"label":"LLR · tomato","fam":"llr","value":0.6825},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8609},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.982},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9853},{"label":"TIS · arabidopsis","fam":"tis","value":0.9088},{"label":"TTS · arabidopsis","fam":"tts","value":0.8808}]},{"id":"botanic1-S","name":"Botanic1-S","org":"Living Models","group":"ours","ours":true,"objective":"masked","domain":"plant","params":318000000.0,"params_str":"318M","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":314600000000.0,"bp_equivalent":"314.6B","s_bal":0.7578,"families":{"chromatin":0.4666,"grc":0.5243,"conservation":0.7939,"splicing":0.9819,"tis":0.9052,"tts":0.8754,"proseq":0.8588,"llr":0.7115,"causal":0.7024},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7024},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4173},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3954},{"label":"Chromatin · maize","fam":"chromatin","value":0.4576},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4484},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.419},{"label":"Chromatin · setaria","fam":"chromatin","value":0.561},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5676},{"label":"Conservation · maize","fam":"conservation","value":0.7479},{"label":"Conservation · sorghum","fam":"conservation","value":0.8399},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6385},{"label":"GRC2 · maize","fam":"grc","value":0.4679},{"label":"GRC2 · rice","fam":"grc","value":0.4921},{"label":"GRC2 · tomato","fam":"grc","value":0.4985},{"label":"LLR · arabidopsis","fam":"llr","value":0.7241},{"label":"LLR · maize","fam":"llr","value":0.7227},{"label":"LLR · tomato","fam":"llr","value":0.6875},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8588},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9792},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9846},{"label":"TIS · arabidopsis","fam":"tis","value":0.9052},{"label":"TTS · arabidopsis","fam":"tts","value":0.8754}]},{"id":"PlantCAD2-L","name":"PlantCAD2-L","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":694000000.0,"params_str":"694M","logo":"cornell-seal.png","in_fig1":true,"tokens_seen":4030000000000.001,"bp_equivalent":"4.03T","s_bal":0.7556,"families":{"chromatin":0.4834,"grc":0.5251,"conservation":0.7955,"splicing":0.9775,"tis":0.8893,"tts":0.8817,"proseq":0.8669,"llr":0.7038,"causal":0.677},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.677},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4286},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.41},{"label":"Chromatin · maize","fam":"chromatin","value":0.4759},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4806},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4362},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5745},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5782},{"label":"Conservation · maize","fam":"conservation","value":0.7422},{"label":"Conservation · sorghum","fam":"conservation","value":0.8489},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6636},{"label":"GRC2 · maize","fam":"grc","value":0.4504},{"label":"GRC2 · rice","fam":"grc","value":0.4941},{"label":"GRC2 · tomato","fam":"grc","value":0.4925},{"label":"LLR · arabidopsis","fam":"llr","value":0.7012},{"label":"LLR · maize","fam":"llr","value":0.7263},{"label":"LLR · tomato","fam":"llr","value":0.6839},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8669},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9758},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9793},{"label":"TIS · arabidopsis","fam":"tis","value":0.8893},{"label":"TTS · arabidopsis","fam":"tts","value":0.8817}]},{"id":"PlantCaduceus","name":"PlantCaduceus-l32","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":225000000.0,"params_str":"225M","logo":"cornell-seal.png","in_fig1":true,"tokens_seen":251700000000.0,"bp_equivalent":"251.7B","s_bal":0.7521,"families":{"chromatin":0.4573,"grc":0.5236,"conservation":0.7937,"splicing":0.9781,"tis":0.8965,"tts":0.863,"proseq":0.8177,"llr":0.7132,"causal":0.7256},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7256},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3992},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.38},{"label":"Chromatin · maize","fam":"chromatin","value":0.4406},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4466},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4217},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5527},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5605},{"label":"Conservation · maize","fam":"conservation","value":0.7548},{"label":"Conservation · sorghum","fam":"conservation","value":0.8326},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6746},{"label":"GRC2 · maize","fam":"grc","value":0.4677},{"label":"GRC2 · rice","fam":"grc","value":0.4867},{"label":"GRC2 · tomato","fam":"grc","value":0.4655},{"label":"LLR · arabidopsis","fam":"llr","value":0.7431},{"label":"LLR · maize","fam":"llr","value":0.7169},{"label":"LLR · tomato","fam":"llr","value":0.6795},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8177},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9715},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9846},{"label":"TIS · arabidopsis","fam":"tis","value":0.8965},{"label":"TTS · arabidopsis","fam":"tts","value":0.863}]},{"id":"PlantCAD2-S","name":"PlantCAD2-S","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":88000000.0,"params_str":"88M","logo":"cornell-seal.png","in_fig1":false,"tokens_seen":4030000000000.001,"bp_equivalent":"4.03T","s_bal":0.742,"families":{"chromatin":0.4732,"grc":0.5085,"conservation":0.7698,"splicing":0.9796,"tis":0.891,"tts":0.8632,"proseq":0.8574,"llr":0.6895,"causal":0.6458},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.6458},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4271},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.4073},{"label":"Chromatin · maize","fam":"chromatin","value":0.4766},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4544},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4198},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5591},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5681},{"label":"Conservation · maize","fam":"conservation","value":0.7188},{"label":"Conservation · sorghum","fam":"conservation","value":0.8208},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6351},{"label":"GRC2 · maize","fam":"grc","value":0.4491},{"label":"GRC2 · rice","fam":"grc","value":0.485},{"label":"GRC2 · tomato","fam":"grc","value":0.4649},{"label":"LLR · arabidopsis","fam":"llr","value":0.6902},{"label":"LLR · maize","fam":"llr","value":0.7091},{"label":"LLR · tomato","fam":"llr","value":0.6694},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8574},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9783},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9809},{"label":"TIS · arabidopsis","fam":"tis","value":0.891},{"label":"TTS · arabidopsis","fam":"tts","value":0.8632}]},{"id":"PlantCAD2-M","name":"PlantCAD2-M","org":"Cornell University","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":311000000.0,"params_str":"311M","logo":"cornell-seal.png","in_fig1":false,"tokens_seen":4030000000000.001,"bp_equivalent":"4.03T","s_bal":0.7331,"families":{"chromatin":0.4201,"grc":0.4885,"conservation":0.7508,"splicing":0.9763,"tis":0.8955,"tts":0.8571,"proseq":0.8387,"llr":0.6933,"causal":0.6777},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.6777},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3582},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3565},{"label":"Chromatin · maize","fam":"chromatin","value":0.3777},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4084},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4006},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5163},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5232},{"label":"Conservation · maize","fam":"conservation","value":0.7035},{"label":"Conservation · sorghum","fam":"conservation","value":0.798},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6312},{"label":"GRC2 · maize","fam":"grc","value":0.4092},{"label":"GRC2 · rice","fam":"grc","value":0.4693},{"label":"GRC2 · tomato","fam":"grc","value":0.4441},{"label":"LLR · arabidopsis","fam":"llr","value":0.686},{"label":"LLR · maize","fam":"llr","value":0.707},{"label":"LLR · tomato","fam":"llr","value":0.6869},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8387},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.969},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9836},{"label":"TIS · arabidopsis","fam":"tis","value":0.8955},{"label":"TTS · arabidopsis","fam":"tts","value":0.8571}]},{"id":"GPN","name":"GPN","org":"UC Berkeley","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":66000000.0,"params_str":"66M","logo":"berkeley.png","in_fig1":true,"tokens_seen":157300000000.0,"bp_equivalent":"157.3B","s_bal":0.7314,"families":{"chromatin":0.3796,"grc":0.4975,"conservation":0.7286,"splicing":0.9786,"tis":0.9014,"tts":0.8877,"proseq":0.8456,"llr":0.7093,"causal":0.6539},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.6539},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.422},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3129},{"label":"Chromatin · maize","fam":"chromatin","value":0.2724},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.3632},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3608},{"label":"Chromatin · setaria","fam":"chromatin","value":0.4753},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.4506},{"label":"Conservation · maize","fam":"conservation","value":0.6958},{"label":"Conservation · sorghum","fam":"conservation","value":0.7614},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.675},{"label":"GRC2 · maize","fam":"grc","value":0.3945},{"label":"GRC2 · rice","fam":"grc","value":0.4526},{"label":"GRC2 · tomato","fam":"grc","value":0.4679},{"label":"LLR · arabidopsis","fam":"llr","value":0.7153},{"label":"LLR · maize","fam":"llr","value":0.7352},{"label":"LLR · tomato","fam":"llr","value":0.6773},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8456},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9773},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9799},{"label":"TIS · arabidopsis","fam":"tis","value":0.9014},{"label":"TTS · arabidopsis","fam":"tts","value":0.8877}]},{"id":"CARBON-3B","name":"Carbon-3B","org":"Hugging Face","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":3450000000.0,"params_str":"3.45B","logo":"huggingface.png","in_fig1":true,"tokens_seen":1050000000000.0,"bp_equivalent":"6.30T","s_bal":0.731,"families":{"chromatin":0.425,"grc":0.4637,"conservation":0.7635,"splicing":0.9513,"tis":0.9012,"tts":0.869,"proseq":0.8298,"llr":0.6936,"causal":0.6816},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.6816},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.4005},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3468},{"label":"Chromatin · maize","fam":"chromatin","value":0.3689},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4253},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4016},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5154},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5162},{"label":"Conservation · maize","fam":"conservation","value":0.7159},{"label":"Conservation · sorghum","fam":"conservation","value":0.8112},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5949},{"label":"GRC2 · maize","fam":"grc","value":0.3851},{"label":"GRC2 · rice","fam":"grc","value":0.4322},{"label":"GRC2 · tomato","fam":"grc","value":0.4425},{"label":"LLR · arabidopsis","fam":"llr","value":0.6833},{"label":"LLR · maize","fam":"llr","value":0.7283},{"label":"LLR · tomato","fam":"llr","value":0.6692},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8298},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9476},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.955},{"label":"TIS · arabidopsis","fam":"tis","value":0.9012},{"label":"TTS · arabidopsis","fam":"tts","value":0.869}]},{"id":"CARBON-8B","name":"Carbon-8B","org":"Hugging Face","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":8260000000.0,"params_str":"8.26B","logo":"huggingface.png","in_fig1":true,"tokens_seen":1050000000000.0,"bp_equivalent":"6.30T","s_bal":0.7272,"families":{"chromatin":0.416,"grc":0.4441,"conservation":0.7583,"splicing":0.9323,"tis":0.8962,"tts":0.869,"proseq":0.8141,"llr":0.6941,"causal":0.7207},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7207},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3898},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.328},{"label":"Chromatin · maize","fam":"chromatin","value":0.3419},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4235},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3864},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5162},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5266},{"label":"Conservation · maize","fam":"conservation","value":0.7247},{"label":"Conservation · sorghum","fam":"conservation","value":0.7918},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5892},{"label":"GRC2 · maize","fam":"grc","value":0.3335},{"label":"GRC2 · rice","fam":"grc","value":0.4266},{"label":"GRC2 · tomato","fam":"grc","value":0.4271},{"label":"LLR · arabidopsis","fam":"llr","value":0.6979},{"label":"LLR · maize","fam":"llr","value":0.7163},{"label":"LLR · tomato","fam":"llr","value":0.6682},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8141},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9253},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9394},{"label":"TIS · arabidopsis","fam":"tis","value":0.8962},{"label":"TTS · arabidopsis","fam":"tts","value":0.869}]},{"id":"NTv3-pre","name":"NTv3-650M-pre","org":"InstaDeep","group":"mlm","ours":false,"objective":"masked","domain":"generalist","params":652000000.0,"params_str":"652M","logo":"instadeep.png","in_fig1":true,"tokens_seen":null,"bp_equivalent":null,"s_bal":0.7245,"families":{"chromatin":0.4192,"grc":0.4839,"conservation":0.7252,"splicing":0.9506,"tis":0.8662,"tts":0.8286,"proseq":0.8151,"llr":0.6922,"causal":0.7393},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7393},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3757},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3464},{"label":"Chromatin · maize","fam":"chromatin","value":0.3601},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4172},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3892},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5235},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5222},{"label":"Conservation · maize","fam":"conservation","value":0.7046},{"label":"Conservation · sorghum","fam":"conservation","value":0.7458},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5926},{"label":"GRC2 · maize","fam":"grc","value":0.4512},{"label":"GRC2 · rice","fam":"grc","value":0.4586},{"label":"GRC2 · tomato","fam":"grc","value":0.4332},{"label":"LLR · arabidopsis","fam":"llr","value":0.6953},{"label":"LLR · maize","fam":"llr","value":0.7167},{"label":"LLR · tomato","fam":"llr","value":0.6646},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8151},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.946},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9552},{"label":"TIS · arabidopsis","fam":"tis","value":0.8662},{"label":"TTS · arabidopsis","fam":"tts","value":0.8286}]},{"id":"CARBON-500M","name":"Carbon-500M","org":"Hugging Face","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":512000000.0,"params_str":"512M","logo":"huggingface.png","in_fig1":true,"tokens_seen":600000000000.0,"bp_equivalent":"3.60T","s_bal":0.7087,"families":{"chromatin":0.3941,"grc":0.4228,"conservation":0.7342,"splicing":0.9262,"tis":0.8947,"tts":0.8514,"proseq":0.7872,"llr":0.6657,"causal":0.7025},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7025},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3734},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3061},{"label":"Chromatin · maize","fam":"chromatin","value":0.3174},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.3968},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3834},{"label":"Chromatin · setaria","fam":"chromatin","value":0.4961},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.4858},{"label":"Conservation · maize","fam":"conservation","value":0.6963},{"label":"Conservation · sorghum","fam":"conservation","value":0.7721},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5716},{"label":"GRC2 · maize","fam":"grc","value":0.3024},{"label":"GRC2 · rice","fam":"grc","value":0.4167},{"label":"GRC2 · tomato","fam":"grc","value":0.4005},{"label":"LLR · arabidopsis","fam":"llr","value":0.6926},{"label":"LLR · maize","fam":"llr","value":0.6536},{"label":"LLR · tomato","fam":"llr","value":0.6509},{"label":"PRO-seq · cassava","fam":"proseq","value":0.7872},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9198},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9326},{"label":"TIS · arabidopsis","fam":"tis","value":0.8947},{"label":"TTS · arabidopsis","fam":"tts","value":0.8514}]},{"id":"PlantBiMoE","name":"PlantBiMoE","org":"Huazhong Agric. Univ.","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":116000000.0,"params_str":"116M","logo":"huazhong.png","in_fig1":true,"tokens_seen":241300000000.0,"bp_equivalent":"241.3B","s_bal":0.7036,"families":{"chromatin":0.4388,"grc":0.4813,"conservation":0.7163,"splicing":0.9619,"tis":0.8796,"tts":0.7914,"proseq":0.83,"llr":0.6669,"causal":0.5664},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.5664},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3926},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3636},{"label":"Chromatin · maize","fam":"chromatin","value":0.3901},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4323},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4157},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5312},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5461},{"label":"Conservation · maize","fam":"conservation","value":0.6858},{"label":"Conservation · sorghum","fam":"conservation","value":0.7468},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5992},{"label":"GRC2 · maize","fam":"grc","value":0.4269},{"label":"GRC2 · rice","fam":"grc","value":0.4515},{"label":"GRC2 · tomato","fam":"grc","value":0.4478},{"label":"LLR · arabidopsis","fam":"llr","value":0.6846},{"label":"LLR · maize","fam":"llr","value":0.6612},{"label":"LLR · tomato","fam":"llr","value":0.6548},{"label":"PRO-seq · cassava","fam":"proseq","value":0.83},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9609},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9629},{"label":"TIS · arabidopsis","fam":"tis","value":0.8796},{"label":"TTS · arabidopsis","fam":"tts","value":0.7914}]},{"id":"Evo2-7b","name":"Evo 2-7B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":7000000000.0,"params_str":"7B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":2400000000000.0,"bp_equivalent":"2.40T","s_bal":0.695,"families":{"chromatin":0.239,"grc":0.3974,"conservation":0.7373,"splicing":0.9354,"tis":0.8937,"tts":0.8847,"proseq":0.7492,"llr":0.7236,"causal":0.6946},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.6946},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.1793},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.2159},{"label":"Chromatin · maize","fam":"chromatin","value":0.1882},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.1935},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.2273},{"label":"Chromatin · setaria","fam":"chromatin","value":0.3539},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.3146},{"label":"Conservation · maize","fam":"conservation","value":0.7043},{"label":"Conservation · sorghum","fam":"conservation","value":0.7703},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.4949},{"label":"GRC2 · maize","fam":"grc","value":0.3423},{"label":"GRC2 · rice","fam":"grc","value":0.3677},{"label":"GRC2 · tomato","fam":"grc","value":0.3846},{"label":"LLR · arabidopsis","fam":"llr","value":0.7546},{"label":"LLR · maize","fam":"llr","value":0.7165},{"label":"LLR · tomato","fam":"llr","value":0.6996},{"label":"PRO-seq · cassava","fam":"proseq","value":0.7492},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9385},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9324},{"label":"TIS · arabidopsis","fam":"tis","value":0.8937},{"label":"TTS · arabidopsis","fam":"tts","value":0.8847}]},{"id":"NTv3","name":"NTv3-650M-post","org":"InstaDeep","group":"mlm","ours":false,"objective":"masked","domain":"generalist","params":650000000.0,"params_str":"650M","logo":"instadeep.png","in_fig1":true,"tokens_seen":12104000000000.0,"bp_equivalent":"12.10T","s_bal":0.6903,"families":{"chromatin":0.4674,"grc":0.4565,"conservation":0.7456,"splicing":0.9784,"tis":0.8951,"tts":0.8813,"proseq":0.8129,"llr":0.5024,"causal":0.4735},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.4735},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.5007},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.4003},{"label":"Chromatin · maize","fam":"chromatin","value":0.4182},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4405},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.431},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5555},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5254},{"label":"Conservation · maize","fam":"conservation","value":0.6904},{"label":"Conservation · sorghum","fam":"conservation","value":0.8008},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.6207},{"label":"GRC2 · maize","fam":"grc","value":0.3585},{"label":"GRC2 · rice","fam":"grc","value":0.4298},{"label":"GRC2 · tomato","fam":"grc","value":0.4171},{"label":"LLR · arabidopsis","fam":"llr","value":0.5596},{"label":"LLR · maize","fam":"llr","value":0.5362},{"label":"LLR · tomato","fam":"llr","value":0.4113},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8129},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9778},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.979},{"label":"TIS · arabidopsis","fam":"tis","value":0.8951},{"label":"TTS · arabidopsis","fam":"tts","value":0.8813}]},{"id":"Botanic0-L","name":"Botanic0-L","org":"Living Models","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":991000000.0,"params_str":"991M","logo":"living-models-icon.png","in_fig1":true,"tokens_seen":226100000000.0,"bp_equivalent":"1.36T","s_bal":0.681,"families":{"chromatin":0.4273,"grc":0.4579,"conservation":0.6831,"splicing":0.9396,"tis":0.8471,"tts":0.7142,"proseq":0.8551,"llr":0.6488,"causal":0.5565},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.5565},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3602},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3531},{"label":"Chromatin · maize","fam":"chromatin","value":0.3816},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4309},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4046},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5258},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5349},{"label":"Conservation · maize","fam":"conservation","value":0.6659},{"label":"Conservation · sorghum","fam":"conservation","value":0.7003},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5716},{"label":"GRC2 · maize","fam":"grc","value":0.3995},{"label":"GRC2 · rice","fam":"grc","value":0.4248},{"label":"GRC2 · tomato","fam":"grc","value":0.4355},{"label":"LLR · arabidopsis","fam":"llr","value":0.6717},{"label":"LLR · maize","fam":"llr","value":0.6659},{"label":"LLR · tomato","fam":"llr","value":0.6087},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8551},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.919},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9601},{"label":"TIS · arabidopsis","fam":"tis","value":0.8471},{"label":"TTS · arabidopsis","fam":"tts","value":0.7142}]},{"id":"Evo2-20b","name":"Evo 2-20B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":20000000000.0,"params_str":"20B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":9300000000000.0,"bp_equivalent":"9.30T","s_bal":0.6776,"families":{"chromatin":0.2323,"grc":0.4103,"conservation":0.7355,"splicing":0.8658,"tis":0.852,"tts":0.8313,"proseq":0.7148,"llr":0.7178,"causal":0.7388},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7388},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.167},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.222},{"label":"Chromatin · maize","fam":"chromatin","value":0.1924},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.1827},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.1968},{"label":"Chromatin · setaria","fam":"chromatin","value":0.3514},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.3138},{"label":"Conservation · maize","fam":"conservation","value":0.7028},{"label":"Conservation · sorghum","fam":"conservation","value":0.7682},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.513},{"label":"GRC2 · maize","fam":"grc","value":0.3742},{"label":"GRC2 · rice","fam":"grc","value":0.3831},{"label":"GRC2 · tomato","fam":"grc","value":0.3712},{"label":"LLR · arabidopsis","fam":"llr","value":0.7361},{"label":"LLR · maize","fam":"llr","value":0.7194},{"label":"LLR · tomato","fam":"llr","value":0.6979},{"label":"PRO-seq · cassava","fam":"proseq","value":0.7148},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.8955},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.8361},{"label":"TIS · arabidopsis","fam":"tis","value":0.852},{"label":"TTS · arabidopsis","fam":"tts","value":0.8313}]},{"id":"Evo2-1b","name":"Evo 2-1B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":1000000000.0,"params_str":"1B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":1000000000000.0,"bp_equivalent":"1.00T","s_bal":0.6695,"families":{"chromatin":0.1807,"grc":0.3438,"conservation":0.7137,"splicing":0.9181,"tis":0.9057,"tts":0.8722,"proseq":0.6946,"llr":0.6915,"causal":0.7055},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7055},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.1386},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.1613},{"label":"Chromatin · maize","fam":"chromatin","value":0.1145},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.134},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.1563},{"label":"Chromatin · setaria","fam":"chromatin","value":0.3113},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.2486},{"label":"Conservation · maize","fam":"conservation","value":0.6761},{"label":"Conservation · sorghum","fam":"conservation","value":0.7513},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.4592},{"label":"GRC2 · maize","fam":"grc","value":0.2741},{"label":"GRC2 · rice","fam":"grc","value":0.3445},{"label":"GRC2 · tomato","fam":"grc","value":0.2974},{"label":"LLR · arabidopsis","fam":"llr","value":0.7078},{"label":"LLR · maize","fam":"llr","value":0.6976},{"label":"LLR · tomato","fam":"llr","value":0.6691},{"label":"PRO-seq · cassava","fam":"proseq","value":0.6946},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.9129},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.9233},{"label":"TIS · arabidopsis","fam":"tis","value":0.9057},{"label":"TTS · arabidopsis","fam":"tts","value":0.8722}]},{"id":"PlantGFM","name":"PlantGFM","org":"Huazhong Agric. Univ.","group":"ar","ours":false,"objective":"autoregressive","domain":"plant","params":220000000.0,"params_str":"220M","logo":"huazhong.png","in_fig1":true,"tokens_seen":430600000000.0,"bp_equivalent":"430.6B","s_bal":0.6497,"families":{"chromatin":0.4219,"grc":0.4497,"conservation":0.6758,"splicing":0.8167,"tis":0.7695,"tts":0.712,"proseq":0.8115,"llr":0.6004,"causal":0.5898},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.5898},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3767},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3326},{"label":"Chromatin · maize","fam":"chromatin","value":0.3826},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4136},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.4047},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5155},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5277},{"label":"Conservation · maize","fam":"conservation","value":0.6582},{"label":"Conservation · sorghum","fam":"conservation","value":0.6935},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.55},{"label":"GRC2 · maize","fam":"grc","value":0.4097},{"label":"GRC2 · rice","fam":"grc","value":0.4303},{"label":"GRC2 · tomato","fam":"grc","value":0.4088},{"label":"LLR · arabidopsis","fam":"llr","value":0.689},{"label":"LLR · maize","fam":"llr","value":0.5714},{"label":"LLR · tomato","fam":"llr","value":0.5407},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8115},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.8507},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.7828},{"label":"TIS · arabidopsis","fam":"tis","value":0.7695},{"label":"TTS · arabidopsis","fam":"tts","value":0.712}]},{"id":"Botanic0-M","name":"Botanic0-M","org":"Living Models","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":260000000.0,"params_str":"260M","logo":"living-models-icon.png","in_fig1":false,"tokens_seen":226100000000.0,"bp_equivalent":"1.36T","s_bal":0.6435,"families":{"chromatin":0.415,"grc":0.4231,"conservation":0.6702,"splicing":0.8461,"tis":0.7865,"tts":0.6049,"proseq":0.8212,"llr":0.6279,"causal":0.5965},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.5965},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3544},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3337},{"label":"Chromatin · maize","fam":"chromatin","value":0.3879},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.4153},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3842},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5002},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5294},{"label":"Conservation · maize","fam":"conservation","value":0.6465},{"label":"Conservation · sorghum","fam":"conservation","value":0.694},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.535},{"label":"GRC2 · maize","fam":"grc","value":0.3565},{"label":"GRC2 · rice","fam":"grc","value":0.4059},{"label":"GRC2 · tomato","fam":"grc","value":0.3949},{"label":"LLR · arabidopsis","fam":"llr","value":0.6569},{"label":"LLR · maize","fam":"llr","value":0.6165},{"label":"LLR · tomato","fam":"llr","value":0.6104},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8212},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.7782},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.914},{"label":"TIS · arabidopsis","fam":"tis","value":0.7865},{"label":"TTS · arabidopsis","fam":"tts","value":0.6049}]},{"id":"Evo2-40b","name":"Evo 2-40B","org":"Arc Institute","group":"ar","ours":false,"objective":"autoregressive","domain":"generalist","params":40000000000.0,"params_str":"40B","logo":"arc-institute.png","in_fig1":true,"tokens_seen":9300000000000.0,"bp_equivalent":"9.30T","s_bal":0.6287,"families":{"chromatin":0.2384,"grc":0.4084,"conservation":0.6922,"splicing":0.7181,"tis":0.7202,"tts":0.7187,"proseq":0.7012,"llr":0.7185,"causal":0.7426},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.7426},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.1718},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.2274},{"label":"Chromatin · maize","fam":"chromatin","value":0.1964},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.1939},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.2141},{"label":"Chromatin · setaria","fam":"chromatin","value":0.3483},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.3166},{"label":"Conservation · maize","fam":"conservation","value":0.6638},{"label":"Conservation · sorghum","fam":"conservation","value":0.7205},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.5197},{"label":"GRC2 · maize","fam":"grc","value":0.351},{"label":"GRC2 · rice","fam":"grc","value":0.3828},{"label":"GRC2 · tomato","fam":"grc","value":0.38},{"label":"LLR · arabidopsis","fam":"llr","value":0.7412},{"label":"LLR · maize","fam":"llr","value":0.7168},{"label":"LLR · tomato","fam":"llr","value":0.6975},{"label":"PRO-seq · cassava","fam":"proseq","value":0.7012},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.776},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.6602},{"label":"TIS · arabidopsis","fam":"tis","value":0.7202},{"label":"TTS · arabidopsis","fam":"tts","value":0.7187}]},{"id":"AgroNT","name":"AgroNT","org":"InstaDeep","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":1000000000.0,"params_str":"1.0B","logo":"instadeep.png","in_fig1":true,"tokens_seen":472500000000.0,"bp_equivalent":"2.83T","s_bal":0.5748,"families":{"chromatin":0.3946,"grc":0.3559,"conservation":0.6056,"splicing":0.7815,"tis":0.6564,"tts":0.3745,"proseq":0.8075,"llr":0.6258,"causal":0.5713},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.5713},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3169},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.3235},{"label":"Chromatin · maize","fam":"chromatin","value":0.3387},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.3808},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3846},{"label":"Chromatin · setaria","fam":"chromatin","value":0.5057},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.5119},{"label":"Conservation · maize","fam":"conservation","value":0.5973},{"label":"Conservation · sorghum","fam":"conservation","value":0.6139},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.4521},{"label":"GRC2 · maize","fam":"grc","value":0.2893},{"label":"GRC2 · rice","fam":"grc","value":0.3599},{"label":"GRC2 · tomato","fam":"grc","value":0.3224},{"label":"LLR · arabidopsis","fam":"llr","value":0.6716},{"label":"LLR · maize","fam":"llr","value":0.6207},{"label":"LLR · tomato","fam":"llr","value":0.5851},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8075},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.7227},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.8402},{"label":"TIS · arabidopsis","fam":"tis","value":0.6564},{"label":"TTS · arabidopsis","fam":"tts","value":0.3745}]},{"id":"Botanic0-S","name":"Botanic0-S","org":"Living Models","group":"mlm","ours":false,"objective":"masked","domain":"plant","params":114000000.0,"params_str":"114M","logo":"living-models-icon.png","in_fig1":false,"tokens_seen":226100000000.0,"bp_equivalent":"1.36T","s_bal":0.5478,"families":{"chromatin":0.3609,"grc":0.3371,"conservation":0.5885,"splicing":0.7347,"tis":0.6091,"tts":0.3663,"proseq":0.8109,"llr":0.5961,"causal":0.527},"metrics":[{"label":"GWAS · recall_auc","fam":"causal","value":0.527},{"label":"Chromatin · arabidopsis","fam":"chromatin","value":0.3004},{"label":"Chromatin · brachypodium","fam":"chromatin","value":0.2841},{"label":"Chromatin · maize","fam":"chromatin","value":0.2944},{"label":"Chromatin · rice MH63","fam":"chromatin","value":0.3456},{"label":"Chromatin · rice ZS97","fam":"chromatin","value":0.3693},{"label":"Chromatin · setaria","fam":"chromatin","value":0.4469},{"label":"Chromatin · sorghum","fam":"chromatin","value":0.4853},{"label":"Conservation · maize","fam":"conservation","value":0.5798},{"label":"Conservation · sorghum","fam":"conservation","value":0.5973},{"label":"GRC2 · arabidopsis","fam":"grc","value":0.4469},{"label":"GRC2 · maize","fam":"grc","value":0.2677},{"label":"GRC2 · rice","fam":"grc","value":0.3317},{"label":"GRC2 · tomato","fam":"grc","value":0.3022},{"label":"LLR · arabidopsis","fam":"llr","value":0.6338},{"label":"LLR · maize","fam":"llr","value":0.5615},{"label":"LLR · tomato","fam":"llr","value":0.5929},{"label":"PRO-seq · cassava","fam":"proseq","value":0.8109},{"label":"Splice acceptor · arabidopsis","fam":"splicing","value":0.6956},{"label":"Splice donor · arabidopsis","fam":"splicing","value":0.7739},{"label":"TIS · arabidopsis","fam":"tis","value":0.6091},{"label":"TTS · arabidopsis","fam":"tts","value":0.3663}]}],"note":"S_bal^test: species averaged within each of nine task families, then the nine families averaged. 22 metrics from the frozen-model suite."}
data/scaling.json CHANGED
The diff for this file is too large to render. See raw diff
 
data/tf.json CHANGED
@@ -1 +1 @@
1
- {"test":[{"model":"Botanic1-S","initialization":"pretrained","params_m":318.0,"input_bp":2048,"seeds":[0.7229,0.7192,0.721],"mean":0.721},{"model":"Botanic1-M","initialization":"pretrained","params_m":688.0,"input_bp":2048,"seeds":[0.7144,0.7198,0.7178],"mean":0.7173},{"model":"NTv3-652M","initialization":"pretrained","params_m":652.0,"input_bp":2048,"seeds":[0.6909,0.6906,0.6922],"mean":0.6912},{"model":"NTv3-106M","initialization":"pretrained","params_m":106.0,"input_bp":2048,"seeds":[0.6833,0.6795,0.6829],"mean":0.6819},{"model":"AgroNT","initialization":"pretrained","params_m":1000.0,"input_bp":2048,"seeds":[0.6241,0.6296,0.6238],"mean":0.6258},{"model":"DeepCistrome","initialization":"random","params_m":0.731,"input_bp":250,"seeds":[0.6133,0.6181,0.6172],"mean":0.6162}],"validation":[{"model":"Botanic1-S","initialization":"pretrained","mean":0.7187,"seeds":[0.7201,0.7177,0.7184]},{"model":"Botanic1-S","initialization":"random","mean":0.6915,"seeds":[0.6942,0.6879,0.6925]}],"contrasts":[{"comparison":"Botanic1-M","margin":0.0037,"lower":0.0002,"upper":0.0071,"blocks":270,"replicates":500},{"comparison":"NTv3-652M","margin":0.0298,"lower":0.0272,"upper":0.0326,"blocks":270,"replicates":500},{"comparison":"NTv3-106M","margin":0.0391,"lower":0.0356,"upper":0.0424,"blocks":270,"replicates":500},{"comparison":"AgroNT","margin":0.0952,"lower":0.0908,"upper":0.0992,"blocks":270,"replicates":500},{"comparison":"DeepCistrome","margin":0.1048,"lower":0.099,"upper":0.1104,"blocks":270,"replicates":500}],"note":"figures/data/tf_binding/results.csv and contrasts.csv; test = chromosome 5, validation = chromosome 4."}
 
1
+ {"test":[{"model":"Botanic1-S","initialization":"pretrained","params_m":318.0,"input_bp":2048,"seeds":[0.7229,0.7192,0.721],"mean":0.721},{"model":"Botanic1-M","initialization":"pretrained","params_m":688.0,"input_bp":2048,"seeds":[0.7144,0.7198,0.7178],"mean":0.7173},{"model":"NTv3-652M","initialization":"pretrained","params_m":652.0,"input_bp":2048,"seeds":[0.6909,0.6906,0.6922],"mean":0.6912},{"model":"NTv3-106M","initialization":"pretrained","params_m":106.0,"input_bp":2048,"seeds":[0.6833,0.6795,0.6829],"mean":0.6819},{"model":"AgroNT","initialization":"pretrained","params_m":1000.0,"input_bp":2048,"seeds":[0.6241,0.6296,0.6238],"mean":0.6258},{"model":"DeepCistrome","initialization":"random","params_m":0.731,"input_bp":250,"seeds":[0.6133,0.6181,0.6172],"mean":0.6162}],"validation":[{"model":"Botanic1-S","initialization":"pretrained","mean":0.7187,"seeds":[0.7201,0.7177,0.7184]},{"model":"Botanic1-S","initialization":"random","mean":0.6915,"seeds":[0.6942,0.6879,0.6925]}],"contrasts":[{"comparison":"Botanic1-M","margin":0.0037,"lower":0.0002,"upper":0.0071,"blocks":270,"replicates":500},{"comparison":"NTv3-652M","margin":0.0298,"lower":0.0272,"upper":0.0326,"blocks":270,"replicates":500},{"comparison":"NTv3-106M","margin":0.0391,"lower":0.0356,"upper":0.0424,"blocks":270,"replicates":500},{"comparison":"AgroNT","margin":0.0952,"lower":0.0908,"upper":0.0992,"blocks":270,"replicates":500},{"comparison":"DeepCistrome","margin":0.1048,"lower":0.099,"upper":0.1104,"blocks":270,"replicates":500}],"note":"TF-binding results and contrasts; test = chromosome 5, validation = chromosome 4."}