Spaces:
Running
Running
Overview page for the model: what it is, quick start, results in short; report moved to a section
Browse files- README.md +7 -6
- causal-variants.html +1 -1
- corpus.html +1 -1
- factory.html +1 -1
- index.html +67 -42
- leaderboard.html +1 -1
- scripts/patch_headers.py +2 -2
README.md
CHANGED
|
@@ -1,20 +1,21 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: 🌿
|
| 4 |
colorFrom: green
|
| 5 |
colorTo: gray
|
| 6 |
sdk: static
|
| 7 |
app_file: index.html
|
| 8 |
pinned: false
|
| 9 |
-
short_description: Botanic1
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
Static
|
| 15 |
|
| 16 |
-
- `index.html`: the
|
| 17 |
-
|
|
|
|
| 18 |
- `leaderboard.html`: the S_bal^test leaderboard, the nine-family scorecard and the
|
| 19 |
size and training-token views, filterable and sortable.
|
| 20 |
- `causal-variants.html`: the 545-study causal-variant prioritisation benchmark, with
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Botanic1
|
| 3 |
emoji: 🌿
|
| 4 |
colorFrom: green
|
| 5 |
colorTo: gray
|
| 6 |
sdk: static
|
| 7 |
app_file: index.html
|
| 8 |
pinned: false
|
| 9 |
+
short_description: Botanic1 plant genomic foundation models, in short
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Botanic1
|
| 13 |
|
| 14 |
+
Static Space for the Botanic1 models (Living Models): what they are, how to use them, how they perform, with the technical report linked for depth. Pages:
|
| 15 |
|
| 16 |
+
- `index.html`: the overview, quick start, the live overview figure, and one short
|
| 17 |
+
result per message with a live figure.
|
| 18 |
+
- `factory.html`: the model factory, with the report's S1 and data-pipeline figures.
|
| 19 |
- `leaderboard.html`: the S_bal^test leaderboard, the nine-family scorecard and the
|
| 20 |
size and training-token views, filterable and sortable.
|
| 21 |
- `causal-variants.html`: the 545-study causal-variant prioritisation benchmark, with
|
causal-variants.html
CHANGED
|
@@ -12,7 +12,7 @@
|
|
| 12 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 13 |
</head>
|
| 14 |
<body>
|
| 15 |
-
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">
|
| 16 |
<div class="page">
|
| 17 |
|
| 18 |
<main class="prose">
|
|
|
|
| 12 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 13 |
</head>
|
| 14 |
<body>
|
| 15 |
+
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html" aria-current="page">Causal variants</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a></nav></div></header>
|
| 16 |
<div class="page">
|
| 17 |
|
| 18 |
<main class="prose">
|
corpus.html
CHANGED
|
@@ -12,7 +12,7 @@
|
|
| 12 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 13 |
</head>
|
| 14 |
<body>
|
| 15 |
-
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">
|
| 16 |
<div class="page">
|
| 17 |
|
| 18 |
<main class="prose">
|
|
|
|
| 12 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 13 |
</head>
|
| 14 |
<body>
|
| 15 |
+
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html">Causal variants</a><a href="corpus.html" aria-current="page">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a></nav></div></header>
|
| 16 |
<div class="page">
|
| 17 |
|
| 18 |
<main class="prose">
|
factory.html
CHANGED
|
@@ -13,7 +13,7 @@
|
|
| 13 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
-
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">
|
| 17 |
<div class="page">
|
| 18 |
<main class="prose">
|
| 19 |
<p class="kicker">Results</p>
|
|
|
|
| 13 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
+
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html" aria-current="page">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html">Causal variants</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a></nav></div></header>
|
| 17 |
<div class="page">
|
| 18 |
<main class="prose">
|
| 19 |
<p class="kicker">Results</p>
|
index.html
CHANGED
|
@@ -3,8 +3,8 @@
|
|
| 3 |
<head>
|
| 4 |
<meta charset="utf-8">
|
| 5 |
<meta name="viewport" content="width=device-width, initial-scale=1">
|
| 6 |
-
<title>
|
| 7 |
-
<meta name="description" content="
|
| 8 |
<script>(function(){try{var t=new URLSearchParams(location.search).get('__theme');if(t==='dark'||t==='light')document.documentElement.setAttribute('data-theme',t);}catch(e){}})();</script>
|
| 9 |
<link rel="preconnect" href="https://fonts.googleapis.com">
|
| 10 |
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
@@ -13,15 +13,15 @@
|
|
| 13 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
-
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html" aria-current="page">
|
| 17 |
<section class="hero">
|
| 18 |
<div class="inner">
|
| 19 |
<div class="medallion">
|
| 20 |
<div class="hero-art" id="hero-art" aria-hidden="true"></div>
|
| 21 |
<div class="centre">
|
| 22 |
-
<p class="org">Living Models
|
| 23 |
<div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div>
|
| 24 |
-
<p class="eyebrow">
|
| 25 |
<dl class="spec">
|
| 26 |
<div><dt>context</dt><dd><b>100 bp to 128 kbp</b></dd></div>
|
| 27 |
<div><dt>tokens</dt><dd>single nucleotide</dd></div>
|
|
@@ -31,10 +31,10 @@
|
|
| 31 |
</dl>
|
| 32 |
<div class="chips">
|
| 33 |
<a class="primary" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a>
|
|
|
|
|
|
|
| 34 |
<a href="https://huggingface.co/datasets/living-models/Botanic1-pretraining">Pre-training data</a>
|
| 35 |
<a href="https://huggingface.co/datasets/living-models/Botanic1-causal-variants">Causal-variant benchmark</a>
|
| 36 |
-
<a href="botanic1-report.pdf">Tech report <span class="ext">PDF</span></a>
|
| 37 |
-
<a href="https://huggingface.co/spaces/living-models/botanic-gemma-chat">Chat demo <span class="ext">↗</span></a>
|
| 38 |
</div>
|
| 39 |
</div>
|
| 40 |
</div>
|
|
@@ -44,81 +44,106 @@
|
|
| 44 |
<div class="page">
|
| 45 |
<main class="prose">
|
| 46 |
<div class="titleblock">
|
| 47 |
-
<p class="kicker">
|
| 48 |
-
<h1>
|
| 49 |
-
<p class="authors">Amélie Barozet<sup>★</sup>, Vincent Cabeli<sup>★</sup>, Jean Ogier du Terrail<sup>★</sup>, Alexey Rukhovich<sup>★</sup>, Thomas Janssoone<sup>‡</sup>, Gary Klajer<sup>‡</sup>, Zeinab Sheikhitarghi<sup>‡</sup>, Gregory Andrews, Cyril Veran, Léonard Strouk</p>
|
| 50 |
-
<p class="affil">Living Models, Paris, France</p>
|
| 51 |
-
<p class="notes"><sup>★</sup> These authors contributed equally; ordered alphabetically. <sup>‡</sup> Core team; ordered alphabetically.</p>
|
| 52 |
</div>
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
-
<h2 id="
|
| 55 |
-
<
|
| 56 |
-
<
|
| 57 |
-
<
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
|
|
|
|
| 59 |
<figure class="wide" id="figure1">
|
| 60 |
<div class="fig1" id="fig1"></div>
|
| 61 |
-
<figcaption><span class="figlabel">Figure 1</span> <b>
|
| 62 |
</figure>
|
| 63 |
|
| 64 |
-
<h2 id="zeroshot">
|
| 65 |
<p class="kicker">Results</p>
|
| 66 |
-
<p>
|
| 67 |
<figure class="wide">
|
| 68 |
<div id="zeroshot-curve"></div>
|
| 69 |
-
<figcaption><span class="figlabel">Figure</span> <b>
|
| 70 |
</figure>
|
| 71 |
-
<p>
|
| 72 |
|
| 73 |
-
<h2 id="scaling">
|
| 74 |
-
<p>
|
| 75 |
<figure class="wide">
|
| 76 |
<div id="scaling-traces"></div>
|
| 77 |
-
<figcaption><span class="figlabel">Figure</span> <b>Scaling
|
| 78 |
</figure>
|
| 79 |
-
<p>
|
| 80 |
<div class="card">
|
| 81 |
<div class="cardhead"><span class="label">Fig · size against score</span><span class="small">downstream task performance · 22 frozen-model metrics · log scale · hover a mark</span></div>
|
| 82 |
<div id="intro-chart"></div>
|
| 83 |
-
<p class="chartnote">
|
| 84 |
</div>
|
| 85 |
|
| 86 |
-
<h2 id="finetune">
|
| 87 |
-
<p>
|
| 88 |
<figure class="wide">
|
| 89 |
<div id="finetune-chart"></div>
|
| 90 |
-
<figcaption><span class="figlabel">
|
| 91 |
</figure>
|
| 92 |
-
<h3 id="atac">
|
| 93 |
-
<p>
|
| 94 |
<figure class="wide">
|
| 95 |
<div id="atac-chart"></div>
|
| 96 |
-
<figcaption><span class="figlabel">Figure</span> <b>Base-resolution ATAC-seq prediction
|
| 97 |
</figure>
|
| 98 |
|
| 99 |
-
<h3 id="tf">
|
| 100 |
-
<p>
|
| 101 |
<figure class="wide">
|
| 102 |
<div id="tf-chart"></div>
|
| 103 |
<figcaption><span class="figlabel">Figure</span> <b>Genome-wide transcription factor family binding in <em>Arabidopsis thaliana</em>.</b> Macro average precision across three seeds on the chromosome 5 test set; circles are the seeds. The dashed line is the bespoke architecture trained from scratch, DeepCistrome. <span class="small">Hover a bar for the seeds and the paired margin against Botanic1-S.</span></figcaption>
|
| 104 |
</figure>
|
| 105 |
|
| 106 |
-
<h2 id="more">
|
|
|
|
| 107 |
<ul class="also">
|
| 108 |
-
<li><a href="
|
| 109 |
-
<li><a href="botanic1-report.pdf">
|
| 110 |
-
<li><a href="botanic1-report.pdf">
|
| 111 |
-
<li><a href="botanic1-report.pdf">Botanic1
|
| 112 |
<li><a href="leaderboard.html">Leaderboard</a><span class="small">Downstream task performance, the nine-family scorecard, size and tokens seen.</span></li>
|
| 113 |
<li><a href="corpus.html">Pre-training corpus</a><span class="small">The 320 species as a treemap, with the weighting and deduplication passages.</span></li>
|
| 114 |
</ul>
|
| 115 |
|
| 116 |
-
<h2 id="availability">
|
| 117 |
<ul>
|
| 118 |
<li><b>Models.</b> Botanic1-S, Botanic1-M, Botanic1-L and Botanic1-XL in the <a href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Botanic1 collection</a>, with the previous generation <a href="https://huggingface.co/living-models/Botanic0-L">Botanic0</a>. Research use only, under the Living Models research licence.</li>
|
| 119 |
<li><b>Pre-training data.</b> The 8 kbp corpus, with source manifests and assembly metadata: <a href="https://huggingface.co/datasets/living-models/Botanic1-pretraining">living-models/Botanic1-pretraining</a>.</li>
|
| 120 |
<li><b>Causal-variant benchmark.</b> The 545 tasks, their candidate SNPs and the curation audit: <a href="https://huggingface.co/datasets/living-models/Botanic1-causal-variants">living-models/Botanic1-causal-variants</a>.</li>
|
| 121 |
-
<li><b>
|
| 122 |
</ul>
|
| 123 |
|
| 124 |
<h2 id="cite">Citation</h2>
|
|
@@ -137,7 +162,7 @@
|
|
| 137 |
|
| 138 |
<footer class="site-footer">
|
| 139 |
<p>Living Models, Paris. Botanic1 is released for research use under the Living Models research licence. Organisation logos are third-party trademarks shown for attribution.</p>
|
| 140 |
-
<p>
|
| 141 |
</footer>
|
| 142 |
</div>
|
| 143 |
<div class="tip" id="tip" role="tooltip"></div>
|
|
|
|
| 3 |
<head>
|
| 4 |
<meta charset="utf-8">
|
| 5 |
<meta name="viewport" content="width=device-width, initial-scale=1">
|
| 6 |
+
<title>Botanic1</title>
|
| 7 |
+
<meta name="description" content="Botanic1: plant genomic foundation models by Living Models. What the models are, how to use them, and how they perform.">
|
| 8 |
<script>(function(){try{var t=new URLSearchParams(location.search).get('__theme');if(t==='dark'||t==='light')document.documentElement.setAttribute('data-theme',t);}catch(e){}})();</script>
|
| 9 |
<link rel="preconnect" href="https://fonts.googleapis.com">
|
| 10 |
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
|
|
| 13 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
+
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html" aria-current="page">Overview</a><a href="factory.html">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html">Causal variants</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a></nav></div></header>
|
| 17 |
<section class="hero">
|
| 18 |
<div class="inner">
|
| 19 |
<div class="medallion">
|
| 20 |
<div class="hero-art" id="hero-art" aria-hidden="true"></div>
|
| 21 |
<div class="centre">
|
| 22 |
+
<p class="org">Living Models</p>
|
| 23 |
<div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div>
|
| 24 |
+
<p class="eyebrow"><b>Foundation models for plant genomes</b>, from hundreds of base pairs to 128 kbp</p>
|
| 25 |
<dl class="spec">
|
| 26 |
<div><dt>context</dt><dd><b>100 bp to 128 kbp</b></dd></div>
|
| 27 |
<div><dt>tokens</dt><dd>single nucleotide</dd></div>
|
|
|
|
| 31 |
</dl>
|
| 32 |
<div class="chips">
|
| 33 |
<a class="primary" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a>
|
| 34 |
+
<a href="https://huggingface.co/spaces/living-models/botanic-gemma-chat">Try it in the chat demo <span class="ext">↗</span></a>
|
| 35 |
+
<a href="botanic1-report.pdf">Technical report <span class="ext">PDF</span></a>
|
| 36 |
<a href="https://huggingface.co/datasets/living-models/Botanic1-pretraining">Pre-training data</a>
|
| 37 |
<a href="https://huggingface.co/datasets/living-models/Botanic1-causal-variants">Causal-variant benchmark</a>
|
|
|
|
|
|
|
| 38 |
</div>
|
| 39 |
</div>
|
| 40 |
</div>
|
|
|
|
| 44 |
<div class="page">
|
| 45 |
<main class="prose">
|
| 46 |
<div class="titleblock">
|
| 47 |
+
<p class="kicker">Overview</p>
|
| 48 |
+
<h1>What Botanic1 is</h1>
|
|
|
|
|
|
|
|
|
|
| 49 |
</div>
|
| 50 |
+
<p class="lede">Botanic1 is a family of four genomic language models for plants, from 318M to 3.2B parameters. Each one is a bidirectional Mamba2 encoder pre-trained with masked language modelling on single-nucleotide tokens, over 320 species spanning the land plants, on 8,192 bp windows. A context-extended version reads windows up to 131,072 bp.</p>
|
| 51 |
+
<p>The models read raw DNA and return two things. An embedding per nucleotide, which trains small probes or fine-tunes to a new task. And a likelihood for every base, which scores a variant with no annotation, alignment or population data. All four sizes rank at the top of the frozen-model benchmark against 20 plant and generalist genomic language models, after 314.6B training tokens where the closest competitor used about 4 trillion.</p>
|
| 52 |
+
<p>Botanic1-S, Botanic1-M and Botanic1-L are released for research use. The <a href="botanic1-report.pdf">technical report</a> has the full evaluation; this page is the short version.</p>
|
| 53 |
|
| 54 |
+
<h2 id="start">Get started</h2>
|
| 55 |
+
<div class="tablewrap">
|
| 56 |
+
<table class="models-table">
|
| 57 |
+
<thead><tr><th>Model</th><th class="n">Parameters</th><th class="n">Blocks</th><th class="n">Hidden size</th><th class="n">Context</th><th>Repository</th></tr></thead>
|
| 58 |
+
<tbody>
|
| 59 |
+
<tr class="ours"><td class="name">Botanic1-S</td><td class="n">318M</td><td class="n">24</td><td class="n">1,024</td><td class="n">8,192 bp</td><td><a href="https://huggingface.co/living-models/Botanic1-S">living-models/Botanic1-S</a></td></tr>
|
| 60 |
+
<tr class="ours"><td class="name">Botanic1-M</td><td class="n">688M</td><td class="n">52</td><td class="n">1,024</td><td class="n">8,192 bp</td><td><a href="https://huggingface.co/living-models/Botanic1-M">living-models/Botanic1-M</a></td></tr>
|
| 61 |
+
<tr class="ours"><td class="name">Botanic1-L</td><td class="n">2.11B</td><td class="n">72</td><td class="n">1,536</td><td class="n">8,192 bp</td><td><a href="https://huggingface.co/living-models/Botanic1-L">living-models/Botanic1-L</a></td></tr>
|
| 62 |
+
<tr class="ours"><td class="name">Botanic1-XL</td><td class="n">3.18B</td><td class="n">80</td><td class="n">1,792</td><td class="n">8,192 bp</td><td><a href="https://huggingface.co/living-models/Botanic1-XL">living-models/Botanic1-XL</a></td></tr>
|
| 63 |
+
</tbody>
|
| 64 |
+
<caption>Single-nucleotide tokeniser (A, T, C, G, N, one token per base). Pure PyTorch; runs on CPU, Apple Silicon and CUDA, where the <code>mamba_ssm</code> kernels are used when installed. Research use only, under the Living Models research licence.</caption>
|
| 65 |
+
</table>
|
| 66 |
+
</div>
|
| 67 |
+
<pre><code>from transformers import AutoModel, AutoTokenizer
|
| 68 |
+
|
| 69 |
+
name = "living-models/Botanic1-S"
|
| 70 |
+
model = AutoModel.from_pretrained(name, trust_remote_code=True)
|
| 71 |
+
tokenizer = AutoTokenizer.from_pretrained(name, trust_remote_code=True)
|
| 72 |
+
|
| 73 |
+
# embeddings, (batch, seq_len, hidden): frozen representations for probing or fine-tuning
|
| 74 |
+
emb = model.embed_sequences(["ACGTACGTNNACGT"], tokenizer,
|
| 75 |
+
average_with_reverse_complement=True)["post_encoder_norm"]
|
| 76 |
+
|
| 77 |
+
# zero-shot variant score: log P(alt) - log P(ref) at one position; deleterious variants score negative
|
| 78 |
+
llr = model.score_variant_zero_shot(sequence=sequence, tokenizer=tokenizer,
|
| 79 |
+
variant_pos=251, variant_char="C")</code></pre>
|
| 80 |
+
<p>Botanic1 also runs as a tool inside a Gemma 4 chat agent, with FASTA and VCF upload, in the <a href="https://huggingface.co/spaces/living-models/botanic-gemma-chat">Botanic × Gemma Space</a>. The <a href="factory.html">model factory</a> page describes how the models were built.</p>
|
| 81 |
|
| 82 |
+
<h2 id="overview-figure">How the models were trained and evaluated</h2>
|
| 83 |
<figure class="wide" id="figure1">
|
| 84 |
<div class="fig1" id="fig1"></div>
|
| 85 |
+
<figcaption><span class="figlabel">Figure 1</span> <b>From pre-training to the leaderboard.</b> <b>a</b>, Training: plant genomes only, masked language modelling on single nucleotides with 15% of positions hidden, a stack of bidirectional Mamba2 blocks released at four sizes. <b>b</b>, The six evaluation regimes, from probes on frozen representations to needle-in-a-haystack retrieval. <b>c</b>, Context sizes, from 100 bp probe tasks to 131,072 bp. <b>d</b>, Downstream task performance for 20 models, each bar topped by its organisation's logo; the axis starts at 0.5. <span class="small">Drawn live from the report's data; every mark carries its numbers on hover.</span></figcaption>
|
| 86 |
</figure>
|
| 87 |
|
| 88 |
+
<h2 id="zeroshot">Zero-shot variant scoring</h2>
|
| 89 |
<p class="kicker">Results</p>
|
| 90 |
+
<p>Given the reference sequence around a variant, Botanic1 returns the log-likelihood ratio of the alternate allele against the reference. On 545 experimentally validated causal variants across 14 plant species, each hidden among every documented SNV of its 100 kbp locus, Botanic1-XL places the causal variant in the top 1% of candidates in half of the studies (49.9%) and in the top 0.1% in 16.5%, ahead of PlantCAD2-L (44.6% and 9.0%) and of PhyloP, PhastCons and Ensembl VEP, which need alignments or annotations that Botanic1 does not.</p>
|
| 91 |
<figure class="wide">
|
| 92 |
<div id="zeroshot-curve"></div>
|
| 93 |
+
<figcaption><span class="figlabel">Figure</span> <b>Recovering experimentally validated causal variants.</b> 545 studies, each scored against every documented SNV in a 100 kb region around the causal variant: the fraction of studies whose causal variant falls inside a shortlist of the given size. Baselines that need alignments or annotations are dashed; the dotted line is a shuffled ranking. Every study is on <a href="causal-variants.html">the causal-variants page</a>.</figcaption>
|
| 94 |
</figure>
|
| 95 |
+
<p>Because the score comes from the sequence alone, it is not affected by linkage disequilibrium or by allele frequencies in any population. Used as a tool by a Gemma 4 chat agent on a published melon sex-determination locus, Botanic1-L let the agent rank the validated causal variant first among 3,061 candidates in all 20 runs; with conventional bioinformatics tools instead, the agent reached it within its top 20 in at most 35% of runs.</p>
|
| 96 |
|
| 97 |
+
<h2 id="scaling">Bigger is better, and small is already strong</h2>
|
| 98 |
+
<p>The four sizes share one recipe and one 314.6B-token horizon. Training loss and downstream task performance both improve with size, from 0.735 for Botanic1-S to 0.748 for Botanic1-XL on the aggregate over nine task families. Under the leaderboard's scoring, the four checkpoints take the top four positions, 0.758 to 0.769, above PlantCAD2-L at 0.756.</p>
|
| 99 |
<figure class="wide">
|
| 100 |
<div id="scaling-traces"></div>
|
| 101 |
+
<figcaption><span class="figlabel">Figure</span> <b>Scaling.</b> Training loss and downstream task performance against tokens seen for the four sizes; the dashed curve is the shared learning-rate schedule and stars mark the released checkpoints at 314.6B tokens. <span class="small">Hover for the four models at one token count.</span></figcaption>
|
| 102 |
</figure>
|
| 103 |
+
<p>Botanic1-S, at 318M parameters, outperforms models more than ten times larger, including Carbon-8B (0.727) and Evo 2 at 7B (0.695), and does so after 314.6B base pairs where PlantCAD2-L processed 4.03T and NTv3 12.1T. Botanic1-L and Botanic1-XL are significantly above PlantCAD2-L under a paired bootstrap over the nine families; Botanic1-S and Botanic1-M are not.</p>
|
| 104 |
<div class="card">
|
| 105 |
<div class="cardhead"><span class="label">Fig · size against score</span><span class="small">downstream task performance · 22 frozen-model metrics · log scale · hover a mark</span></div>
|
| 106 |
<div id="intro-chart"></div>
|
| 107 |
+
<p class="chartnote">Organisation logos are shown for attribution. <a href="leaderboard.html">Full leaderboard</a>, with the nine-family scorecard and the tokens-seen view.</p>
|
| 108 |
</div>
|
| 109 |
|
| 110 |
+
<h2 id="finetune">Fine-tuning: easier, and better than other foundation models</h2>
|
| 111 |
+
<p>Botanic1-S adapts with full fine-tuning, LoRA or IA<sup>3</sup>. On four regulatory tasks from the Plant Genomic Benchmark it beats AgroNT and PlantCAD2-L in regression (Pearson <span class="math">r</span> 0.861 against 0.834 and 0.790 for terminator strength) and leads poly(A) and lncRNA classification in every setting but one. Fine-tuning PlantCAD2-L with the authors' code needed model modifications to run stably; Botanic1-S and AgroNT did not.</p>
|
| 112 |
<figure class="wide">
|
| 113 |
<div id="finetune-chart"></div>
|
| 114 |
+
<figcaption><span class="figlabel">Figure</span> <b>Fine-tuning across four plant-regulatory tasks.</b> Held-out test metric per model and adaptation regime; regression tasks report Pearson <span class="math">r</span> and R<sup>2</sup>, classification tasks the mean AUROC over six species. The dashed line is the best task-specific model trained from scratch. <span class="small">Pick a task and metric above; hover a bar for every number of its cell.</span></figcaption>
|
| 115 |
</figure>
|
| 116 |
+
<h3 id="atac">Base-resolution chromatin accessibility</h3>
|
| 117 |
+
<p>Fine-tuned to predict ATAC-seq profiles at single-base resolution, Botanic1-S beats ChromBPNet, the specialised model for this task, in Arabidopsis and in maize, on read-count correlation and on profile shape. The same architecture trained from random weights does not, so the gain comes from pre-training.</p>
|
| 118 |
<figure class="wide">
|
| 119 |
<div id="atac-chart"></div>
|
| 120 |
+
<figcaption><span class="figlabel">Figure</span> <b>Base-resolution ATAC-seq prediction against ChromBPNet.</b> Held-out chromosomes: Arabidopsis chromosome 4 (4,646 peaks) and maize chromosomes 1 and 9 (23,220 peaks). Whiskers are 95% bootstrap intervals; ChromBPNet is scored by its own pipeline, without intervals. <span class="small">Switch the metric above; hover a bar for all four.</span></figcaption>
|
| 121 |
</figure>
|
| 122 |
|
| 123 |
+
<h3 id="tf">Genome-wide transcription factor binding</h3>
|
| 124 |
+
<p>Fine-tuned on 568 <em>Arabidopsis thaliana</em> DAP-seq assays grouped into 46 transcription factor families, the context-extended Botanic1-S reaches a chromosome 5 macro average precision of 0.72, above NTv3 (0.69 and 0.68), AgroNT (0.63) and DeepCistrome, a bespoke architecture trained from scratch (0.62), at a lower compute budget than all but the smallest NTv3.</p>
|
| 125 |
<figure class="wide">
|
| 126 |
<div id="tf-chart"></div>
|
| 127 |
<figcaption><span class="figlabel">Figure</span> <b>Genome-wide transcription factor family binding in <em>Arabidopsis thaliana</em>.</b> Macro average precision across three seeds on the chromosome 5 test set; circles are the seeds. The dashed line is the bespoke architecture trained from scratch, DeepCistrome. <span class="small">Hover a bar for the seeds and the paired margin against Botanic1-S.</span></figcaption>
|
| 128 |
</figure>
|
| 129 |
|
| 130 |
+
<h2 id="more">More in the technical report</h2>
|
| 131 |
+
<p><b>BOTANIC-1: a series of long-context plant genomic foundation models in the agentic era.</b> Amélie Barozet<sup>★</sup>, Vincent Cabeli<sup>★</sup>, Jean Ogier du Terrail<sup>★</sup>, Alexey Rukhovich<sup>★</sup>, Thomas Janssoone<sup>‡</sup>, Gary Klajer<sup>‡</sup>, Zeinab Sheikhitarghi<sup>‡</sup>, Gregory Andrews, Cyril Veran, Léonard Strouk. Living Models, Paris. <a href="botanic1-report.pdf">PDF</a>. <span class="small"><sup>★</sup> equal contribution, alphabetical; <sup>‡</sup> core team, alphabetical.</span></p>
|
| 132 |
<ul class="also">
|
| 133 |
+
<li><a href="factory.html">The model factory</a><span class="small">How agents run the experiments, and the data pipeline behind the 320-species corpus.</span></li>
|
| 134 |
+
<li><a href="botanic1-report.pdf">Long context up to 131 kbp</a><span class="small">Pseudo-perplexity, needle-in-a-haystack retrieval and chromatin accessibility at long context.</span></li>
|
| 135 |
+
<li><a href="botanic1-report.pdf">What the model has learned</a><span class="small">Sparse autoencoder features that read as splice sites, introns and codon position.</span></li>
|
| 136 |
+
<li><a href="botanic1-report.pdf">Botanic1 as a tool for an agent</a><span class="small">The melon <em>CmEIN3</em> locus with a Gemma 4 agent calling Botanic1-L.</span></li>
|
| 137 |
<li><a href="leaderboard.html">Leaderboard</a><span class="small">Downstream task performance, the nine-family scorecard, size and tokens seen.</span></li>
|
| 138 |
<li><a href="corpus.html">Pre-training corpus</a><span class="small">The 320 species as a treemap, with the weighting and deduplication passages.</span></li>
|
| 139 |
</ul>
|
| 140 |
|
| 141 |
+
<h2 id="availability">Downloads</h2>
|
| 142 |
<ul>
|
| 143 |
<li><b>Models.</b> Botanic1-S, Botanic1-M, Botanic1-L and Botanic1-XL in the <a href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Botanic1 collection</a>, with the previous generation <a href="https://huggingface.co/living-models/Botanic0-L">Botanic0</a>. Research use only, under the Living Models research licence.</li>
|
| 144 |
<li><b>Pre-training data.</b> The 8 kbp corpus, with source manifests and assembly metadata: <a href="https://huggingface.co/datasets/living-models/Botanic1-pretraining">living-models/Botanic1-pretraining</a>.</li>
|
| 145 |
<li><b>Causal-variant benchmark.</b> The 545 tasks, their candidate SNPs and the curation audit: <a href="https://huggingface.co/datasets/living-models/Botanic1-causal-variants">living-models/Botanic1-causal-variants</a>.</li>
|
| 146 |
+
<li><b>Technical report.</b> <a href="botanic1-report.pdf">PDF</a>.</li>
|
| 147 |
</ul>
|
| 148 |
|
| 149 |
<h2 id="cite">Citation</h2>
|
|
|
|
| 162 |
|
| 163 |
<footer class="site-footer">
|
| 164 |
<p>Living Models, Paris. Botanic1 is released for research use under the Living Models research licence. Organisation logos are third-party trademarks shown for attribution.</p>
|
| 165 |
+
<p>Numbers on this page are the technical report's; figures are drawn live from its data tables.</p>
|
| 166 |
</footer>
|
| 167 |
</div>
|
| 168 |
<div class="tip" id="tip" role="tooltip"></div>
|
leaderboard.html
CHANGED
|
@@ -12,7 +12,7 @@
|
|
| 12 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 13 |
</head>
|
| 14 |
<body>
|
| 15 |
-
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">
|
| 16 |
<div class="page">
|
| 17 |
|
| 18 |
<main class="prose">
|
|
|
|
| 12 |
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script>
|
| 13 |
</head>
|
| 14 |
<body>
|
| 15 |
+
<header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org">Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html">Model factory</a><a href="leaderboard.html" aria-current="page">Leaderboard</a><a href="causal-variants.html">Causal variants</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d">Models</a></nav></div></header>
|
| 16 |
<div class="page">
|
| 17 |
|
| 18 |
<main class="prose">
|
scripts/patch_headers.py
CHANGED
|
@@ -12,13 +12,13 @@ from pathlib import Path
|
|
| 12 |
ROOT = Path(__file__).resolve().parents[1]
|
| 13 |
COLLECTION = "https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d"
|
| 14 |
PAGES = {
|
| 15 |
-
"index.html": "
|
| 16 |
"factory.html": "Model factory",
|
| 17 |
"leaderboard.html": "Leaderboard",
|
| 18 |
"causal-variants.html": "Causal variants",
|
| 19 |
"corpus.html": "Corpus",
|
| 20 |
}
|
| 21 |
-
TABS = [("index.html", "
|
| 22 |
("causal-variants.html", "Causal variants"), ("corpus.html", "Corpus")]
|
| 23 |
LOCKUP = ('<div class="wordmark" aria-label="BOTANIC-1">'
|
| 24 |
+ "".join(f'<span class="cell">{ch}</span>' for ch in "BOTANIC")
|
|
|
|
| 12 |
ROOT = Path(__file__).resolve().parents[1]
|
| 13 |
COLLECTION = "https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d"
|
| 14 |
PAGES = {
|
| 15 |
+
"index.html": "Overview",
|
| 16 |
"factory.html": "Model factory",
|
| 17 |
"leaderboard.html": "Leaderboard",
|
| 18 |
"causal-variants.html": "Causal variants",
|
| 19 |
"corpus.html": "Corpus",
|
| 20 |
}
|
| 21 |
+
TABS = [("index.html", "Overview"), ("factory.html", "Model factory"), ("leaderboard.html", "Leaderboard"),
|
| 22 |
("causal-variants.html", "Causal variants"), ("corpus.html", "Corpus")]
|
| 23 |
LOCKUP = ('<div class="wordmark" aria-label="BOTANIC-1">'
|
| 24 |
+ "".join(f'<span class="cell">{ch}</span>' for ch in "BOTANIC")
|