Spaces:
Running
Running
Jean Ogier du Terrail
Point technical report links and BibTeX at bioRxiv v2 (2026.09.04.749355v2, posted 2026-09-09)
fd66581 | <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>Botanic1 model factory</title> | |
| <meta name="description" content="The agent-powered Model Factory behind Botanic1: orchestration, the pre-training data pipeline and its ablations."> | |
| <script>(function(){try{var t=new URLSearchParams(location.search).get('__theme');if(t==='dark'||t==='light')document.documentElement.setAttribute('data-theme',t);}catch(e){}})();</script> | |
| <link rel="preconnect" href="https://fonts.googleapis.com"> | |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> | |
| <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Manrope:wght@500;600;700;800&family=Spectral:ital,wght@0,400;0,500;0,600;1,400&family=IBM+Plex+Mono:wght@400;500;600&display=swap"> | |
| <link rel="stylesheet" href="assets/site.css?v=20260908-use-case"> | |
| <link rel="stylesheet" href="assets/factory.css?v=20260908-gallery"> | |
| <link rel="stylesheet" href="assets/factory-actors.css?v=20260908-stream"> | |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/7.9.0/d3.min.js"></script> | |
| </head> | |
| <body class="factory"> | |
| <header class="topbar"><div class="inner"><a class="lockup" href="index.html"><div class="wordmark" aria-label="BOTANIC-1"><span class="cell">B</span><span class="cell">O</span><span class="cell">T</span><span class="cell">A</span><span class="cell">N</span><span class="cell">I</span><span class="cell">C</span><span class="cell tok" data-state="masked"><span class="probs"><i></i><i class="top"></i><i></i><i></i></span>?</span></div><span class="org"><svg class="living-models-mark" width="18" height="20" fill="currentColor" aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 85.90 91.50"><rect x="55.1000" y="2.1000" width="3.2000" height="53.2000" /><rect x="55.1000" y="61.3000" width="3.2000" height="28.1000" /><rect x="68.9000" y="9.2000" width="3.2000" height="15.5000" /><rect x="68.9000" y="30.6000" width="3.2000" height="51.7000" /><rect x="82.7000" y="24.7000" width="3.2000" height="42.2000" /><rect x="41.4000" y="0.0000" width="3.2000" height="11.6000" /><rect x="41.4000" y="17.5000" width="3.2000" height="74.0000" /><rect x="27.6000" y="2.1000" width="3.2000" height="70.7000" /><rect x="27.6000" y="78.7000" width="3.2000" height="10.6000" /><rect x="13.8000" y="9.2000" width="3.2000" height="25.0000" /><rect x="13.8000" y="40.2000" width="3.2000" height="42.2000" /><rect x="0.0000" y="24.7000" width="3.2000" height="42.2000" /></svg>Living Models</span></a><nav aria-label="Pages"><a href="index.html">Overview</a><a href="factory.html" aria-current="page">Model factory</a><a href="leaderboard.html">Leaderboard</a><a href="causal-variants.html">Causal variants</a><a href="sae-features.html">SAE features</a><a href="use-case.html">Use case</a><a href="corpus.html">Corpus</a><a class="ext" href="https://huggingface.co/collections/living-models/botanic1-6a97f4e3c33f3d109a75057d" target="_blank" rel="noopener">Models</a></nav></div></header> | |
| <div class="page"> | |
| <main class="prose"> | |
| <p class="kicker">Methods</p> | |
| <h1>The Model Factory</h1> | |
| <p class="lede">Botanic1 is developed through repeated experiments on training data, model architecture, and evaluation. Researchers define the questions; agents manage the experimental infrastructure.</p> | |
| <p>The plan agent prepares configurations and code changes. An orchestrator dispatches and monitors training jobs, while a separate cluster evaluates the saved checkpoints. Researchers review the results and intervene throughout the process.</p> | |
| <figure class="wide authored-figure orchestration"> | |
| <div class="figure-scroll"> | |
| <div class="figstack" id="s1-fig"><img src="assets/figures/agentic_orchestration.svg" alt="Agentic orchestration of the experimentation loop: a plan agent, an orchestrator agent on a gateway, the training cluster, the evaluation cluster and the results server, with the research team intervening at each stage."></div> | |
| </div> | |
| </figure> | |
| <h2 id="data">Pre-training data</h2> | |
| <p>The pipeline controls which genomes enter the corpus, how sequence windows are sampled within each genome, and which windows are retained. Each choice is evaluated at a fixed model size and training budget.</p> | |
| <p><b>Window sampling.</b> Sampling more functional regions improves downstream performance across the tested species pools. We use a 1:1 mixture of functional and other windows, retaining more unique sequence than a stronger functional bias.</p> | |
| <p><b>Redundancy and species coverage.</b> A MinHash threshold of 0.3 reduces repeated windows within each species. Genome-length correction also improves performance. Broadening the species pool retains performance on the crop-focused evaluation suite; phylogenetic weighting and N-content filtering do not show consistent gains.</p> | |
| <figure class="wide authored-figure pipeline-figure"> | |
| <div class="figure-scroll"> | |
| <div class="figstack" id="pipeline-fig"><img src="assets/figures/data_pipeline.svg" alt="Pre-training-data preparation in three columns: genome selection, window construction, and window filtering."></div> | |
| </div> | |
| <div class="figure-tools" hidden><div class="figure-choices" role="group" aria-label="Select a data-pipeline stage"></div><button type="button" class="figure-zoom" aria-pressed="false">Enlarge figure</button></div> | |
| <figcaption><span class="figlabel">Figure</span> <b>Pre-training-data preparation.</b> Select a column to follow genome selection, window construction, or filtering.</figcaption> | |
| <div class="figure-detail" aria-live="polite" aria-atomic="true"><p>Genome selection controls redundancy between species. Window construction samples functional and background regions. Filtering removes ambiguous or redundant windows and adjusts representation by genome size.</p></div> | |
| </figure> | |
| <h2 id="ablations">Data ablations</h2> | |
| <p>The overview below shows the effect of each data choice. Select a row to inspect the individual comparisons and both seeds.</p> | |
| <figure class="wide"> | |
| <div class="figure-scroll ablation-scroll"><div class="figwrap" id="ablation-effects"></div></div> | |
| <figcaption><span class="figlabel">Figure</span> <b>Effects of data choices.</b> Change in S<sub>bal</sub><sup>train</sup>, in percentage points. Circles show individual comparison means, open circles and whiskers show the two seeds, and diamonds show the mean across comparisons.</figcaption> | |
| </figure> | |
| <details class="comparison-details" id="comparison-details"> | |
| <summary>Inspect individual comparisons</summary> | |
| <div class="ablation-explorer" id="ablation-explorer" aria-busy="true"> | |
| <fieldset class="decision-options" disabled> | |
| <legend>Explore a data choice</legend> | |
| <label><input type="radio" name="decision" value="sampling" checked><span>Window sampling</span></label> | |
| <label><input type="radio" name="decision" value="species"><span>Species coverage</span></label> | |
| <label><input type="radio" name="decision" value="redundancy"><span>Redundancy</span></label> | |
| <label><input type="radio" name="decision" value="length"><span>Genome weighting</span></label> | |
| <label><input type="radio" name="decision" value="phylogeny"><span>Phylogenetic weighting</span></label> | |
| <label><input type="radio" name="decision" value="ambiguous"><span>Ambiguous bases</span></label> | |
| </fieldset> | |
| <p class="factory-note" id="ablation-status" role="status">Loading measured comparisons…</p> | |
| <noscript><p><a href="https://www.biorxiv.org/content/10.64898/2026.09.04.749355v2" target="_blank" rel="noopener">Read the data ablations in the report</a>.</p></noscript> | |
| <div id="ablation-result" hidden> | |
| <div class="decision-summary" aria-live="polite" aria-atomic="true"> | |
| <div><p class="decision-stage" id="decision-stage"></p><h3 id="decision-title"></h3><p id="decision-description"></p></div> | |
| <div class="effect-summary"><span class="effect-number" id="effect-mean"></span><span class="small" id="effect-scope"></span></div> | |
| </div> | |
| <div class="comparison-control"> | |
| <label for="comparison-context">Other settings held fixed</label> | |
| <select id="comparison-context"></select> | |
| </div> | |
| <figure class="comparison-figure" aria-labelledby="decision-title"> | |
| <div class="effect-chart" id="effect-chart"></div> | |
| <figcaption>Change in S<sub>bal</sub><sup>train</sup>, in percentage points. Positive values favour the first setting named above. <span class="chart-legend"><span><i class="seed-mark"></i> Individual seeds</span><span><i class="mean-mark"></i> Two-seed mean</span></span></figcaption> | |
| </figure> | |
| <p class="decision-reading" id="decision-reading"></p> | |
| <details class="exact-values"><summary>See the values for both seeds</summary><div class="tablewrap"><table id="comparison-table"><caption>Differences in percentage points. Each row is one controlled comparison.</caption><thead><tr><th scope="col">Other settings</th><th scope="col" class="n">Seed 42</th><th scope="col" class="n">Seed 43</th><th scope="col" class="n">Mean</th></tr></thead><tbody></tbody></table></div></details> | |
| </div> | |
| </div> | |
| </details> | |
| <p class="factory-note">Each comparison uses a 300M-parameter model and two seeds, with matched optimiser steps, training tokens, and effective epochs. Seed ranges are not confidence intervals. The <a href="https://www.biorxiv.org/content/10.64898/2026.09.04.749355v2" target="_blank" rel="noopener">technical report</a> gives the methods and limitations; the <a href="corpus.html">corpus page</a> shows the resulting species mix.</p> | |
| </main> | |
| <footer class="site-footer"> | |
| <p>Living Models · Paris</p> | |
| <p><a href="https://www.biorxiv.org/content/10.64898/2026.09.04.749355v2" target="_blank" rel="noopener">Technical report</a> · <a href="index.html#models">Models & data</a></p> | |
| </footer> | |
| </div> | |
| <div class="tip" id="tip" role="tooltip"></div> | |
| <script src="assets/hero.js"></script> | |
| <script src="assets/sections.js?v=20260908-seoul-history"></script> | |
| <script src="assets/factory.js?v=20260908-seoul-history"></script> | |
| <script src="assets/factory-actors.js?v=20260908-stream"></script> | |
| </body> | |
| </html> | |