Update README.md

README.md

@@ -16,78 +16,57 @@ tags:
   - insilicomedicine
 language:
   - en
-model-index:
-  - name: longevity-llm
-    results:
-      - task:
-          type: text-generation
-        dataset:
-          name: LongeBench (aging clocks, methylation, proteomics, NHANES)
-          type: in-house
-        metrics:
-          - type: accuracy
-            value: 0.565
-            name: Classification avg accuracy (29 tasks)
-          - type: mae
-            value: 21.71
-            name: Regression avg MAE (8 tasks)
-          - type: jaccard
-            value: 0.510
-            name: Generation avg Jaccard (7 tasks)
 ---
 
 # Longevity-LLM (L-LLM)
 
-A domain-adapted **Qwen3.5-9B** for aging and longevity biology. L-LLM is the
-phase-1 continued-pretraining + phase-2 supervised fine-tuning (with a third
-reasoning-augmented continuation pass) of Qwen3.5-9B on ≈1.88M examples
-(≈7.0B tokens) drawn from clinical aging, epigenomics, transcriptomics,
-proteomics, and genetics tasks. The two trained LoRA adapters were
-concatenated into a single rank-64 LoRA and merged into the base weights to
-produce this standalone bf16 checkpoint.
+A domain-adapted **Qwen3.5-9B** for aging and longevity biology. L-LLM is
+the result of continued pretraining + supervised fine-tuning + a
+reasoning-augmented continuation pass on a multi-domain corpus spanning
+clinical aging, epigenomics, transcriptomics, proteomics, and genetics.
+The two trained LoRA adapters were concatenated into a single rank-64 LoRA
+and merged into the base weights to produce this standalone bf16
+checkpoint.
 
 ## Methods
 
 L-LLM was built by LoRA fine-tuning Qwen3.5-9B, a 9B-parameter hybrid
 transformer that interleaves Gated DeltaNet linear-attention layers with
-standard self-attention in a 3:1 ratio.
-
-Training data: ≈1.88M examples / ≈7.0B tokens over 50 tasks in seven datasets.
-Two were domain-knowledge priors — **QPT_Gene** (≈224K examples: UniProt
-annotations, GO terms, PPIs, pathway membership) and **QPT_Clock** (≈926K
-examples: published aging-clock formulas, CpG-site coefficients, gene-to-clock
-mappings, feature importance from Biolearn). The remaining ≈727K examples
-covered prediction tasks across **Clinical** (NHANES age/mortality),
-**Epigenomics** (GEO DNAm age, CpG methylation, clock proxies),
-**Transcriptomics** (GTEx age, TCGA survival, expression generation),
-**Proteomics** (Olink age, protein generation, clock proxies), and
-**Genetics** (OpenGenes expression directionality, SynergyAge lifespan,
-CellAge senescence, anti-aging target classification). ≈90K prediction
-examples carried frontier-model thinking-trace reasoning. A continuation
-corpus of 286K reasoning-augmented examples (≈1.13B tokens) was assembled by
-re-mixing prediction tasks with additional reasoning traces.
-
-Three training stages on Qwen3.5-9B:
-
-1. **Continued pretraining** (phase 1) — rank 32, α=32, rsLoRA scaling,
-   dropout 0.0, LR 2 × 10⁻⁵, 3 epochs, raw text packed into 4,096-token
-   blocks, effective BS 32.
-2. **Supervised fine-tuning** (phase 2) — rank 32, α=32, standard α/r
-   scaling, dropout 0.05, LR 1 × 10⁻⁴, 3 epochs, conversation format,
-   effective BS 16. Initialized from the phase-1 adapter via LLaMA-Factory's
-   `adapter_name_or_path`.
-3. **Reasoning continuation** (phase 2-continue) — continued the SFT adapter
-   from its 9,500-step checkpoint for ≈2,504 additional steps (≈1 epoch) on
-   the 286K reasoning-augmented corpus, LR 3 × 10⁻⁵, 3% warmup, context
-   32,768 with example packing, effective BS 16.
+standard self-attention in a 3:1 ratio. Training data was assembled across
+three main domains:
+
+| Domain | Sources |
+|---|---|
+| Knowledge priors | UniProt protein/gene annotations, Gene Ontology, protein–protein interactions, pathway membership; published aging-clock formulas and CpG-site coefficients (Biolearn) |
+| Clinical & epidemiology | NHANES (age, mortality) |
+| Epigenomics | GEO DNA-methylation cohorts, CpG methylation profiles, aging-clock proxy tasks |
+| Transcriptomics | GTEx (tissue age), TCGA (cancer survival), expression-profile generation |
+| Proteomics | Olink plasma-proteomics panels, proteomic clock proxy tasks |
+| Genetics & longevity | OpenGenes (expression directionality), SynergyAge (lifespan), CellAge (senescence), anti-aging target classification |
+| Reasoning corpus | Prediction tasks augmented with frontier-model chain-of-thought traces |
+
+Approximate scale across all domains: ≈10⁶ training prompts at the order of
+several billion tokens total. Exact composition, prompt counts, and token
+counts will be reported in the forthcoming preprint.
+
+Training proceeded in three stages on Qwen3.5-9B:
+
+1. **Continued pretraining** — knowledge priors only, raw text packed into
+   4,096-token blocks. Rank-32 LoRA with rsLoRA scaling, LR 2 × 10⁻⁵,
+   3 epochs.
+2. **Supervised fine-tuning** — aging prediction tasks in conversation
+   format. Rank-32 LoRA initialized from the phase-1 adapter, standard α/r
+   scaling, LR 1 × 10⁻⁴, 3 epochs.
+3. **Reasoning continuation** — continued the SFT adapter on the reasoning
+   corpus, LR 3 × 10⁻⁵, ≈1 epoch, context length 32,768 with example
+   packing.
 
 All adapters targeted the 12 linear projections including the GatedDeltaNet
-modules (`in_proj_qkv`, `in_proj_z`, `in_proj_a`, `in_proj_b`, `out_proj`).
-After training, the CPT adapter and the continued-SFT adapter were
-**concatenated into a single rank-64 LoRA and merged into the base weights**
-to produce this checkpoint. DeepSpeed ZeRO-2 on 2× NVIDIA H100 NVL 94GB GPUs,
-bf16, tf32 matmul, gradient checkpointing, Liger fused kernels (continuation
-stage), flash-attention 2 for self-attention layers.
+modules. After training, the CPT and continued-SFT adapters were
+concatenated into a single rank-64 LoRA and merged into the base weights to
+produce this checkpoint. DeepSpeed ZeRO-2 on 2× NVIDIA H100 NVL 94 GB,
+bf16, flash-attention 2 on self-attention layers. Full details in the
+forthcoming preprint.
 
 ## Example usage