README.md

# Bio-ACDC: Biological Sequence Model Coevolution

An adaptation of [AC/DC (Assessment Coevolving with Diverse Capabilities)](https://acdc-llm.github.io) for biological language models.

## Overview

Bio-ACDC coevolves populations of biological language models (for DNA, RNA, and Protein sequences) with synthetic sequence tasks to discover specialized model experts.

## Core Components

### 1. Coevolution Loop (`core.py`)
- **Initialization**: Seed models are evaluated on base tasks
- **Offspring Generation**: Parents are merged and optionally mutated
- **Evaluation**: New models are tested on current task pool
- **Archive Update**: Dominated Novelty Search (DNS) maintains diverse Pareto archive
- **Task Generation**: New tasks target weaknesses discovered in the archive

### 2. Task Pool (`tasks.py`)
Generates and manages biological sequence tasks:
- **Protein tasks**: Motif recognition, sequence completion, structure prediction
- **DNA tasks**: Regulatory element detection, motif localization
- **RNA tasks**: Secondary structure prediction, motif finding

Tasks are auto-generated targeting archive weaknesses.

### 3. Model Merging (`mergers.py`)
Multiple merging strategies:
- **Linear Merge**: Weighted average of parent parameters
- **SLERP**: Spherical linear interpolation
- **Task Vector**: Arithmetic with base model subtraction

### 4. Mutation (`mutators.py`)
Controlled perturbation operators:
- **Gaussian Noise**: Add random noise to weights
- **Layer Scale**: Randomly scale specific layers
- **Dropout**: Structured pruning of weights

### 5. Archive (`archive.py`)
Dominated Novelty Search maintains a Pareto archive:
- Maximizes fitness + novelty
- Novelty based on unique capabilities vs fitter solutions
- Difficulty-aware weighting

### 6. Evaluator (`evaluator.py`)
Evaluates models on biological tasks:
- Sequence identity/similarity
- Motif containment
- Perplexity
- RNA structure prediction

## Usage

### Quick Start

```python
from bio_acdc import BioACDC, BioACDCConfig
from bio_acdc.tasks import BioTaskPool
from bio_acdc.mergers import LinearMerge
from bio_acdc.mutators import GaussianNoiseMutator
from bio_acdc.evaluator import BioEvaluator

# Configuration
config = BioACDCConfig(
    seed_model_paths=[
        "facebook/esm2_t33_650M_UR50D",
        "InstaDeepAI/nucleotide-transformer-v2-500m-multi-species",
    ],
    archive_size=20,
    num_generations=10,
    offspring_per_gen=5,
    output_dir="./bio_acdc_output",
)

# Components
task_pool = BioTaskPool(seed=42)
evaluator = BioEvaluator()
merger = LinearMerge()
mutator = GaussianNoiseMutator(std=0.01)

# Create Bio-ACDC
bio_acdc = BioACDC(
    config=config,
    task_pool=task_pool,
    evaluator=evaluator,
    merger=merger,
    mutator=mutator,
)

# Run evolution
final_archive = bio_acdc.evolve()

# Best model
best = bio_acdc.archive.get_best()
print(f"Best model: {best.model_path}, Fitness: {best.fitness:.4f}")
```

### Using with Custom Models

```python
# For ESM-2 protein models
config = BioACDCConfig(
    seed_model_paths=[
        "facebook/esm2_t33_650M_UR50D",
        "facebook/esm2_t30_150M_UR50D",
    ],
)

# For Nucleotide Transformer DNA models
config = BioACDCConfig(
    seed_model_paths=[
        "InstaDeepAI/nucleotide-transformer-v2-500m-multi-species",
        "InstaDeepAI/nucleotide-transformer-v2-100m-multi-species",
    ],
)
```

## Architecture Comparison: ACDC vs Bio-ACDC

| Feature | ACDC (SakanaAI) | Bio-ACDC |
|---------|-----------------|----------|
| Domain | General NLP (text) | Biological sequences |
| Seed Models | Qwen/Llama LLMs | ESM-2, NT, protein/DNA LMs |
| Tasks | Synthetic code/math/text | Motif detection, sequence completion, structure prediction |
| Evaluation | LLM-as-judge + code sandbox | Sequence similarity, biological metrics |
| Merging | SLERP, Linear, Task Vectors | Same (adapted for masked LMs) |
| Archive | Dominated Novelty Search | Same |
| Task Gen | LLM-based task creation | Rule-based + evolutionary targeting |

## Key Biological Adaptations

1. **Sequence-Specific Tasks**: Motifs, regulatory elements, structure prediction
2. **Token-Aware Evaluation**: Handles amino acids (20 AA), nucleotides (4 NT)
3. **Masked LM Support**: Works with ESM-2 and Nucleotide Transformer (masked language models)
4. **Motif-Based Difficulty**: Tasks target specific biological motifs
5. **Structure Evaluation**: RNA secondary structure comparison

## Requirements

```
torch>=2.0
transformers>=4.30
datasets
numpy
safetensors
biopython  # Optional, for advanced sequence analysis
```

## Citation

```bibtex
@software{bio_acdc_2024,
  title = {Bio-ACDC: Coevolution of Biological Language Models and Sequence Tasks},
  author = {Adapted from SakanaAI AC/DC},
  year = {2024},
  url = {https://acdc-llm.github.io}
}
```