Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,113 @@
+---
+base_model: yahmaachi/omni-dna-multitask-1b
+library_name: peft
+pipeline_tag: text-generation
+tags:
+  - lora
+  - dna
+  - genomic
+  - horizontal-gene-transfer
+  - hgt
+  - omnidna
+  - qlora
+---
+
+# Omni-DNA-Multitask-1B HGT Detection LoRA (Step 1500)
+
+## Model Description
+
+QLoRA adapter for Omni-DNA-Multitask-1B fine-tuned for **Horizontal Gene Transfer (HGT) detection**.
+
+Task: Binary classification - detect genomic islands (horizontally transferred genes).
+
+- Training data: IslandViewer 4 (11,182 train / 2,796 eval, balanced)
+- Format: DNA_sequence + HGT_detection_token + label (0 or 1)
+- Novel token: Token 4117 (HGT_detection) added to vocabulary
+
+## Training Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| LoRA Rank | 64 |
+| LoRA Alpha | 128 |
+| Target Modules | att_proj, attn_out, ff_proj, ff_out |
+| LoRA Dropout | 0.05 |
+| Quantization | 4-bit NF4 (QLoRA) |
+| Trainable Params | ~46.1M |
+| Learning Rate | 2e-4 (cosine) |
+| Batch Size | 8 (grad accum 4) |
+
+## Best Performance (Step 1300, Epoch 3.72)
+
+| Metric | Value |
+|--------|-------|
+| AUC | 0.8736 |
+| AP | 0.8735 |
+| F1 | 0.6857 |
+| Accuracy | 0.736 |
+
+## AUC Trajectory
+
+| Step | Epoch | AUC | F1 |
+|------|-------|-----|-----|
+| 100 | 0.29 | 0.6146 | 0.6967 |
+| 200 | 0.57 | 0.7620 | 0.7399 |
+| 300 | 0.86 | 0.8101 | 0.7069 |
+| 400 | 1.14 | 0.8244 | 0.7172 |
+| 500 | 1.43 | 0.8359 | 0.6844 |
+| 600 | 1.72 | 0.8381 | 0.6697 |
+| 700 | 2.00 | 0.8166 | 0.5854 |
+| 800 | 2.29 | 0.8015 | 0.3593 |
+| 900 | 2.57 | 0.8224 | 0.7445 |
+| 1000 | 2.86 | 0.8213 | 0.5407 |
+| 1100 | 3.14 | 0.8227 | 0.7000 |
+| 1200 | 3.43 | 0.7214 | 0.3933 |
+| 1300 | 3.72 | 0.8736 | 0.6857 **BEST** |
+| 1400 | 4.00 | 0.8267 | 0.7701 |
+| 1500 | 4.29 | 0.8389 | 0.7795 |
+| 1600 | 4.57 | 0.8048 | 0.5260 |
+| 1700 | 4.86 | 0.7646 | 0.4451 |
+| 1800 | 5.14 | 0.7948 | 0.4070 |
+| 1900 | 5.43 | 0.8115 | 0.5714 |
+| 2000 | 5.72 | 0.7920 | 0.5168 |
+| 2100 | 6.00 | 0.7985 | 0.5528 |
+| 2200 | 6.29 | 0.7355 | 0.3631 |
+| 2300 | 6.57 | 0.7689 | 0.3873 |
+
+## Usage
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+base = AutoModelForCausalLM.from_pretrained('yahmaachi/omni-dna-multitask-1b')
+tokenizer = AutoTokenizer.from_pretrained('yahmaachi/omni-dna-multitask-1b')
+tokenizer.add_tokens(['HGT_detection'])
+base.resize_token_embeddings(len(tokenizer))
+
+model = PeftModel.from_pretrained(base, 'Nhoodie/omni-dna-hgt-lora-best')
+model.eval()
+
+dna = 'ATGCGATCGATCGATCGATC...'  # your sequence
+inputs = tokenizer(dna + 'HGT_detection', return_tensors='pt')
+with torch.no_grad():
+    logits = model(**inputs).logits[:, -1, :]
+    prob = torch.softmax(logits, dim=-1)
+    # token 4097 = 1 (HGT), token 4096 = 0 (not HGT)
+    hgt_prob = prob[0, 4097].item()
+print(f'HGT probability: {hgt_prob:.4f}')
+```
+
+## Training Notes
+
+The training exhibited an interesting collapse-recovery pattern: after initial overfitting
+around epoch 2-3.4, the novel token (HGT_detection, #4117) underwent a representational
+reorganization, leading to a new best AUC of 0.8736 at step 1300.
+
+Steps 1300 and 1500 are sibling models - different representational equilibria rather than
+descendant relationships. Step 1300 optimizes ranking (AUC), step 1500 optimizes classification (F1).
+
+## License
+
+MIT