Add prominent warning about model effectiveness and unreliable SAD coefficient
Browse files
README.md
CHANGED
|
@@ -10,6 +10,17 @@ base_model: Nhoodie/omni-dna-ici-dc
|
|
| 10 |
pipeline_tag: text-generation
|
| 11 |
---
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
# Omni-DNA SAD Checkpoint
|
| 14 |
|
| 15 |
**Sequential Attenuation Denoising (SAD)** — ICI-DC checkpoint fine-tuned on real mutation data.
|
|
@@ -47,7 +58,7 @@ real data persist from ICI-DC; contradicted patterns get corrected.
|
|
| 47 |
### Axis 5: Mutation Surprise
|
| 48 |
|
| 49 |
| Model | Surprise | p-value | Interpretation |
|
| 50 |
-
|-------|:---:|:---:|---|
|
| 51 |
| Base Omni | +0.777 | <0.0001 | Expects parent (mutations are surprising) |
|
| 52 |
| ICI-DC | -0.216 | 0.0001 | Expects mutations everywhere |
|
| 53 |
| **SAD coeff=4.89** | -0.203 | 0.0003 | Partially attenuated |
|
|
|
|
| 10 |
pipeline_tag: text-generation
|
| 11 |
---
|
| 12 |
|
| 13 |
+
## ⚠️ IMPORTANT WARNING — Model Effectiveness
|
| 14 |
+
|
| 15 |
+
**This model's reported SAD coefficient is unreliable.** The ICI-DC (S1) and SAD (S2) training stages both underfit near-instantly. The best S1 checkpoint was selected at epoch 0.59 (step 150 of 2530), meaning the model had barely begun learning before checkpoint selection occurred. The SAD coefficient was calculated from training hyperparameters, NOT from converged training.
|
| 16 |
+
|
| 17 |
+
Subsequent testing revealed that the unmodified base Omni-DNA-20M model achieves **0.951 AUC** on discriminative mutation detection with raw DNA prompts — significantly outperforming ALL fine-tuned variants including this one. Fine-tuning with ICI-DC synthetic data systematically degraded the base model's innate mutation discrimination capability.
|
| 18 |
+
|
| 19 |
+
**This model is preserved for historical and reproducibility purposes only.**
|
| 20 |
+
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
|
| 24 |
# Omni-DNA SAD Checkpoint
|
| 25 |
|
| 26 |
**Sequential Attenuation Denoising (SAD)** — ICI-DC checkpoint fine-tuned on real mutation data.
|
|
|
|
| 58 |
### Axis 5: Mutation Surprise
|
| 59 |
|
| 60 |
| Model | Surprise | p-value | Interpretation |
|
| 61 |
+
|-------|:---:|:---:|:---|
|
| 62 |
| Base Omni | +0.777 | <0.0001 | Expects parent (mutations are surprising) |
|
| 63 |
| ICI-DC | -0.216 | 0.0001 | Expects mutations everywhere |
|
| 64 |
| **SAD coeff=4.89** | -0.203 | 0.0003 | Partially attenuated |
|