README: full TDC results table + wins summary; SKBP/Trillion co-development

README.md

@@ -22,6 +22,8 @@ base_model:
 
 **TxGravity-30B-A6B-Open** is a therapeutics-focused, task-specialized model built on the **Gravity-30B-A5B** base. It predicts a broad range of therapeutic properties — small-molecule ADMET, toxicity, drug–target interaction, protein–protein and peptide–MHC interaction, and more — following the instruction format of the **Therapeutic Data Commons (TDC)** benchmark, in the same spirit as Google's TxGemma.
 
+TxGravity is **co-developed by SK Biopharmaceuticals (SKBP) and Trillion Labs**.
+
 It is trained on TDC therapeutic instruction data (single-turn instruction → answer) covering 57 tasks. Answers are formatted as `(A)`/`(B)` for binary classification or a normalized `000`–`1000` bin for regression.
 
 | Property | Value |
@@ -39,26 +41,86 @@ It is trained on TDC therapeutic instruction data (single-turn instruction → a
 
 ## Performance
 
-Evaluated on the TDC therapeutic benchmark against a reproduced **TxGemma-27B** baseline across 60 comparable tasks:
-
-**16 wins · 26 ties · 18 losses**
-
-Selected results (TxGravity vs TxGemma-27B):
-
-| Task | Metric | TxGravity | TxGemma-27B |
-|---|---|---|---|
-| bindingdb_ki | PCC ↑ | **0.750** | −0.112 |
-| bindingdb_ic50 | Spearman ↑ | **0.766** | 0.643 |
-| protein_sabdab | MAE ↓ | **0.969** | 2.332 |
-| ld50_zhu | MAE ↓ | **0.651** | 0.776 |
-| drugcomb_css | MAE ↓ | **7.838** | 9.418 |
-| bioavailability_ma | AUROC ↑ | **0.749** | 0.697 |
-| bbb_martins | AUROC ↑ | **0.919** | 0.897 |
-| ames | AUROC ↑ | **0.841** | 0.830 |
-| davis | MSE ↓ | 0.760 | **0.643** |
-| kiba | MSE ↓ | 0.637 | **0.460** |
-
-Full per-task results (66 tasks, metric, N, margin, verdict) are in [`summary.csv`](./summary.csv).
+Evaluated on the TDC therapeutic benchmark against a reproduced **TxGemma-27B** baseline. Over 60 comparable tasks: **16 wins / 26 ties / 18 losses** (6 tasks excluded for lack of a paper-reported comparison). A win/loss requires the margin to exceed the tie threshold; otherwise the task is scored a tie.
+
+**Wins (16 tasks)** — TxGravity beats TxGemma-27B on: `bindingdb_ki`, `bindingdb_ic50`, `protein_sabdab`, `ppbr_az`, `caco2_wang`, `lipophilicity_astrazeneca`, `half_life_obach`, `clearance_hepatocyte_az`, `ld50_zhu`, `bioavailability_ma`, `buchwald_hartwig`, `drugcomb_css`, `drugcomb_loewe`, `drugcomb_zip`, `drugcomb_hsa`, `drugcomb_bliss` — spanning binding affinity, ADMET, toxicity, and drug-combination synergy.
+
+Highlights: `bindingdb_ki` PCC **0.750** vs -0.112, `bindingdb_ic50` Spearman **0.766** vs 0.643, `protein_sabdab` MAE **0.969** vs 2.332, `ld50_zhu` MAE **0.651** vs 0.776.
+
+### Full results
+
+Margin is TxGravity - TxGemma, in percentage points (%p) for score metrics or relative percent (%rel) for error metrics (MAE/MSE, lower is better). Up-arrow = higher is better, down-arrow = lower is better.
+
+| Task | Metric | N | TxGravity | TxGemma-27B | Margin | Verdict |
+|---|---|---|---|---|---|---|
+| bindingdb_ic50 | Spearman ↑ | 35725 | **0.7660** | 0.6430 | +12.3%p | WIN |
+| bindingdb_ki | PCC ↑ | 11857 | **0.7500** | -0.1120 | +86.2%p | WIN |
+| bioavailability_ma | AUROC ↑ | 128 | **0.7490** | 0.6970 | +5.2%p | WIN |
+| buchwald_hartwig | PCC ↑ | 791 | **0.8920** | 0.8550 | +3.7%p | WIN |
+| caco2_wang | MAE ↓ | 182 | **0.4160** | 0.4660 | +10.7%rel | WIN |
+| clearance_hepatocyte_az | Spearman ↑ | 243 | **0.3240** | 0.2910 | +3.3%p | WIN |
+| drugcomb_bliss | MAE ↓ | 59708 | **3.7370** | 3.9280 | +4.9%rel | WIN |
+| drugcomb_css | MAE ↓ | 59708 | **7.8380** | 9.4180 | +16.8%rel | WIN |
+| drugcomb_hsa | MAE ↓ | 59708 | **3.5840** | 3.7880 | +5.4%rel | WIN |
+| drugcomb_loewe | MAE ↓ | 59708 | **6.4130** | 7.6720 | +16.4%rel | WIN |
+| drugcomb_zip | MAE ↓ | 59708 | **3.0610** | 3.4100 | +10.2%rel | WIN |
+| half_life_obach | Spearman ↑ | 135 | **0.3600** | 0.2890 | +7.1%p | WIN |
+| ld50_zhu | MAE ↓ | 1478 | **0.6510** | 0.7760 | +16.1%rel | WIN |
+| lipophilicity_astrazeneca | MAE ↓ | 840 | **0.5700** | 0.5950 | +4.2%rel | WIN |
+| ppbr_az | MAE ↓ | 559 | **8.2900** | 9.4180 | +12.0%rel | WIN |
+| protein_sabdab | MAE ↓ | 99 | **0.9690** | 2.3320 | +58.4%rel | WIN |
+| ames | AUROC ↑ | 1457 | **0.8410** | 0.8300 | +1.1%p | TIE |
+| bbb_martins | AUROC ↑ | 406 | **0.9190** | 0.8970 | +2.2%p | TIE |
+| bindingdb_patent | PCC ↑ | 49028 | **0.5410** | 0.5140 | +2.7%p | TIE |
+| carcinogens_lagunin | Accuracy ↑ | 56 | **0.8750** | 0.8930 | -1.8%p | TIE |
+| clearance_microsome_az | Spearman ↑ | 221 | **0.5000** | 0.4870 | +1.3%p | TIE |
+| clintox | AUROC ↑ | 297 | **0.7610** | 0.7560 | +0.5%p | TIE |
+| cyp1a2_veith | AUPRC ↑ | 2517 | **0.9310** | 0.9350 | -0.4%p | TIE |
+| cyp2c19_veith | AUROC ↑ | 2534 | **0.8940** | 0.8910 | +0.3%p | TIE |
+| cyp2c9_veith | AUPRC ↑ | 2419 | **0.7920** | 0.7820 | +1.0%p | TIE |
+| cyp2d6_veith | AUPRC ↑ | 2626 | **0.6840** | 0.6620 | +2.2%p | TIE |
+| cyp3a4_veith | AUPRC ↑ | 2467 | **0.8610** | 0.8410 | +2.0%p | TIE |
+| dili | AUROC ↑ | 96 | **0.8930** | 0.8890 | +0.4%p | TIE |
+| herg | AUROC ↑ | 132 | **0.8930** | 0.9000 | -0.7%p | TIE |
+| herg_central | AUROC ↑ | 61379 | **0.8760** | 0.8830 | -0.7%p | TIE |
+| herg_karim | Accuracy ↑ | 2690 | **0.7920** | 0.8000 | -0.8%p | TIE |
+| hia_hou | AUROC ↑ | 117 | **0.9860** | 0.9860 | +0.0%p | TIE |
+| hiv | AUROC ↑ | 8227 | **0.7640** | 0.7930 | -2.9%p | TIE |
+| huri | AUPRC ↑ | 4204 | **0.8080** | 0.7920 | +1.6%p | TIE |
+| mhc2_iedb_jensen | AUROC ↑ | 26856 | **0.8680** | 0.8500 | +1.8%p | TIE |
+| pgp_broccatelli | AUROC ↑ | 245 | **0.9380** | 0.9320 | +0.6%p | TIE |
+| sarscov2_3clpro_diamond | AUROC ↑ | 176 | **0.7470** | 0.7250 | +2.2%p | TIE |
+| skin_reaction | AUROC ↑ | 82 | **0.6510** | 0.6370 | +1.4%p | TIE |
+| tox21 | AUROC ↑ | 15600 | **0.8230** | 0.8430 | -2.0%p | TIE |
+| toxcast | AUROC ↑ | 307282 | **0.8970** | 0.9050 | -0.8%p | TIE |
+| vdss_lombardo | Spearman ↑ | 226 | **0.5820** | 0.5660 | +1.6%p | TIE |
+| weber | AUROC ↑ | 9417 | **0.7180** | 0.7390 | -2.1%p | TIE |
+| bindingdb_kd | PCC ↑ | 1630 | **0.5220** | 0.5960 | -7.4%p | LOSS |
+| butkiewicz | AUROC ↑ | 401997 | **0.7540** | 0.8620 | -10.8%p | LOSS |
+| cyp2c9_substrate_carbonmangels | AUPRC ↑ | 135 | **0.3460** | 0.4730 | -12.7%p | LOSS |
+| cyp2d6_substrate_carbonmangels | AUPRC ↑ | 135 | **0.7080** | 0.7380 | -3.0%p | LOSS |
+| cyp3a4_substrate_carbonmangels | AUROC ↑ | 135 | **0.6410** | 0.6950 | -5.4%p | LOSS |
+| davis | MSE ↓ | 1064 | **0.7600** | 0.6430 | -18.2%rel | LOSS |
+| disgenet | MAE | 10495 | — | — | — | excluded |
+| gdsc1 | PCC | 35462 | — | — | — | excluded |
+| gdsc2 | PCC | 18541 | — | — | — | excluded |
+| kiba | MSE ↓ | 4537 | **0.6370** | 0.4600 | -38.5%rel | LOSS |
+| leenay | Spearman ↑ | 1520 | **0.1510** | 0.2260 | -7.5%p | LOSS |
+| mhc1_iedb_imgt_nielsen | AUROC ↑ | 37197 | **0.9200** | 0.9680 | -4.8%p | LOSS |
+| mirtarbase | Accuracy ↑ | 160033 | **0.5550** | 0.8040 | -24.9%p | LOSS |
+| oncopolypharmacology | PCC ↑ | 4647 | **0.5050** | 0.5690 | -6.4%p | LOSS |
+| pampa_ncats | AUROC ↑ | 408 | **0.6630** | 0.7080 | -4.5%p | LOSS |
+| phase1 | AUROC | 561 | — | — | — | excluded |
+| phase2 | AUROC | 1279 | — | — | — | excluded |
+| phase3 | AUROC | 1200 | — | — | — | excluded |
+| sabdab_chen | AUPRC ↑ | 482 | **0.6580** | 0.7080 | -5.0%p | LOSS |
+| sarscov2_vitro_touret | AUROC ↑ | 298 | **0.4580** | 0.5300 | -7.2%p | LOSS |
+| solubility_aqsoldb | MAE ↓ | 1997 | **0.8550** | 0.8050 | -6.2%rel | LOSS |
+| tap | MAE ↓ | 240 | **5.4400** | 4.7850 | -13.7%rel | LOSS |
+| uspto | Accuracy ↑ | 221648 | **0.0000** | 0.0880 | -8.8%p | LOSS |
+| uspto_yields | PCC ↑ | 170728 | **-0.0571** | 0.1860 | -24.3%p | LOSS |
+
+Also included in this repo as [`summary.csv`](./summary.csv).
 
 ## Usage