README.md

---
license: apache-2.0
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - moe
  - mixture-of-experts
  - gravity
  - txgravity
  - therapeutics
  - tdc
  - trillion-labs
base_model:
  - trillionlabs/Gravity-30B-A5B
---

# TxGravity-30B-A6B-Open

## Model Summary

**TxGravity-30B-A6B-Open** is a therapeutics-focused, task-specialized model built on the **Gravity-30B-A5B** base. It predicts a broad range of therapeutic properties — small-molecule ADMET, toxicity, drug–target interaction, protein–protein and peptide–MHC interaction, and more — by converting the **Therapeutic Data Commons (TDC)** benchmark tasks into an instruction format for LLMs, in the same spirit as Google's TxGemma.

TxGravity is **co-developed by SK Biopharmaceuticals (SKBP) and Trillion Labs**.

It is trained on TDC tasks reformatted as single-turn instructions (instruction → answer), covering 57 tasks. Answers are formatted as `(A)`/`(B)` for binary classification or a normalized `000`–`1000` bin for regression.

This release is the open-license version, with any datasets restricted to non-commercial use excluded from training.

| Property | Value |
|---|---|
| Total Parameters | 29.56B |
| Active Parameters | ~6B |
| Architecture | GravityMoE (DeepSeek-V3-compatible) |
| Layers | 52 |
| Routed / Shared Experts | 64 (top-8) / 1 |
| Context Length | 8,192 tokens |
| Precision | bf16 |
| Base model | Gravity-30B-A5B |

> ⚠️ This is a task-specialized property predictor, not a general instruction-tuned or safety-aligned assistant. Its outputs are intended for the TDC-style therapeutic prediction prompts it was trained on. Predictions may be inaccurate, biased, or incomplete and must be independently verified before any experimental, clinical, or decision-making use.

## Performance

Evaluated on the TDC therapeutic benchmark against a reproduced **TxGemma-27B** baseline. Over 60 comparable tasks: **16 wins / 26 ties / 18 losses** (6 tasks excluded for lack of a paper-reported comparison). A win/loss requires the margin to exceed the tie threshold; otherwise the task is scored a tie.

**Wins (16 tasks)** — TxGravity beats TxGemma-27B on: `bindingdb_ki`, `bindingdb_ic50`, `protein_sabdab`, `ppbr_az`, `caco2_wang`, `lipophilicity_astrazeneca`, `half_life_obach`, `clearance_hepatocyte_az`, `ld50_zhu`, `bioavailability_ma`, `buchwald_hartwig`, `drugcomb_css`, `drugcomb_loewe`, `drugcomb_zip`, `drugcomb_hsa`, `drugcomb_bliss` — spanning binding affinity, ADMET, toxicity, and drug-combination synergy.

Highlights: `bindingdb_ki` PCC **0.750** vs -0.112, `bindingdb_ic50` Spearman **0.766** vs 0.643, `protein_sabdab` MAE **0.969** vs 2.332, `ld50_zhu` MAE **0.651** vs 0.776.

### Full results

Margin is TxGravity - TxGemma, in percentage points (%p) for score metrics or relative percent (%rel) for error metrics (MAE/MSE, lower is better). Up-arrow = higher is better, down-arrow = lower is better.

| Task | Metric | N | TxGravity | TxGemma-27B | Margin | Verdict |
|---|---|---|---|---|---|---|
| bindingdb_ic50 | Spearman ↑ | 35725 | **0.7660** | 0.6430 | +12.3%p | WIN |
| bindingdb_ki | PCC ↑ | 11857 | **0.7500** | -0.1120 | +86.2%p | WIN |
| bioavailability_ma | AUROC ↑ | 128 | **0.7490** | 0.6970 | +5.2%p | WIN |
| buchwald_hartwig | PCC ↑ | 791 | **0.8920** | 0.8550 | +3.7%p | WIN |
| caco2_wang | MAE ↓ | 182 | **0.4160** | 0.4660 | +10.7%rel | WIN |
| clearance_hepatocyte_az | Spearman ↑ | 243 | **0.3240** | 0.2910 | +3.3%p | WIN |
| drugcomb_bliss | MAE ↓ | 59708 | **3.7370** | 3.9280 | +4.9%rel | WIN |
| drugcomb_css | MAE ↓ | 59708 | **7.8380** | 9.4180 | +16.8%rel | WIN |
| drugcomb_hsa | MAE ↓ | 59708 | **3.5840** | 3.7880 | +5.4%rel | WIN |
| drugcomb_loewe | MAE ↓ | 59708 | **6.4130** | 7.6720 | +16.4%rel | WIN |
| drugcomb_zip | MAE ↓ | 59708 | **3.0610** | 3.4100 | +10.2%rel | WIN |
| half_life_obach | Spearman ↑ | 135 | **0.3600** | 0.2890 | +7.1%p | WIN |
| ld50_zhu | MAE ↓ | 1478 | **0.6510** | 0.7760 | +16.1%rel | WIN |
| lipophilicity_astrazeneca | MAE ↓ | 840 | **0.5700** | 0.5950 | +4.2%rel | WIN |
| ppbr_az | MAE ↓ | 559 | **8.2900** | 9.4180 | +12.0%rel | WIN |
| protein_sabdab | MAE ↓ | 99 | **0.9690** | 2.3320 | +58.4%rel | WIN |
| ames | AUROC ↑ | 1457 | **0.8410** | 0.8300 | +1.1%p | TIE |
| bbb_martins | AUROC ↑ | 406 | **0.9190** | 0.8970 | +2.2%p | TIE |
| bindingdb_patent | PCC ↑ | 49028 | **0.5410** | 0.5140 | +2.7%p | TIE |
| carcinogens_lagunin | Accuracy ↑ | 56 | **0.8750** | 0.8930 | -1.8%p | TIE |
| clearance_microsome_az | Spearman ↑ | 221 | **0.5000** | 0.4870 | +1.3%p | TIE |
| clintox | AUROC ↑ | 297 | **0.7610** | 0.7560 | +0.5%p | TIE |
| cyp1a2_veith | AUPRC ↑ | 2517 | **0.9310** | 0.9350 | -0.4%p | TIE |
| cyp2c19_veith | AUROC ↑ | 2534 | **0.8940** | 0.8910 | +0.3%p | TIE |
| cyp2c9_veith | AUPRC ↑ | 2419 | **0.7920** | 0.7820 | +1.0%p | TIE |
| cyp2d6_veith | AUPRC ↑ | 2626 | **0.6840** | 0.6620 | +2.2%p | TIE |
| cyp3a4_veith | AUPRC ↑ | 2467 | **0.8610** | 0.8410 | +2.0%p | TIE |
| dili | AUROC ↑ | 96 | **0.8930** | 0.8890 | +0.4%p | TIE |
| herg | AUROC ↑ | 132 | **0.8930** | 0.9000 | -0.7%p | TIE |
| herg_central | AUROC ↑ | 61379 | **0.8760** | 0.8830 | -0.7%p | TIE |
| herg_karim | Accuracy ↑ | 2690 | **0.7920** | 0.8000 | -0.8%p | TIE |
| hia_hou | AUROC ↑ | 117 | **0.9860** | 0.9860 | +0.0%p | TIE |
| hiv | AUROC ↑ | 8227 | **0.7640** | 0.7930 | -2.9%p | TIE |
| huri | AUPRC ↑ | 4204 | **0.8080** | 0.7920 | +1.6%p | TIE |
| mhc2_iedb_jensen | AUROC ↑ | 26856 | **0.8680** | 0.8500 | +1.8%p | TIE |
| pgp_broccatelli | AUROC ↑ | 245 | **0.9380** | 0.9320 | +0.6%p | TIE |
| sarscov2_3clpro_diamond | AUROC ↑ | 176 | **0.7470** | 0.7250 | +2.2%p | TIE |
| skin_reaction | AUROC ↑ | 82 | **0.6510** | 0.6370 | +1.4%p | TIE |
| tox21 | AUROC ↑ | 15600 | **0.8230** | 0.8430 | -2.0%p | TIE |
| toxcast | AUROC ↑ | 307282 | **0.8970** | 0.9050 | -0.8%p | TIE |
| vdss_lombardo | Spearman ↑ | 226 | **0.5820** | 0.5660 | +1.6%p | TIE |
| weber | AUROC ↑ | 9417 | **0.7180** | 0.7390 | -2.1%p | TIE |
| bindingdb_kd | PCC ↑ | 1630 | **0.5220** | 0.5960 | -7.4%p | LOSS |
| butkiewicz | AUROC ↑ | 401997 | **0.7540** | 0.8620 | -10.8%p | LOSS |
| cyp2c9_substrate_carbonmangels | AUPRC ↑ | 135 | **0.3460** | 0.4730 | -12.7%p | LOSS |
| cyp2d6_substrate_carbonmangels | AUPRC ↑ | 135 | **0.7080** | 0.7380 | -3.0%p | LOSS |
| cyp3a4_substrate_carbonmangels | AUROC ↑ | 135 | **0.6410** | 0.6950 | -5.4%p | LOSS |
| davis | MSE ↓ | 1064 | **0.7600** | 0.6430 | -18.2%rel | LOSS |
| disgenet | MAE | 10495 | — | — | — | excluded |
| gdsc1 | PCC | 35462 | — | — | — | excluded |
| gdsc2 | PCC | 18541 | — | — | — | excluded |
| kiba | MSE ↓ | 4537 | **0.6370** | 0.4600 | -38.5%rel | LOSS |
| leenay | Spearman ↑ | 1520 | **0.1510** | 0.2260 | -7.5%p | LOSS |
| mhc1_iedb_imgt_nielsen | AUROC ↑ | 37197 | **0.9200** | 0.9680 | -4.8%p | LOSS |
| mirtarbase | Accuracy ↑ | 160033 | **0.5550** | 0.8040 | -24.9%p | LOSS |
| oncopolypharmacology | PCC ↑ | 4647 | **0.5050** | 0.5690 | -6.4%p | LOSS |
| pampa_ncats | AUROC ↑ | 408 | **0.6630** | 0.7080 | -4.5%p | LOSS |
| phase1 | AUROC | 561 | — | — | — | excluded |
| phase2 | AUROC | 1279 | — | — | — | excluded |
| phase3 | AUROC | 1200 | — | — | — | excluded |
| sabdab_chen | AUPRC ↑ | 482 | **0.6580** | 0.7080 | -5.0%p | LOSS |
| sarscov2_vitro_touret | AUROC ↑ | 298 | **0.4580** | 0.5300 | -7.2%p | LOSS |
| solubility_aqsoldb | MAE ↓ | 1997 | **0.8550** | 0.8050 | -6.2%rel | LOSS |
| tap | MAE ↓ | 240 | **5.4400** | 4.7850 | -13.7%rel | LOSS |
| uspto | Accuracy ↑ | 221648 | **0.0000** | 0.0880 | -8.8%p | LOSS |
| uspto_yields | PCC ↑ | 170728 | **-0.0571** | 0.1860 | -24.3%p | LOSS |

Also included in this repo as [`summary.csv`](./summary.csv).

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("trillionlabs/TxGravity-30B-A6B-Open")
model = AutoModelForCausalLM.from_pretrained(
    "trillionlabs/TxGravity-30B-A6B-Open", dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content": "<TDC-formatted instruction here>"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=16)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

The model expects TDC-style prompts and answers in TDC format: `(A)`/`(B)` for classification, or a `000`–`1000` normalized bin for regression.

## License
Apache 2.0

## Citation

```bibtex
@misc{txgravity2026,
  title        = {TxGravity-30B-A6B-Open},
  author       = {SK Biopharmaceuticals and Trillion Labs},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/trillionlabs/TxGravity-30B-A6B-Open}}
}
```

This model builds on the Therapeutic Data Commons (TDC) benchmark:

```bibtex
@article{huang2021therapeutics,
  title   = {Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development},
  author  = {Huang, Kexin and Fu, Tianfan and Gao, Wenhao and Zhao, Yue and Roohani, Yusuf and Leskovec, Jure and Coley, Connor W and Xiao, Cao and Sun, Jimeng and Zitnik, Marinka},
  journal = {Proceedings of Neural Information Processing Systems, NeurIPS Datasets and Benchmarks},
  year    = {2021}
}
```