jiosephlee commited on
Commit
382b399
·
verified ·
1 Parent(s): 1ee112e

Add model card and evaluation results

Browse files
Files changed (1) hide show
  1. README.md +28 -7
README.md CHANGED
@@ -1,19 +1,40 @@
1
  ---
2
  base_model: jiosephlee/Intern-S1-mini-lm
 
3
  datasets:
4
  - jiosephlee/assay-transfer-record-level-v27-bbb-martins-l3-intern
5
  tags:
 
6
  - assay-transfer
7
- - bbb-martins
8
- - molecular-ranking
9
  ---
 
10
 
11
- # Intern-S1-mini: BBB Martins record-level assay transfer V27 L3
12
 
13
- This is the selected full-model checkpoint from a 10-epoch assay-transfer training run on the [BBB Martins V27 L3 dataset](https://huggingface.co/datasets/jiosephlee/assay-transfer-record-level-v27-bbb-martins-l3-intern), pinned to dataset revision `2a4c6848de474f0c9152e3acdf4087ce1fe16db8`. It is trained to score A/B assay-transfer prompts, not as a general-purpose molecule predictor.
14
 
15
- The checkpoint was selected at optimizer step 510 by the lowest validation ranking **joint ID/OOD level-macro KNN regression MAE@3** (0.5804). Ranking validation ran every 10 optimizer steps; the effective batch size was 128 (8 GPUs × device batch 4 × gradient accumulation 4). Training details and curves are in the [W&B training run](https://wandb.ai/upenn-ml/record-level-assay-transfer/runs/peyddfo8).
 
 
 
 
 
 
 
 
 
 
16
 
17
- On the held-out test ranking split, KNN binary macro F1@3/F1@5 was **0.8611/0.8611** for this model, **0.7432/0.7677** for vanilla Morgan, and **0.7345/0.7677** for weighted Morgan (higher is better). Overall Spearman was **0.5149**, versus -0.0088 and 0.0238 for the Morgan baselines. Full comparison metrics are in the [W&B test run](https://wandb.ai/upenn-ml/record-level-assay-transfer/runs/gelk5fi5).
18
 
19
- The repository includes the tokenizer, model weights, and `metric.json` with checkpoint-selection details. Loading requires `trust_remote_code=True` for the custom tokenizer.
 
 
 
 
 
 
 
 
 
1
  ---
2
  base_model: jiosephlee/Intern-S1-mini-lm
3
+ library_name: transformers
4
  datasets:
5
  - jiosephlee/assay-transfer-record-level-v27-bbb-martins-l3-intern
6
  tags:
7
+ - chemistry
8
  - assay-transfer
9
+ - record-level
10
+ - ranking
11
  ---
12
+ # Intern-S1-mini assay transfer V27 — BBB Martins L3
13
 
14
+ Validation-selected checkpoint from the V27 BBB Martins L3 record-level assay-transfer run.
15
 
16
+ ## Provenance
17
 
18
+ - Base model: `jiosephlee/Intern-S1-mini-lm`
19
+ - Base model revision: `fcb667c380ae01f57693a45b4b5c2d331052a107`
20
+ - Training dataset: `jiosephlee/assay-transfer-record-level-v27-bbb-martins-l3-intern`
21
+ - Dataset revision: `2a4c6848de474f0c9152e3acdf4087ce1fe16db8`
22
+ - Training configuration: 10 scheduled epochs, seed 42, 8 GPUs, device batch size 4, gradient accumulation 4, Flash Attention 2, BFD packing, and padding-free training
23
+ - Selection metric: validation `L3/overall/knn_regression_id_ood_level_macro_mae_at_3` (lower is better)
24
+ - Selected optimizer step: 510
25
+ - Selected validation metric: `0.5804`
26
+ - The run was intentionally stopped before completing the full scheduled training; this repository contains its saved validation-selected checkpoint.
27
+ - [Training run](https://wandb.ai/upenn-ml/record-level-assay-transfer/runs/peyddfo8)
28
+ - [Held-out model/Morgan comparison](https://wandb.ai/upenn-ml/record-level-assay-transfer/runs/gelk5fi5)
29
 
30
+ ## Held-out test comparison
31
 
32
+ | Ranker | Binary macro-F1@3 | Binary macro-F1@5 | Regression MAE@3 | Spearman@3 | Top-1 hit@3 |
33
+ |---|---:|---:|---:|---:|---:|
34
+ | Model | 0.8611 | 0.8611 | 0.7717 | -0.0802 | 0.1408 |
35
+ | Morgan vanilla | 0.7432 | 0.7677 | 1.1246 | -0.0382 | 0.1127 |
36
+ | Morgan weighted | 0.7345 | 0.7677 | 1.0652 | -0.1077 | 0.1408 |
37
+
38
+ Lower regression MAE is better; higher values are better for the other metrics. All W&B metric names retain the `L3/` prefix.
39
+
40
+ The repository contains the full Transformers checkpoint and tokenizer files. `metric.json` is the complete validation-selection record stored with the checkpoint.