Joesh1 commited on
Commit
f6328a1
·
verified ·
1 Parent(s): 13ca273

Upload 2 files

Browse files
Files changed (2) hide show
  1. README.md +120 -5
  2. assets/onca-logo-horizontal.svg +30 -0
README.md CHANGED
@@ -1,22 +1,52 @@
1
  ---
2
  base_model: Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled
3
  library_name: transformers
 
4
  pipeline_tag: text-generation
 
 
5
  tags:
6
  - oncology
7
  - medical
 
8
  - qwen
9
  - sft
 
10
  ---
11
 
 
 
 
 
12
  # ONCA 1.5
13
 
14
- This repository contains the merged BF16 ONCA 1.5 release.
15
- It is based on `Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled`.
 
 
 
 
 
16
 
17
- Performance tables and benchmark details will be attached later.
 
 
 
 
 
 
 
18
 
19
- This release is intended for research use and is not for direct clinical decision-making.
 
 
 
 
 
 
 
 
 
20
 
21
  ## Quick Start
22
 
@@ -26,5 +56,90 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
26
  model_id = "YOUR_USERNAME/onca-1.5"
27
 
28
  tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
29
- model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
 
 
 
 
 
30
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  base_model: Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled
3
  library_name: transformers
4
+ license: apache-2.0
5
  pipeline_tag: text-generation
6
+ language:
7
+ - en
8
  tags:
9
  - oncology
10
  - medical
11
+ - pancreatic-cancer
12
  - qwen
13
  - sft
14
+ - research
15
  ---
16
 
17
+ <p align="center">
18
+ <img src="./assets/onca-logo-horizontal.svg" alt="Onca logo" width="520">
19
+ </p>
20
+
21
  # ONCA 1.5
22
 
23
+ > Open pancreatic cancer language model for four research-oriented workflows: trial screening, clinical reasoning, pathology extraction, and variant evidence interpretation.
24
+
25
+ ONCA 1.5 is the current continued-SFT release in the ONCA model line. It builds on `Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled` and follows the project direction established in [ONCA 1.0](https://huggingface.co/Joesh1/onca-1.0-9B): focus on pancreatic cancer workflows, open-data training assets, and practical structured prompting for clinical research use.
26
+
27
+ For ONCA 1.5, the training emphasis was to improve all four existing task families at once, with particular attention to parser-safe trial screening, preserved pathology extraction performance, concise clinical reasoning, and stronger variant evidence handling.
28
+
29
+ ## At a Glance
30
 
31
+ | Field | Value |
32
+ | --- | --- |
33
+ | Release | BF16 reference release |
34
+ | Base model | `Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled` |
35
+ | Architecture | Qwen3.5-class causal LM (`Qwen3_5ForCausalLM`) |
36
+ | Context window | 262,144 tokens |
37
+ | Training recipe | Continued SFT, no DPO |
38
+ | Domain focus | Pancreatic cancer and oncology-adjacent research workflows |
39
 
40
+ ## What This Release Is Good At
41
+
42
+ - Criterion-aware pancreatic cancer trial screening with explicit eligibility framing.
43
+ - Concise oncology reasoning and question answering for research workflows.
44
+ - Structured pathology abstraction with field-oriented prompting.
45
+ - Variant evidence interpretation with oncology context and uncertainty signaling.
46
+
47
+ ## Release Note
48
+
49
+ This is the main reference checkpoint in the ONCA 1.5 family and the best starting point if you want the least altered release.
50
 
51
  ## Quick Start
52
 
 
56
  model_id = "YOUR_USERNAME/onca-1.5"
57
 
58
  tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
59
+ model = AutoModelForCausalLM.from_pretrained(
60
+ model_id,
61
+ torch_dtype="auto",
62
+ device_map="auto",
63
+ trust_remote_code=True,
64
+ )
65
  ```
66
+
67
+ Use the included tokenizer and chat template for best results:
68
+
69
+ ```python
70
+ messages = [
71
+ {"role": "system", "content": "You are Onca, a pancreatic cancer clinical research assistant."},
72
+ {"role": "user", "content": "Extract tumor grade, margin status, pT, and pN from this pathology report as JSON."},
73
+ ]
74
+ text = tokenizer.apply_chat_template(
75
+ messages,
76
+ tokenize=False,
77
+ add_generation_prompt=True,
78
+ )
79
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
80
+ outputs = model.generate(
81
+ **inputs,
82
+ max_new_tokens=256,
83
+ do_sample=False,
84
+ temperature=0.2,
85
+ )
86
+ answer = outputs[0][inputs["input_ids"].shape[1]:]
87
+ print(tokenizer.decode(answer, skip_special_tokens=True))
88
+ ```
89
+
90
+ ## Prompting Tips
91
+
92
+ - Use the included chat template and ask for a specific output structure.
93
+ - For extraction tasks, request exact JSON keys or field names.
94
+ - For screening tasks, include both the patient summary and the trial criteria.
95
+ - Ask the model to state uncertainty and missing information explicitly.
96
+
97
+ ## Training Scope
98
+
99
+ The active ONCA 1.5 continued-SFT stack uses openly available data only and does not use DPO. The data mixture keeps all four task families in play, with weighting geared toward variant evidence repair and trial-screening stability while guarding against pathology regression.
100
+
101
+ | Task family | Active rows | Train | Val | Test |
102
+ | --- | ---: | ---: | ---: | ---: |
103
+ | Trial Screening | 12,137 | 10,921 | 608 | 608 |
104
+ | Clinical Reasoning | 3,496 | 3,146 | 174 | 176 |
105
+ | Pathology Extraction | 7,642 | 6,583 | 414 | 405 |
106
+ | Variant Evidence | 2,432 | 2,191 | 116 | 125 |
107
+ | Total | 25,707 | 22,841 | 1,312 | 1,314 |
108
+
109
+ Initial task weights in the active prepare stack are 27% trial screening, 18% clinical reasoning, 27% pathology extraction, and 28% variant evidence.
110
+
111
+ ## Benchmarks
112
+
113
+ Benchmark tables and comparative evaluation plots will be added later.
114
+
115
+ ## Repository Contents
116
+
117
+ - `model-*.safetensors`: sharded weights for this release.
118
+ - `model.safetensors.index.json`: shard map for the checkpoint files.
119
+ - `config.json`: architecture config and, for quantized variants, quantization metadata.
120
+ - `generation_config.json`: default generation settings.
121
+ - `tokenizer.json` and `tokenizer_config.json`: tokenizer assets.
122
+ - `chat_template.jinja`: chat formatting template for inference.
123
+ - `assets/onca-logo-horizontal.svg`: ONCA family logo used at the top of the model card.
124
+
125
+ ## Related Releases
126
+
127
+ - `onca-1.5`: BF16 reference release (this page).
128
+ - `onca-1.5-8bit`: 8-bit merged release.
129
+ - `onca-1.5-4bit`: 4-bit merged release.
130
+
131
+ ## Limitations and Safety
132
+
133
+ - This is a research model and not a clinical decision system.
134
+ - Outputs should be reviewed by qualified experts before any real-world use.
135
+ - The model is specialized for pancreatic cancer and oncology-adjacent workflows rather than broad general medicine.
136
+ - Variant evidence training includes broader oncology signal, but the intended framing of the model remains pancreatic cancer research support.
137
+ - Quantized releases are convenience variants and may behave slightly differently from the BF16 reference checkpoint.
138
+
139
+ ## Citation
140
+
141
+ A formal ONCA 1.5 citation block will be added with the accompanying manuscript. Until then, please cite the model repository and version used in your work.
142
+
143
+ ## Acknowledgments
144
+
145
+ ONCA 1.5 continues the ONCA project lineage from ONCA 1.0 and builds on the Qwen/Qwopus ecosystem plus the open-data contributors whose datasets made this release possible.
assets/onca-logo-horizontal.svg ADDED