Hugging Face's logo

soleaf
/
carbon-500m-lora-promoter-expression-rank

PEFT
Safetensors
biology
genomics
dna
promoter
expression
carbon
lora
adapter
sequence-regression
Model card Files
xet
 Community
soleaf commited on
Commit
93d150c
·
verified ·
1 Parent(s): b3da82d

Initial Carbon-500M LoRA promoter ranking adapters

Browse files
Files changed (24) hide show
  1. README.md +98 -0
  2. label_stats.json +7 -0
  3. metrics/final_test_metrics.csv +7 -0
  4. metrics/final_test_summary.json +31 -0
  5. metrics/freeze_manifest.json +31 -0
  6. metrics/low_label_summary_by_budget.csv +3 -0
  7. metrics/phase_stability_summary.json +14 -0
  8. metrics/validation_per_seed_metrics.csv +4 -0
  9. release_manifest.json +17 -0
  10. seed_20260523/adapter_config.json +25 -0
  11. seed_20260523/adapter_model.safetensors +3 -0
  12. seed_20260523/config_snapshot.json +27 -0
  13. seed_20260523/metrics.json +78 -0
  14. seed_20260523/model_lora_audit.json +114 -0
  15. seed_20260524/adapter_config.json +25 -0
  16. seed_20260524/adapter_model.safetensors +3 -0
  17. seed_20260524/config_snapshot.json +27 -0
  18. seed_20260524/metrics.json +78 -0
  19. seed_20260524/model_lora_audit.json +114 -0
  20. seed_20260525/adapter_config.json +25 -0
  21. seed_20260525/adapter_model.safetensors +3 -0
  22. seed_20260525/config_snapshot.json +27 -0
  23. seed_20260525/metrics.json +78 -0
  24. seed_20260525/model_lora_audit.json +114 -0
README.md ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: HuggingFaceBio/Carbon-500M
4
+ tags:
5
+ - biology
6
+ - genomics
7
+ - dna
8
+ - promoter
9
+ - expression
10
+ - carbon
11
+ - lora
12
+ - sequence-classification
13
+ pipeline_tag: text-classification
14
+ library_name: transformers
15
+ ---
16
+
17
+ # Carbon-500M LoRA for promoter expression ranking
18
+
19
+ This repository contains three LoRA sequence-classification adapters for `HuggingFaceBio/Carbon-500M`, trained to rank synthetic yeast promoter sequences by expression under the Random Promoter DREAM Challenge 2022 protocol.
20
+
21
+ The intended use is **rank prediction**, not calibrated absolute expression prediction.
22
+
23
+ ## Contents
24
+
25
+ - `seed_20260523/adapter_model.safetensors`
26
+ - `seed_20260524/adapter_model.safetensors`
27
+ - `seed_20260525/adapter_model.safetensors`
28
+ - `label_stats.json`
29
+ - `metrics/`
30
+ - `release_manifest.json`
31
+
32
+ The Carbon-500M base weights are not included. Load them from `HuggingFaceBio/Carbon-500M`.
33
+
34
+ ## Main locked final-test result
35
+
36
+ | Model | Spearman rho |
37
+ |---|---:|
38
+ | simple ridge | 0.585197 |
39
+ | k-mer 1-4 ridge | 0.710897 |
40
+ | Carbon LoRA seed 20260523 | 0.778652 |
41
+ | Carbon LoRA seed 20260524 | 0.790764 |
42
+ | Carbon LoRA seed 20260525 | 0.771563 |
43
+ | Carbon LoRA 3-seed mean prediction | 0.783603 |
44
+
45
+ The 3-seed mean prediction improves over the k-mer baseline by +0.072706 Spearman on the locked final test.
46
+
47
+ ## Training setup
48
+
49
+ - Base model: `HuggingFaceBio/Carbon-500M`
50
+ - Task wrapper: `LlamaForSequenceClassification`
51
+ - LoRA rank: 8
52
+ - LoRA alpha: 16
53
+ - LoRA dropout: 0.05
54
+ - Target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`
55
+ - Modules saved: `score`
56
+ - Max length: 256
57
+ - Train rows: 1,000,000
58
+ - Epochs: 1
59
+ - Primary metric: Spearman rank correlation
60
+
61
+ ## Loading notes
62
+
63
+ Each seed directory is structured like a PEFT adapter directory. A typical loading flow is:
64
+
65
+ ```python
66
+ from transformers import AutoModelForSequenceClassification, AutoTokenizer
67
+ from peft import PeftModel
68
+ import torch
69
+
70
+ base_id = "HuggingFaceBio/Carbon-500M"
71
+ adapter_id = "soleaf/carbon-500m-lora-promoter-expression-rank"
72
+ seed = "seed_20260524"
73
+
74
+ tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
75
+ model = AutoModelForSequenceClassification.from_pretrained(
76
+ base_id,
77
+ num_labels=1,
78
+ torch_dtype=torch.bfloat16,
79
+ trust_remote_code=True,
80
+ )
81
+ model = PeftModel.from_pretrained(model, adapter_id, subfolder=seed)
82
+ model.eval()
83
+ ```
84
+
85
+ Promoter DNA should be tokenized consistently with the experiment code: max length 256 and `add_special_tokens=False`.
86
+
87
+ ## Limitations
88
+
89
+ - This is a task-specific adapter for promoter-expression ranking under one experimental protocol.
90
+ - The final-test expression distribution is shifted relative to train normalization; absolute expression calibration is poor.
91
+ - Low-label LoRA runs at 6,044 and 60,436 rows underperformed matched k-mer controls.
92
+ - Phase/window perturbation diagnostics showed substantial prediction sensitivity.
93
+ - No robustness or broad SOTA claim is made.
94
+
95
+ ## Data and provenance
96
+
97
+ The raw and processed datasets are not redistributed in this model repo. See the associated GitHub reproducibility package for data provenance, code, evaluation protocol, and compact metrics.
98
+
label_stats.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "expression_mean": 11.145822088831892,
3
+ "expression_std": 2.371304223468006,
4
+ "normalization": "zscore_train_only",
5
+ "source_split": "train",
6
+ "train_valid_finite_expression_rows": 6043623
7
+ }
metrics/final_test_metrics.csv ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ model,spearman_rho,pearson_r,rmse_z,mae_z,rmse_expression,mae_expression,prediction_std_z
2
+ simple_ridge,0.5851972035445128,0.5791107774484756,4.620064325929989,4.590839999896166,10.95557804877165,10.88627828101964,0.6336010021720072
3
+ kmer_1_4_ridge,0.7108970672182182,0.6935416205345623,4.70507578492027,4.662317118639126,11.157166080518483,11.055772274576142,0.8106230965256014
4
+ carbon_seed_20260523,0.7786515821346187,0.7779813645407481,4.647999524958891,4.600902347790893,11.021820904212303,10.910139169080413,0.8765903812217523
5
+ carbon_seed_20260524,0.7907638801265369,0.7910450329283952,4.655807559149224,4.60832800615873,11.040336128664824,10.927747664130095,0.88576526361997
6
+ carbon_seed_20260525,0.7715627424009701,0.771221204350323,4.629860550766288,4.5848592528036285,10.97880787810001,10.872096110179614,0.8567131144146685
7
+ carbon_3seed_mean_prediction,0.7836031250366325,0.7834443465086167,4.643861775207591,4.598029868917751,11.012009040751392,10.903327647796706,0.8693944554117747
metrics/final_test_summary.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "primary_metric": "final_test_spearman_rho",
3
+ "primary_value": 0.7803260682207086,
4
+ "higher_is_better": true,
5
+ "status": "done",
6
+ "final_test_policy_satisfied": true,
7
+ "freeze_manifest_written_before_test_read": true,
8
+ "test_rows": 71103,
9
+ "kmer_final_spearman": 0.7108970672182182,
10
+ "carbon_3seed_mean_final_spearman": 0.7803260682207086,
11
+ "carbon_3seed_std_final_spearman": 0.009709472191032171,
12
+ "carbon_minus_kmer_mean_final_spearman": 0.06942900100249039,
13
+ "carbon_3seed_mean_prediction_metrics": {
14
+ "model": "carbon_3seed_mean_prediction",
15
+ "spearman_rho": 0.7836031250366325,
16
+ "pearson_r": 0.7834443465086167,
17
+ "rmse_z": 4.643861775207591,
18
+ "mae_z": 4.598029868917751,
19
+ "rmse_expression": 11.012009040751392,
20
+ "mae_expression": 10.903327647796706,
21
+ "prediction_std_z": 0.8693944554117747
22
+ },
23
+ "metric_table": "final_test_metrics.csv",
24
+ "prediction_table": "final_test_predictions.csv",
25
+ "device": "cuda",
26
+ "cuda_device_name": "NVIDIA GB10",
27
+ "use_bf16": true,
28
+ "peak_cuda_memory_gb": 5.036571979522705,
29
+ "elapsed_seconds": 202.0921413898468,
30
+ "final_test_evaluated": true
31
+ }
metrics/freeze_manifest.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "frozen_at": "2026-05-24T00:05:36+0900",
3
+ "policy": "This manifest is written before reading test_final.parquet metrics. No model/config/report-choice changes after final metrics.",
4
+ "test_file": "data/processed/test_final.parquet",
5
+ "primary_final_metric": "final_test_spearman_rho",
6
+ "frozen_models": {
7
+ "simple_ridge": {
8
+ "source_run": "exp_01_simple_and_kmer_baselines/r002",
9
+ "alpha": "from ridge_model_coefficients.npz"
10
+ },
11
+ "kmer_1_4_ridge": {
12
+ "source_run": "exp_01_simple_and_kmer_baselines/r002",
13
+ "alpha": "from ridge_model_coefficients.npz",
14
+ "role": "primary non-Carbon baseline"
15
+ },
16
+ "carbon_500m_lora_3seed_condition": {
17
+ "source_runs": [
18
+ "experiments/exp_04_carbon_500m_lora_validation/artifacts/runs/r005",
19
+ "experiments/exp_05_carbon_500m_lora_seed_confirm/artifacts/runs/r006/seed_20260524",
20
+ "experiments/exp_05_carbon_500m_lora_seed_confirm/artifacts/runs/r006/seed_20260525"
21
+ ],
22
+ "role": "primary Carbon condition reported as per-seed metrics plus 3-seed mean"
23
+ }
24
+ },
25
+ "known_caveats_before_final_read": [
26
+ "Carbon low-label 6k/60k underperforms matched k-mer controls.",
27
+ "Phase/window diagnostic found substantial prediction sensitivity.",
28
+ "Small-CNN comparison is one seed only.",
29
+ "No robustness or SOTA claim is frozen."
30
+ ]
31
+ }
metrics/low_label_summary_by_budget.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ budget_rows,n_seeds,carbon_spearman_mean,carbon_spearman_std,carbon_spearman_min,carbon_spearman_max,kmer_control_mean_spearman,carbon_minus_kmer_control_mean,carbon_minus_kmer_control_min,final_test_evaluated_any
2
+ 6044,3,0.2304413079762786,0.09716890100007153,0.11836000110935953,0.29096869851083296,0.6152879943955233,-0.38484668641924474,-0.49692799328616377,False
3
+ 60436,3,0.4574703317855265,0.05862353304806437,0.4068496177318754,0.5217020981979507,0.6294478678994059,-0.17197753611387934,-0.2225982501675305,False
metrics/phase_stability_summary.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "n_rows": 8192,
3
+ "n_perturbations": 15,
4
+ "row_prediction_std_mean_z": 0.12435458417137643,
5
+ "row_prediction_std_median_z": 0.12066128198420133,
6
+ "row_prediction_std_p95_z": 0.20003382875743392,
7
+ "row_max_abs_delta_mean_z": 0.37548579306167085,
8
+ "row_max_abs_delta_median_z": 0.3499755859375,
9
+ "row_max_abs_delta_p95_z": 0.68359375,
10
+ "row_max_abs_delta_max_z": 1.4296875,
11
+ "fraction_rows_max_abs_delta_gt_0_10": 0.9964599609375,
12
+ "fraction_rows_max_abs_delta_gt_0_25": 0.7169189453125,
13
+ "fraction_rows_max_abs_delta_gt_0_50": 0.234375
14
+ }
metrics/validation_per_seed_metrics.csv ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ source,seed,run_dir,status,validation_spearman_rho,validation_pearson_r,validation_rmse_z,validation_mae_z,validation_loss,train_rows_seen,validation_rows,steps,peak_cuda_memory_gb,elapsed_seconds,carbon_minus_kmer,carbon_minus_cnn_one_seed,carbon_minus_strongest,final_test_evaluated
2
+ prior_r005,20260523,/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression/experiments/exp_04_carbon_500m_lora_validation/artifacts/runs/r005,done,0.6906407227853265,0.6757242933965104,0.7476240361692528,0.5652980054250197,0.5589417028199415,1000000,671567,489,6.957329750061035,4076.657002687454,0.059191759097768526,0.05780048541578775,0.05780048541578775,False
3
+ new_seed,20260524,/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression/experiments/exp_05_carbon_500m_lora_seed_confirm/artifacts/runs/r006/seed_20260524,done,0.6973705211571226,0.6826701938315765,0.7393169153868578,0.5583845155808999,0.5465895029432161,1000000,671567,489,6.957329750061035,4080.9787974357605,0.06592155746956463,0.06453028378758385,0.06453028378758385,False
4
+ new_seed,20260525,/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression/experiments/exp_05_carbon_500m_lora_seed_confirm/artifacts/runs/r006/seed_20260525,done,0.6868102547720921,0.6721345373748435,0.7496320223954087,0.5673071180500381,0.5619481679525792,1000000,671567,489,6.957329750061035,4075.8555772304535,0.05536129108453414,0.05397001740255336,0.05397001740255336,False
release_manifest.json ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_model": "HuggingFaceBio/Carbon-500M",
3
+ "adapter_type": "LoRA sequence-classification adapters",
4
+ "primary_use": "rank promoter sequences by predicted expression under the Random Promoter DREAM Challenge 2022 protocol",
5
+ "seeds": [
6
+ "seed_20260523",
7
+ "seed_20260524",
8
+ "seed_20260525"
9
+ ],
10
+ "format": "per-seed adapter_model.safetensors plus adapter_config.json",
11
+ "not_included": [
12
+ "raw Zenodo data",
13
+ "processed parquet splits",
14
+ "full prediction dumps",
15
+ "Carbon-500M base weights"
16
+ ]
17
+ }
seed_20260523/adapter_config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_model_name_or_path": "HuggingFaceBio/Carbon-500M",
3
+ "bias": "none",
4
+ "fan_in_fan_out": false,
5
+ "inference_mode": true,
6
+ "init_lora_weights": true,
7
+ "layers_pattern": null,
8
+ "layers_to_transform": null,
9
+ "loftq_config": {},
10
+ "lora_alpha": 16,
11
+ "lora_dropout": 0.05,
12
+ "modules_to_save": [
13
+ "score"
14
+ ],
15
+ "peft_type": "LORA",
16
+ "r": 8,
17
+ "revision": null,
18
+ "target_modules": [
19
+ "q_proj",
20
+ "k_proj",
21
+ "v_proj",
22
+ "o_proj"
23
+ ],
24
+ "task_type": "SEQ_CLS"
25
+ }
seed_20260523/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a4247ddcb47397fa8f754ca3a23815232b8014b13d0888215f6c5af06f92c96
3
+ size 6458128
seed_20260523/config_snapshot.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "args": {
3
+ "project_root": "/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression",
4
+ "run_dir": "/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression/experiments/exp_04_carbon_500m_lora_validation/artifacts/runs/r005",
5
+ "model_id": "HuggingFaceBio/Carbon-500M",
6
+ "seed": 20260523,
7
+ "max_train_rows": 1000000,
8
+ "max_val_rows": null,
9
+ "epochs": 1,
10
+ "max_steps": null,
11
+ "train_batch_size": 2048,
12
+ "eval_batch_size": 8192,
13
+ "lr": 0.0005,
14
+ "weight_decay": 0.0001,
15
+ "max_length": 256,
16
+ "lora_r": 8,
17
+ "lora_alpha": 16,
18
+ "lora_dropout": 0.05,
19
+ "add_special_tokens": false,
20
+ "no_bf16": false,
21
+ "log_every": 25,
22
+ "baseline_kmer_spearman": 0.631448963687558,
23
+ "baseline_cnn_spearman": 0.6328402373695388,
24
+ "meaningful_delta": 0.01
25
+ },
26
+ "final_test_policy": "locked; this validation pilot reads train.parquet and val.parquet only"
27
+ }
seed_20260523/metrics.json ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "primary_metric": "validation_spearman_rho",
3
+ "primary_value": 0.6906407227853265,
4
+ "higher_is_better": true,
5
+ "status": "done",
6
+ "model": "HuggingFaceBio/Carbon-500M",
7
+ "model_info_sha": "5d31d59b3c845b288a13aedb1358934196852eec",
8
+ "seed": 20260523,
9
+ "train_rows_seen": 1000000,
10
+ "validation_rows": 671567,
11
+ "epochs_requested": 1,
12
+ "steps": 489,
13
+ "max_train_rows": 1000000,
14
+ "max_val_rows": null,
15
+ "train_batch_size": 2048,
16
+ "eval_batch_size": 8192,
17
+ "max_length": 256,
18
+ "add_special_tokens": false,
19
+ "device": "cuda",
20
+ "cuda_device_name": "NVIDIA GB10",
21
+ "use_bf16": true,
22
+ "peak_cuda_memory_gb": 6.957329750061035,
23
+ "elapsed_seconds": 4076.657002687454,
24
+ "sanity_checks": {
25
+ "tokenizer_audit_recorded": true,
26
+ "model_lora_audit_recorded": true,
27
+ "model_loaded": true,
28
+ "lora_targets_verified": true,
29
+ "pad_or_eos_recorded": true,
30
+ "token_count_and_truncation_recorded": true,
31
+ "train_loss_finite": true,
32
+ "validation_loss_finite": true,
33
+ "predictions_nonconstant": true,
34
+ "cuda_bf16_no_nan": true,
35
+ "final_test_evaluated_false": true
36
+ },
37
+ "baseline_comparison": {
38
+ "baseline_kmer_spearman": 0.631448963687558,
39
+ "baseline_cnn_one_seed_spearman": 0.6328402373695388,
40
+ "strongest_baseline_spearman": 0.6328402373695388,
41
+ "carbon_minus_kmer": 0.059191759097768526,
42
+ "carbon_minus_cnn_one_seed": 0.05780048541578775,
43
+ "carbon_minus_strongest": 0.05780048541578775,
44
+ "meaningful_delta_threshold": 0.01,
45
+ "meets_meaningful_validation_win": true
46
+ },
47
+ "validation_metrics": {
48
+ "spearman_rho": 0.6906407227853265,
49
+ "pearson_r": 0.6757242933965104,
50
+ "rmse_z": 0.7476240361692528,
51
+ "mae_z": 0.5652980054250197,
52
+ "rmse_expression": 1.7728440467044884,
53
+ "mae_expression": 1.3404935566003675,
54
+ "prediction_std_z": 0.7133016114168015
55
+ },
56
+ "validation_loss": 0.5589417028199415,
57
+ "loss_summary": {
58
+ "train_loss_first": 12.107651710510254,
59
+ "train_loss_last": 0.5358932614326477,
60
+ "train_loss_min": 0.520319402217865,
61
+ "validation_loss": 0.5589417028199415,
62
+ "train_loss_finite": true,
63
+ "validation_loss_finite": true,
64
+ "train_loss_decreased": true,
65
+ "train_loss_improved_at_least_once": true
66
+ },
67
+ "checkpoint": {
68
+ "path": "trainable_state.pt",
69
+ "n_tensors": 226,
70
+ "n_parameters": 1607680
71
+ },
72
+ "final_test_evaluated": false,
73
+ "training_loss_logging": {
74
+ "status": "available",
75
+ "loss_history": "loss_history.csv",
76
+ "loss_summary": "loss_summary.json"
77
+ }
78
+ }
seed_20260523/model_lora_audit.json ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "HuggingFaceBio/Carbon-500M",
3
+ "config_class": "LlamaConfig",
4
+ "architectures": [
5
+ "LlamaForCausalLM"
6
+ ],
7
+ "hidden_size": 1024,
8
+ "pad_token_id": 151643,
9
+ "sequence_classification_attempt": {
10
+ "success": true,
11
+ "class": "LlamaForSequenceClassification",
12
+ "error": null
13
+ },
14
+ "fallback": null,
15
+ "pooling_strategy": "AutoModelForSequenceClassification internal pooling",
16
+ "lora_target_audit": {
17
+ "target_source": "preferred_llama_attention",
18
+ "linear_suffix_counts": {
19
+ "q_proj": 28,
20
+ "k_proj": 28,
21
+ "v_proj": 28,
22
+ "o_proj": 28,
23
+ "gate_proj": 28,
24
+ "up_proj": 28,
25
+ "down_proj": 28
26
+ },
27
+ "selected_target_modules": [
28
+ "q_proj",
29
+ "k_proj",
30
+ "v_proj",
31
+ "o_proj"
32
+ ],
33
+ "matched_module_count": 112,
34
+ "matched_module_examples": [
35
+ "model.layers.0.self_attn.q_proj",
36
+ "model.layers.0.self_attn.k_proj",
37
+ "model.layers.0.self_attn.v_proj",
38
+ "model.layers.0.self_attn.o_proj",
39
+ "model.layers.1.self_attn.q_proj",
40
+ "model.layers.1.self_attn.k_proj",
41
+ "model.layers.1.self_attn.v_proj",
42
+ "model.layers.1.self_attn.o_proj",
43
+ "model.layers.2.self_attn.q_proj",
44
+ "model.layers.2.self_attn.k_proj",
45
+ "model.layers.2.self_attn.v_proj",
46
+ "model.layers.2.self_attn.o_proj",
47
+ "model.layers.3.self_attn.q_proj",
48
+ "model.layers.3.self_attn.k_proj",
49
+ "model.layers.3.self_attn.v_proj",
50
+ "model.layers.3.self_attn.o_proj",
51
+ "model.layers.4.self_attn.q_proj",
52
+ "model.layers.4.self_attn.k_proj",
53
+ "model.layers.4.self_attn.v_proj",
54
+ "model.layers.4.self_attn.o_proj",
55
+ "model.layers.5.self_attn.q_proj",
56
+ "model.layers.5.self_attn.k_proj",
57
+ "model.layers.5.self_attn.v_proj",
58
+ "model.layers.5.self_attn.o_proj",
59
+ "model.layers.6.self_attn.q_proj",
60
+ "model.layers.6.self_attn.k_proj",
61
+ "model.layers.6.self_attn.v_proj",
62
+ "model.layers.6.self_attn.o_proj",
63
+ "model.layers.7.self_attn.q_proj",
64
+ "model.layers.7.self_attn.k_proj",
65
+ "model.layers.7.self_attn.v_proj",
66
+ "model.layers.7.self_attn.o_proj",
67
+ "model.layers.8.self_attn.q_proj",
68
+ "model.layers.8.self_attn.k_proj",
69
+ "model.layers.8.self_attn.v_proj",
70
+ "model.layers.8.self_attn.o_proj",
71
+ "model.layers.9.self_attn.q_proj",
72
+ "model.layers.9.self_attn.k_proj",
73
+ "model.layers.9.self_attn.v_proj",
74
+ "model.layers.9.self_attn.o_proj"
75
+ ]
76
+ },
77
+ "lora_config": {
78
+ "task_type": "SEQ_CLS",
79
+ "r": 8,
80
+ "lora_alpha": 16,
81
+ "lora_dropout": 0.05,
82
+ "bias": "none",
83
+ "target_modules": [
84
+ "q_proj",
85
+ "k_proj",
86
+ "v_proj",
87
+ "o_proj"
88
+ ]
89
+ },
90
+ "parameter_counts": {
91
+ "total_parameters": 513502208,
92
+ "trainable_parameters": 1607680,
93
+ "trainable_fraction": 0.0031308141911631274
94
+ },
95
+ "model_info_sha": "5d31d59b3c845b288a13aedb1358934196852eec",
96
+ "model_tags": [
97
+ "transformers",
98
+ "safetensors",
99
+ "llama",
100
+ "text-generation",
101
+ "dna",
102
+ "genomic",
103
+ "speculative-decoding",
104
+ "license:apache-2.0",
105
+ "text-generation-inference",
106
+ "endpoints_compatible",
107
+ "region:us"
108
+ ],
109
+ "gradient_checkpointing": {
110
+ "attempted": true,
111
+ "enabled": true,
112
+ "error": null
113
+ }
114
+ }
seed_20260524/adapter_config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_model_name_or_path": "HuggingFaceBio/Carbon-500M",
3
+ "bias": "none",
4
+ "fan_in_fan_out": false,
5
+ "inference_mode": true,
6
+ "init_lora_weights": true,
7
+ "layers_pattern": null,
8
+ "layers_to_transform": null,
9
+ "loftq_config": {},
10
+ "lora_alpha": 16,
11
+ "lora_dropout": 0.05,
12
+ "modules_to_save": [
13
+ "score"
14
+ ],
15
+ "peft_type": "LORA",
16
+ "r": 8,
17
+ "revision": null,
18
+ "target_modules": [
19
+ "q_proj",
20
+ "k_proj",
21
+ "v_proj",
22
+ "o_proj"
23
+ ],
24
+ "task_type": "SEQ_CLS"
25
+ }
seed_20260524/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14a360cbe8b351a92f9c8dc5e9986da2dc5b8fbc3735ebe8d302b4a84c06b0e8
3
+ size 6458144
seed_20260524/config_snapshot.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "args": {
3
+ "project_root": "/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression",
4
+ "run_dir": "/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression/experiments/exp_05_carbon_500m_lora_seed_confirm/artifacts/runs/r006/seed_20260524",
5
+ "model_id": "HuggingFaceBio/Carbon-500M",
6
+ "seed": 20260524,
7
+ "max_train_rows": 1000000,
8
+ "max_val_rows": null,
9
+ "epochs": 1,
10
+ "max_steps": null,
11
+ "train_batch_size": 2048,
12
+ "eval_batch_size": 8192,
13
+ "lr": 0.0005,
14
+ "weight_decay": 0.0001,
15
+ "max_length": 256,
16
+ "lora_r": 8,
17
+ "lora_alpha": 16,
18
+ "lora_dropout": 0.05,
19
+ "add_special_tokens": false,
20
+ "no_bf16": false,
21
+ "log_every": 25,
22
+ "baseline_kmer_spearman": 0.631448963687558,
23
+ "baseline_cnn_spearman": 0.6328402373695388,
24
+ "meaningful_delta": 0.01
25
+ },
26
+ "final_test_policy": "locked; this validation pilot reads train.parquet and val.parquet only"
27
+ }
seed_20260524/metrics.json ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "primary_metric": "validation_spearman_rho",
3
+ "primary_value": 0.6973705211571226,
4
+ "higher_is_better": true,
5
+ "status": "done",
6
+ "model": "HuggingFaceBio/Carbon-500M",
7
+ "model_info_sha": "5d31d59b3c845b288a13aedb1358934196852eec",
8
+ "seed": 20260524,
9
+ "train_rows_seen": 1000000,
10
+ "validation_rows": 671567,
11
+ "epochs_requested": 1,
12
+ "steps": 489,
13
+ "max_train_rows": 1000000,
14
+ "max_val_rows": null,
15
+ "train_batch_size": 2048,
16
+ "eval_batch_size": 8192,
17
+ "max_length": 256,
18
+ "add_special_tokens": false,
19
+ "device": "cuda",
20
+ "cuda_device_name": "NVIDIA GB10",
21
+ "use_bf16": true,
22
+ "peak_cuda_memory_gb": 6.957329750061035,
23
+ "elapsed_seconds": 4080.9787974357605,
24
+ "sanity_checks": {
25
+ "tokenizer_audit_recorded": true,
26
+ "model_lora_audit_recorded": true,
27
+ "model_loaded": true,
28
+ "lora_targets_verified": true,
29
+ "pad_or_eos_recorded": true,
30
+ "token_count_and_truncation_recorded": true,
31
+ "train_loss_finite": true,
32
+ "validation_loss_finite": true,
33
+ "predictions_nonconstant": true,
34
+ "cuda_bf16_no_nan": true,
35
+ "final_test_evaluated_false": true
36
+ },
37
+ "baseline_comparison": {
38
+ "baseline_kmer_spearman": 0.631448963687558,
39
+ "baseline_cnn_one_seed_spearman": 0.6328402373695388,
40
+ "strongest_baseline_spearman": 0.6328402373695388,
41
+ "carbon_minus_kmer": 0.06592155746956463,
42
+ "carbon_minus_cnn_one_seed": 0.06453028378758385,
43
+ "carbon_minus_strongest": 0.06453028378758385,
44
+ "meaningful_delta_threshold": 0.01,
45
+ "meets_meaningful_validation_win": true
46
+ },
47
+ "validation_metrics": {
48
+ "spearman_rho": 0.6973705211571226,
49
+ "pearson_r": 0.6826701938315765,
50
+ "rmse_z": 0.7393169153868578,
51
+ "mae_z": 0.5583845155808999,
52
+ "rmse_expression": 1.7531453361407565,
53
+ "mae_expression": 1.324099568973164,
54
+ "prediction_std_z": 0.7100825596695095
55
+ },
56
+ "validation_loss": 0.5465895029432161,
57
+ "loss_summary": {
58
+ "train_loss_first": 1.6106956005096436,
59
+ "train_loss_last": 0.5188584327697754,
60
+ "train_loss_min": 0.5094407200813293,
61
+ "validation_loss": 0.5465895029432161,
62
+ "train_loss_finite": true,
63
+ "validation_loss_finite": true,
64
+ "train_loss_decreased": true,
65
+ "train_loss_improved_at_least_once": true
66
+ },
67
+ "checkpoint": {
68
+ "path": "trainable_state.pt",
69
+ "n_tensors": 226,
70
+ "n_parameters": 1607680
71
+ },
72
+ "final_test_evaluated": false,
73
+ "training_loss_logging": {
74
+ "status": "available",
75
+ "loss_history": "loss_history.csv",
76
+ "loss_summary": "loss_summary.json"
77
+ }
78
+ }
seed_20260524/model_lora_audit.json ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "HuggingFaceBio/Carbon-500M",
3
+ "config_class": "LlamaConfig",
4
+ "architectures": [
5
+ "LlamaForCausalLM"
6
+ ],
7
+ "hidden_size": 1024,
8
+ "pad_token_id": 151643,
9
+ "sequence_classification_attempt": {
10
+ "success": true,
11
+ "class": "LlamaForSequenceClassification",
12
+ "error": null
13
+ },
14
+ "fallback": null,
15
+ "pooling_strategy": "AutoModelForSequenceClassification internal pooling",
16
+ "lora_target_audit": {
17
+ "target_source": "preferred_llama_attention",
18
+ "linear_suffix_counts": {
19
+ "q_proj": 28,
20
+ "k_proj": 28,
21
+ "v_proj": 28,
22
+ "o_proj": 28,
23
+ "gate_proj": 28,
24
+ "up_proj": 28,
25
+ "down_proj": 28
26
+ },
27
+ "selected_target_modules": [
28
+ "q_proj",
29
+ "k_proj",
30
+ "v_proj",
31
+ "o_proj"
32
+ ],
33
+ "matched_module_count": 112,
34
+ "matched_module_examples": [
35
+ "model.layers.0.self_attn.q_proj",
36
+ "model.layers.0.self_attn.k_proj",
37
+ "model.layers.0.self_attn.v_proj",
38
+ "model.layers.0.self_attn.o_proj",
39
+ "model.layers.1.self_attn.q_proj",
40
+ "model.layers.1.self_attn.k_proj",
41
+ "model.layers.1.self_attn.v_proj",
42
+ "model.layers.1.self_attn.o_proj",
43
+ "model.layers.2.self_attn.q_proj",
44
+ "model.layers.2.self_attn.k_proj",
45
+ "model.layers.2.self_attn.v_proj",
46
+ "model.layers.2.self_attn.o_proj",
47
+ "model.layers.3.self_attn.q_proj",
48
+ "model.layers.3.self_attn.k_proj",
49
+ "model.layers.3.self_attn.v_proj",
50
+ "model.layers.3.self_attn.o_proj",
51
+ "model.layers.4.self_attn.q_proj",
52
+ "model.layers.4.self_attn.k_proj",
53
+ "model.layers.4.self_attn.v_proj",
54
+ "model.layers.4.self_attn.o_proj",
55
+ "model.layers.5.self_attn.q_proj",
56
+ "model.layers.5.self_attn.k_proj",
57
+ "model.layers.5.self_attn.v_proj",
58
+ "model.layers.5.self_attn.o_proj",
59
+ "model.layers.6.self_attn.q_proj",
60
+ "model.layers.6.self_attn.k_proj",
61
+ "model.layers.6.self_attn.v_proj",
62
+ "model.layers.6.self_attn.o_proj",
63
+ "model.layers.7.self_attn.q_proj",
64
+ "model.layers.7.self_attn.k_proj",
65
+ "model.layers.7.self_attn.v_proj",
66
+ "model.layers.7.self_attn.o_proj",
67
+ "model.layers.8.self_attn.q_proj",
68
+ "model.layers.8.self_attn.k_proj",
69
+ "model.layers.8.self_attn.v_proj",
70
+ "model.layers.8.self_attn.o_proj",
71
+ "model.layers.9.self_attn.q_proj",
72
+ "model.layers.9.self_attn.k_proj",
73
+ "model.layers.9.self_attn.v_proj",
74
+ "model.layers.9.self_attn.o_proj"
75
+ ]
76
+ },
77
+ "lora_config": {
78
+ "task_type": "SEQ_CLS",
79
+ "r": 8,
80
+ "lora_alpha": 16,
81
+ "lora_dropout": 0.05,
82
+ "bias": "none",
83
+ "target_modules": [
84
+ "q_proj",
85
+ "k_proj",
86
+ "v_proj",
87
+ "o_proj"
88
+ ]
89
+ },
90
+ "parameter_counts": {
91
+ "total_parameters": 513502208,
92
+ "trainable_parameters": 1607680,
93
+ "trainable_fraction": 0.0031308141911631274
94
+ },
95
+ "model_info_sha": "5d31d59b3c845b288a13aedb1358934196852eec",
96
+ "model_tags": [
97
+ "transformers",
98
+ "safetensors",
99
+ "llama",
100
+ "text-generation",
101
+ "dna",
102
+ "genomic",
103
+ "speculative-decoding",
104
+ "license:apache-2.0",
105
+ "text-generation-inference",
106
+ "endpoints_compatible",
107
+ "region:us"
108
+ ],
109
+ "gradient_checkpointing": {
110
+ "attempted": true,
111
+ "enabled": true,
112
+ "error": null
113
+ }
114
+ }
seed_20260525/adapter_config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_model_name_or_path": "HuggingFaceBio/Carbon-500M",
3
+ "bias": "none",
4
+ "fan_in_fan_out": false,
5
+ "inference_mode": true,
6
+ "init_lora_weights": true,
7
+ "layers_pattern": null,
8
+ "layers_to_transform": null,
9
+ "loftq_config": {},
10
+ "lora_alpha": 16,
11
+ "lora_dropout": 0.05,
12
+ "modules_to_save": [
13
+ "score"
14
+ ],
15
+ "peft_type": "LORA",
16
+ "r": 8,
17
+ "revision": null,
18
+ "target_modules": [
19
+ "q_proj",
20
+ "k_proj",
21
+ "v_proj",
22
+ "o_proj"
23
+ ],
24
+ "task_type": "SEQ_CLS"
25
+ }
seed_20260525/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4120db2cd5cb7cb7faf1d18cb62307548b1fff66b51a62d76580d1de8b9c3253
3
+ size 6458144
seed_20260525/config_snapshot.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "args": {
3
+ "project_root": "/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression",
4
+ "run_dir": "/home/soleaf/jupyterlab/lab/experiment-lab/projects/carbon-promoter-expression/experiments/exp_05_carbon_500m_lora_seed_confirm/artifacts/runs/r006/seed_20260525",
5
+ "model_id": "HuggingFaceBio/Carbon-500M",
6
+ "seed": 20260525,
7
+ "max_train_rows": 1000000,
8
+ "max_val_rows": null,
9
+ "epochs": 1,
10
+ "max_steps": null,
11
+ "train_batch_size": 2048,
12
+ "eval_batch_size": 8192,
13
+ "lr": 0.0005,
14
+ "weight_decay": 0.0001,
15
+ "max_length": 256,
16
+ "lora_r": 8,
17
+ "lora_alpha": 16,
18
+ "lora_dropout": 0.05,
19
+ "add_special_tokens": false,
20
+ "no_bf16": false,
21
+ "log_every": 25,
22
+ "baseline_kmer_spearman": 0.631448963687558,
23
+ "baseline_cnn_spearman": 0.6328402373695388,
24
+ "meaningful_delta": 0.01
25
+ },
26
+ "final_test_policy": "locked; this validation pilot reads train.parquet and val.parquet only"
27
+ }
seed_20260525/metrics.json ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "primary_metric": "validation_spearman_rho",
3
+ "primary_value": 0.6868102547720921,
4
+ "higher_is_better": true,
5
+ "status": "done",
6
+ "model": "HuggingFaceBio/Carbon-500M",
7
+ "model_info_sha": "5d31d59b3c845b288a13aedb1358934196852eec",
8
+ "seed": 20260525,
9
+ "train_rows_seen": 1000000,
10
+ "validation_rows": 671567,
11
+ "epochs_requested": 1,
12
+ "steps": 489,
13
+ "max_train_rows": 1000000,
14
+ "max_val_rows": null,
15
+ "train_batch_size": 2048,
16
+ "eval_batch_size": 8192,
17
+ "max_length": 256,
18
+ "add_special_tokens": false,
19
+ "device": "cuda",
20
+ "cuda_device_name": "NVIDIA GB10",
21
+ "use_bf16": true,
22
+ "peak_cuda_memory_gb": 6.957329750061035,
23
+ "elapsed_seconds": 4075.8555772304535,
24
+ "sanity_checks": {
25
+ "tokenizer_audit_recorded": true,
26
+ "model_lora_audit_recorded": true,
27
+ "model_loaded": true,
28
+ "lora_targets_verified": true,
29
+ "pad_or_eos_recorded": true,
30
+ "token_count_and_truncation_recorded": true,
31
+ "train_loss_finite": true,
32
+ "validation_loss_finite": true,
33
+ "predictions_nonconstant": true,
34
+ "cuda_bf16_no_nan": true,
35
+ "final_test_evaluated_false": true
36
+ },
37
+ "baseline_comparison": {
38
+ "baseline_kmer_spearman": 0.631448963687558,
39
+ "baseline_cnn_one_seed_spearman": 0.6328402373695388,
40
+ "strongest_baseline_spearman": 0.6328402373695388,
41
+ "carbon_minus_kmer": 0.05536129108453414,
42
+ "carbon_minus_cnn_one_seed": 0.05397001740255336,
43
+ "carbon_minus_strongest": 0.05397001740255336,
44
+ "meaningful_delta_threshold": 0.01,
45
+ "meets_meaningful_validation_win": true
46
+ },
47
+ "validation_metrics": {
48
+ "spearman_rho": 0.6868102547720921,
49
+ "pearson_r": 0.6721345373748435,
50
+ "rmse_z": 0.7496320223954087,
51
+ "mae_z": 0.5673071180500381,
52
+ "rmse_expression": 1.7776055932289672,
53
+ "mae_expression": 1.3452577739372926,
54
+ "prediction_std_z": 0.6922592808091649
55
+ },
56
+ "validation_loss": 0.5619481679525792,
57
+ "loss_summary": {
58
+ "train_loss_first": 10.797410011291504,
59
+ "train_loss_last": 0.5414774417877197,
60
+ "train_loss_min": 0.521086573600769,
61
+ "validation_loss": 0.5619481679525792,
62
+ "train_loss_finite": true,
63
+ "validation_loss_finite": true,
64
+ "train_loss_decreased": true,
65
+ "train_loss_improved_at_least_once": true
66
+ },
67
+ "checkpoint": {
68
+ "path": "trainable_state.pt",
69
+ "n_tensors": 226,
70
+ "n_parameters": 1607680
71
+ },
72
+ "final_test_evaluated": false,
73
+ "training_loss_logging": {
74
+ "status": "available",
75
+ "loss_history": "loss_history.csv",
76
+ "loss_summary": "loss_summary.json"
77
+ }
78
+ }
seed_20260525/model_lora_audit.json ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "HuggingFaceBio/Carbon-500M",
3
+ "config_class": "LlamaConfig",
4
+ "architectures": [
5
+ "LlamaForCausalLM"
6
+ ],
7
+ "hidden_size": 1024,
8
+ "pad_token_id": 151643,
9
+ "sequence_classification_attempt": {
10
+ "success": true,
11
+ "class": "LlamaForSequenceClassification",
12
+ "error": null
13
+ },
14
+ "fallback": null,
15
+ "pooling_strategy": "AutoModelForSequenceClassification internal pooling",
16
+ "lora_target_audit": {
17
+ "target_source": "preferred_llama_attention",
18
+ "linear_suffix_counts": {
19
+ "q_proj": 28,
20
+ "k_proj": 28,
21
+ "v_proj": 28,
22
+ "o_proj": 28,
23
+ "gate_proj": 28,
24
+ "up_proj": 28,
25
+ "down_proj": 28
26
+ },
27
+ "selected_target_modules": [
28
+ "q_proj",
29
+ "k_proj",
30
+ "v_proj",
31
+ "o_proj"
32
+ ],
33
+ "matched_module_count": 112,
34
+ "matched_module_examples": [
35
+ "model.layers.0.self_attn.q_proj",
36
+ "model.layers.0.self_attn.k_proj",
37
+ "model.layers.0.self_attn.v_proj",
38
+ "model.layers.0.self_attn.o_proj",
39
+ "model.layers.1.self_attn.q_proj",
40
+ "model.layers.1.self_attn.k_proj",
41
+ "model.layers.1.self_attn.v_proj",
42
+ "model.layers.1.self_attn.o_proj",
43
+ "model.layers.2.self_attn.q_proj",
44
+ "model.layers.2.self_attn.k_proj",
45
+ "model.layers.2.self_attn.v_proj",
46
+ "model.layers.2.self_attn.o_proj",
47
+ "model.layers.3.self_attn.q_proj",
48
+ "model.layers.3.self_attn.k_proj",
49
+ "model.layers.3.self_attn.v_proj",
50
+ "model.layers.3.self_attn.o_proj",
51
+ "model.layers.4.self_attn.q_proj",
52
+ "model.layers.4.self_attn.k_proj",
53
+ "model.layers.4.self_attn.v_proj",
54
+ "model.layers.4.self_attn.o_proj",
55
+ "model.layers.5.self_attn.q_proj",
56
+ "model.layers.5.self_attn.k_proj",
57
+ "model.layers.5.self_attn.v_proj",
58
+ "model.layers.5.self_attn.o_proj",
59
+ "model.layers.6.self_attn.q_proj",
60
+ "model.layers.6.self_attn.k_proj",
61
+ "model.layers.6.self_attn.v_proj",
62
+ "model.layers.6.self_attn.o_proj",
63
+ "model.layers.7.self_attn.q_proj",
64
+ "model.layers.7.self_attn.k_proj",
65
+ "model.layers.7.self_attn.v_proj",
66
+ "model.layers.7.self_attn.o_proj",
67
+ "model.layers.8.self_attn.q_proj",
68
+ "model.layers.8.self_attn.k_proj",
69
+ "model.layers.8.self_attn.v_proj",
70
+ "model.layers.8.self_attn.o_proj",
71
+ "model.layers.9.self_attn.q_proj",
72
+ "model.layers.9.self_attn.k_proj",
73
+ "model.layers.9.self_attn.v_proj",
74
+ "model.layers.9.self_attn.o_proj"
75
+ ]
76
+ },
77
+ "lora_config": {
78
+ "task_type": "SEQ_CLS",
79
+ "r": 8,
80
+ "lora_alpha": 16,
81
+ "lora_dropout": 0.05,
82
+ "bias": "none",
83
+ "target_modules": [
84
+ "q_proj",
85
+ "k_proj",
86
+ "v_proj",
87
+ "o_proj"
88
+ ]
89
+ },
90
+ "parameter_counts": {
91
+ "total_parameters": 513502208,
92
+ "trainable_parameters": 1607680,
93
+ "trainable_fraction": 0.0031308141911631274
94
+ },
95
+ "model_info_sha": "5d31d59b3c845b288a13aedb1358934196852eec",
96
+ "model_tags": [
97
+ "transformers",
98
+ "safetensors",
99
+ "llama",
100
+ "text-generation",
101
+ "dna",
102
+ "genomic",
103
+ "speculative-decoding",
104
+ "license:apache-2.0",
105
+ "text-generation-inference",
106
+ "endpoints_compatible",
107
+ "region:us"
108
+ ],
109
+ "gradient_checkpointing": {
110
+ "attempted": true,
111
+ "enabled": true,
112
+ "error": null
113
+ }
114
+ }