Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - ATAC
+  - heart
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: ATAC in heart left ventricle (ENCSR745CGG)
+- Model: ChromBPNet
+- Assay: ATAC-seq
+- Experiment: [ENCSR745CGG](https://www.encodeproject.org/experiments/ENCSR745CGG/)
+- Model annotation: [ENCSR302PTC](https://www.encodeproject.org/annotations/ENCSR302PTC/)
+- Biosample: heart left ventricle (Full name: Homo sapiens heart left ventricle tissue male adult (54 years))
+- Cell slim(s): None
+- Organ slim(s): heart
+- Developmental slim(s): mesoderm
+- System slim(s): circulatory-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR745CGG/logfile.modelling.fold_0.ENCSR745CGG.args.json

@@ -0,0 +1,23 @@
+{
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference/hg38.genome.fa",
+    "bigwig": "/scratch/groups/akundaje/anusri/chromatin_atlas/ATAC/ENCSR745CGG//preprocessing/bigWigs/ENCSR745CGG.bigWig",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/ATAC/ENCSR745CGG//chrombpnet_model_feb15//filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/ATAC/ENCSR745CGG//chrombpnet_model_feb15//filtered.nonpeaks.bed",
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/ATAC/ENCSR745CGG//chrombpnet_model_feb15//chrombpnet",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json",
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "epochs": 50,
+    "early_stop": 5,
+    "batch_size": 64,
+    "learning_rate": 0.001,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/ATAC/ENCSR745CGG//chrombpnet_model_feb15//chrombpnet_model_params.tsv",
+    "seed": 1234,
+    "architecture_from_file": "/home/users/anusri/chromatin-atlas-anvil/sherlock/chrombpnet/src/training/models/chrombpnet_with_bias_model.py"
+}

fold_0/logs.models.fold_0.ENCSR745CGG/logfile.modelling.fold_0.ENCSR745CGG.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	19.0
+counts_sum_max_thresh	1548.0
+trainings_pts_post_thresh	150836

fold_0/logs.models.fold_0.ENCSR745CGG/logfile.modelling.fold_0.ENCSR745CGG.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	15.3
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/ATAC/ENCSR745CGG//chrombpnet_model_feb15/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR745CGG/logfile.modelling.fold_0.ENCSR745CGG.epoch_loss.csv

@@ -0,0 +1,15 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,0.44608309864997864,356.3638610839844,363.1895446777344,0.34995314478874207,348.2383117675781,353.59271240234375
+1,0.35449835658073425,343.2564392089844,348.67974853515625,0.30353981256484985,344.474853515625,349.1188049316406
+2,0.3212377727031708,340.2561340332031,345.17041015625,0.3039330244064331,343.08807373046875,347.73834228515625
+3,0.30749833583831787,338.0492858886719,342.7536315917969,0.31002068519592285,343.0008544921875,347.7442321777344
+4,0.2845754623413086,336.5156555175781,340.8693542480469,0.2656431496143341,342.3581848144531,346.4224548339844
+5,0.27536502480506897,335.02850341796875,339.2415466308594,0.29356274008750916,342.6092224121094,347.10076904296875
+6,0.26146334409713745,333.99920654296875,337.9997253417969,0.26989084482192993,342.4691162109375,346.5985107421875
+7,0.25363364815711975,333.1258239746094,337.00579833984375,0.2649283707141876,342.5541076660156,346.6075744628906
+8,0.22175979614257812,329.23712158203125,332.6299133300781,0.2581357955932617,341.575927734375,345.5254211425781
+9,0.20746880769729614,327.1663513183594,330.3407287597656,0.26081982254981995,341.74676513671875,345.7373046875
+10,0.1936865597963333,325.8124084472656,328.7756042480469,0.2684626281261444,342.19775390625,346.3051452636719
+11,0.18290485441684723,324.9057312011719,327.7044677734375,0.273595929145813,342.6397399902344,346.8258056640625
+12,0.16137783229351044,322.6907043457031,325.16009521484375,0.2661406993865967,342.7127685546875,346.78472900390625
+13,0.15418533980846405,321.59710693359375,323.95574951171875,0.26349204778671265,343.31817626953125,347.3497009277344

fold_0/model.bias_scaled.fold_0.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90b4b58a21f2238a7255d356f3310d89ec0bbc2f6dcdfac032b72b5f7a027848
+size 2688440

fold_0/model.bias_scaled.fold_0.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd849f06a3d975930b9862319e4ef48ea59a4f4d123520927eae5d9dbbb68172
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17ae4cb09aa5b564d213b82cea2a4ca276497a76e9efb21fc5b37d33188d56f6
+size 26448128

fold_0/model.chrombpnet.fold_0.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2930f0c2a406d50a529ea807ad08c36df2780fe2c2a85e080f7e21206d49fc63
+size 27535360

fold_0/model.chrombpnet_nobias.fold_0.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:597aa31314d2f6bea82416866f9fb0aee671082c185b411a4cdfc56f0b184927
+size 25583536

fold_0/model.chrombpnet_nobias.fold_0.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4991475890dbc433521dd6688a060981c5210462f9c9367129e41ad185e8abb1
+size 26060800

fold_1/model.bias_scaled.fold_1.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90b4b58a21f2238a7255d356f3310d89ec0bbc2f6dcdfac032b72b5f7a027848
+size 2688440

fold_1/model.bias_scaled.fold_1.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:325e56a565856846902ebc5a351a819e3446a08f76f39ca14550f38ca030f5c4
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7179b589f6734f27db17aac4c760f4c23e7de67c654564ab782d38800d9893cb
+size 26448128

fold_1/model.chrombpnet.fold_1.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:269a2a3c34a52d7a864c5f06515e5fa16bad2228d241a99eab5e5572ca4f1be3
+size 27535360

fold_1/model.chrombpnet_nobias.fold_1.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30c8241f88c704fc6531e557f7b72ed000462e3f70609d5c91f05d80d76c46b6
+size 25583536

fold_1/model.chrombpnet_nobias.fold_1.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e493a4890f49b03a4c86bb31e536628bcb4043f3721aa53b20d4409b89f783c1
+size 26060800

fold_2/model.bias_scaled.fold_2.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90b4b58a21f2238a7255d356f3310d89ec0bbc2f6dcdfac032b72b5f7a027848
+size 2688440

fold_2/model.bias_scaled.fold_2.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:138b8c3696767c68d1f730e57372da0f9f404e55245af8f6b4674a7ec4778713
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19eb267675387da6f76790652e38c5f66148dc79c6c76d6affa8219b1426fd9c
+size 26448128

fold_2/model.chrombpnet.fold_2.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6202ae04984f662e9ec48c738fa78f4d81baed7988b20b2213e9cefc1e02ba4b
+size 27535360

fold_2/model.chrombpnet_nobias.fold_2.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6380ab82b8db80f3fe066a0f2efaf8d7b8cfb9ccda35761d904274a403126d5
+size 25583536

fold_2/model.chrombpnet_nobias.fold_2.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1c08a8b4f78ec55f86c7b0b8c1ae2f601dd7c2c8bcdac0623d40cd4d5875557
+size 26060800

fold_3/model.bias_scaled.fold_3.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90b4b58a21f2238a7255d356f3310d89ec0bbc2f6dcdfac032b72b5f7a027848
+size 2688440

fold_3/model.bias_scaled.fold_3.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee4f720486660425d56ff0da1f3512a13114ad1250fe520dd4b5edfd48f2f9f5
+size 1198080

fold_3/model.chrombpnet.fold_3.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ae1b3aae30823aacaa74dc56d39b9f6973e8b6ad04d7b44a12b8ba505375537
+size 26448128

fold_3/model.chrombpnet.fold_3.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:648bfdf11b9d0faa8a56165dd9b85bd03940a7eb3ff5a6acab9719b32c7b2c03
+size 27535360

fold_3/model.chrombpnet_nobias.fold_3.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f913227ed55d62e4ef2fa0c44a78f03e0f476a9683e669325c6152e6f36fecbb
+size 25583536

fold_3/model.chrombpnet_nobias.fold_3.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20c081fa812f14c81f3be94bd699cbf5843c4d4b89b9e21d9be5dd0eac559c82
+size 26060800

fold_4/model.bias_scaled.fold_4.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90b4b58a21f2238a7255d356f3310d89ec0bbc2f6dcdfac032b72b5f7a027848
+size 2688440

fold_4/model.bias_scaled.fold_4.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98d7d6c3e6b509e1b47aa76f3c0f9af4810d58ee78951a946a7397659543067c
+size 1198080

fold_4/model.chrombpnet.fold_4.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af53986085db18cdb85c19e2b30f3fb0be5561032f8610ac91bbea6cad4e8071
+size 26448128

fold_4/model.chrombpnet.fold_4.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7a13965f5c7a0fdb6855d8908206314ba0bd3324f1e6ea965767d5940b6ce3f
+size 27535360

fold_4/model.chrombpnet_nobias.fold_4.ENCSR745CGG.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40f7f3d92b355a83fa417a036b23c39b29e273e82ad61a4df7c01bbf72d906df
+size 25583536

fold_4/model.chrombpnet_nobias.fold_4.ENCSR745CGG.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4138b6675513cd9aec0918db3c240f87e05845a43733fac81d59451935260bd3
+size 26060800