Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - kidney
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in kidney (ENCSR437KLY)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR437KLY](https://www.encodeproject.org/experiments/ENCSR437KLY/)
+- Model annotation: [ENCSR950QAL](https://www.encodeproject.org/annotations/ENCSR950QAL/)
+- Biosample: kidney (Full name: Homo sapiens kidney tissue female embryo (85 days))
+- Cell slim(s): None
+- Organ slim(s): kidney
+- Developmental slim(s): mesoderm
+- System slim(s): excretory-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "pipeline",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR437KLY//preprocessing/bigWigs/ENCSR437KLY.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR437KLY",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR437KLY_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR437KLY_chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/chrombpnet_model/logs/ENCSR437KLY_chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	752.48
+trainings_pts_post_thresh	170341

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR437KLY_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	5.9
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR437KLY_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR437KLY/logfile.modelling.fold_0.ENCSR437KLY.epoch_loss.csv

@@ -0,0 +1,13 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.3513398170471191,226.99900817871094,234.97215270996094,0.7196467518806458,231.76414489746094,236.01004028320312
+1,0.5413776636123657,223.22837829589844,226.42266845703125,0.493980348110199,230.57302856445312,233.4874725341797
+2,0.4949847161769867,221.5220184326172,224.4422149658203,0.44884130358695984,229.64564514160156,232.29379272460938
+3,0.4654676020145416,220.42959594726562,223.17578125,0.4278126358985901,228.3895721435547,230.91378784179688
+4,0.4374271631240845,219.5720977783203,222.15277099609375,0.42110317945480347,228.67225646972656,231.15684509277344
+5,0.4230702221393585,218.80874633789062,221.30494689941406,0.41530048847198486,228.6090545654297,231.05923461914062
+6,0.40258416533470154,217.98252868652344,220.35794067382812,0.3933229446411133,228.4912567138672,230.81190490722656
+7,0.38745996356010437,217.29856872558594,219.5846710205078,0.4546775817871094,229.073974609375,231.75662231445312
+8,0.3823310136795044,216.68527221679688,218.94085693359375,0.38269031047821045,229.99880981445312,232.2567596435547
+9,0.36824607849121094,215.92501831054688,218.0977783203125,0.3953729271888733,229.2761688232422,231.60879516601562
+10,0.359809547662735,215.02931213378906,217.15203857421875,0.3771713972091675,230.16635131835938,232.39157104492188
+11,0.34928208589553833,214.6013946533203,216.66226196289062,0.37758153676986694,230.08160400390625,232.309326171875

fold_0/model.bias_scaled.fold_0.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56e2c8a3d19f115d4a69b45f9e622cd71dc8e80ceadea739c80b685879e10058
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7617ddb3af1aa43089ee09b39543b506292071ab6f7a2133f967bbf7e52e7f3d
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15e69e6d10bf3c75fa6ba0deba6170ca5d4f03a827ea6cfa0da58c94403b2961
+size 77538904

fold_0/model.chrombpnet.fold_0.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9d74808244f9c9967ba938c7c04bb75e269753bb96df544ff92964a500503e5
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3746134c22ab6454f4a724df56962bc3462ed33f39aaa3fa25e065c32229fe06
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7cbc4b6b78ac218664290195fd8e2bb7ace1e89ff56df1cd3ecf99699d5ce7c
+size 26060800

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR437KLY//preprocessing/bigWigs/ENCSR437KLY.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_1/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_1/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_1/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR437KLY",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR437KLY_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR437KLY_chrombpnet",
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_1/chrombpnet_model/logs/ENCSR437KLY_chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	788.45
+trainings_pts_post_thresh	174704

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR437KLY_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	5.9
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR437KLY_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR437KLY/logfile.modelling.fold_1.ENCSR437KLY.epoch_loss.csv

@@ -0,0 +1,13 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,4.07789945602417,228.28863525390625,252.34793090820312,0.6067625880241394,250.43167114257812,254.01145935058594
+1,0.555264413356781,224.4690399169922,227.7447052001953,0.5201427340507507,249.0332794189453,252.10227966308594
+2,0.508330762386322,222.70494079589844,225.70413208007812,0.5625291466712952,246.4462127685547,249.76490783691406
+3,0.48692482709884644,221.7659149169922,224.63868713378906,0.732347846031189,247.46066284179688,251.78146362304688
+4,0.4619123637676239,220.9099884033203,223.63565063476562,0.44248974323272705,247.07371520996094,249.68447875976562
+5,0.439741313457489,220.57952880859375,223.1739959716797,0.4354651868343353,246.0287628173828,248.59803771972656
+6,0.4186106026172638,219.6305389404297,222.10011291503906,0.43538135290145874,245.51219177246094,248.0808563232422
+7,0.4029938578605652,218.98036193847656,221.35800170898438,0.44861486554145813,246.53903198242188,249.18582153320312
+8,0.39204248785972595,218.3465576171875,220.65972900390625,0.42624524235725403,248.4443359375,250.95925903320312
+9,0.3776220381259918,217.552978515625,219.78099060058594,0.4043189585208893,247.4713134765625,249.85690307617188
+10,0.3676249086856842,216.64613342285156,218.81507873535156,0.4088645577430725,248.0261688232422,250.43849182128906
+11,0.3607848584651947,215.97488403320312,218.10337829589844,0.39533811807632446,248.56988525390625,250.90223693847656

fold_1/model.bias_scaled.fold_1.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2915fc55a5d6ef6989eb3a0c64d11e441aa98c896395a08f42e3d8f7d82883dc
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2afbd37372bcd0083e8bffd9340cf363ec3f7933969f9cc17e6f4031ecfba990
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac362bfed4be89e4fa1b17e8e39d0c0e05eb7620e4713aa9f83ebbdaba1c423b
+size 77538904

fold_1/model.chrombpnet.fold_1.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a728b821cb7f03c6d9a2b35c15db6ef19fc706cbd81fb0ecceea77baf2574eeb
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f42ba871ce88cdf5c5e66081cb3b7df6546dd1e79f2c68790bcca77883307a2
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27c146c212d9b6259f1161159e42124d8d0c3a5e44e8a4e7f4588fa0d85a116c
+size 26060800

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR437KLY//preprocessing/bigWigs/ENCSR437KLY.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_2/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_2/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_2/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR437KLY",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR437KLY_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR437KLY_chrombpnet",
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_2/chrombpnet_model/logs/ENCSR437KLY_chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	780.45
+trainings_pts_post_thresh	176481

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR437KLY_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	5.9
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR437KLY_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR437KLY/logfile.modelling.fold_2.ENCSR437KLY.epoch_loss.csv

@@ -0,0 +1,14 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.8064446449279785,228.8628387451172,239.52056884765625,0.5464224219322205,234.06915283203125,237.29298400878906
+1,0.546389102935791,224.61676025390625,227.8406219482422,0.511306643486023,232.09249877929688,235.10928344726562
+2,0.4977969229221344,222.89198303222656,225.82891845703125,0.458994060754776,231.95155334472656,234.6595001220703
+3,0.4695129096508026,221.77149963378906,224.54153442382812,0.5847147703170776,230.6614227294922,234.1112060546875
+4,0.44502079486846924,220.9047393798828,223.5301971435547,0.4154255986213684,229.58119201660156,232.0321502685547
+5,0.4201901853084564,220.35728454589844,222.83612060546875,0.4532342255115509,230.1028594970703,232.7770538330078
+6,0.4072771370410919,219.44017028808594,221.84307861328125,0.43129274249076843,230.32110595703125,232.8657684326172
+7,0.38729387521743774,218.83251953125,221.11709594726562,0.4131564199924469,229.50759887695312,231.94520568847656
+8,0.37462112307548523,218.0452880859375,220.255615234375,0.4061313569545746,230.61099243164062,233.00714111328125
+9,0.36423155665397644,217.07762145996094,219.2268524169922,0.45279616117477417,230.40252685546875,233.07412719726562
+10,0.35821473598480225,216.89488220214844,219.00845336914062,0.380096435546875,231.29176330566406,233.5342254638672
+11,0.3512309491634369,216.0052490234375,218.0773468017578,0.39174848794937134,231.8896026611328,234.2010498046875
+12,0.3405075669288635,215.4766082763672,217.4857635498047,0.38279271125793457,232.6078643798828,234.8662109375

fold_2/model.bias_scaled.fold_2.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fb3ece65298a82a786c18fd604848dbf0e39d28229d1cbb15dc54ccb5061599
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5f3420daac8c7ab83cae3e9c65143e6af4991589ea2e4e2b32ad0a899174990
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c501f03bada2ea2e811a0e4c1b56e7b866afe4289dacdf6a3375bb3fc82dc57a
+size 77538904

fold_2/model.chrombpnet.fold_2.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6d6fd5aee71ba2f4524968c7cc352cfd3d795ddf9df518ba1a175d9d78ab9e1
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR437KLY.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d5a263cf3a43ac32b878cba1d03809ecf386bbe89f0c309995cf814a566ee26
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR437KLY.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e298ca3cdc4c57de0b5d0ac4b88dd1441f8011e6f938200aac70ab77c713c42
+size 26060800

fold_3/logs.models.fold_3.ENCSR437KLY/logfile.modelling.fold_3.ENCSR437KLY.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR437KLY//preprocessing/bigWigs/ENCSR437KLY.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_3/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_3/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_3/chrombpnet_model/auxiliary/ENCSR437KLY_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR437KLY",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR437KLY_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR437KLY_chrombpnet",
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_3/chrombpnet_model/logs/ENCSR437KLY_chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR437KLY/logfile.modelling.fold_3.ENCSR437KLY.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR437KLY/logfile.modelling.fold_3.ENCSR437KLY.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	784.28
+trainings_pts_post_thresh	172226

fold_3/logs.models.fold_3.ENCSR437KLY/logfile.modelling.fold_3.ENCSR437KLY.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR437KLY_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR437KLY/logfile.modelling.fold_3.ENCSR437KLY.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	6.0
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR437KLY//chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR437KLY_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR437KLY/logfile.modelling.fold_3.ENCSR437KLY.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR437KLY/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR437KLY_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR437KLY/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py