Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - CD14-positive-monocyte
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in CD14-positive monocyte (ENCSR674JIL)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR674JIL](https://www.encodeproject.org/experiments/ENCSR674JIL/)
+- Model annotation: [ENCSR279GNT](https://www.encodeproject.org/annotations/ENCSR279GNT/)
+- Biosample: CD14-positive monocyte (Full name: Homo sapiens CD14-positive monocyte male adult (37 years))
+- Cell slim(s): myeloid-cell,monocyte,hematopoietic-cell,leukocyte
+- Organ slim(s): bodily-fluid,blood
+- Developmental slim(s): mesoderm
+- System slim(s): immune-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "pipeline",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR674JIL//preprocessing/bigWigs/ENCSR674JIL.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR674JIL",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR674JIL_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR674JIL_chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/chrombpnet_model/logs/ENCSR674JIL_chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	1804.41
+trainings_pts_post_thresh	109122

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR674JIL_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	2.8
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR674JIL_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR674JIL/logfile.modelling.fold_0.ENCSR674JIL.epoch_loss.csv

@@ -0,0 +1,20 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.3461129665374756,192.0008087158203,198.5699005126953,1.3042008876800537,186.2135772705078,189.8652801513672
+1,1.1968731880187988,177.4316864013672,180.78277587890625,1.1969919204711914,180.55384826660156,183.90536499023438
+2,1.0800861120224,173.75209045410156,176.7763214111328,1.024059772491455,177.39735412597656,180.26475524902344
+3,0.9888692498207092,171.796630859375,174.56541442871094,0.9292765259742737,176.5570831298828,179.1591033935547
+4,0.926807165145874,170.03799438476562,172.6331024169922,0.8608449697494507,176.75845336914062,179.1687774658203
+5,0.8934528827667236,169.02511596679688,171.5267333984375,0.815356433391571,175.10476684570312,177.3877716064453
+6,0.8515987992286682,168.11282348632812,170.49732971191406,0.9296345114707947,175.8961639404297,178.49913024902344
+7,0.8216729760169983,166.50509643554688,168.8059539794922,0.8833080530166626,177.3710479736328,179.8442840576172
+8,0.8026081323623657,166.56387329101562,168.8112030029297,0.7884612083435059,174.58595275878906,176.79364013671875
+9,0.7706258296966553,165.67308044433594,167.83082580566406,0.8425867557525635,175.69805908203125,178.05728149414062
+10,0.7569814920425415,164.7798309326172,166.89935302734375,0.8655059337615967,175.3828125,177.80624389648438
+11,0.7409766912460327,164.3773193359375,166.45213317871094,0.7600317001342773,175.28366088867188,177.41171264648438
+12,0.7201640605926514,163.30206298828125,165.31871032714844,0.7970102429389954,174.37313842773438,176.6048126220703
+13,0.7133830785751343,162.71946716308594,164.71702575683594,0.8331853151321411,174.1439971923828,176.47694396972656
+14,0.6961377263069153,162.4920654296875,164.44114685058594,0.7694048285484314,176.41043090820312,178.56480407714844
+15,0.6750816702842712,161.88980102539062,163.78030395507812,0.7451543807983398,176.80596923828125,178.89244079589844
+16,0.6689637303352356,161.01454162597656,162.88768005371094,0.7478750944137573,176.9388885498047,179.03294372558594
+17,0.6539497375488281,160.7453155517578,162.5763702392578,0.7559962272644043,176.25653076171875,178.3732147216797
+18,0.6362802982330322,159.86634826660156,161.64768981933594,0.7507581114768982,176.62380981445312,178.72593688964844

fold_0/model.bias_scaled.fold_0.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c5a2ac904ffd9b9cfe00493acf6b1a3c50c399441f8cfab738d884e4fe99f6d
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2967f3d0327c94b37844c1607730b807ee1d7ac610a7b4c31040fa1564b9ac49
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b02ddcab1c5a75e6a538db6020e7d9a1bce40ee30cf9bfcb434815ff06cdea69
+size 77538904

fold_0/model.chrombpnet.fold_0.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8c774f1adf3aa95ab4d83933e1e5db9de9508e3feb874f7538c4c2854b4df06
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2322f849338087f7d9c32dafc834144813b2f161b5d0e7687bd25fb89416235
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea735fd72e874a086c1e70ac8921b1adb21875d2168cf9385cf78b3601e1cc86
+size 26060800

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR674JIL//preprocessing/bigWigs/ENCSR674JIL.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_1/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_1/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_1/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR674JIL",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR674JIL_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR674JIL_chrombpnet",
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_1/chrombpnet_model/logs/ENCSR674JIL_chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	1760.54
+trainings_pts_post_thresh	112625

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR674JIL_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	2.8
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR674JIL_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR674JIL/logfile.modelling.fold_1.ENCSR674JIL.epoch_loss.csv

@@ -0,0 +1,16 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,3.0046021938323975,191.80792236328125,200.220703125,1.3716617822647095,201.6514129638672,205.49224853515625
+1,1.3223270177841187,179.2430419921875,182.9457244873047,1.1625068187713623,197.09812927246094,200.35302734375
+2,1.1418167352676392,174.74505615234375,177.9420928955078,0.9773170948028564,194.98670959472656,197.72323608398438
+3,1.0197829008102417,172.2552032470703,175.1107940673828,0.9200223684310913,193.1016387939453,195.6776885986328
+4,0.9648198485374451,171.0035858154297,173.7049560546875,0.8527940511703491,193.4988250732422,195.88656616210938
+5,0.9221975207328796,170.18446350097656,172.76661682128906,0.8484418392181396,191.1975860595703,193.5732421875
+6,0.8779483437538147,168.81051635742188,171.2686767578125,0.8083779215812683,191.65037536621094,193.91375732421875
+7,0.8535568118095398,167.7147216796875,170.1047821044922,0.7742857933044434,191.7469940185547,193.9149627685547
+8,0.8105877637863159,167.28192138671875,169.55154418945312,0.794975996017456,190.99276733398438,193.2186279296875
+9,0.8025133013725281,166.3924102783203,168.6394500732422,0.7690487504005432,189.5332794189453,191.6866455078125
+10,0.7856358885765076,165.6908416748047,167.89041137695312,0.7633747458457947,191.75448608398438,193.89199829101562
+11,0.7651742100715637,164.7729034423828,166.91549682617188,0.787261962890625,190.95632934570312,193.1605987548828
+12,0.7470319867134094,164.29794311523438,166.38958740234375,0.7528258562088013,193.03030395507812,195.13816833496094
+13,0.7376905679702759,163.5843505859375,165.6499481201172,0.731789767742157,191.054443359375,193.10350036621094
+14,0.7130024433135986,163.2798309326172,165.27647399902344,0.7342199683189392,190.3644256591797,192.4202117919922

fold_1/model.bias_scaled.fold_1.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd167f8f8d72f21897d068ea012779cb23ee62f6077af261704dd8ca3b994ac9
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2430649544590c50613ac4c0e889f357986e5f7a62ac801698fb0804cff9090
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c47065e21c586427a68114e0144367a19e1afe39461364227e3d0f5ac200201a
+size 77538904

fold_1/model.chrombpnet.fold_1.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfea8dd68d8e42540c31f3b49b704e85b58bcd117c2761cc90662e3c0101c000
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:853fa93b54a08040899f92afc9a689141abf1c0f590479b149f344efef5563b3
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c5838a161cdd15b6e5828c9cbe957e05940842f184127c359bc17fe34fab827
+size 26060800

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR674JIL//preprocessing/bigWigs/ENCSR674JIL.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_2/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_2/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_2/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR674JIL",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR674JIL_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR674JIL_chrombpnet",
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_2/chrombpnet_model/logs/ENCSR674JIL_chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	1736.78
+trainings_pts_post_thresh	115831

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR674JIL_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	2.8
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR674JIL_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR674JIL/logfile.modelling.fold_2.ENCSR674JIL.epoch_loss.csv

@@ -0,0 +1,13 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.0728917121887207,190.3804931640625,196.1847381591797,1.383104920387268,187.44093322753906,191.31370544433594
+1,1.2402335405349731,177.52976989746094,181.00241088867188,1.2124886512756348,182.24595642089844,185.64088439941406
+2,1.0729236602783203,173.44032287597656,176.44459533691406,1.0709589719772339,180.2384033203125,183.2371063232422
+3,0.9782441258430481,171.82562255859375,174.5647735595703,0.9578427672386169,179.1140594482422,181.7960968017578
+4,0.9234975576400757,170.17689514160156,172.76280212402344,0.8574806451797485,179.1420135498047,181.54293823242188
+5,0.8869812488555908,169.1129608154297,171.59637451171875,0.8397234082221985,179.0846405029297,181.4358367919922
+6,0.8463179469108582,168.2548370361328,170.62448120117188,0.8032829165458679,176.82032775878906,179.0695343017578
+7,0.8205516338348389,167.46290588378906,169.760498046875,0.8269631862640381,177.48165893554688,179.7972412109375
+8,0.7970538139343262,166.66456604003906,168.8963165283203,0.7986038327217102,177.10684204101562,179.34286499023438
+9,0.7789639830589294,165.76239013671875,167.94346618652344,0.8498391509056091,178.20388793945312,180.58340454101562
+10,0.7528854012489319,165.4779815673828,167.5862274169922,0.7921310067176819,178.14112854003906,180.3590545654297
+11,0.744307816028595,165.01498413085938,167.0992889404297,0.7813647389411926,177.6256561279297,179.81344604492188

fold_2/model.bias_scaled.fold_2.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa5fee7cd98007f161fcc65c9e7b1acf5d6f07f044621dd9640fe24a8deb6a68
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a535356a088b2281cbae1d7397a4c7418537d3fa5c393cc1242cfc0548b1ab96
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c001ba82909d7b9680f3e59b5a4a2e00ad6ebe4e053251b21bc1591eb651932c
+size 77538904

fold_2/model.chrombpnet.fold_2.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3aeabbcfe75831c22fbcaa9398e03cfecf60e7f0dc7ec35db817e263c0c64425
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR674JIL.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8105aeea9d3e47e0d3d153ad6d0b6c0e2e0a33b30b0f86582fb514b182f6c310
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR674JIL.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e6d3074d5861af581291dc131bdb1e8c46cdcd04a391dc1825407dfba3a317c
+size 26060800

fold_3/logs.models.fold_3.ENCSR674JIL/logfile.modelling.fold_3.ENCSR674JIL.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR674JIL//preprocessing/bigWigs/ENCSR674JIL.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_3/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_3/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_3/chrombpnet_model/auxiliary/ENCSR674JIL_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR674JIL",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR674JIL_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR674JIL_chrombpnet",
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_3/chrombpnet_model/logs/ENCSR674JIL_chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR674JIL/logfile.modelling.fold_3.ENCSR674JIL.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR674JIL/logfile.modelling.fold_3.ENCSR674JIL.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	1761.02
+trainings_pts_post_thresh	112270

fold_3/logs.models.fold_3.ENCSR674JIL/logfile.modelling.fold_3.ENCSR674JIL.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR674JIL_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR674JIL/logfile.modelling.fold_3.ENCSR674JIL.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	2.8
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR674JIL//chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR674JIL_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR674JIL/logfile.modelling.fold_3.ENCSR674JIL.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR674JIL/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR674JIL_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR674JIL/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py