Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - medialis
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in gastrocnemius medialis (ENCSR171ADO)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR171ADO](https://www.encodeproject.org/experiments/ENCSR171ADO/)
+- Model annotation: [ENCSR968OAS](https://www.encodeproject.org/annotations/ENCSR968OAS/)
+- Biosample: gastrocnemius medialis (Full name: Homo sapiens gastrocnemius medialis tissue male adult (37 years))
+- Cell slim(s): None
+- Organ slim(s): musculature-of-body,limb
+- Developmental slim(s): mesoderm
+- System slim(s): musculature
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.args.json

@@ -0,0 +1,42 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR171ADO/preprocessing/bigWigs/ENCSR171ADO.bigWig",
+    "output_dir": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/ziwei75/anaconda3/envs/chrombpnet/lib/python3.8/site-packages/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.6/ENCSR171ADO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/models/chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	21214.28
+trainings_pts_post_thresh	169410

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.6
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.6/ENCSR171ADO/fold0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR171ADO/logfile.modelling.fold_0.ENCSR171ADO.epoch_loss.csv

@@ -0,0 +1,19 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.536555290222168,518.3924560546875,545.280029296875,0.6992325186729431,509.8704528808594,517.2820434570312
+1,0.7472174763679504,485.0947570800781,493.0145568847656,0.6407674551010132,489.2275695800781,496.02001953125
+2,0.7088358998298645,473.2037658691406,480.7182922363281,0.604375958442688,484.83575439453125,491.2423095703125
+3,0.669227123260498,466.6120910644531,473.7061767578125,0.6095032691955566,482.1376037597656,488.59832763671875
+4,0.6475526690483093,464.2513427734375,471.11572265625,0.5731508135795593,484.3200988769531,490.3952941894531
+5,0.6253441572189331,461.3341064453125,467.96246337890625,0.5801279544830322,478.7237243652344,484.8734130859375
+6,0.6102584600448608,459.1107177734375,465.5791015625,0.573215663433075,478.6899108886719,484.76605224609375
+7,0.5958330631256104,456.9884948730469,463.30419921875,0.5372841358184814,480.59130859375,486.2864685058594
+8,0.5846879482269287,455.97906494140625,462.17559814453125,0.71718829870224,481.509765625,489.11199951171875
+9,0.5701289176940918,454.58251953125,460.62567138671875,0.5328907370567322,480.59222412109375,486.2405090332031
+10,0.559242844581604,453.6734313964844,459.6019287109375,0.5158951282501221,478.2969665527344,483.7655944824219
+11,0.5528007745742798,451.8666687011719,457.7258605957031,0.5066434741020203,478.301513671875,483.67181396484375
+12,0.5419078469276428,451.92962646484375,457.6737365722656,0.5025016069412231,475.5536804199219,480.880126953125
+13,0.5319560766220093,451.3692932128906,457.00762939453125,0.5383791327476501,482.9111022949219,488.61773681640625
+14,0.5262241363525391,449.7715759277344,455.3492736816406,0.49824094772338867,478.11212158203125,483.39337158203125
+15,0.5207111239433289,449.1120910644531,454.6326904296875,0.4981905221939087,478.5921936035156,483.87298583984375
+16,0.5207120180130005,449.16241455078125,454.6815490722656,0.5163971781730652,479.9621887207031,485.4356384277344
+17,0.5161631107330322,447.83062744140625,453.3021545410156,0.5192790031433105,477.34130859375,482.8456115722656

fold_0/model.bias_scaled.fold_0.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0752e5cb61c15c31ec9dde8bd5d74cb4e0d2ada75f3895e6e61e9f8a97e5baba
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71c95517c68e2cff8538138e0f0db690721fef6f06febe98b1bf7150026c4c96
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69e38d186c08d34caa2ab77d4468446a62882fca7c5ddc36a42d0b3f6d4af2a5
+size 77538952

fold_0/model.chrombpnet.fold_0.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f76c0da121943b9687b9f8d26d010e1ca9125a45c0d4228dcb00d4711a77e1b2
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3de4cf7f9badcf6f8ea3c92ec140fc6a198d68084323c156fa2a8e0e3d75904e
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9abe7797424df457e0b5e4c833cb111949b74522949d3b2dc751cc07b8f97c7
+size 26060800

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR171ADO/preprocessing/bigWigs/ENCSR171ADO.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.6/ENCSR171ADO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	13854.85
+trainings_pts_post_thresh	174904

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.5
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR171ADO/logfile.modelling.fold_1.ENCSR171ADO.epoch_loss.csv

@@ -0,0 +1,15 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.5149168968200684,506.3731994628906,522.2805786132812,0.8380930423736572,535.2066040039062,544.006103515625
+1,0.7206115126609802,474.6837158203125,482.25067138671875,0.7022082805633545,520.9627075195312,528.3361206054688
+2,0.6802833080291748,465.70233154296875,472.8462219238281,0.6689457893371582,524.9368896484375,531.9608154296875
+3,0.6503600478172302,460.764404296875,467.5935363769531,0.6593528389930725,510.7728271484375,517.6958618164062
+4,0.6234098672866821,456.62603759765625,463.1720886230469,0.6796773672103882,514.2591552734375,521.3958740234375
+5,0.609252393245697,455.2967529296875,461.6944274902344,0.6562322378158569,508.3978271484375,515.2876586914062
+6,0.5866826772689819,453.1040954589844,459.26422119140625,0.5881223678588867,511.7095031738281,517.8848876953125
+7,0.5693681240081787,451.4240417480469,457.4021301269531,0.5438624024391174,508.0320739746094,513.7426147460938
+8,0.5548412203788757,449.6385498046875,455.46484375,0.5268691182136536,504.8546447753906,510.3863525390625
+9,0.5490727424621582,449.6761474609375,455.441162109375,0.5792406797409058,506.5152893066406,512.5972900390625
+10,0.539961576461792,448.540283203125,454.2098083496094,0.5577627420425415,510.4911804199219,516.34716796875
+11,0.5360916256904602,447.82879638671875,453.4580078125,0.5195603370666504,506.5582580566406,512.0139770507812
+12,0.5252586603164673,446.5688781738281,452.08441162109375,0.6294057369232178,511.4520568847656,518.0606689453125
+13,0.5271204710006714,446.71954345703125,452.2536926269531,0.5093127489089966,505.1814880371094,510.529541015625

fold_1/model.bias_scaled.fold_1.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d50b4aa9859cc69e23c07394950efe1d23e40bb75982736256267c2d4a44c58
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03209443dcc76d028f73b78ac68dc8052c4baefe71bc8512d84e541115c8e6ad
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fa63b244ed4c27a5efe9f8a66d599b0861b6eb935261abbbbe7b9115f519ee7
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6ffe857ac3ba661cdfff1e4078abe9ab250a75455428d6fd15f33ee645bca74
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff4bfd7db4588654ada1f030818c7c4a0e1ce04c6c380f05d21a63d909cf4ccf
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9578f4565cf9f11bbef918a59682edb5649f0722a30f52fe37032a7a9c96b73c
+size 26060800

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR171ADO/preprocessing/bigWigs/ENCSR171ADO.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.6/ENCSR171ADO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	12468.06
+trainings_pts_post_thresh	178663

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.7
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR171ADO/logfile.modelling.fold_2.ENCSR171ADO.epoch_loss.csv

@@ -0,0 +1,16 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.6608757972717285,511.4304504394531,529.2012329101562,0.7841984033584595,519.0613403320312,527.4521484375
+1,0.7211304903030396,478.9271240234375,486.64288330078125,0.7195712327957153,506.7756652832031,514.4749755859375
+2,0.6640215516090393,468.800048828125,475.9046630859375,0.6473183035850525,500.5389404296875,507.4651794433594
+3,0.6408696174621582,464.09393310546875,470.9507751464844,0.621464729309082,500.65966796875,507.3092956542969
+4,0.6073433756828308,461.42852783203125,467.9273376464844,0.5822290182113647,493.78759765625,500.0173645019531
+5,0.5924136638641357,457.99884033203125,464.336669921875,0.5967719554901123,495.5419921875,501.92742919921875
+6,0.5724568963050842,456.3086242675781,462.4327392578125,0.5870542526245117,492.4405822753906,498.72198486328125
+7,0.559049665927887,455.3253173828125,461.30682373046875,0.5632758736610413,491.5528869628906,497.5796813964844
+8,0.5463001728057861,454.03936767578125,459.884765625,0.5493103861808777,491.4401550292969,497.3177185058594
+9,0.5415032505989075,453.3243408203125,459.11773681640625,0.5893075466156006,490.23406982421875,496.53924560546875
+10,0.5350534319877625,452.2706604003906,457.9949035644531,0.528380274772644,491.81964111328125,497.47344970703125
+11,0.5261486172676086,452.07843017578125,457.7088317871094,0.5385718941688538,491.98419189453125,497.7469177246094
+12,0.5228468179702759,450.90374755859375,456.4984436035156,0.5267296433448792,493.58306884765625,499.21893310546875
+13,0.5127570033073425,449.99298095703125,455.4798889160156,0.522074818611145,491.1232604980469,496.7095642089844
+14,0.5100924968719482,449.2792053222656,454.7370910644531,0.5567551255226135,494.7644958496094,500.72149658203125

fold_2/model.bias_scaled.fold_2.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:532853cfa4893d72deceee3c4ef010b7caa7f22d04132cbe09c0fe57adc988e8
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0e2a15d09d29ad8283a4ae7ca81ce5f0341d8af59023fcc18f2aae48bca707f
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:494fbc9ba0f9cbf82e52572dcf4c416066baf1a2b52eac5b935c4ba0c54178d5
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57aaa115f23e526b9f7697200691a9e0531267f096b1c68d6a8f70703f583879
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR171ADO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b99c5be2013aea5f9ab94c48cb3bd18824295490fd59461349dbbba38987faa9
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR171ADO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb914c0b295b78176a1b9c81f2c8646f4e10e06c1c7478192989e1ba8e39ddea
+size 26060800

fold_3/logs.models.fold_3.ENCSR171ADO/logfile.modelling.fold_3.ENCSR171ADO.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR171ADO/preprocessing/bigWigs/ENCSR171ADO.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.6/ENCSR171ADO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR171ADO/logfile.modelling.fold_3.ENCSR171ADO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR171ADO/logfile.modelling.fold_3.ENCSR171ADO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	15121.75
+trainings_pts_post_thresh	172529

fold_3/logs.models.fold_3.ENCSR171ADO/logfile.modelling.fold_3.ENCSR171ADO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR171ADO/logfile.modelling.fold_3.ENCSR171ADO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.7
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR171ADO/logfile.modelling.fold_3.ENCSR171ADO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR171ADO/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR171ADO/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py