Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - kidney
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in left kidney (ENCSR295DUI)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR295DUI](https://www.encodeproject.org/experiments/ENCSR295DUI/)
+- Model annotation: [ENCSR162OQQ](https://www.encodeproject.org/annotations/ENCSR162OQQ/)
+- Biosample: left kidney (Full name: Homo sapiens left kidney tissue female embryo (87 days))
+- Cell slim(s): None
+- Organ slim(s): kidney
+- Developmental slim(s): mesoderm
+- System slim(s): excretory-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.args.json

@@ -0,0 +1,42 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR295DUI/preprocessing/bigWigs/ENCSR295DUI.bigWig",
+    "output_dir": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/ziwei75/anaconda3/envs/chrombpnet/lib/python3.8/site-packages/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.4/ENCSR295DUI/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/models/chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2217.37
+trainings_pts_post_thresh	171600

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	7.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.4/ENCSR295DUI/fold0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR295DUI/logfile.modelling.fold_0.ENCSR295DUI.epoch_loss.csv

@@ -0,0 +1,14 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.361537218093872,302.7167663574219,312.5200500488281,0.6580591201782227,293.7565002441406,298.49456787109375
+1,0.683074951171875,289.9215087890625,294.8400573730469,0.5895107984542847,290.1676330566406,294.4121398925781
+2,0.6268962025642395,286.61529541015625,291.1289978027344,0.5629411935806274,286.1910095214844,290.2442626953125
+3,0.5890056490898132,284.13043212890625,288.37109375,0.544750988483429,286.3458251953125,290.2682189941406
+4,0.560980498790741,282.7547912597656,286.79400634765625,0.6236740350723267,286.28564453125,290.77593994140625
+5,0.5350519418716431,281.4083251953125,285.26055908203125,0.6154831647872925,286.65411376953125,291.0854797363281
+6,0.5152902603149414,280.0320739746094,283.7428283691406,0.5811012983322144,285.4978332519531,289.6818542480469
+7,0.5004095435142517,279.4696350097656,283.07232666015625,0.5026824474334717,285.63214111328125,289.2516174316406
+8,0.4927681088447571,278.407470703125,281.95550537109375,0.4798538088798523,285.99981689453125,289.4547119140625
+9,0.4760380983352661,277.6470031738281,281.0748291015625,0.5006421804428101,286.79119873046875,290.39581298828125
+10,0.4675215780735016,277.0452575683594,280.41131591796875,0.4736482501029968,287.1547546386719,290.56494140625
+11,0.45836058259010315,276.2262268066406,279.5263671875,0.4715127646923065,285.9847717285156,289.37969970703125
+12,0.4561891555786133,275.203369140625,278.4879150390625,0.46034130454063416,286.2226867675781,289.5373229980469

fold_0/model.bias_scaled.fold_0.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b7cea5b8fe8abd608799874cbbee2210263c97f9ffbf0788cdd233756d01100
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:546632496f68ae106e903d1baa5afaaa52629bf67eafacb611d795b254a347ce
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9ad4cced5d587a7c36616a3b8744b9c1b45f35f96f2d1c50faad6aefbfb0c2
+size 77538952

fold_0/model.chrombpnet.fold_0.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29f91591400bc1496b721d0825d2c699efef69992762d46d730a43988372b7e4
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:345c8932212f4e2d4dae1bc9ef8f4fc53cfb4ba860d458841f565256d0ff90c9
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb05a046d14167797b2531471ac0d76a87eda928f77ce8f20124dee327fa5d51
+size 26060800

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR295DUI/preprocessing/bigWigs/ENCSR295DUI.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.4/ENCSR295DUI/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2253.11
+trainings_pts_post_thresh	173376

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	7.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR295DUI/logfile.modelling.fold_1.ENCSR295DUI.epoch_loss.csv

@@ -0,0 +1,18 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,3.5269157886505127,305.2493896484375,330.9957580566406,0.8127281665802002,345.3192443847656,351.251953125
+1,0.7392527461051941,291.1426696777344,296.5395812988281,0.649209201335907,338.33392333984375,343.07275390625
+2,0.6720671057701111,287.5921936035156,292.4980163574219,0.6708300113677979,333.2124938964844,338.10931396484375
+3,0.6279041767120361,285.0049743652344,289.5887145996094,0.6827467679977417,329.2738342285156,334.25799560546875
+4,0.595315158367157,283.1436462402344,287.48980712890625,0.5808502435684204,330.5541076660156,334.7940979003906
+5,0.5679532289505005,281.64984130859375,285.79547119140625,0.5726357102394104,330.8240661621094,335.004150390625
+6,0.5405636429786682,280.4794006347656,284.4255065917969,0.6021061539649963,329.689697265625,334.0851745605469
+7,0.5223551392555237,279.0945129394531,282.9081726074219,0.6933891177177429,329.78271484375,334.8444519042969
+8,0.5061928629875183,278.0325927734375,281.7285461425781,0.507735550403595,329.6365661621094,333.3431091308594
+9,0.4887116551399231,276.996337890625,280.5641174316406,0.5316151976585388,329.35595703125,333.23699951171875
+10,0.48383015394210815,276.3247985839844,279.8570556640625,0.5302810072898865,328.93798828125,332.80914306640625
+11,0.46648669242858887,275.6193542480469,279.0247497558594,0.483746737241745,328.8585205078125,332.3897399902344
+12,0.4596062898635864,274.62249755859375,277.9772033691406,0.5788987874984741,330.10394287109375,334.3299560546875
+13,0.44969213008880615,274.20819091796875,277.4914855957031,0.4903797507286072,330.14013671875,333.7197265625
+14,0.44017916917800903,273.3440246582031,276.5577087402344,0.48508355021476746,330.3311767578125,333.8724670410156
+15,0.43485239148139954,272.50604248046875,275.6802062988281,0.4845436215400696,332.66387939453125,336.2010498046875
+16,0.429773211479187,272.0438537597656,275.180908203125,0.5088933706283569,333.7913818359375,337.5061950683594

fold_1/model.bias_scaled.fold_1.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84ec4d532028fdefe9f29c2cff30e870af3ef6b865aa56ba5a0486e03c540171
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:751e4a19d91cbfc0b1a3ead61c521fba0f525056b70fcbcd357e2e8912a133cb
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df2a4084447af285988b579b14254ae9bf5f5f6d153391271add1d05804d9f30
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0471ef24d88548afbb327849e07509c0d2016b7c8aa11737f26a763b91c5848
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7efe2c188a3bab94e58c0d6869c8e5a830b7feea62af53b7c44982d8370706d
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71c75e6f4a54c76c43e7c8902f9195ed39ab267361478d583f19682cbaf99ae2
+size 26060800

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR295DUI/preprocessing/bigWigs/ENCSR295DUI.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.4/ENCSR295DUI/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2216.1
+trainings_pts_post_thresh	178090

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	7.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR295DUI/logfile.modelling.fold_2.ENCSR295DUI.epoch_loss.csv

@@ -0,0 +1,12 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.731676459312439,303.5727844238281,316.04071044921875,0.7188676595687866,306.24658203125,311.4224853515625
+1,0.6872384548187256,290.5625,295.5108947753906,0.9276742339134216,301.6647033691406,308.3442687988281
+2,0.6310428380966187,286.7873840332031,291.3313293457031,0.6035119891166687,299.1141357421875,303.45947265625
+3,0.5899078845977783,284.3901062011719,288.6376647949219,0.5847938656806946,298.0536193847656,302.2638854980469
+4,0.5593740344047546,283.1529846191406,287.18048095703125,0.5584679841995239,297.99139404296875,302.012451171875
+5,0.5354822278022766,281.23193359375,285.0873107910156,0.5555868744850159,295.1190185546875,299.1193542480469
+6,0.5150731205940247,279.82501220703125,283.533447265625,0.686077892780304,296.9570617675781,301.8971252441406
+7,0.49978846311569214,279.1483154296875,282.7471008300781,0.49894970655441284,296.95916748046875,300.5516662597656
+8,0.4887351393699646,277.9928283691406,281.5121765136719,0.5067560076713562,298.276611328125,301.9250793457031
+9,0.4735479950904846,277.1428527832031,280.5519714355469,0.5104485154151917,295.8547058105469,299.52984619140625
+10,0.4652867913246155,276.65496826171875,280.0050354003906,0.4857771098613739,296.9376525878906,300.4353332519531

fold_2/model.bias_scaled.fold_2.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed76c401cb15038e3518777564be42f83cd5223d73609e9d9d6c31f05b1b20fb
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63605dfda1390168fa4c89f96149c0c37497c627bffe12c3bae86c105bee8dfc
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:791a734c910356afbb4fd5fbf7aad3f3bed091937d0a7548bbf344c94f2f0961
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:416906051765ef3678a9b6f34c9a5a677341bac866f3f0d510a8c40d3bafc9b3
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR295DUI.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:368cc3ae47bb615945e6204b8008eabd9a14f7bb3afdb50bb3290871f09286c9
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR295DUI.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7feb4c06bdca3ae030b63cbf6d18c5b55e7ce6fcd9cc086c381a8ede777a4661
+size 26060800

fold_3/logs.models.fold_3.ENCSR295DUI/logfile.modelling.fold_3.ENCSR295DUI.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR295DUI/preprocessing/bigWigs/ENCSR295DUI.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.4/ENCSR295DUI/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR295DUI/logfile.modelling.fold_3.ENCSR295DUI.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR295DUI/logfile.modelling.fold_3.ENCSR295DUI.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2253.9
+trainings_pts_post_thresh	171948

fold_3/logs.models.fold_3.ENCSR295DUI/logfile.modelling.fold_3.ENCSR295DUI.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR295DUI/logfile.modelling.fold_3.ENCSR295DUI.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	7.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR295DUI/logfile.modelling.fold_3.ENCSR295DUI.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR295DUI/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR295DUI/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py