Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - LHCN-M2
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in LHCN-M2 (ENCSR000EPE)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR000EPE](https://www.encodeproject.org/experiments/ENCSR000EPE/)
+- Model annotation: [ENCSR606CFD](https://www.encodeproject.org/annotations/ENCSR606CFD/)
+- Biosample: LHCN-M2 (Full name: Homo sapiens LHCN-M2)
+- Cell slim(s): embryonic-cell,myoblast
+- Organ slim(s): musculature-of-body
+- Developmental slim(s): mesoderm
+- System slim(s): musculature
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000EPE/preprocessing/bigWigs/ENCSR000EPE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_0/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	7.0
+counts_sum_max_thresh	17073.5
+trainings_pts_post_thresh	170383

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	34.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000EPE/logfile.modelling.fold_0.ENCSR000EPE.epoch_loss.csv

@@ -0,0 +1,21 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.9445836544036865,1023.5791625976562,1124.2847900390625,1.6119002103805542,953.57275390625,1008.6997680664062
+1,1.4718959331512451,899.6307373046875,949.9680786132812,1.4565200805664062,903.751953125,953.5648193359375
+2,1.3480651378631592,867.8142700195312,913.9171752929688,1.2540333271026611,879.202392578125,922.09033203125
+3,1.2571237087249756,850.6494140625,893.6420288085938,1.371833086013794,875.961181640625,922.8778686523438
+4,1.1909083127975464,837.4822998046875,878.212890625,1.1334929466247559,874.5881958007812,913.3538818359375
+5,1.1372253894805908,824.7689208984375,863.6611328125,1.236807942390442,867.5013427734375,909.80029296875
+6,1.1018739938735962,815.9937133789062,853.6785888671875,1.0637025833129883,861.0537719726562,897.4326782226562
+7,1.0672630071640015,810.8377075195312,847.3388061523438,1.0563066005706787,859.5890502929688,895.71435546875
+8,1.0268057584762573,804.8967895507812,840.0138549804688,1.0532679557800293,868.1295166015625,904.1514282226562
+9,1.0074785947799683,800.5968627929688,835.0529174804688,1.0356289148330688,847.44775390625,882.86669921875
+10,0.9703565239906311,797.1519165039062,830.3394165039062,1.0189262628555298,850.0604858398438,884.90771484375
+11,0.9489843249320984,791.9637451171875,824.4190673828125,1.0913525819778442,854.0580444335938,891.3820190429688
+12,0.9317097067832947,789.4688110351562,821.3320922851562,1.0126314163208008,851.400390625,886.0319213867188
+13,0.9120619297027588,788.9315185546875,820.1239013671875,0.9972705841064453,843.994873046875,878.1016845703125
+14,0.8874508738517761,785.7880859375,816.1377563476562,1.021747350692749,832.8167114257812,867.7597045898438
+15,0.8656544089317322,783.4223022460938,813.0276489257812,1.0317660570144653,855.4952392578125,890.781494140625
+16,0.8466562032699585,781.5924072265625,810.5489501953125,1.0308942794799805,846.3069458007812,881.5631713867188
+17,0.8281238079071045,778.8102416992188,807.1322021484375,0.9913411736488342,851.1090087890625,885.0122680664062
+18,0.818230152130127,777.7882080078125,805.7716674804688,1.0112730264663696,849.4122314453125,883.9979248046875
+19,0.7956740260124207,776.3364868164062,803.5491333007812,1.157671332359314,844.6720581054688,884.2650146484375

fold_0/model.bias_scaled.fold_0.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca9cd31810a963ed1e93d71f7371aadf145caa3334dcbf0e64e4ed1ee9e8c5b2
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2ac93298911ba46ecee6265b9a5a59082eb7bf49a89fcfe0b8da003bd197669
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1697b86f48d42d2b0871c283b6420d6ca636e6aa75acfaa6b2cd42fa90edb869
+size 77538840

fold_0/model.chrombpnet.fold_0.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c7d50c862023cee1ce80fec7d4509dba7c24694f1c90e5dee869577d5ab8457
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99348c42ae7229d35d0863a5ba4ec9aa4246f6f7aa2f22f6ef20aa3fc999af58
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:837b5b2fb7da1512d2a3d91610d4d8d9ce34f72c1c0870234b2df723c15c01af
+size 26060800

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000EPE/preprocessing/bigWigs/ENCSR000EPE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_1/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	7.0
+counts_sum_max_thresh	17180.9
+trainings_pts_post_thresh	174646

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	34.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000EPE/logfile.modelling.fold_1.ENCSR000EPE.epoch_loss.csv

@@ -0,0 +1,21 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,3.313202381134033,1019.60888671875,1132.9200439453125,1.6845664978027344,1060.6953125,1118.3074951171875
+1,1.5100511312484741,902.6466064453125,954.2891845703125,1.2971818447113037,990.9057006835938,1035.2684326171875
+2,1.3524338006973267,864.390380859375,910.6450805664062,1.2485491037368774,975.5470581054688,1018.2468872070312
+3,1.249580979347229,847.0264892578125,889.7630004882812,1.1875312328338623,972.7573852539062,1013.3705444335938
+4,1.1973341703414917,831.5838012695312,872.5321655273438,1.1396404504776,944.0316772460938,983.007568359375
+5,1.1419285535812378,821.548095703125,860.6023559570312,1.0680185556411743,948.0595703125,984.5857543945312
+6,1.0947855710983276,812.16455078125,849.6054077148438,1.046137809753418,936.9451904296875,972.7234497070312
+7,1.0572582483291626,804.2713623046875,840.429931640625,1.051464557647705,940.7793579101562,976.7393188476562
+8,1.02833890914917,798.788330078125,833.9566040039062,1.2322684526443481,936.3848876953125,978.5291748046875
+9,1.0038411617279053,796.7512817382812,831.0826416015625,0.9772108793258667,935.4361572265625,968.8564453125
+10,0.9697701930999756,791.4698486328125,824.635009765625,0.9999576210975647,931.6832275390625,965.8821411132812
+11,0.9453245997428894,787.238525390625,819.5682373046875,1.0421044826507568,940.4213256835938,976.0616455078125
+12,0.9223175048828125,783.4965209960938,815.037841796875,0.9963768124580383,938.4547119140625,972.5307006835938
+13,0.9013230800628662,782.4265747070312,813.2509765625,0.9656319618225098,928.7477416992188,961.7728271484375
+14,0.8819543719291687,778.9901123046875,809.1536254882812,0.9853487014770508,923.2518920898438,956.9503784179688
+15,0.8596621155738831,777.4097290039062,806.8096923828125,1.017833948135376,936.9788208007812,971.7889404296875
+16,0.8447396755218506,775.1742553710938,804.0634155273438,1.0093826055526733,932.2338256835938,966.7543334960938
+17,0.828711986541748,773.3983764648438,801.740966796875,1.0102819204330444,930.8262329101562,965.3782958984375
+18,0.8110905885696411,772.0269775390625,799.7672729492188,1.0033918619155884,931.6796875,965.9957275390625
+19,0.7932707071304321,769.099365234375,796.2296752929688,0.9758096933364868,927.3411254882812,960.7141723632812

fold_1/model.bias_scaled.fold_1.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8dc4ea450779d6ef83dd93df2dadbd55964067f15832c1b005ce0c7593d7145
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a924754ab172e9fe64b18d227c428623a1255d3c4d3bf424ba8ab55e2e980e0c
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49488054ccc374ffb71657535a0a50f2ec7c1fb917c5f0b74ae4d2755d5266e2
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f3d72a085e2a91dd07a1d3faa45dddef37730598c008f28c9d36b8d716b4e0f
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35f31c2df6645b988b73254b8c8a06fcf4ebe6596eb54aa24d4eca2eec843eb1
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84b88d3c4a0cb37aedc9a26c899a01a549e91c3e03ccb094b94594396397c9f2
+size 26060800

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000EPE/preprocessing/bigWigs/ENCSR000EPE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_2/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	7.0
+counts_sum_max_thresh	17285.33
+trainings_pts_post_thresh	178494

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	34.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000EPE/logfile.modelling.fold_2.ENCSR000EPE.epoch_loss.csv

@@ -0,0 +1,14 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,7.785579681396484,1037.84912109375,1304.89501953125,1.8026044368743896,1110.33056640625,1172.1600341796875
+1,1.6143593788146973,928.16650390625,983.5377807617188,1.5285779237747192,1039.434814453125,1091.865234375
+2,1.4436542987823486,881.4264526367188,930.9428100585938,1.4020311832427979,1019.1576538085938,1067.247314453125
+3,1.3264812231063843,859.8426513671875,905.3414306640625,1.2664366960525513,984.5139770507812,1027.953125
+4,1.226823329925537,844.0672607421875,886.1469116210938,1.1918400526046753,967.78857421875,1008.669189453125
+5,1.1757076978683472,830.9891357421875,871.3152465820312,1.1296268701553345,958.3993530273438,997.144775390625
+6,1.1252219676971436,820.8365478515625,859.4326171875,1.1461697816848755,953.6272583007812,992.941162109375
+7,1.0901944637298584,813.123046875,850.5158081054688,1.091018795967102,941.36669921875,978.7886962890625
+8,1.056318998336792,807.5006103515625,843.7323608398438,1.0955793857574463,960.120361328125,997.698486328125
+9,1.0277924537658691,801.6011962890625,836.8550415039062,1.0936375856399536,950.0849609375,987.5960083007812
+10,0.9989309906959534,797.0277099609375,831.2910766601562,1.1101351976394653,950.2481689453125,988.326171875
+11,0.9784485101699829,794.3346557617188,827.895751953125,1.075147032737732,945.5729370117188,982.4505004882812
+12,0.9527466893196106,789.779296875,822.4590454101562,1.0239659547805786,948.1858520507812,983.3082275390625

fold_2/model.bias_scaled.fold_2.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4c09a0d0d6cfae87335fc92b54334a3bffc2c1ef99fd661842c75a7fd8b2381
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:749e5e82d33e37f2ff07497a597a61aba3a91acb1829298491350c4091c6fda1
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24303f9927b44b23423520ad4fe42667130c66fe4333a465a068f288dd8ac36e
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae24161ee57c78da25ca3e3b2f5fd98de59865c1cdf55e1f347bba3570b71c5e
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR000EPE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4539d8fe240b12df4d5c78973a50d1917d3d7fbbc4c288c5c0d62d02c6fed4d
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR000EPE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b581fcc0fc62fa479daa60957d8c4e6ce755167ca152c176dfaa0f912cce351a
+size 26060800

fold_3/logs.models.fold_3.ENCSR000EPE/logfile.modelling.fold_3.ENCSR000EPE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000EPE/preprocessing/bigWigs/ENCSR000EPE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_3/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR000EPE/logfile.modelling.fold_3.ENCSR000EPE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR000EPE/logfile.modelling.fold_3.ENCSR000EPE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	7.0
+counts_sum_max_thresh	17578.46
+trainings_pts_post_thresh	176670

fold_3/logs.models.fold_3.ENCSR000EPE/logfile.modelling.fold_3.ENCSR000EPE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR000EPE/logfile.modelling.fold_3.ENCSR000EPE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	34.4
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR000EPE/logfile.modelling.fold_3.ENCSR000EPE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000EPE/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000EPE/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py