Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - t-cell
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in stimulated activated CD4-positive, alpha-beta T cell (ENCSR703AYZ)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR703AYZ](https://www.encodeproject.org/experiments/ENCSR703AYZ/)
+- Model annotation: [ENCSR444BKO](https://www.encodeproject.org/annotations/ENCSR444BKO/)
+- Biosample: stimulated activated CD4-positive, alpha-beta T cell (Full name: Homo sapiens stimulated activated CD4-positive, alpha-beta T cell male adult (42 years) treated with anti-CD3 and anti-CD28 coated beads for 24 hours, 100 ng/mL Interleukin-4 for 24 hours)
+- Cell slim(s): CD4+-T-cell,T-cell,hematopoietic-cell,leukocyte
+- Organ slim(s): blood,bodily-fluid
+- Developmental slim(s): mesoderm,endoderm
+- System slim(s): immune-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR703AYZ/preprocessing/bigWigs/ENCSR703AYZ.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/ENCSR283TME_bias_0.9.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	12.0
+counts_sum_max_thresh	16711.49
+trainings_pts_post_thresh	172032

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	12.1
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR703AYZ/logfile.modelling.fold_0.ENCSR703AYZ.epoch_loss.csv

@@ -0,0 +1,12 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.0827419757843018,751.307861328125,764.4078979492188,0.8572027087211609,665.76611328125,676.1384887695312
+1,0.845602810382843,683.6397094726562,693.8713989257812,0.6694642901420593,652.9652099609375,661.0660400390625
+2,0.7617568373680115,667.935791015625,677.1534423828125,0.7553879022598267,644.161376953125,653.3013305664062
+3,0.7057015895843506,656.7761840820312,665.3139038085938,0.6205217838287354,642.4993286132812,650.0074462890625
+4,0.6788067817687988,649.1011352539062,657.3140869140625,0.6620478630065918,651.0892333984375,659.0996704101562
+5,0.653669536113739,643.1796875,651.0885009765625,0.6510060429573059,631.9671630859375,639.8441772460938
+6,0.6330723762512207,638.0779418945312,645.7382202148438,0.6848334074020386,635.7083740234375,643.9949340820312
+7,0.6156825423240662,633.7621459960938,641.210693359375,0.7128728032112122,640.7117309570312,649.33740234375
+8,0.5967041254043579,631.4541015625,638.6735229492188,0.6746969223022461,640.0288696289062,648.1929931640625
+9,0.5823125839233398,627.1688842773438,634.2163696289062,0.6186822652816772,645.2136840820312,652.699951171875
+10,0.5740126967430115,624.2861328125,631.2310180664062,0.549136757850647,634.9204711914062,641.5647583007812

fold_0/model.bias_scaled.fold_0.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf2b55e831fa04b44ae5c3eb3f27fcd8263310ed25c2226cec6111075042ef05
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d41494d2bc90d968b159b0c2d621832fb8f0c1881a3ef44330980d85fcf2c9d
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c23e891d3f45e6bc522f48ed5817bbd973a8cb3a7ccffc84d9d8709af5702035
+size 77538896

fold_0/model.chrombpnet.fold_0.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd66ff9aeb311b35cbfd484c1b49fe8aa9f0ed8fd03378728d87bcda557ebaaa
+size 27535360

fold_0/model.chrombpnet_nobias.fold_0.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a4c637bc21948972512ccccfa8a078cf3b260e1e3443d254ba46e39001ba37e
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbc8c4ebb6889615b12725544352062f86a1eaa365edf1c1ec01f9ec19f0b565
+size 26060800

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR703AYZ/preprocessing/bigWigs/ENCSR703AYZ.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/ENCSR283TME/fold_1/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	13.0
+counts_sum_max_thresh	16731.84
+trainings_pts_post_thresh	175559

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	12.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR703AYZ/logfile.modelling.fold_1.ENCSR703AYZ.epoch_loss.csv

@@ -0,0 +1,16 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.006342649459839,733.7158813476562,758.19384765625,0.8872881531715393,823.7530517578125,834.5769653320312
+1,0.8676131367683411,673.468505859375,684.0526123046875,0.7431106567382812,789.9581909179688,799.0235595703125
+2,0.7843101620674133,659.2339477539062,668.8028564453125,0.8686099648475647,772.9908447265625,783.5878295898438
+3,0.7279632687568665,647.5464477539062,656.4271240234375,0.8358080983161926,766.7789916992188,776.9756469726562
+4,0.6897042989730835,640.362548828125,648.7762451171875,0.6488600969314575,769.7171630859375,777.633544921875
+5,0.6539513468742371,633.6682739257812,641.6463623046875,0.6227163076400757,769.2222900390625,776.8199462890625
+6,0.6390420794487,629.68212890625,637.4786987304688,0.6088941693305969,762.7314453125,770.1600341796875
+7,0.6163201928138733,625.236572265625,632.7560424804688,0.6148807406425476,771.7636108398438,779.2648315429688
+8,0.6076939702033997,620.65185546875,628.0656127929688,0.6142629981040955,760.9061279296875,768.399658203125
+9,0.590556263923645,618.56640625,625.771240234375,0.6117562651634216,756.8997802734375,764.3631591796875
+10,0.580071210861206,615.50244140625,622.5781860351562,0.603762149810791,785.3663330078125,792.7327270507812
+11,0.5707601308822632,611.7620239257812,618.7250366210938,0.58961021900177,766.6492309570312,773.84228515625
+12,0.5604268908500671,610.0900268554688,616.9273681640625,0.6501576900482178,776.3265991210938,784.2589111328125
+13,0.549530565738678,607.0639038085938,613.7681274414062,0.5785014629364014,768.4296264648438,775.4871826171875
+14,0.5449245572090149,605.8052368164062,612.452392578125,0.6155508160591125,774.2535400390625,781.7637939453125

fold_1/model.bias_scaled.fold_1.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd5d5c5d14e8e89274a19566536958ec1f8a3c255a3f1180e38e1f1b9768ef72
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4151dcf2761ba3d7b08feb8858d97f993b3316cf6e9c504e3fe33ed2e1038fcb
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8f897c9391c77f3c8bbc61dafcfa92c21994eb698a613bc302db152c5084e05
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:169e859dbc8bbff2a75d7216565c4c76dbc210463751bbf088ecae09599165d9
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06c42ce06199cfc844cfb1621ce592a489e8c3ffa1f154fe36b44b86af87a4e5
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce5ca9802d5e6d74393c0cc4b847851252de07a9065794f2865e8b73231baa18
+size 26060800

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR703AYZ/preprocessing/bigWigs/ENCSR703AYZ.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/ENCSR283TME/fold_2/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	13.0
+counts_sum_max_thresh	17145.87
+trainings_pts_post_thresh	181831

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	12.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR703AYZ/logfile.modelling.fold_2.ENCSR703AYZ.epoch_loss.csv

@@ -0,0 +1,11 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.3215160369873047,750.5100708007812,778.8341064453125,0.9200870394706726,722.3521118164062,733.5771484375
+1,0.8874126672744751,687.4766845703125,698.3028564453125,0.7808356285095215,696.6070556640625,706.1329956054688
+2,0.784990668296814,667.87939453125,677.4554443359375,0.7271524667739868,697.7908935546875,706.6622314453125
+3,0.7288228869438171,656.89208984375,665.7831420898438,0.7941270470619202,679.7894287109375,689.4780883789062
+4,0.6871240139007568,649.1480102539062,657.5302124023438,0.6786439418792725,674.0465087890625,682.3260498046875
+5,0.6619958281517029,643.3701171875,651.4459838867188,0.784225344657898,676.4708862304688,686.0384521484375
+6,0.6377147436141968,639.193603515625,646.9736938476562,0.6738148331642151,683.4007568359375,691.621826171875
+7,0.6186647415161133,633.6163330078125,641.1636352539062,0.7591723203659058,682.0910034179688,691.35302734375
+8,0.5995228290557861,630.2840576171875,637.5984497070312,0.7135933637619019,681.6788330078125,690.384521484375
+9,0.5899399518966675,627.0406494140625,634.23876953125,0.6749047040939331,680.4163818359375,688.64990234375

fold_2/model.bias_scaled.fold_2.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:237c28647b942fe0832fdfada8e2b8cff925e9afee2166fb88eeb977b710a645
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6456e2118e688c14a7a48129b320f23599dd4e4eb1e52df9a5070ead28ba5eae
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5cd3d04062ef951320062fe6bd301e08c95a54a324121d1025a590e98049f7c
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3641081a25f21618715dcdd6a6510297c30e24460ea322f1884abe5676892364
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR703AYZ.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:baf3400ee7d05d34cf1404d83abb37c92fbb2a63621918482b18c4ca63e23712
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR703AYZ.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d5701057a4e54354c498eabeec26365a445755fd1352787699d82c162d3b5a6
+size 26060800

fold_3/logs.models.fold_3.ENCSR703AYZ/logfile.modelling.fold_3.ENCSR703AYZ.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR703AYZ/preprocessing/bigWigs/ENCSR703AYZ.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/ENCSR283TME/fold_3/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR703AYZ/logfile.modelling.fold_3.ENCSR703AYZ.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR703AYZ/logfile.modelling.fold_3.ENCSR703AYZ.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	12.0
+counts_sum_max_thresh	17213.08
+trainings_pts_post_thresh	173095

fold_3/logs.models.fold_3.ENCSR703AYZ/logfile.modelling.fold_3.ENCSR703AYZ.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR703AYZ/logfile.modelling.fold_3.ENCSR703AYZ.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	12.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR703AYZ/logfile.modelling.fold_3.ENCSR703AYZ.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR703AYZ/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR703AYZ/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py