Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - endothelial
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in brain microvascular endothelial cell (ENCSR000ENE)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR000ENE](https://www.encodeproject.org/experiments/ENCSR000ENE/)
+- Model annotation: [ENCSR323JJT](https://www.encodeproject.org/annotations/ENCSR323JJT/)
+- Biosample: brain microvascular endothelial cell (Full name: Homo sapiens brain microvascular endothelial cell)
+- Cell slim(s): epithelial-cell,neural-cell,endothelial-cell
+- Organ slim(s): epithelium,brain,vasculature,blood-vessel
+- Developmental slim(s): mesoderm,ectoderm
+- System slim(s): central-nervous-system,circulatory-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "pipeline",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR000ENE//preprocessing/bigWigs/ENCSR000ENE.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/chrombpnet_model/auxiliary/ENCSR000ENE_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/chrombpnet_model/auxiliary/ENCSR000ENE_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR000ENE",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/bias_model/models/ENCSR000ENE_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/chrombpnet_model/models/ENCSR000ENE_chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/chrombpnet_model/logs/ENCSR000ENE_chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/failed_models_retrained/chrombpnet_model_feb22_fold_0/chrombpnet_model/models/ENCSR000ENE_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3307.96
+trainings_pts_post_thresh	169251

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/failed_models_retrained/chrombpnet_model_feb22_fold_0/chrombpnet_model/models/ENCSR000ENE_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.2
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENE//chrombpnet_model_feb22_fold_0/chrombpnet_model/models/ENCSR000ENE_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/failed_models_retrained/chrombpnet_model_feb22_fold_0/chrombpnet_model/models/ENCSR000ENE_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000ENE/logfile.modelling.fold_0.ENCSR000ENE.epoch_loss.csv

@@ -0,0 +1,18 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.9634382724761963,377.73748779296875,397.7646789550781,1.0293521881103516,385.5916748046875,396.0910339355469
+1,0.9573468565940857,355.93017578125,365.695556640625,0.9524052739143372,376.9235534667969,386.6379699707031
+2,0.8791595697402954,350.7334899902344,359.70068359375,0.9935187101364136,378.2821960449219,388.4160461425781
+3,0.8236331343650818,346.8332214355469,355.2343444824219,0.7963070273399353,371.296630859375,379.41900634765625
+4,0.7789008021354675,343.8221130371094,351.7666931152344,0.7410964965820312,367.70269775390625,375.26190185546875
+5,0.7531371116638184,342.4731750488281,350.15557861328125,0.7754468321800232,376.4928283691406,384.4020690917969
+6,0.7246262431144714,340.3036193847656,347.69451904296875,0.9618564248085022,366.8589782714844,376.6697082519531
+7,0.6907596588134766,338.6612243652344,345.70623779296875,0.7200509905815125,368.45355224609375,375.79803466796875
+8,0.672538161277771,337.91680908203125,344.77703857421875,0.7421784996986389,370.3897399902344,377.96002197265625
+9,0.649025022983551,336.1482849121094,342.7685241699219,0.7076439261436462,367.19427490234375,374.4123229980469
+10,0.630963921546936,335.5350036621094,341.97088623046875,0.714817464351654,369.24444580078125,376.53546142578125
+11,0.6167646050453186,334.36260986328125,340.6532897949219,0.6734544634819031,366.47320556640625,373.34222412109375
+12,0.6043353080749512,333.6611022949219,339.8254699707031,0.7203248143196106,367.19439697265625,374.54193115234375
+13,0.5880498886108398,333.0064697265625,339.0044250488281,0.6749315857887268,369.14056396484375,376.0249328613281
+14,0.5701214075088501,331.98297119140625,337.7980651855469,0.6750552654266357,369.53485107421875,376.4205017089844
+15,0.5577042102813721,330.668212890625,336.3565673828125,0.670957624912262,368.8642883300781,375.7078857421875
+16,0.5420794486999512,330.5259704589844,336.0548400878906,0.6870818734169006,367.4170227050781,374.42523193359375

fold_0/model.bias_scaled.fold_0.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12766724b6ca86143125ebffd04b5d1700ff32bb277794efb8f842417de389b3
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fcb142c5a5c73ee5f985483296d7de998aae90f1c08fac1326455bba3e66a0c
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f32771c4d065d549f30496d060401c92dc535d94722ef028d8fec60a18aa3ea4
+size 77538904

fold_0/model.chrombpnet.fold_0.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e02349ba8d1e2d8a0d20e45761502ba9f042f34af54d1102c3b57b0a5cdc752
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1213e58247b41dc256f5f7721371248a2bc2f61717d063124db6d3be59df6fb
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b908fa779d064824892adb87d14994a37d659611a9dcbf650c2ce7a30c7f4442
+size 26060800

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/preprocessing/bigWigs/ENCSR000ENE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/failed_models_retrained/chrombpnet_model_feb22_fold_0/bias_model/models/ENCSR000ENE_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3290.67
+trainings_pts_post_thresh	174037

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000ENE/logfile.modelling.fold_1.ENCSR000ENE.epoch_loss.csv

@@ -0,0 +1,14 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.038708448410034,375.0960693359375,396.09490966796875,0.9744908809661865,400.5649719238281,410.6021423339844
+1,0.9260057210922241,354.6159973144531,364.15447998046875,0.8501225113868713,391.496337890625,400.2525329589844
+2,0.8437067866325378,348.9721374511719,357.662841796875,0.8907222151756287,389.9610900878906,399.1355285644531
+3,0.7992215752601624,345.1899108886719,353.4217834472656,0.8650943636894226,388.578857421875,397.4893798828125
+4,0.7531725168228149,343.2485656738281,351.0065612792969,0.7216870784759521,384.1022033691406,391.5354919433594
+5,0.721278190612793,341.3906555175781,348.8194274902344,0.8492127656936646,388.3586120605469,397.1056213378906
+6,0.6922621726989746,339.34930419921875,346.4794006347656,0.7176091074943542,385.44525146484375,392.8366394042969
+7,0.6730366349220276,338.34161376953125,345.27374267578125,0.7038235068321228,382.5793151855469,389.8283996582031
+8,0.6537469625473022,336.6867980957031,343.42034912109375,0.6990910172462463,383.94647216796875,391.147216796875
+9,0.6320470571517944,335.7017517089844,342.2120056152344,0.7917746901512146,385.61859130859375,393.7739562988281
+10,0.6124797463417053,334.90447998046875,341.2132263183594,0.7016525864601135,383.5893859863281,390.8166198730469
+11,0.5997549295425415,334.00482177734375,340.1816711425781,0.776370108127594,386.4659423828125,394.46258544921875
+12,0.5797068476676941,333.1341552734375,339.104736328125,0.6754305362701416,385.54766845703125,392.5044860839844

fold_1/model.bias_scaled.fold_1.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc4141643884473b2259170db491c9c91c984d339b0344b736651363052cccb0
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da860393904d58a078c32e477068d3f5642b0eedfb0c2046a2fa9adb227c5676
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6d34c0894d6455fd0b567e094560ca5da8c53b8964bd0555e401473f8ab3c7f
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:519b92665c52ff82eee28d4d52547608b1991b08ce44e9cd95054d55c566fbf6
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7b4e3299a32484da6d695b36d7b738b1904b53e06e70c2021da85c7e6389a81
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:564bb3381b0ae7556f9cd6b6ee4da588f428d475993ca48f49f2e44ba0c7f924
+size 26060800

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/preprocessing/bigWigs/ENCSR000ENE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/failed_models_retrained/chrombpnet_model_feb22_fold_0/bias_model/models/ENCSR000ENE_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3328.23
+trainings_pts_post_thresh	176679

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000ENE/logfile.modelling.fold_2.ENCSR000ENE.epoch_loss.csv

@@ -0,0 +1,19 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.4371490478515625,384.8477478027344,409.9506530761719,1.0961838960647583,381.1049499511719,392.3955078125
+1,1.0061450004577637,361.5724792480469,371.9356384277344,1.4472131729125977,375.780029296875,390.6862487792969
+2,0.8968560695648193,355.1752624511719,364.412841796875,0.8448379635810852,366.8259582519531,375.52764892578125
+3,0.842910885810852,351.5809631347656,360.2619323730469,0.8583584427833557,369.255126953125,378.096435546875
+4,0.7956525683403015,348.2622985839844,356.4571228027344,1.0439527034759521,366.5329895019531,377.2856140136719
+5,0.759153425693512,346.119873046875,353.9394836425781,0.787948727607727,363.9100341796875,372.0258483886719
+6,0.7269721031188965,344.1510314941406,351.6393127441406,0.7308609485626221,362.328125,369.8561706542969
+7,0.6999543309211731,342.52447509765625,349.7337951660156,0.7027519345283508,367.4123840332031,374.6509704589844
+8,0.6806836724281311,341.14678955078125,348.1579284667969,0.7374311089515686,362.099609375,369.6953430175781
+9,0.6598536372184753,339.5876770019531,346.3836669921875,0.7129265069961548,362.18914794921875,369.53240966796875
+10,0.6419927477836609,338.6297302246094,345.2422790527344,0.7178053855895996,360.5595703125,367.952880859375
+11,0.6212935447692871,338.03070068359375,344.43023681640625,0.6917955875396729,363.1435852050781,370.2690734863281
+12,0.6079497337341309,337.1772155761719,343.4392395019531,0.710781455039978,360.1853942871094,367.50628662109375
+13,0.5983754396438599,336.4161376953125,342.5795593261719,0.7058741450309753,364.906982421875,372.17730712890625
+14,0.5813228487968445,335.7602844238281,341.74786376953125,0.7089843153953552,367.9651794433594,375.26788330078125
+15,0.5650802254676819,334.5577392578125,340.3778381347656,0.682396650314331,362.4888916015625,369.51763916015625
+16,0.5493679046630859,333.91632080078125,339.5751953125,0.6725011467933655,371.606689453125,378.5335693359375
+17,0.5455114245414734,333.5422058105469,339.1609191894531,0.7573755383491516,367.0349426269531,374.83599853515625

fold_2/model.bias_scaled.fold_2.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f7be00f775b807f6e4592608381b45849cd666a7faba1c676470e0410e3be52
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a460ed497a7b47f713760ffac5dc1b248881a42de595a39fbd89f30b53aeb56c
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b942e89562d40b769af80c8159d5fa984e4724d63927e3cc34fa6ed4c7ede4f
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4e7c89ab77534433ae9a5d3d952812f4d638451a996776ab4ff5b3f725b3147
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR000ENE.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eed0cee72a3ce0774afb48ee4372c8a27fb3a4171e4310f9b61a99d61e459cef
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR000ENE.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30d994b584478105d698626e88861fc3bbd6750b0ce150b6cc0b8dddc0f2f4df
+size 26060800

fold_3/logs.models.fold_3.ENCSR000ENE/logfile.modelling.fold_3.ENCSR000ENE.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/preprocessing/bigWigs/ENCSR000ENE.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENE/failed_models_retrained/chrombpnet_model_feb22_fold_0/bias_model/models/ENCSR000ENE_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR000ENE/logfile.modelling.fold_3.ENCSR000ENE.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR000ENE/logfile.modelling.fold_3.ENCSR000ENE.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3353.35
+trainings_pts_post_thresh	174968

fold_3/logs.models.fold_3.ENCSR000ENE/logfile.modelling.fold_3.ENCSR000ENE.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR000ENE/logfile.modelling.fold_3.ENCSR000ENE.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	10.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR000ENE/logfile.modelling.fold_3.ENCSR000ENE.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR000ENE/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENE/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py