Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - ureter
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in ureter (ENCSR398JUP)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR398JUP](https://www.encodeproject.org/experiments/ENCSR398JUP/)
+- Model annotation: [ENCSR078RRP](https://www.encodeproject.org/annotations/ENCSR078RRP/)
+- Biosample: ureter (Full name: Homo sapiens ureter tissue female adult (47 years))
+- Cell slim(s): None
+- Organ slim(s): kidney,ureter
+- Developmental slim(s): mesoderm
+- System slim(s): excretory-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.args.json

@@ -0,0 +1,42 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR398JUP/preprocessing/bigWigs/ENCSR398JUP.bigWig",
+    "output_dir": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/ziwei75/anaconda3/envs/chrombpnet/lib/python3.8/site-packages/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR398JUP/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/models/chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	8300.0
+trainings_pts_post_thresh	172659

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	21.4
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR398JUP/fold0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR398JUP/logfile.modelling.fold_0.ENCSR398JUP.epoch_loss.csv

@@ -0,0 +1,16 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.270136833190918,729.0353393554688,777.61669921875,0.34775885939598083,685.9082641601562,693.3505249023438
+1,0.3959747850894928,699.333251953125,707.8074340820312,0.39723026752471924,681.5462036132812,690.0473022460938
+2,0.35579779744148254,692.23046875,699.8438720703125,0.2830733358860016,672.081787109375,678.1397705078125
+3,0.330522745847702,687.5692749023438,694.641845703125,0.28537896275520325,674.0271606445312,680.1347045898438
+4,0.30846652388572693,684.5039672851562,691.1065063476562,0.26029717922210693,671.4427490234375,677.0133056640625
+5,0.2935991585254669,681.2141723632812,687.4976196289062,0.2604922652244568,666.3526611328125,671.9271240234375
+6,0.2852952480316162,678.90771484375,685.0130004882812,0.2711145877838135,665.263916015625,671.0660400390625
+7,0.2721380889415741,677.1530151367188,682.9769897460938,0.2834313213825226,669.9794921875,676.0453491210938
+8,0.2711276710033417,674.7201538085938,680.5216064453125,0.2607209384441376,666.2208251953125,671.8001098632812
+9,0.2595021724700928,672.8922729492188,678.4457397460938,0.24041514098644257,663.8956298828125,669.040283203125
+10,0.2515006363391876,671.9090576171875,677.2919311523438,0.24255456030368805,664.1221313476562,669.312744140625
+11,0.2495410293340683,669.61279296875,674.952880859375,0.2533259093761444,665.923095703125,671.3439331054688
+12,0.2404443770647049,669.1378784179688,674.28271484375,0.2333649843931198,665.3543090820312,670.3486938476562
+13,0.23877988755702972,667.38037109375,672.4904174804688,0.24638620018959045,670.58984375,675.86279296875
+14,0.22964882850646973,666.3124389648438,671.2267456054688,0.26498857140541077,671.663330078125,677.3339233398438

fold_0/model.bias_scaled.fold_0.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f87e7bd434d880b5bf75a1136d1ea0be0e6616ea88946e3f6bcc3c4dfe512ff1
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a73d2c429820008dcf6f18fcbd7cd3c6ce0419af5c6296ab3518d00db4e8dd64
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4def7f33d4620855bd32d4b4f1fbe2946637d89430f5edf9d44bd179b4715301
+size 77538952

fold_0/model.chrombpnet.fold_0.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ec75c760ff49a04a156ed192d1eadc7bca97078e0213fd9761cca9e296d12f7
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfdf92ebfa1f070980238e49e25f62235037af8cd5df9c777509d4169c1dd48d
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1b4083beac4e9eb66c06c278dc69b4bebafc07756db007c3223136d8380f36d
+size 26060800

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR398JUP/preprocessing/bigWigs/ENCSR398JUP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR398JUP/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	8397.76
+trainings_pts_post_thresh	174367

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	21.4
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR398JUP/logfile.modelling.fold_1.ENCSR398JUP.epoch_loss.csv

@@ -0,0 +1,14 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.417996883392334,706.693115234375,737.037841796875,0.4395836889743805,811.732666015625,821.13916015625
+1,0.3534804582595825,679.9891967773438,687.553955078125,0.31035947799682617,790.0065307617188,796.6482543945312
+2,0.32262834906578064,673.6705932617188,680.5743408203125,0.4627419710159302,791.4224243164062,801.3252563476562
+3,0.299799382686615,669.1775512695312,675.592529296875,0.3137473165988922,788.3442993164062,795.058349609375
+4,0.28259676694869995,666.5296630859375,672.57861328125,0.28528711199760437,794.6881713867188,800.79296875
+5,0.2710038721561432,664.108642578125,669.91015625,0.32998526096343994,786.3973999023438,793.458984375
+6,0.2629711627960205,661.2449340820312,666.8720092773438,0.2736140191555023,787.1406860351562,792.9961547851562
+7,0.2538187503814697,659.598876953125,665.0314331054688,0.2543548345565796,783.3096923828125,788.7526245117188
+8,0.2464998960494995,658.122314453125,663.3978271484375,0.2758314907550812,786.6483764648438,792.5516357421875
+9,0.24107618629932404,656.1837768554688,661.3427124023438,0.2690506875514984,783.68115234375,789.4387817382812
+10,0.23911215364933014,655.0431518554688,660.1600952148438,0.28271931409835815,789.5341186523438,795.5846557617188
+11,0.23142847418785095,653.0767822265625,658.0291137695312,0.252614289522171,797.3333129882812,802.7391967773438
+12,0.22813889384269714,652.1227416992188,657.0045776367188,0.25479939579963684,792.9594116210938,798.412109375

fold_1/model.bias_scaled.fold_1.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:113e087df3c9ca8cb811f6194e1ebf5f49b699d219d149912988506fabfbafc6
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d0a27ed976398d4ee432c30b0c74f245ee1bf93ba8fadbaf913ef2e8474a060
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b14f25b523553e6dc5b7a2a2679f1be706cea04874803438bcb04ac60c6f091f
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ef68d378b5ad6470bc6163f1660e1c1fefe63c951c1fb696733f5a3b69a6941
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76c33d89bcc39d89b9abbbf98a0adf4d795b4d02e5d501e78e30b51f0ae3cb8a
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d2b66e0a94e481be1d7951b14e286e1b751c5857bed3200de7c509bd59b5846
+size 26060800

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR398JUP/preprocessing/bigWigs/ENCSR398JUP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR398JUP/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	8415.85
+trainings_pts_post_thresh	181111

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	21.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR398JUP/logfile.modelling.fold_2.ENCSR398JUP.epoch_loss.csv

@@ -0,0 +1,11 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.2550675868988037,717.3446655273438,744.0780639648438,0.3554855287075043,724.8363037109375,732.4078369140625
+1,0.35337191820144653,689.1173095703125,696.6448974609375,0.31485092639923096,712.843994140625,719.5502319335938
+2,0.31334805488586426,682.93505859375,689.6091918945312,0.33162033557891846,707.4268188476562,714.490234375
+3,0.2962327003479004,677.8681640625,684.177978515625,0.4955995976924896,709.009521484375,719.5656127929688
+4,0.279708594083786,674.9435424804688,680.90185546875,0.2691021263599396,704.7195434570312,710.451416015625
+5,0.2709655165672302,672.4837646484375,678.2545776367188,0.28000742197036743,706.3897094726562,712.3538818359375
+6,0.25781744718551636,669.9029541015625,675.3947143554688,0.301303505897522,710.14892578125,716.566650390625
+7,0.253516286611557,668.8029174804688,674.2022094726562,0.2666979432106018,706.0806884765625,711.76123046875
+8,0.24501381814479828,666.7236328125,671.9425048828125,0.26215890049934387,705.2216796875,710.8054809570312
+9,0.2404337376356125,664.7976684570312,669.9189453125,0.26186466217041016,707.5818481445312,713.1597900390625

fold_2/model.bias_scaled.fold_2.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1405182441a5ddb10b1f87be8cde1a0ef0d5d77506ca6dc7be93897419de1c0
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3d675649aeea06e082504f8c27c5c1f0c27cde05e276de0895d0cfdccf8277e
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f186c381faecb1355911396cccc4ff65a236baceb0a721936e1afef9d58f871
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d82fbeb0ef21c69dd4116893f11641771aa723d1038583ace34b9bc5580296b
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR398JUP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1aace441c5ef9325bcb4703b568d47c92a9e99ae5d176af2e1b662966c88340
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR398JUP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71d850de19a81a750a36f41fd84ef0e764876582b3c1532b295e2d6e92ebc195
+size 26060800

fold_3/logs.models.fold_3.ENCSR398JUP/logfile.modelling.fold_3.ENCSR398JUP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR398JUP/preprocessing/bigWigs/ENCSR398JUP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR398JUP/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR398JUP/logfile.modelling.fold_3.ENCSR398JUP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR398JUP/logfile.modelling.fold_3.ENCSR398JUP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	8428.12
+trainings_pts_post_thresh	173018

fold_3/logs.models.fold_3.ENCSR398JUP/logfile.modelling.fold_3.ENCSR398JUP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR398JUP/logfile.modelling.fold_3.ENCSR398JUP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	21.3
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR398JUP/logfile.modelling.fold_3.ENCSR398JUP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR398JUP/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR398JUP/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py