Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - SK-N-DZ
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in SK-N-DZ (ENCSR946MVP)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR946MVP](https://www.encodeproject.org/experiments/ENCSR946MVP/)
+- Model annotation: [ENCSR822RPX](https://www.encodeproject.org/annotations/ENCSR822RPX/)
+- Biosample: SK-N-DZ (Full name: Homo sapiens SK-N-DZ)
+- Cell slim(s): neural-cell,neuroblastoma-cell,cancer-cell,stem-cell
+- Organ slim(s): brain
+- Developmental slim(s): ectoderm
+- System slim(s): central-nervous-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR946MVP/preprocessing/bigWigs/ENCSR946MVP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_0/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	76.0
+counts_sum_max_thresh	130743.87
+trainings_pts_post_thresh	174246

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	62.2
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR946MVP/logfile.modelling.fold_0.ENCSR946MVP.epoch_loss.csv

@@ -0,0 +1,23 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,3.3408985137939453,1569.5146484375,1777.31787109375,0.7607563138008118,1523.5791015625,1570.898681640625
+1,0.817518413066864,1457.8642578125,1508.7152099609375,0.6993418335914612,1481.41943359375,1524.919189453125
+2,0.7375103831291199,1424.630859375,1470.5040283203125,0.6234272122383118,1465.500732421875,1504.278076171875
+3,0.6971818208694458,1398.2735595703125,1441.637451171875,0.586941659450531,1435.0777587890625,1471.5853271484375
+4,0.6710680723190308,1383.2015380859375,1424.9422607421875,0.6695260405540466,1462.4451904296875,1504.089599609375
+5,0.6545649766921997,1371.6959228515625,1412.4097900390625,0.5865715146064758,1436.6544189453125,1473.1392822265625
+6,0.6223482489585876,1354.7679443359375,1393.4776611328125,0.5573334097862244,1432.057861328125,1466.72509765625
+7,0.600297749042511,1346.12841796875,1383.4683837890625,0.5476023554801941,1423.6650390625,1457.7255859375
+8,0.5839160084724426,1337.8392333984375,1374.1571044921875,0.5335003733634949,1422.065673828125,1455.2489013671875
+9,0.5754469037055969,1335.318603515625,1371.1109619140625,0.5360093116760254,1428.0177001953125,1461.357421875
+10,0.5617044568061829,1325.8817138671875,1360.8187255859375,0.5290147066116333,1419.35693359375,1452.260986328125
+11,0.5504777431488037,1322.1944580078125,1356.4354248046875,0.5562748312950134,1410.5281982421875,1445.127197265625
+12,0.5362228751182556,1317.9296875,1351.280029296875,0.5557938814163208,1406.1627197265625,1440.732421875
+13,0.5153308510780334,1311.536376953125,1343.59228515625,0.6103193759918213,1417.6871337890625,1455.6480712890625
+14,0.5097391605377197,1309.56201171875,1341.2686767578125,0.5304195880889893,1422.282470703125,1455.2747802734375
+15,0.4910423159599304,1302.97607421875,1333.52001953125,0.6298614740371704,1430.53515625,1469.7132568359375
+16,0.48071396350860596,1300.89453125,1330.7957763671875,0.5169614553451538,1403.47705078125,1435.6317138671875
+17,0.4658091068267822,1294.78369140625,1323.7562255859375,0.5229767560958862,1417.7498779296875,1450.278076171875
+18,0.46031418442726135,1296.05517578125,1324.6873779296875,0.5943333506584167,1404.44091796875,1441.4088134765625
+19,0.4426235258579254,1287.035400390625,1314.5657958984375,0.5321539044380188,1417.418212890625,1450.517578125
+20,0.43555396795272827,1284.5155029296875,1311.60888671875,0.5651055574417114,1405.9163818359375,1441.0657958984375
+21,0.422941118478775,1281.12109375,1307.4296875,0.557479977607727,1411.754150390625,1446.42919921875

fold_0/model.bias_scaled.fold_0.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd7e8261627158af0cd9c7ae671bf26a2955ead5621b502d749cda458d801be6
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47f82897ee19fc08ee8553b18e25026dd4e4713a803864f4b1af52429467691d
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d277acc4d711b8ce33221d161a1d0088106663e808fa0f6749a4801a574e80dc
+size 77538840

fold_0/model.chrombpnet.fold_0.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb02dc83404ba9f4d57acb984a795c52ec3104219eb9b91e9726a215051359a6
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:965e504d7835c8475da763d4c59814e3969198e4525655ba74de9d75172f8685
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b87161990d3309836c24c9886ecc1a9b9f1ed23429e238dbe315d54689dbe174
+size 26060800

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR946MVP/preprocessing/bigWigs/ENCSR946MVP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_1/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	81.0
+counts_sum_max_thresh	36074.39
+trainings_pts_post_thresh	175309

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	61.6
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR946MVP/logfile.modelling.fold_1.ENCSR946MVP.epoch_loss.csv

@@ -0,0 +1,21 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.3684048652648926,1490.039306640625,1635.931884765625,0.7094875574111938,1701.8970947265625,1745.6015625
+1,0.6943105459213257,1375.7669677734375,1418.539794921875,0.8139530420303345,1629.2471923828125,1679.386474609375
+2,0.641025960445404,1340.318115234375,1379.8070068359375,0.5814587473869324,1593.913818359375,1629.732177734375
+3,0.6039670705795288,1317.8988037109375,1355.1038818359375,0.5769849419593811,1594.0040283203125,1629.5458984375
+4,0.5682508945465088,1299.4324951171875,1334.438232421875,0.5232350826263428,1592.658447265625,1624.8900146484375
+5,0.5487430095672607,1288.740234375,1322.543212890625,0.5314211249351501,1589.271728515625,1622.0059814453125
+6,0.5242558121681213,1280.162841796875,1312.4564208984375,0.5569776296615601,1587.9989013671875,1622.3084716796875
+7,0.5041025280952454,1272.3577880859375,1303.41015625,0.5234900712966919,1579.322509765625,1611.569091796875
+8,0.4834202527999878,1262.3170166015625,1292.0933837890625,0.506224513053894,1591.6429443359375,1622.8272705078125
+9,0.46609368920326233,1257.8046875,1286.51708984375,0.5357299447059631,1575.9801025390625,1608.98046875
+10,0.4468473792076111,1252.2025146484375,1279.7281494140625,0.6076387763023376,1618.3629150390625,1655.7938232421875
+11,0.4332374334335327,1247.6934814453125,1274.3826904296875,0.5840591192245483,1592.3050537109375,1628.2821044921875
+12,0.41503313183784485,1243.22607421875,1268.7900390625,0.5765385627746582,1587.8804931640625,1623.3951416015625
+13,0.4009878635406494,1239.258544921875,1263.9598388671875,0.5711628198623657,1584.5693359375,1619.753173828125
+14,0.3853195607662201,1234.741455078125,1258.47607421875,0.5138239860534668,1568.4058837890625,1600.0574951171875
+15,0.3711819350719452,1231.5904541015625,1254.453857421875,0.55119389295578,1606.011474609375,1639.96337890625
+16,0.3613501489162445,1228.643798828125,1250.902587890625,0.5278586149215698,1583.441162109375,1615.957763671875
+17,0.3515424132347107,1227.999267578125,1249.6553955078125,0.5198220610618591,1615.6337890625,1647.654296875
+18,0.3389403820037842,1223.6053466796875,1244.4874267578125,0.5943402647972107,1604.2091064453125,1640.8206787109375
+19,0.3315652906894684,1221.289306640625,1241.71533203125,0.6190791726112366,1636.8507080078125,1674.9849853515625

fold_1/model.bias_scaled.fold_1.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06571508c0aebcfa19e02eef29d1c224ffeb71efa5d2bdec0c1c38546fb8c3b3
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff19b36071a9f36ee36cc538144684b0ede837e061f5d9c035effd0a5e93695c
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16034f3771b23231fb5f17401941d86a15ac15d8158de77a231edea32ec23f31
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3ba1720c3bce5cdb2e2f4f2a64e2620aae1860d675d053410f819983111eae2
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0ef83dda35d484d208571b40e9175ec39dde52d72671190bc0e9b28479556e7
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35db77c47d941720f92ce7df9f8ffbfe63b6a7bbef392d2815c6177bd33273e4
+size 26060800

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR946MVP/preprocessing/bigWigs/ENCSR946MVP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_2/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	84.0
+counts_sum_max_thresh	118851.09
+trainings_pts_post_thresh	179799

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	61.7
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR946MVP/logfile.modelling.fold_2.ENCSR946MVP.epoch_loss.csv

@@ -0,0 +1,15 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.1548190116882324,1563.392333984375,1696.3427734375,0.7163865566253662,1509.2984619140625,1553.5003662109375
+1,0.7454506754875183,1440.9544677734375,1486.947265625,0.6480811238288879,1458.094482421875,1498.082763671875
+2,0.6904349327087402,1403.986572265625,1446.5869140625,0.6113530397415161,1444.2376708984375,1481.9578857421875
+3,0.6631267070770264,1379.0728759765625,1419.9866943359375,0.582589864730835,1420.88916015625,1456.834228515625
+4,0.6360616683959961,1362.1868896484375,1401.4339599609375,0.5865843892097473,1431.7354736328125,1467.92822265625
+5,0.619316041469574,1353.734375,1391.9453125,0.5594961643218994,1421.7593994140625,1456.280029296875
+6,0.5971941947937012,1343.603515625,1380.450439453125,0.5832591652870178,1401.140625,1437.1285400390625
+7,0.5787097215652466,1333.5576171875,1369.265625,0.5406956672668457,1399.4354248046875,1432.796142578125
+8,0.565818727016449,1326.3594970703125,1361.269775390625,0.5329533815383911,1387.907470703125,1420.7916259765625
+9,0.5529313683509827,1319.619140625,1353.7333984375,0.5572875142097473,1419.972900390625,1454.3583984375
+10,0.5405921339988708,1317.59716796875,1350.9532470703125,0.5339509844779968,1401.4144287109375,1434.3609619140625
+11,0.5257632732391357,1313.2142333984375,1345.6551513671875,0.5193160772323608,1410.4613037109375,1442.503173828125
+12,0.5112573504447937,1309.2041015625,1340.7493896484375,0.5160146951675415,1429.40966796875,1461.24755859375
+13,0.5039511919021606,1304.263427734375,1335.3590087890625,0.5250393152236938,1416.2845458984375,1448.67822265625

fold_2/model.bias_scaled.fold_2.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f4b6031d43e4e06516f5800188686e210385d5a9dfdc8b83520e0d98341497b
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7023e3799f84811fb486278c09d7ca427d64677824de6904365f5260569803b3
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed9a10f834e45de8d90123f6a597f9b0eb3c912a9728498925419b697dc48a24
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06a1dbe1caac163f5dd78d980faad9e8ed2d795c3b8381047df36e55064f6e53
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR946MVP.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:399b44905756ceabc98e830eafabe300f42ac8a329689c86330ed853c5fab8a7
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR946MVP.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69e9905ab91063861b94cfbe84fc4f051c96697c54f27f30afc1aa94927d1b44
+size 26060800

fold_3/logs.models.fold_3.ENCSR946MVP/logfile.modelling.fold_3.ENCSR946MVP.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR946MVP/preprocessing/bigWigs/ENCSR946MVP.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/reference/HEPG2_DNASE_PE/fold_3/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR946MVP/logfile.modelling.fold_3.ENCSR946MVP.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR946MVP/logfile.modelling.fold_3.ENCSR946MVP.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	77.0
+counts_sum_max_thresh	132416.93
+trainings_pts_post_thresh	173450

fold_3/logs.models.fold_3.ENCSR946MVP/logfile.modelling.fold_3.ENCSR946MVP.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR946MVP/logfile.modelling.fold_3.ENCSR946MVP.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	61.8
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR946MVP/logfile.modelling.fold_3.ENCSR946MVP.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR946MVP/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR946MVP/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py