Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - brain
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in brain pericyte (ENCSR000ENF)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR000ENF](https://www.encodeproject.org/experiments/ENCSR000ENF/)
+- Model annotation: [ENCSR630WHT](https://www.encodeproject.org/annotations/ENCSR630WHT/)
+- Biosample: brain pericyte (Full name: Homo sapiens brain pericyte)
+- Cell slim(s): neural-cell,connective-tissue-cell,hematopoietic-cell,pericyte
+- Organ slim(s): brain,connective-tissue
+- Developmental slim(s): ectoderm
+- System slim(s): central-nervous-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "pipeline",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR000ENF//preprocessing/bigWigs/ENCSR000ENF.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR000ENF",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR000ENF_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR000ENF_chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/chrombpnet_model/logs/ENCSR000ENF_chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3039.97
+trainings_pts_post_thresh	169207

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR000ENF_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	4.1
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_0/chrombpnet_model/models/ENCSR000ENF_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR000ENF/logfile.modelling.fold_0.ENCSR000ENF.epoch_loss.csv

@@ -0,0 +1,11 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.629178762435913,250.4822540283203,257.1619873046875,0.8461863398551941,261.1580810546875,264.6274108886719
+1,0.7611991763114929,242.47784423828125,245.5985107421875,0.6575604677200317,254.47784423828125,257.1738586425781
+2,0.6791106462478638,239.2318572998047,242.01596069335938,0.6155365109443665,251.6620635986328,254.18576049804688
+3,0.6315810084342957,237.5257568359375,240.1151885986328,0.5840906500816345,251.3649444580078,253.75967407226562
+4,0.6041699647903442,236.07386779785156,238.55076599121094,0.5627414584159851,249.65647888183594,251.9636688232422
+5,0.5858009457588196,234.9840850830078,237.3857879638672,0.55000239610672,250.21380615234375,252.4688262939453
+6,0.5664205551147461,233.88441467285156,236.20631408691406,0.5597932934761047,250.23451232910156,252.5298614501953
+7,0.550638735294342,233.08253479003906,235.34030151367188,0.5349343419075012,250.06442260742188,252.25775146484375
+8,0.5346351861953735,232.13027954101562,234.3227996826172,0.5437079668045044,250.39532470703125,252.62457275390625
+9,0.5236569046974182,231.54933166503906,233.69622802734375,0.5397088527679443,250.79942321777344,253.01229858398438

fold_0/model.bias_scaled.fold_0.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37d7cf6a2a1a88b0c29093ac0262a0ab6e3f708228218faa0963330f8bd72e99
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:743ef4dddc1e13112c13cbfbfa9ca5f33320c350eb0070fa7d29f7944a969745
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5cc31353a6dd643696664ff09e0b84dd17cb841d0c8cb3103ab5cb48efbd369
+size 77538904

fold_0/model.chrombpnet.fold_0.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fee98b908dfe5cc62ca4e630ee67b8aa1f162f006cdaec5b0cbf24cb22d9421e
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7df9c32b0e22e4a8b87dd8f6132f67fcc3f47f458aabef3a1c42808dd6a5a9b2
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a56b294133c8037056e100e5dc0b1b48a88bac9f3309f75c65f07540024a0b4f
+size 26060800

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR000ENF//preprocessing/bigWigs/ENCSR000ENF.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_1/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_1/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_1/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR000ENF",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR000ENF_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR000ENF_chrombpnet",
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_1/chrombpnet_model/logs/ENCSR000ENF_chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3090.03
+trainings_pts_post_thresh	174228

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR000ENF_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	4.1
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_1/chrombpnet_model/models/ENCSR000ENF_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR000ENF/logfile.modelling.fold_1.ENCSR000ENF.epoch_loss.csv

@@ -0,0 +1,11 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.4874112606048584,246.261962890625,252.3603515625,0.839512050151825,281.59039306640625,285.0323791503906
+1,0.7267096638679504,238.3378448486328,241.31736755371094,0.8189385533332825,273.7025451660156,277.0600891113281
+2,0.6540318131446838,235.69781494140625,238.3795623779297,0.6436423659324646,273.8876647949219,276.52691650390625
+3,0.6149494051933289,233.7967987060547,236.3182830810547,0.6444804072380066,272.8840637207031,275.5263366699219
+4,0.5813332200050354,232.37100219726562,234.75460815429688,0.6265383362770081,271.0336608886719,273.6025390625
+5,0.560298502445221,230.83607482910156,233.1332550048828,0.5835089683532715,272.5917053222656,274.98382568359375
+6,0.5448749661445618,230.0653076171875,232.29928588867188,0.5529177784919739,272.20892333984375,274.4757080078125
+7,0.530339241027832,229.45623779296875,231.6304168701172,0.5787375569343567,273.669189453125,276.0419006347656
+8,0.5223525166511536,228.04080200195312,230.182373046875,0.560637891292572,273.4392395019531,275.7376708984375
+9,0.5039920210838318,227.62759399414062,229.6942596435547,0.5304937958717346,274.4872131347656,276.6622009277344

fold_1/model.bias_scaled.fold_1.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a79776b71d41c4d1b83cf15714f40322a735509f186ed887af32296ad1dc313
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a59dd55254a40b1e77ba892838b561ab8b03f60020ef2fd05cfdc4332d8868b
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db936b6b0fd2b0749ca633de88b166566a80362557ea160b5a54b76cf5462ce5
+size 77538904

fold_1/model.chrombpnet.fold_1.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c898fb5afdf25fa1c45f26f8b10bb533b9e6f789e5d46259f9481f85086a89b6
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97d9115d086124475888397488930005892c77435f2e0095a9d3b10a26cf06a1
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c983d2ac4c9c625696c1d0106e294a2c9e2a39d5b7feb26e816bf66f68f85f3a
+size 26060800

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR000ENF//preprocessing/bigWigs/ENCSR000ENF.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_2/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_2/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_2/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR000ENF",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR000ENF_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR000ENF_chrombpnet",
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_2/chrombpnet_model/logs/ENCSR000ENF_chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3064.32
+trainings_pts_post_thresh	177001

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR000ENF_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	4.1
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_2/chrombpnet_model/models/ENCSR000ENF_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR000ENF/logfile.modelling.fold_2.ENCSR000ENF.epoch_loss.csv

@@ -0,0 +1,13 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,5.835454940795898,251.99522399902344,275.9204406738281,0.9046844840049744,261.0357666015625,264.7450256347656
+1,0.8393650650978088,244.6654510498047,248.10655212402344,0.7370827794075012,258.1161804199219,261.13818359375
+2,0.7226832509040833,241.01611328125,243.979248046875,0.7408095598220825,255.114013671875,258.15142822265625
+3,0.673505425453186,239.39170837402344,242.15284729003906,0.6614046692848206,253.31546020507812,256.02728271484375
+4,0.6356587409973145,238.02334594726562,240.62936401367188,0.5759359002113342,251.69740295410156,254.05874633789062
+5,0.6021857857704163,236.577392578125,239.04681396484375,0.6037001013755798,253.15846252441406,255.63372802734375
+6,0.5826375484466553,235.63462829589844,238.02337646484375,0.5763080716133118,250.24549865722656,252.60838317871094
+7,0.5641195178031921,234.96743774414062,237.2799072265625,0.6442228555679321,251.83645629882812,254.47799682617188
+8,0.545741617679596,233.51828002929688,235.7558135986328,0.6148302555084229,251.04176330566406,253.56271362304688
+9,0.5287032723426819,233.19129943847656,235.35899353027344,0.6770491600036621,251.37728881835938,254.15330505371094
+10,0.5199392437934875,232.3419647216797,234.47418212890625,0.5116053223609924,251.88186645507812,253.97946166992188
+11,0.5078042149543762,231.50143432617188,233.58319091796875,0.4985535144805908,250.71559143066406,252.75967407226562

fold_2/model.bias_scaled.fold_2.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ce4ce3054698d887b8e61423f7fac679948c205258f31ca67b8c598078d6bd2
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:837979233910350f96950ae58d550bcbec0d911c26d6fda108cccfff2ec9c040
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90ef91715985dd7ca95860ed22210e6649c4dcc9e19bdac64c89681ab29a516c
+size 77538904

fold_2/model.chrombpnet.fold_2.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01e2087b5255e1c707a3684e6aae3381e59a674e6131b55e2fb0f486b98e84b0
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR000ENF.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4c1e00f91c4c6186b28c74fe715279b4ad2e0ab8f67251c4da8712f91ab32ad
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR000ENF.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5df88bf452e701976aac65864081518b803239b8466ed4ec8a8b7f5b183591b5
+size 26060800

fold_3/logs.models.fold_3.ENCSR000ENF/logfile.modelling.fold_3.ENCSR000ENF.args.json

@@ -0,0 +1,45 @@
+{
+    "cmd": "train",
+    "genome": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//hg38.genome.fa",
+    "chrom_sizes": "/scratch/groups/akundaje/anusri/chromatin_atlas/reference//chrom.sizes",
+    "input_bam_file": null,
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE//ENCSR000ENF//preprocessing/bigWigs/ENCSR000ENF.bigWig",
+    "output_dir": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_3/chrombpnet_model/",
+    "data_type": "DNASE",
+    "peaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_3/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.peaks.bed",
+    "nonpeaks": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_3/chrombpnet_model/auxiliary/ENCSR000ENF_filtered.nonpeaks.bed",
+    "chr_fold_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/anusri/simg/chrombpnet_latest/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": "ENCSR000ENF",
+    "html_prefix": "./",
+    "bias_model_path": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_0/bias_model/models/ENCSR000ENF_bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR000ENF_chrombpnet",
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_3/chrombpnet_model/logs/ENCSR000ENF_chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR000ENF/logfile.modelling.fold_3.ENCSR000ENF.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR000ENF/logfile.modelling.fold_3.ENCSR000ENF.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	3246.48
+trainings_pts_post_thresh	172736

fold_3/logs.models.fold_3.ENCSR000ENF/logfile.modelling.fold_3.ENCSR000ENF.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR000ENF_chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR000ENF/logfile.modelling.fold_3.ENCSR000ENF.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	4.1
+filters	512
+n_dil_layers	8
+bias_model_path	/scratch/groups/akundaje/anusri/chromatin_atlas/DNASE/ENCSR000ENF//chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR000ENF_bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/scratch/groups/akundaje/anusri/chromatin_atlas/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR000ENF/logfile.modelling.fold_3.ENCSR000ENF.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR000ENF/failed_models_retrained/chrombpnet_model_feb15_fold_3/chrombpnet_model/models/ENCSR000ENF_chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR000ENF/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py