Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+library_name: chrombpnet
+tags:
+  - encode
+  - chrombpnet
+  - chromatin-accessibility
+  - DNASE
+  - neuron
+  - hg38
+---
+# ENCODE ChromBPNet Atlas
+As part of the ENCODE 4 Project, we trained ChromBPNet models on 1,512 ENCODE DNAse-seq and ATAC-seq across 408 biosamples. Here, we provide all models for open-source use.
+
+For more information about the models, see:
+- Main ENCODE 4 Paper
+- [A unified lexicon of predictive DNA sequence motifs from ENCODE transcription factor binding and chromatin accessibility assays](https://doi.org/10.5281/zenodo.17123347) (Deshpande et al., Zenodo 2025)
+- [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (Pampari et al., bioRxiv 2024)
+
+## ChromBPNet model: DNASE in neuronal stem cell (ENCSR278FVO)
+- Model: ChromBPNet
+- Assay: DNASE-seq
+- Experiment: [ENCSR278FVO](https://www.encodeproject.org/experiments/ENCSR278FVO/)
+- Model annotation: [ENCSR127MYY](https://www.encodeproject.org/annotations/ENCSR127MYY/)
+- Biosample: neuronal stem cell (Full name: Homo sapiens neuronal stem cell originated from H1)
+- Cell slim(s): neural-cell,stem-cell
+- Organ slim(s): embryo
+- Developmental slim(s): ectoderm
+- System slim(s): central-nervous-system
+- Assembly: hg38
+
+## Directory structure
+- `fold_0`: Model of 5-fold cross-validation: Fold 0
+    - `model.chrombpnet.fold_0.encid.h5`: full chrombpnet model that combines both bias and corrected model in .h5 format
+    - `model.chrombpnet_nobias.fold_0.encid.h5`: bias-corrected accessibility model in .h5 format (Use for all biological discovery)
+    - `model.bias_scaled.fold_0.encid.h5`: bias model in .h5 format
+    - `model.chrombpnet.fold_0.encid.tar`: full chrombpnet model that combines both bias and  corrected model in SavedModel format. After being untarred, it results in a directory named "chrombpnet".
+    - `model.chrombpnet_nobias.fold_0.encid.tar`: bias-corrected accessibility model in SavedModel format (Use for all biological discovery). After being untarred, it results in a directory named "chrombpnet_wo_bias".
+    - `model.bias_scaled.fold_0.encid.tar`: bias model in SavedModel format. After being untarred, it results in a directory named "bias_model_scaled".
+    - `logs.models.fold_0.encid`: folder containing log files for training models
+- `fold_1`: Model of 5-fold coss-validation: Fold 1
+- `fold_2`: Model of 5-fold cross-validation: Fold 2
+- `fold_3`: Model of 5-fold cross-validation: Fold 3
+- `fold_4`: Model of 5-fold cross-validation: Fold 4
+
+# Instructions
+## 1. Pseudocode for loading models in .h5 format 
+
+(1) Use the code in python after appropriately defining `model_in_h5_format` and `inputs`. \
+(2) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the 
+number of tested sequences, 2114 is the input sequence length and 4 corresponds to [A,C,G,T].
+
+```python
+import tensorflow as tf
+from tensorflow.keras.utils import get_custom_objects
+from tensorflow.keras.models import load_model
+
+custom_objects={"tf": tf}
+get_custom_objects().update(custom_objects)
+
+model=load_model(model_in_h5_format,compile=False)
+outputs = model(inputs)
+```
+
+The list `outputs` consists of two elements. The first element has a shape of (N, 1000) and
+contains logit predictions for a 1000-base-pair output. The second element, with a shape of
+(N, 1), contains logcount predictions. To transform these predictions into per-base signals, 
+follow the provided pseudo code lines below.
+
+```python
+import numpy as np
+
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs[0]) * (np.exp(outputs[1])-1)
+```
+
+## 2. Pseudocode for loading models in .tar format
+
+(1) First untar the directory as follows `tar -xvf model.tar`. \
+(2) Use the code below in python after appropriately defining `model_dir_untared` and `inputs`. \
+(3) `inputs` is a one hot encoded sequence of shape (N,2114,4). Here N corresponds to the number
+of tested sequences, 2114 is the input sequence length and 4 corresponds to ACGT.
+
+Reference: https://www.tensorflow.org/api_docs/python/tf/saved_model/load
+
+```python
+import tensorflow as tf
+
+model = tf.saved_model.load('model_dir_untared')
+outputs = model.signatures['serving_default'](**{'sequence':inputs.astype('float32')})
+```
+
+The variable `outputs` represents a dictionary containing two key-value pairs. The first key
+is `logits_profile_predictions`, holding a value with a shape of (N, 1000). This value corresponds
+to logit predictions for a 1000-base-pair output. The second key, named `logcount_predictions``, 
+is associated with a value of shape (N, 1), representing logcount predictions. To transform these
+predictions into per-base signals, utilize the provided pseudo code lines mentioned below.
+
+```python
+import numpy as np
+def softmax(x, temp=1):
+    norm_x = x - np.mean(x,axis=1, keepdims=True)
+    return np.exp(temp*norm_x)/np.sum(np.exp(temp*norm_x), axis=1, keepdims=True)
+    
+predictions = softmax(outputs["logits_profile_predictions"]) * (np.exp(outputs["logcount_predictions"])-1)
+```
+
+## Docker image to load and use the models
+- https://hub.docker.com/r/kundajelab/chrombpnet-atlas/ (tag:v1)
+
+## Code for ChromBPNet
+- https://github.com/kundajelab/chrombpnet/
+
+# License & citation
+External data users may freely download, analyze and publish results based on any ENCODE data without restrictions.
+
+Released under the [ENCODE data-use policy](https://www.encodeproject.org/about/data-use-policy/). Please cite the ENCODE Project Consortium and the model software: [ChromBPNet](https://github.com/kundajelab/chrombpnet) (Pampari et al., bioRxiv 2024).

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.args.json

@@ -0,0 +1,42 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "bigwig": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR278FVO/preprocessing/bigWigs/ENCSR278FVO.bigWig",
+    "output_dir": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/groups/akundaje/ziwei75/anaconda3/envs/chrombpnet/lib/python3.8/site-packages/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR278FVO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/models/chrombpnet",
+    "chr": "chr8",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/logs/chrombpnet_model_params.tsv"
+}

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_0/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2442.75
+trainings_pts_post_thresh	171558

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_0/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	6.7
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_0.json
+negative_sampling_ratio	0.1

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_model/bias_threshold_0.8/ENCSR278FVO/fold0/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_0/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_0/logs.models.fold_0.ENCSR278FVO/logfile.modelling.fold_0.ENCSR278FVO.epoch_loss.csv

@@ -0,0 +1,12 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.4102703332901,325.218017578125,334.6663513183594,0.3813888728618622,321.29534912109375,323.8506164550781
+1,0.4281182289123535,316.085205078125,318.9539794921875,0.38000908493995667,318.6742248535156,321.2201232910156
+2,0.3805522322654724,314.0189514160156,316.56854248046875,0.3180951178073883,316.71954345703125,318.850830078125
+3,0.35770732164382935,312.3800048828125,314.7763671875,0.306389182806015,314.06353759765625,316.115966796875
+4,0.33813202381134033,311.1996154785156,313.4645690917969,0.33221134543418884,316.496337890625,318.7221984863281
+5,0.3216957747936249,310.1456604003906,312.3011474609375,0.2937024235725403,313.5032043457031,315.4708557128906
+6,0.3058374226093292,309.0191955566406,311.0677185058594,0.3113885223865509,315.2349548339844,317.3212890625
+7,0.29721835255622864,308.22015380859375,310.2113342285156,0.2823430895805359,316.5437927246094,318.4356689453125
+8,0.28732213377952576,307.4718933105469,309.3963317871094,0.2848535180091858,317.08587646484375,318.9943542480469
+9,0.27642977237701416,307.0738830566406,308.9261779785156,0.27006158232688904,319.8281555175781,321.6376037597656
+10,0.26445838809013367,306.21661376953125,307.98876953125,0.28300338983535767,316.69940185546875,318.5956726074219

fold_0/model.bias_scaled.fold_0.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8cb5185591bdff6e9c0b4bd552870212003d9810c171875649f5f9445c959f3
+size 2691928

fold_0/model.bias_scaled.fold_0.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd6ad56ea3754cca3bf204478677bbb6a6a21b7f8c6094b8f680d71a1ccb39bf
+size 1198080

fold_0/model.chrombpnet.fold_0.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6db07d98c30df4d6b96add0e1cbfd2e35839782591a88a716ce2b4cb2e332fb8
+size 77538952

fold_0/model.chrombpnet.fold_0.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9176222acb81a34fc2b635b800d94a3f8244028b2c79a90a50db4914af00c45d
+size 27525120

fold_0/model.chrombpnet_nobias.fold_0.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3744405814cd68a98a0d9901b32c95ffdd4f44f9dddee30f9cff975d5565169
+size 25582648

fold_0/model.chrombpnet_nobias.fold_0.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95e8f58d1b34f5a932347943a39607bb73f2f7cd50af2d7e3236942f44b28842
+size 26060800

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR278FVO/preprocessing/bigWigs/ENCSR278FVO.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR278FVO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr12",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/logs/chrombpnet_model_params.tsv"
+}

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_1/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2390.48
+trainings_pts_post_thresh	174239

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_1/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	6.8
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_1.json
+negative_sampling_ratio	0.1

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_1/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_1/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_1/logs.models.fold_1.ENCSR278FVO/logfile.modelling.fold_1.ENCSR278FVO.epoch_loss.csv

@@ -0,0 +1,15 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,2.32427978515625,324.6566467285156,340.46148681640625,0.47158172726631165,371.6412353515625,374.8477783203125
+1,0.45296937227249146,315.4248352050781,318.5047607421875,0.4795067310333252,367.2979431152344,370.5586853027344
+2,0.4110468626022339,312.74237060546875,315.53729248046875,0.4272199273109436,368.3455810546875,371.2503967285156
+3,0.3759203255176544,310.80322265625,313.3590087890625,0.3455599546432495,363.351806640625,365.7015380859375
+4,0.35406461358070374,309.810302734375,312.2189025878906,0.3779771327972412,363.4830627441406,366.0533752441406
+5,0.3319687247276306,308.4212341308594,310.6784973144531,0.3151719570159912,365.0776672363281,367.2206726074219
+6,0.3194902539253235,307.57818603515625,309.7507019042969,0.345096617937088,365.4154052734375,367.7619934082031
+7,0.30275431275367737,306.3904724121094,308.44921875,0.3025529086589813,367.5987243652344,369.65606689453125
+8,0.2929040789604187,306.1372375488281,308.1289367675781,0.29988449811935425,363.22601318359375,365.2650146484375
+9,0.28521981835365295,305.0465393066406,306.9850769042969,0.3027079999446869,366.43341064453125,368.491943359375
+10,0.2723214626312256,304.649169921875,306.5011291503906,0.29497188329696655,365.9818115234375,367.98760986328125
+11,0.2654109001159668,303.86077880859375,305.6653747558594,0.2934381663799286,366.6458435058594,368.6412658691406
+12,0.2566681504249573,302.7627868652344,304.50799560546875,0.2978667616844177,365.6770935058594,367.7025451660156
+13,0.2529032230377197,302.5738220214844,304.29364013671875,0.34071412682533264,367.9002685546875,370.2170715332031

fold_1/model.bias_scaled.fold_1.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14bb77a043647fb5c814d3901bac55957cd1e6d6c61fd4cf30dd21b1f29294b0
+size 2691928

fold_1/model.bias_scaled.fold_1.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44ee48651a2d633bbc23133fdaac0e55b968c706c3cb310861f1166db19ce56f
+size 1198080

fold_1/model.chrombpnet.fold_1.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f05ea5d29201c3c6cf4af41f1d87ec6cbb0e42a42678ff785638842f511f8736
+size 77538840

fold_1/model.chrombpnet.fold_1.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:150c4a0afb17291848840207a868c843a737101ebb08504ba35ac74cf24e52c0
+size 27525120

fold_1/model.chrombpnet_nobias.fold_1.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3be36b968f25dcea14144a67142afb4704564103f17311b30587575ea81d4719
+size 25582648

fold_1/model.chrombpnet_nobias.fold_1.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:258118ce8f505319933d50e4062f9351f1000dafed9c50b1f99424eed712bd8d
+size 26060800

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR278FVO/preprocessing/bigWigs/ENCSR278FVO.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR278FVO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr22",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/logs/chrombpnet_model_params.tsv"
+}

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_2/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2443.32
+trainings_pts_post_thresh	177964

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_2/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	6.7
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_2.json
+negative_sampling_ratio	0.1

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_2/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_2/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py

fold_2/logs.models.fold_2.ENCSR278FVO/logfile.modelling.fold_2.ENCSR278FVO.epoch_loss.csv

@@ -0,0 +1,11 @@
+epoch,logcount_predictions_loss,logits_profile_predictions_loss,loss,val_logcount_predictions_loss,val_logits_profile_predictions_loss,val_loss
+0,1.3059190511703491,323.7770080566406,332.5262145996094,0.4313778877258301,329.74481201171875,332.6352844238281
+1,0.411296010017395,315.90838623046875,318.66351318359375,0.4331924319267273,327.8535461425781,330.7558898925781
+2,0.3688298165798187,313.6708984375,316.1425476074219,0.35643401741981506,326.6015319824219,328.9896545410156
+3,0.34542515873908997,312.3286437988281,314.6433410644531,0.37865155935287476,325.76263427734375,328.2995300292969
+4,0.32850611209869385,311.4726867675781,313.67364501953125,0.34736835956573486,323.45843505859375,325.78558349609375
+5,0.31198441982269287,310.1813659667969,312.2723388671875,0.3778809607028961,324.5369567871094,327.0688781738281
+6,0.29785436391830444,309.5487976074219,311.5443420410156,0.35927173495292664,325.8230285644531,328.2304992675781
+7,0.2863939106464386,308.7279052734375,310.6470642089844,0.3780200779438019,324.4637756347656,326.9964294433594
+8,0.2784799337387085,307.7500915527344,309.6161193847656,0.3039402365684509,324.0560607910156,326.0924987792969
+9,0.2725285589694977,306.8572082519531,308.6830749511719,0.3551125228404999,324.528564453125,326.9077453613281

fold_2/model.bias_scaled.fold_2.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bad7b7ffab3c40f1378f8bc9570570f2108fa412079410088ede9d29f94593b
+size 2691928

fold_2/model.bias_scaled.fold_2.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c1ce7ce808bcebf7651fcc4df62826423b8cee53d4be10463461bd9e801f822
+size 1198080

fold_2/model.chrombpnet.fold_2.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64f1ba3661d05b3ecf39ecd3f3dac1d5b79ed493cfd502e408f560e872175adc
+size 77538840

fold_2/model.chrombpnet.fold_2.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4146f0fca6e481497a5c725b3b9b6e9e25707fa6aa3aa1541bda2fa780051ab4
+size 27525120

fold_2/model.chrombpnet_nobias.fold_2.ENCSR278FVO.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6be71b4d5c2ae599eb1dcf0ddb0b4cfefbe840c8d1dad347bcf108907d0abfaa
+size 25582648

fold_2/model.chrombpnet_nobias.fold_2.ENCSR278FVO.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e5393c219fd304b77aadc9af1859b16c16563f25ad667635a9da1ab804826e1
+size 26060800

fold_3/logs.models.fold_3.ENCSR278FVO/logfile.modelling.fold_3.ENCSR278FVO.args.json

@@ -0,0 +1,50 @@
+{
+    "cmd": "pipeline",
+    "genome": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta",
+    "chrom_sizes": "/oak/stanford/groups/akundaje/ziwei75/atac_seq_pipeline/hg38/GRCh38_EBV.chrom.sizes.tsv",
+    "input_bam_file": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/DNASE/ENCSR278FVO/preprocessing/bigWigs/ENCSR278FVO.bigWig",
+    "input_fragment_file": null,
+    "input_tagalign_file": null,
+    "output_dir": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3",
+    "data_type": "DNASE",
+    "peaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/auxiliary/filtered.peaks.bed",
+    "nonpeaks": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/auxiliary/filtered.nonpeaks.bed",
+    "chr_fold_path": "/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json",
+    "outlier_threshold": 0.9999,
+    "ATAC_ref_path": null,
+    "DNASE_ref_path": null,
+    "num_samples": 10000,
+    "inputlen": 2114,
+    "outputlen": 1000,
+    "seed": 1234,
+    "epochs": 50,
+    "early_stop": 5,
+    "learning_rate": 0.001,
+    "trackables": [
+        "logcount_predictions_loss",
+        "loss",
+        "logits_profile_predictions_loss",
+        "val_logcount_predictions_loss",
+        "val_loss",
+        "val_logits_profile_predictions_loss"
+    ],
+    "architecture_from_file": "/home/users/vhecht/chrombpnet/chrombpnet/chrombpnet/training/models/chrombpnet_with_bias_model.py",
+    "file_prefix": null,
+    "html_prefix": "./",
+    "bsort": false,
+    "tmpdir": null,
+    "no_st": false,
+    "bias_model_path": "/oak/stanford/groups/akundaje/ziwei75/chromatin_atlas_bias/DNase_bias_model/bias_threshold_0.8/ENCSR278FVO/models/bias.h5",
+    "negative_sampling_ratio": 0.1,
+    "filters": 512,
+    "n_dilation_layers": 8,
+    "max_jitter": 500,
+    "batch_size": 64,
+    "output_prefix": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/models/chrombpnet",
+    "bigwig": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/auxiliary/data_unstranded.bw",
+    "plus_shift": null,
+    "minus_shift": null,
+    "chr": "chr6",
+    "pwm_width": 24,
+    "params": "/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/logs/chrombpnet_model_params.tsv"
+}

fold_3/logs.models.fold_3.ENCSR278FVO/logfile.modelling.fold_3.ENCSR278FVO.bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/models/bias_model_scaled.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_3/new_model_format/bias_model_scaled.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR278FVO/logfile.modelling.fold_3.ENCSR278FVO.chrombpnet_data_params.tsv

@@ -0,0 +1,3 @@
+counts_sum_min_thresh	0.0
+counts_sum_max_thresh	2442.67
+trainings_pts_post_thresh	172304

fold_3/logs.models.fold_3.ENCSR278FVO/logfile.modelling.fold_3.ENCSR278FVO.chrombpnet_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/models/chrombpnet.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_3/new_model_format/chrombpnet.tar with get_new_tf_model_format.py

fold_3/logs.models.fold_3.ENCSR278FVO/logfile.modelling.fold_3.ENCSR278FVO.chrombpnet_model_params.tsv

@@ -0,0 +1,9 @@
+counts_loss_weight	6.8
+filters	512
+n_dil_layers	8
+bias_model_path	/oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/models/bias_model_scaled.h5
+inputlen	2114
+outputlen	1000
+max_jitter	500
+chr_fold_path	/oak/stanford/groups/akundaje/projects/chromatin-atlas-2022/splits/fold_3.json
+negative_sampling_ratio	0.1

fold_3/logs.models.fold_3.ENCSR278FVO/logfile.modelling.fold_3.ENCSR278FVO.chrombpnet_no_bias_formatting.stdout.txt

@@ -0,0 +1 @@
+Converting /oak/stanford/groups/akundaje/vhecht/chromatin_atlas_bias_corrected/DNASE_model/ENCSR278FVO/fold_3/models/chrombpnet_nobias.h5 to /oak/stanford/groups/akundaje/vhecht/chromatin-atlas-2022/DNASE/ENCSR278FVO/fold_3/new_model_format/chrombpnet_nobias.tar with get_new_tf_model_format.py