text:
# Juicer

Juicer is a platform for analyzing kilobase resolution Hi-C data. In this
distribution, we include the pipeline for generating Hi-C maps from fastq raw
data files and command line tools for feature annotation on the Hi-C maps.

Juicer is currently in its beta release, Juicer version 1.5.
For general questions, please use
[the Google Group](https://groups.google.com/forum/#!forum/3d-genomics).
If you have further difficulties using Juicer, please do not
hesitate to contact us<EMAIL_ADDRESS>
**If you use Juicer in your research, please cite:
Neva C. Durand, Muhammad S. Shamim, Ido Machol, Suhas S. P. Rao, Miriam H. Huntley, Eric S. Lander, and Erez Lieberman Aiden. "Juicer provides a one-click system for analyzing loop-resolution Hi-C experiments." Cell Systems 3(1), 2016.**

------------
Quick Start
------------
Run the Juicer pipeline on your cluster of choice with "juicer.sh [options]"

```
Usage: juicer.sh [-g genomeID] [-d topDir] [-q queue] [-l long queue] [-s site]
                 [-a about] [-R end] [-S stage] [-p chrom.sizes path]
                 [-y restriction site file] [-z reference genome file]
                 [-C chunk size] [-D Juicer scripts directory]
                 [-Q queue time limit] [-L long queue time limit] [-r] [-h] [-x]
* [genomeID] must be defined in the script, e.g. "hg19" or "mm10" (default
  "hg19"); alternatively, it can be defined using the -z command
* [topDir] is the top level directory (default
  "/Users/nchernia/Downloads/neva-muck/UGER")
     [topDir]/fastq must contain the fastq files
     [topDir]/splits will be created to contain the temporary split files
     [topDir]/aligned will be created for the final alignment
* [queue] is the queue for running alignments (default "short")
* [long queue] is the queue for running longer jobs such as the hic file
  creation (default "long")
* [site] must be defined in the script, e.g.  "HindIII" or "MboI"
  (default "MboI")
* [about]: enter description of experiment, enclosed in single quotes
* -r: use the short read version of the aligner, bwa aln
  (default: long read, bwa mem)
* [end]: use the short read aligner on read end, must be one of 1 or 2
* [stage]: must be one of "merge", "dedup", "final", "postproc", or "early".
    -Use "merge" when alignment has finished but the merged_sort file has not
     yet been created.
    -Use "dedup" when the files have been merged into merged_sort but
     merged_nodups has not yet been created.
    -Use "final" when the reads have been deduped into merged_nodups but the
     final stats and hic files have not yet been created.
    -Use "postproc" when the hic files have been created and only
     postprocessing feature annotation remains to be completed.
    -Use "early" for an early exit, before the final creation of the stats and
     hic files
* [chrom.sizes path]: enter path for chrom.sizes file
* [restriction site file]: enter path for restriction site file (locations of
  restriction sites in genome; can be generated with the script
  (misc/generate_site_positions.py) )
* [reference genome file]: enter path for reference sequence file, BWA index
  files must be in same directory
* [chunk size]: number of lines in split files, must be multiple of 4
  (default 90000000, which equals 22.5 million reads)
* [Juicer scripts directory]: set the Juicer directory,
  which should have scripts/ references/ and restriction_sites/ underneath it
  (default /broad/aidenlab)
* [queue time limit]: time limit for queue, i.e. -W 12:00 is 12 hours
  (default 1200)
* [long queue time limit]: time limit for long queue, i.e. -W 168:00 is one week
  (default 3600)
* -x: exclude fragment-delimited maps from hic file creation
* -h: print this help and exit
```

See below and the cluster-specific READMEs for more details.

------------
Distribution
------------

In this repository, we include the scripts for running Juicer on LSF,
Univa Grid Engine, and SLURM

/AWS - scripts for running pipeline and postprocessing on AWS (LSF)

/UGER - scripts for running pipeline and postprocessing on UGER (Univa)

/SLURM - scripts for running pipeline and postprocessing on SLURM

/misc - miscellaneous helpful scripts

----------------------------------
Hardware and Software Requirements
----------------------------------
Juicer is a pipeline optimized for parallel computation on a cluster. Juicer
consists of two parts: the pipeline that creates Hi-C files from raw data,
and the post-processing command line tools.

###Cluster requirements:

Juicer requires the use of a cluster, with ideally >= 4 cores (min 1 core)
and >= 64 GB RAM (min 16 GB RAM)

Juicer currently works with the following resource management software:
- OpenLava (http://www.openlava.org/)
- LSF (http://www-03.ibm.com/systems/services/platformcomputing/lsf.html)
- SLURM (http://slurm.schedmd.com/download.html)
- GridEngine (Univa, etc. any flavor)

###Command line tool requirements

The minimum software requirement to run Juicer is a working Java installation
(version >= 1.7) on Windows, Linux, and Mac OSX.  We recommend using the
latest Java version available, but please do not use the Java Beta Version.
Minimum system requirements for running Java can be found at
http://java.com/en/download/help/sysreq.xml

To download and install the latest Java Runtime Environment (JRE), please go
to http://www.java.com/download

###GNU CoreUtils

The latest version of GNU coreutils can be downloaded from
https://www.gnu.org/software/coreutils/manual/

###Burrows-Wheeler Aligner (BWA)

The latest version of BWA should be installed from
http://bio-bwa.sourceforge.net/

###CUDA (for HiCCUPS peak calling)

You must have an NVIDIA GPU to install CUDA
Instructions for installing the latest version of CUDA can be found
on the NVIDIA Developer site:
   https://developer.nvidia.com/cuda-downloads

The native libraries included with Juicer are compiled for CUDA 7 or CUDA 7.5
(AWS/scripts/juicebox_tools.7.0.jar) or (UGER/scripts/juicebox_tools.7.5.jar)
Other versions of CUDA can be used, but you will need to download the
respective native libraries from
   http://www.jcuda.org/downloads/downloads.html

For best performance, use a dedicated GPU. You may also be able to obtain
access to GPU clusters through Amazon Web Services or a local research
institution.

###Building new jars

See the Juicebox documentation at <https://github.com/theaidenlab/Juicebox> for details on building new jars of the juicebox_tools.

-------------
Documentation
-------------
We have extensive documentation below for how to use Juicer.

------------
Juicer Usage
------------
- **Running Juicer with no arguments** will run it with genomeID hg19 and site MboI
- **Providing a genome ID**: if not defined in the script, you can either directly modify the script or provide the script with the files needed. You would provide the script with the files needed via "-z reference_sequence_path" (needs to have the BWA index files in same directory), "-p chrom_sizes_path" (these are the chromosomes you want included in .hic file), and "-s site_file" (this is the listing of all the restriction site locations, one line per chromosome). Note that ligation junction won't be defined in this case.  The script (misc/generate_site_positions.py) can help you generate the file
- **Providing a restriction enzyme**: if not defined in the script, you can either directly modify the script or provide the files needed via the "-s site_file" flag, as above.  Alternatively, if you don't want to do any fragment-level analysis (as with a DNAse experiment), you should assign the site "none", as in `juicer.sh -s none`
- **Directory structure**: Juicer expects the fastq files to be stored in a directory underneath the top-level directory. E.g. HIC001/fastq.  By default, the top-level directory is the directory where you are when you launch Juicer; you can change this via the -d flag. Fastqs can be zipped. [topDir]/splits will be created to contain the temporary split files and should be deleted once your run is completed.  [topDir]/aligned will be created for the final files, including the hic files, the statistics, the valid pairs (merged_nodups), the collisions, and the feature annotations.
- **Queues** are complicated and it's likely that you'll have to modify the script for your system, though we did our best to avoid this.  By default there's a short queue and a long queue.  We also allow you to pass in wait times for those queues; this is currently ignored by the UGER and SLURM versions.  The short queue should be able to complete alignment of one split file.  The long queue is for jobs that we expect to take a while, like writing out the merged_sort file
- **Chunk size** is intimitely associated with your queues; a smaller chunk size means more alignment jobs that complete in a faster time.  If you have a hard limit on the number of jobs, you don't want too small of a chunk size.  If your short queue has a very limited runtime ceiling, you don't want too big of a chunk size.  Run time for alignment will also depend on the particulars of your cluster.  We launch ~5 jobs per chunk.  Chunk size must be a multiple of 4.
-  **Relaunch** via the same script. Type `juicer.sh [options] -S stage` where "stage" is one of merge, dedup, final, postproc, or early. "merge" is for when alignment has finished but merged_sort hasn't been created; "dedup" is for when merged_sort is there but not merged_nodups (this will relaunch all dedup jobs); "final" is for when merged_nodups is there and you want the stats and hic files; "postproc" is for when you have the hic files and just want feature annotations; and "early" is for early exit, before hic file creation. If your jobs failed at the alignment stage, run `relaunch_prep.sh` and then run juicer.sh.
- **Miscelleaneous options** include -a 'experiment description', which will add the experiment description to the statistics file and the meta data in the hic file; -r, which allows you to use bwa aln instead of bwa mem, useful for shorter reads; -R [end], in case you have one read end that's short and one that's long and you want to align the short end with bwa aln and the long end with bwa mem; and -D [Juicer scripts directory], to set an alternative Juicer directory; must have scripts/, references/, and restriction_sites/ underneath it

------------------------
Command Line Tools Usage
------------------------
To launch the command line tools, use the shell script “juicebox.sh” on Unix/MacOS
or type
```
java -jar juicebox_tools.jar (command...) [flags...] <parameters...>`
```
There are two flavors of juicebox_tools: 7.0 uses CUDA7.0 and 7.5 uses CUDA 7.5

For HiCCUPS loop calling without the shell or bat script, you will need to
call:
		`java -Xms512m -Xmx2048m -Djava.library.path=path/to/natives/ -jar Juicebox_CLT.jar hiccups [flags...] <parameters...>`
   where path/to/natives is the path to the native libraries used for Jcuda
   By default, these are located in the lib/jcuda folder.

In the command line tools, there are 4 functions:
		`apa` for conducting aggregate peak analysis
		`hiccups` for annotating loops
		`motifs` for finding CTCF motifs
		`arrowhead` for annotating contact domains

The `juicebox.sh` (Unix/MacOS) script can be used in place of the unwieldy
		`java -Djava.library.path=path/to/natives/ -jar juicebox_tools.jar`

-------
###Arrowhead
-------

`arrowhead [-c chromosome(s)] [-m matrix size] [-r resolution] [-k normalization (NONE/VC/VC_SQRT/KR)] <HiC file(s)> <output_file> [feature_list] [control_list]`

The required arguments are:

`<HiC file(s)>`: Address of HiC file(s) which should end with ".hic". This is the file you will
   load into Juicebox. URLs or local addresses may be used.

`<output_file>`: Final list of all contact domains found by Arrowhead. Can be visualized directly in Juicebox
  as a 2D annotation.

-- NOTE -- If you want to find scores for a feature and control list, both must be provided:

[feature_list]: Feature list of loops/domains for which block scores are to be calculated

[control_list]: Control list of loops/domains for which block scores are to be calculated

The optional arguments are:

`-c <String(s)>` Chromosome(s) on which Arrowhead will be run. The number/letter for the chromosome can be used with or
  without appending the "chr" string. Multiple chromosomes can be specified using commas (e.g. 1,chr2,X,chrY)

`-m <int>` Size of the sliding window along the diagonal in which contact domains will be found. Must be an even
  number as (m/2) is used as the increment for the sliding window. (Default 2000)

`-r <int>` resolution for which Arrowhead will be run. Generally, 5kB (5000) or 10kB (10000)
  resolution is used depending on the depth of sequencing in the HiC file(s).

`-k <NONE/VC/VC_SQRT/KR>` Normalizations (case sensitive) that can be selected. Generally, KR (Knight-Ruiz)
       balancing should be used when available.

Default settings of optional arguments:

  Medium resolution maps:

   -c (all chromosomes)

   -m 2000

   -r 10000

   -k KR

  High resolution maps:

   -c (all chromosomes)

   -m 2000

   -r 5000

   -k KR

----------------
###Arrowhead Examples
----------------

NOTE: Arrowhead will choose appropriate defaults for HiC files if no specifications are given

		arrowhead https://hicfiles.s3.amazonaws.com/hiseq/ch12-lx-b-lymphoblasts/in-situ/combined_30.hic contact_domains_list
  This command will run Arrowhead on a mouse cell line HiC map (medium resolution) at resolution 10 kB and save all
  contact domains to the contact_domains_list file. These are the settings used to generate the official contact
  domain list on the ch12-lx-b-lymphoblast cell line.

		arrowhead https://hicfiles.s3.amazonaws.com/hiseq/gm12878/in-situ/combined_30.hic contact_domains_list
  This command will run Arrowhead at resolution 5kB on the GM12878 HiC map (high resolution) and save all contact
  domains to the contact_domains_list file. These are the settings used to generate the official GM12878
  contact domain list.

-------
###HiCCUPS
-------

`hiccups [-m matrixSize] [-c chromosome(s)] [-r resolution(s)] [-k normalization (NONE/VC/VC_SQRT/KR)] [-f fdr] [-p peak width] [-i window] [-t thresholds] [-d centroid distances] <HiC file(s)> <outputLoopsList>`

The required arguments are:

`<HiC file(s)>`: Address of HiC file(s) which should end with ".hic". This is the file you will
   load into Juicebox. URLs or local addresses may be used.

`<outputLoopsList>`: Final list of all loops found by HiCCUPS. Can be visualized directly in Juicebox as a 2D annotation.
   By default, various values critical to the HICCUPS algorithm are saved as attributes for each loop found. These can be
   disabled using the suppress flag below.

The optional arguments are:

`-m <int>` Maximum size of the submatrix within the chromosome passed on to GPU (Must be an even number greater than 40
       to prevent issues from running the CUDA kernel). The upper limit will depend on your GPU. Dedicated GPUs
       should be able to use values such as 500, 1000, or 2048 without trouble. Integrated GPUs are unlikely to run
       sizes larger than 90 or 100. Matrix size will not effect the result, merely the time it takes for hiccups.
       Larger values (with a dedicated GPU) will run fastest.

`-c <String(s)>` Chromosome(s) on which HiCCUPS will be run. The number/letter for the chromosome can be used with or
       without appending the "chr" string. Multiple chromosomes can be specified using commas (e.g. 1,chr2,X,chrY)

`-r <int(s)>` Resolution(s) for which HiCCUPS will be run. Multiple resolutions can be specified using commas
       (e.g. 25000,10000,5000). Due to the nature of DNA looping, it is unlikely that loops will be found at
       lower resolutions (i.e. 50kB or 100kB)
       IMPORTANT: if multiple resolutions are used, the flags below can be configured so that different parameters are
       used for the different resolutions.

`-k <NONE/VC/VC_SQRT/KR>` Normalizations (case sensitive) that can be selected. Generally, KR (Knight-Ruiz)
       balancing should be used when available.

`-f <int(s)>` FDR values actually corresponding to max_q_val (i.e. for 1% FDR use 0.01, for 10%FDR use 0.1). Different
       FDR values can be used for each resolution using commas. (e.g "-r 5000,10000 -f 0.1,0.15" would run HiCCUPS at
       10% FDR for resolution 5000 and 15% FDR for resolution 10000)

`-p <int(s)>` Peak width used for finding enriched pixels in HiCCUPS. Different peak widths can be used for each
       resolution using commas. (e.g "-r 5000,10000 -p 4,2" would run at peak width 4 for resolution 5000 and
       peak width 2 for resolution 10000)

`-i <int(s)>` Window width used for finding enriched pixels in HiCCUPS. Different window widths can be used for each
       resolution using commas. (e.g "-r 5000,10000 -p 10,6" would run at window width 10 for resolution 5000 and
       window width 6 for resolution 10000)

`-t <floats>` Thresholds for merging loop lists of different resolutions. Four values must be given, separated by
       commas (e.g. 0.02,1.5,1.75,2). These thresholds (in order) represent:
       - threshold allowed for sum of FDR values of the horizontal, vertical, donut, and bottom left filters
           (an accepted loop must stay below this threshold)
       - threshold ratio that both the horizontal and vertical filters must exceed
       - threshold ratio that both the donut and bottom left filters must exceed
       - threshold ratio that at least one of the donut and bottom left filters must exceed

`-d <ints>` Distances used for merging nearby pixels to a centroid. Different distances can be used for each
       resolution using commas. (e.g "-r 5000,10000 -d 20000,21000” would merge pixels within 20kB of each
       other at 5kB resolution and within 21kB at 10kB resolution.

Defaults:

  Medium resolution maps:

   -m 512

   -c (all chromosomes)

   -r 10000

   -k KR

   -f .1

   -p 2

   -i 5

   -t 0.02,1.5,1.75,2

   -d 20000,20000,50000

  High resolution maps:

   -m 512

   -c (all chromosomes)

   -r 5000,10000

   -k KR

   -f .1,.1

   -p 4,2

   -i 7,5

   -t 0.02,1.5,1.75,2

   -d 20000,20000,50000

----------------
###HiCCUPS Examples
----------------

		hiccups HIC006.hic all_hiccups_loops
This command will run HiCCUPS on HIC006 and save all found loops to the all_hiccups_loops files

		hiccups -m 500 -r 5000,10000 -f 0.1,0.1 -p 4,2 -i 7,5 -d 20000,20000,0  -c 22  HIC006.hic all_hiccups_loops
This command will run HiCCUPS on chromosome 22 of HIC006 at 5kB and 10kB resolution using the following values:

5kB: fdr 10%, peak width 4, window width 7, and centroid distance 20kB

10kB: fdr 10%, peak width 2, window width 5, and centroid distance 20kB

The resulting loop list will be merged and saved as all_hiccups_loops

Note that these are values used for generating the GM12878 loop list

---
###APA
---
The "apa" command takes three required arguments and a number of optional
arguments.

`apa [-n minval] [-x maxval] [-w window]  [-r resolution(s)] [-c chromosome(s)] [-k NONE/VC/VC_SQRT/KR] <HiC file(s)> <PeaksFile> <SaveFolder>`

The required arguments are:

`<HiC file(s)>`: Address of HiC file(s) which should end with ".hic". This is the file you will
   load into Juicebox. URLs or local addresses may be used. To sum multiple HiC Files together,
   use the '+' symbol between the addresses (no whitespace between addresses)

`<PeaksFile>`: List of peaks in standard 2D feature format (chr1 x1 x2 chr2 y1 y2 color ...)

`<SaveFolder>`: Working directory where outputs will be saved

The optional arguments are:

`-n <int>` minimum distance away from the diagonal. Used to filter peaks too close to the diagonal.
       Units are in terms of the provided resolution. (e.g. -n 30 @ resolution 5kB will filter loops
       within 30(5000/sqrt(2)) units of the diagonal)

`-x <int>` maximum distance away from the diagonal. Used to filter peaks too far from the diagonal.
       Units are in terms of the provided resolution. (e.g. -n 30 @ resolution 5kB will filter loops
       further than 30(5000/sqrt(2)) units of the diagonal)

`-w <int>` width of region to be aggregated around the specified loops (units of resolution)

`-r <int(s)>` resolution for APA; multiple resolutions can be specified using commas (e.g. 5000,10000)

`-c <String(s)>` Chromosome(s) on which APA will be run. The number/letter for the chromosome can be
       used with or without appending the "chr" string. Multiple chromosomes can be specified using
       commas (e.g. 1,chr2,X,chrY)

`-k <NONE/VC/VC_SQRT/KR>` Normalizations (case sensitive) that can be selected. Generally, KR (Knight-Ruiz)
       balancing should be used when available.

Default settings of optional arguments:

-n 30

-x (infinity)

-w 10

-r 25000,10000

-c (all chromosomes)

-k KR

------------
###APA Examples
------------

		apa HIC006.hic all_loops.txt results1
This command will run APA on HIC006 using loops from the all_loops files
and save them under the results1 folder.

		apa https://hicfiles.s3.amazonaws.com/hiseq/gm12878/in-situ/combined.hic all_loops.txt results1
This command will run APA on the GM12878 mega map using loops from the all_loops
files and save them under the results1 folder.

		apa -r 10000,5000 -c 17,18 HIC006.hic+HIC007.hic all_loops.txt results
This command will run APA at 50 kB resolution on chromosomes 17 and 18 for the
summed HiC maps (HIC006 and HIC007) using loops from the all_loops files
and save them under the results folder

-----------------------
###Motif Finder
-----------------------

`motifs <genomeID> <bed_file_dir> <looplist> [custom_global_motif_list]`

The required arguments are:

`<genomeID>`: hg19 supported by default. For other genome assemblies, provide a
  custom_global_motif_list in FIMO format.

`<bed_file_dir>` File path to a directory (e.g. ) which contains two folders: "unique" and
  "inferred". These folders should contain a combination of RAD21, SMC3, and CTCF BED files.
  By intersecting these 1D tracks, the strongest peaks will be identified. Unique motifs
  generally use a more stringent combination of BED files than inferred motifs.

`<looplist>`: List of peaks in standard 2D feature format (chr1 x1 x2 chr2 y1 y2 color ...)

-- NOTE -- If you want to use a custom list of potential motifs:

[custom_global_motif_list]: Motif list output using FIMO format can be used as an alternative
  to the internal motif list

-------------------
###Motif Finder Examples
-------------------

Assuming the following file structure is present:

/path/to/local/bed/files/unique/CTCF.bed

/path/to/local/bed/files/unique/RAD21.bed

/path/to/local/bed/files/unique/SMC3.bed

/path/to/local/bed/files/inferred/CTCF.bed

		motifs hg19 /path/to/local/bed/files /gm12878_hiccups_loops.txt
  This command will find motifs from the internal hg19 motif list for the loops in gm12878_hiccups_loops.txt
  and save them to gm12878_hiccups_loops_with_motifs.txt. The CTCF, RAD21, and SMC3 BED files will be used
  together (i.e. intersected) to find unique motifs. Just the CTCF track will be used to infer best motifs.

		motifs hg19 /path/to/local/bed/files gm12878_hiccups_loops.txt hg_19_custom_motif_list.txt
  This command will find motifs from hg_19_custom_motif_list.txt for the loops in gm12878_hiccups_loops.txt
  and save them to gm12878_hiccups_loops_with_motifs.txt. The CTCF, RAD21, and SMC3 BED files will be used
  together (i.e. intersected) to find unique motifs. Just the CTCF track will be used to infer best motifs.

repository: liz-is/juicer

repo_name: liz-is/juicer


text:
---
layout: blog
title: We're back on track with the JAMstack
subTitle: Using Netlify and Gatsby
date: 2018-10-22T20:22:23.640Z
hero: /blog/images/uploads/jamstack.jpg
---
## We are back

And with a new dev stack, called the JAMstack. I've been using a similar approach using [Contentful](https://contentful.com) for a client, [Vogelvrij](https://vogelvrij.be). But this current setup is perfect for home project which require a fast time to market: Easy setup, fast deployment, fast site loading time, free 'til a certain point. G R E A T ! 🤠

The JAMstack stands for Javascript, API's and Markup. And that's exactly what it is.. a static site jammed up with those three ingredients. And it tastes good. 😋

## Netlify, Netlify CMS and Gatsby

The blog part of this site got easily created with [Gatsby](https://gatsbyjs.org), a static site generator which uses [React](https://reactjs.org).

I've decided to write all blog posts in markup. Which is a common way to store blog posts in:

* It makes it easy to change your stack. As markup is a widely accepted way to write blogposts in, it doesn't require you to map from one blogging framework or library into another once you change tech stack. This is a pain, I've been there... 😢

* It allows you to easily add a CMS on top. In this case I've used the open source solutions initiated by [Netlify](https://netlify.com), called [Netlify CMS](https://www.netlifycms.org). Which give you a little CMS, created with React, on top of your static site. You can upload images, and write your markdown pages inside the CMS.

## How it works

There's a github webhook triggering a Netlify deploy at each push to master. Netlify then triggers a new build of the site: converts the markdown files into static html pages. And hosts the complete site on a blazingly fast CDN. It's all a matter of minutes: Creating my new post in the Netlify CMS client (hosted on this site after an /admin page), pushing the 'Publish' button, and seeing my new article live.

## Back to blogging

Whoppa, this was my first small blogpost. Goal is to create more of these very small posts..with things that keep me busy, things I've learned...
Give me a thumbsup, or thumbsdown to show if you appreciate this or not. If there is no thumbsup/thumbsdown button, that means I have not yet implemented this feature...yet. 🙃

license: MIT

branch_name: refs/heads/master

extension: md

gha_language: JavaScript

gha_license_id: MIT

license_type: permissive


text:
---
title: "Git CLI"
description: "Get Up and Running Using GIT"
lead: "Get Up and Running Using GIT"
date: 2020-11-16T13:59:39+01:00
lastmod: 2020-11-16T13:59:39+01:00
draft: false
images: []
menu:
  docs:
    parent: "cli"
weight: 140
toc: true
---

# Typical Git Workflow

Before going through each command lets look at how we would typically use git:

Typical usual steps would be **clone, commit, and then push**.

Or just **commit and then push**.

If its a new repository:

```
git init .
git add .
git commit -am "message"
git remote add origin [url]
git push origin master

```

If its not a new repository:

```
git clone [url]
git commit -am "message"
git push

```

When working locally mostly we will be committing and pushing.

```
git add .
git commit -am "message"
git push

```

When Merging branches:

```

git checkout XXXX

git merge XXXX

```

# Clone a Repo

Cloning a repo is a useful time saving tool.

We can clone a repo with ```git clone```.

You pass a path (usually a URL) of the Git repository you want to clone to the git clone command. EG:

```

git clone  https://github.com/udacity/course-git-blog-project

```

We could also clone and rename instead:

```

git clone https://github.com/udacity/course-git-blog-project  blog-project

```

# Initialize a Repository

The **init** command initializes a new repository. The **dot** syntax does it in current working directory.

```

git init .

```

Running the git init command sets up all of the necessary files and directories that Git will use to keep track of everything.

All of these files are stored in a directory called .git.

The .git directory is the "repo". This is where git records all of the commits and keeps track of everything.

Other than the "hooks" directory, you shouldn't mess with pretty much any of the content in this directory.

# The Git Status Command

```git status```  will tell us what Git is thinking and the state of our repository as Git sees it.

When you're first starting out, you should be using the git status command all of the time. You should get into the habit of running it after any other command.

This will help you learn how Git works and it'll help you from making (possibly) incorrect assumptions about the state of your files/repository.

## Add File to Repository

When we want to add a file to the repo to be tracked, we use the **add** command and then the file we want to track.

```

git add index.html

```

We get a new comand prompt back meaning it worked. If you run ```git status``` you will see we have changes ready to be committed, it will say "new file: index.html",

If we were now to create a commit the file would be included in that commit.

## Commit to Repository

A commit is a record of what the repository looked like at a given point in time.

Each commit has a message to briefly explain the changes we made.

**Commits should contain as small a set of changes as possible.**

The more commits you have, the more flexibility you have to roll back or cherry pick changes - Remember Git does not charge by commit so feel free to make as many as you need!

To create a commit we use the commit command followed by *dash* **m** and our message in quotes:

```

git commit -m "added a new file"

```

## Add all files in Current Working Directory

To add all files we use the **git add** command followed by a **dot**. The dot means the current working directory. Using ```git add  .``` we stage all the files in current directory to be committed.

```
git add .

git commit -m "A line on what changes got made"

```

## Push all files in Current Working Directory

We use ```git push``` to push the changes live. This could be done on one file, here we will do it with the whole all files in Current Working Directory:

```

git add .
git commit -m "A line on what changes got made"
git push

```

## Use Git Add & Git Commit in one step

You can do an **add and commit** at the same time to save time.

Note this will track ALL files not yet staged, so if you have files not ready to be committed just yet, this command is not recommended.

```

git commit -am "message here"

```

## Reviewing a Repo's History

Git log is basically like a journal of commits on the project we are working on.

```
git log

```

Press ``q``` to quit the log.

By default, this command displays:

- the SHA
- the author
- the date
- and the message

The most important line is usually the message.

We can condense what displays by using ```git log --oneline```  This command:

- lists one commit per line
- shows the first 7 characters of the commit's SHA
- shows the commit's message

The ```--stat``` flag is used to alter how git log displays information:

```

$ git log --stat

```

This command:

- displays the file(s) that have been modified
- displays the number of lines that have been added/removed
- displays a summary line with the total number of modified files and lines that have been added/removed

We can also use the ```-p``` flag (which is the same as the ```--patch``` flag) to alter how git log displays information:

```

git log -p

```

Finally, The ```git show``` command will show **only one commit**.

It can be combined with most of the other flags we've looked at/

```

git show --stat 60b6067

```

## Ignoring files with .gitignore

Sometimes we don't want to add or track a file inside a project but we still want it to live inside the project directory and we also don't want it to always show up as untracked. For example a to do list, a file other devs on the team will have that we don't want to over write like their personal notes, or a config file specific to our personal machine that may contain our passwords.

To prevent this:

Create a file in the project root called **.gitignore**

Inside here, on each line we would have the files we want to ignore, eg, something like **dev-notes.txt**

## Branches

Every Git repository has at least one branch. When you intialize a Git repository it creates a master branch.

You can have as many branches as you want, and name them as you wish. Branches are independent of each other, but you can merge them into each other.

The current state of a branch (called Head) always references a commit object.

Always think of the state of the branch as the state of last commit. Until we commit it is not part of the branch.

If you run the command ```git branch``` it will give you a list of available branches.

## Creating Branches

A core aspect of Git is that it allows multiple developers to work on different branches at the same time.

The core reason we use branches is to keep changes isolated and away from the master branch. The master branch is the branch we will have ready to deploy, if we anted to add a new feature we would **branch** off master and then **merge** back in when we are done.

To create a new branch we run the command ```git branch``` followed by the name we will give it - a common naming convention is to prefix your name to the name of the branch:

```

git branch bernard_my_new_branch

```

## Switch Branch

To switch to this branch we use the ```checkout``` command:

```

git checkout bernard_my_new_branch

```

Now we can't impact the master branch until we merge branches.

## Merge Branches

Make sure all changes are committed before merging, using ```git status```.

We should have a clean, working tree.

We should then ```git checkout master``` before merging. Its important to **checkout** the branch we are merging INTO.

Now we are on the master branch. we can check the state of this again using ```git status```.

Now we run ```git merge``` followed by the branch we are merging, and a message:

```

git merge bernard_my_new_branch -m "first merge"

```

## Set up a Remote Repository

Here we would have multiple people collaborating on a project, working locally, and pushing changes to a server where the project lives with ```git push```

To pull down a repository with commits that others have pushed up we would use ```git pull```

After setting up a repository in github we would connect to it by copying the SSH address under "quick set up", and giving the remote a nmae, in the below example its **origin**.

```
git remote add origin REPLACE_WITH_URL_WE_GOT_FROM_GIT_HUB.git

```

Its possible to set up multiple remotes.

To check what remote we are on, we can use ```git remote```

## Push to a Remote Repository

To push all changes to the remote we set up we use ```git push``` followed by the name of the remote and then name of the branch we are pushing. If we don't specify the branch, git will push all available branches.

```

git push origin master

```

If you get a fatal error "...no upstream branch" we can set one with ```git push --set-upstream origin master```

# Clone Remote Repository

You can get the URL of the repository you need to clone from the Green "code" dropdown whe you open it on github.

```

git clone https://github.com/XXXXXXXXXXX/XXXXXXXXX_XXXXXX.git

```

If you do a ```git log``` you can see the while repository history.

We could then push commits to the repository

# Merge Conflicts

Git will flag merge conflicts.

If you open the file it will highlight the conflicts with Left and Right arrows with the names of the Branches, separated by equal signs.


text:
> This document contains documentation for the tracee context-logger modules. Click [here](/README.md) to get an overview that TracEE is about.

# context-logger

> __The TracEE context logger subproject helps you to analyze errors in your application by collecting contextual invocation data and writing it to your log files.

The *TracEE* main project helps you to aggregate all your log files. This will technically enable you to track user requests or session throughout the log but does not have an influence on what will be written to the logs.
In case if an error occurs in your application, the context logger subproject collects contextual log information and writes it to your log files.
Like the *TracEE* main project, it integrates with almost no effort - it contains adapters or interceptors for the most popular JavaEE technologies.

* servlet 2.5+
* jax-ws
* jax-rs2
* cdi / ejb
* jms
* AspectJ

This project is still in an experimental stage and the api may change during further development.

# Getting started

## The contextual information data

So what is this all about?

An example:
- A user is visiting your JavaEE application, after a short time an exception was thrown in your server. This can happen due to a lot of reasons (corrupt data, error in software, infrastructure problems).
- If you have luck, the user notifies you about the error or if you do automated checks against you detect the server on your own.
- Now your operations employees or developers will come into play and are trying to analyze what lead to that error. This can be a very difficult and time consuming task and speed and success is directly affected by the information you can gather from your log files.

The TracEE context logger subproject helps you to provide that contextual invocation data needed to speed up analyzing of error events by providing handlers for most JavaEE technologies.
It is also possible and very easy to add custom functionality to the contextual logger.

# Integrating TracEE into your application

The steps to get TracEE contextual logging up and running pretty much depend on your application scenario. The most common use case would be to
provide contextual invocation data from a servlet container based frontend or an ejb based backend.

## Integration scenarios

| Framework  | Adapter |
| ----------:|:------:|
| Servlet    | Use [context-logger-servlet](context-logger-servlet) as a servlet filter. |
| JAX-RS     | Use [context-logger-servlet](context-logger-servlet) as a servlet filter. |
| JAX-RS2    | TODO |
| JAX-WS     | Use [context-logger-jaxws](context-logger-jaxws) as a message listener. | |
| JMS        | Use [context-logger-jms](context-logger-jms) as an interceptor |
| EJB3/CDI   | Use [context-logger-ejb](context-logger-jms) as an interceptor |

## Modules

TracEE context logger is built highly modular. The modules you need depend on your application and the underlying frameworks and containers.
The following table describes all available TracEE-context-logger modules and their usage scenarios.

| Module                                | Usage |
|--------------------------------------:|:-----:|
| [context-logger-api](context-logger-api)               | Provides a fluent api for contextual logging |
| [context-logger-connectors](context-logger-connectors) | Provides support for writing contextual data to other target as a log (f.e. send error via Http) |
| [context-logger-ejb](context-logger-ejb)               | Provides support for EJB / CDI Interceptors |
| [context-logger-impl](context-logger-impl)             | The implementation of the context logger |
| [context-logger-integration-test](context-logger-integration-test) | Does some integration test for custom data providers |
| [context-logger-jaxws](context-logger-jaxws)           | Provides support for JAXWS via Message handlers. |
| [context-logger-jms](context-logger-jms)               | Provides support for JMS via an interceptor |
| [context-logger-servlet](context-logger-servlet)       | Provides support for Servlets via ServletFilter |
| [context-logger-watchdog](context-logger-watchdog)     | AspectJ powered contextual logging, this can is very useful for development and integration environment.|

repository: DennisLoehmann/tracee

repo_name: DennisLoehmann/tracee


text:
# Pesten

## Rules of Pesten

### General
* Pesten uses 52 cards plus two jokers
* At the start of the game all players get 7 cards

### Special Cards

gha_language: Java


text:
# Quiver

## 1 Overview
The unique characteristic of graph learning systems is that the preprocessing with graph structures and the collection of features are usually the bottlenecks. How to resolve those two bottlenecks? Quiver's core ideas are:
- Graph sampling is a **latency critical problem**, so massive parallelism could hide the latency.
- Feature collection is a **bandwidth critical problem**, so bandwidth should be optimized to achieve high performance.

![GNN training pipeline](multi_medias/imgs/overview.png)

Normally, we choose to use CPU to sample the graph and collect features. This strategy will not only cause performance issues when we use single-GPU training but also cause **poor scalability because of CPU resource sharing**. Below is the benchmark of training `ogbn-product` with `PYG` and `DGL`, on the scalability of multi-GPU training with CPU sampling and feature collection:

`Note`: Benchmark code [here](examples/multi_gpu/pyg/ogb-products/dist_sampling_ogb_products_pyg.py)

| Framework | Device Num | Epoch Time(s) |Scalability|
| ------ | ------ | ------ |------|
| Pyg | 1 | 36.5 |1.0|
| Pyg | 2 | 31.3 |1.16|
| Pyg | 3 | 28.1|1.30|
| Pyg | 4 | 29.1|1.25|
| DGL | 1 |  ||
| DGL | 2 |  ||
| DGL | 3 |  ||
| DGL | 4 |  ||

Our Quiver is a high-performance GNN training add-on which can fully utilize the hardware. Users will have **better end-to-end performance** and **better scalability** of multi-GPU GNN training. We even achieve **super linear scalability** with the help of NVLink. Next we will introduce our contributions on the optimizations of graph sampling and feature collection.

## 2 Graph Sampling

### 2.1 Existing Approaches
Existing graph learning frameworks support both CPU and GPU sampling. GPU sampling needs to store the entire graph structure on GPU memory, which is a limit when the graph is large. CPU sampling has poor performance and bad scalability.

### 2.2 Quiver's Approach
`Note`: Benchmark code[here](benchmarks/sample/bench_sampler.py)

Quiver provide users with **UVA-Based**（Unified Virtual Addressing Based）graph sampling operator, supporting storing graph structure in CPU memory and sampling the graph with GPU when the graph is large. In this way, we not only get performance benefits beyond CPU sampling, but also sample the graph whose size is beyond GPU memory. We evaluate our sampling performance on `ogbn-products`and`reddit` datasets, and we find that the performance of UVA-based sampling is much higher than CPU sampling. Our sampling metrics are the sampling throughput (**S**ampled **E**dges **P**er **S**econd, **SEPS**). **We can observe that without storing the graph on GPU, Quiver get 20x speedup on real datasets**.

| Dataset | Parameter | Sampler |SEPS|Speedup Over CPU|
| ------ | ------ | ------ |------|------|
| ogbn-product | [15, 10, 5] | CPU |1.84 M|1.0|
| ogbn-product | [15, 10, 5] | UVA(`on host memory`) |34.29 M|18.63|
| reddit | [25, 10] | CPU |2 M|1.0|
| reddit | [25, 10] | UVA(`on host memory`) |33.15 M|16.575|

Meanwhile, Quiver's sampling can be configured to enable GPU-only sampling (`mode='GPU'`), which put the whole graph structure to GPU memory to achieve higher performance. Compared to UVA mode, this sampling strategy will 30-40% speedup with extra GPU memory cost.

![uva-sampler](./multi_medias/imgs/UVA-Sampler.png)

```python

    dataset = PygNodePropPredDataset('ogbn-products', root)
    csr_topo = quiver.CSRTopo(dataset[0].edge_index)

    # You can set mode='GPU' to choose place graph data in GPU memory
    quiver_sampler = quiver.pyg.GraphSageSampler(csr_topo, [15, 10, 5], device=0, mode="UVA")
```

Furthermore, UVA sampling will not cost CPU computation resources, which avoid contention when we train with multiple GPUs and have a better scalability. This part will be further explained in our end-to-end performance benchmark.

## 3 Feature Collection
`Note`: Benchmark code[here](benchmarks/feature/bench_feature.py)

### 3.1 Existing Approaches
The feature size of one batch in GNN training is usually hundreds of MB or even several GB. The feature collection time, especially the CPU-to-GPU case, have significant cost. The core idea of optimizing feature collection is to optimise the throughput of feature transfer. Existing approaches are:

1. Collect feature on CPU, and transfer the result to GPU for training
2. When feature size is small, put all feature data on GPU and collect the batch

Approach 1 has a much lower throughput, and it has a poor scalability because multiple workers will share CPU memory bandwidth.

### 3.2 Quiver's Approach

Quiver provides feature collection `quiver.Feature` with high throughput. The implementation of `quiver.Feature` is based on two observations below:

1. Real graph datasets usually follow the power low, which means most of the edges in the graph are associated with a small portion of nodes. Most sampling algorithms based on graph structure will sample more nodes whose degree are higher. We show in the table below that two datasets follow our observations.

| Dataset | #(degree > average_degree)/total_nodes_count | #edges(with node degree > average)/total_edges_count |
| ------ | ------ | ------ |
| ogbn-product | 31.3%| 76.8% |
| reddit | 29.8% | 77.1% |

2. Memory bandwidth relationship of a GPU server is: GPU Global Memory > GPU P2P With NVLink > Pinned Memory > Pageble Memory.

Considering the bandwidth and the power law, Quiver's `quiver.Feature` can allocate features automatically to GPU memory and CPU Pinned Memory. It can also host hot data on GPU and cold data on CPU (if `csr_topo` is provided). We collect features with GPU.

![single_devce](./multi_medias/imgs/single_device.png)

#### 3.2.1 Single-GPU

First we show our performance of feature collection using one GPU. The table below shows the performance of `quiver.Feature` when we cache 20% hot data on GPU cpmpare with normal CPU feature collection.

| Dataset | Feature | Throughput(GB/s) |Speedup Over CPU|
| ------ | ------ | ------ |------|
| ogbn-product | CPU |  1.27|1.0|
| ogbn-product | Quiver | 14.82 |10|
| reddit | CPU | 2.98 |1.0|
| reddit | Quiver|11.66  |3.91|

#### 3.2.2 Multi-GPU

Multi-GPU caching strategy can be classified as `device_replicate` and `p2p_clique_replicate`, depending on whether NVLink is supported between GPUs. First, we take a look at `device_replicate`.

**device replicate**

`device_replicate` strategy is to replicate hot data across GPUs, and they will share the cold data on CPU memory.

![device_replicate](multi_medias/imgs/device_replicate.png)

**p2p_clique_replicate**

When NVLink is supported, feature collection with P2P between GPUs has a higher bandwidth than collecting features from CPU, so GPUs inside a `p2p_clique` will share a hot feature cache. The hot data cache will be larger when each GPU hold a partition of it.

![p2p_clique_replicate](./multi_medias/imgs/p2p_clique_replicate.png)

![p2p_access](multi_medias/imgs/p2p_access.png)

In this way we provide larger cache size, and we achieve **super-linear scalability** because the interconnects between hot cache partitions are much higher.

| Dataset | Device Num | Total Throughput(GB/s) |Speedup Over Single Device|
| ------ | ------ | ------ |------|
| ogbn-product | 1 |  20.29|1.0|
| ogbn-product | 2 | 108.6 |5.35|
| reddit | 1 | |1.0|21.38|1.0|
| reddit | 2|  |3.91|88.34|4.13|

The super-linear scalability of feature collection will make the super-linear speedup of the whole training pipeline.

``` python
# set cache_policy="device_replicate" if you dont have NVLink
quiver_feature = quiver.Feature(rank=0, device_list=[0, 1], device_cache_size="110M", cache_policy="p2p_clique_replicate", csr_topo=csr_topo)
quiver_feature.from_cpu_tensor(dataset[0].x)
```

## 4 End-to-end Performance

Quiver provides high-performance components for GNN training. Users will have the freedom to make choices of sampling and feature collection combined strategy.

### 4.1 cache_policy = device_replicate

We evaluate on the `ogbn-product` dataset, and we compare Quiver with CPU feature collection and sampling from Pyg. The parallelism of each sampling process is 5.

| Device Num | Pyg's Epoch Time | Pyg's  Scalability |Quiver's Epoch Time | Quiver's  Scalability|Quiver Over Pyg|
| ------ | ------ | ------ |------|------|------|
| 1 | 36.5 |  1.0|11.1|1.0|3.23|
| 2| 30 | 1.22 |5.8|1.91|5.17|
| 3 | 27.7 | 1.32|4.7|2.71|6.75|
| 4 | 28.2|  1.28|3.25|3.42|8.68|

Even when we put all feature on GPU and use GPU to collect feature, Quiver still have 3x speedup over Pyg.

| Device Num | Pyg's Epoch Time | Pyg's  Scalability |Quiver Over Pyg|
| ------ | ------ | ------ |------|
| 1 | 23.3 |  1.0|2.1|
| 2| 14.7 | 1.59 |2.53|
| 3 | 11.4 | 2.04|2.78|
| 4 | 9.5|  2.45|2.92|

We show that Quiver has a better performance and a better scalability.

### 4.2 cache_policy = p2p_clique_replicate

When we train `ogb-product` and `reddit` datasets on the machine with NVLink, the speedup with 2 GPUs are 2.25 and 2.75(batch_size=2048), which shows the super-linear scalability of our feature collection strategy.

## 4 Future Works

This time we open-source the single-machine version of Quiver. Next we will work on new features below:

1. **Adaptive allocation**: Now the strategy choice of feature collection and sampling is manually configured. This could be improved by providing the mechanism of making such decisions automatically according to the characteristics of the workload and hardware.
2. **Distributed Quiver**：Next we will support multi-machine distributed version of Quiver to train larger datasets.

## 5 End

The **quiver-team** community will work on adaptive and scalable AI systems. Quiver will fully utilize hardware to help users train GNN efficiently. Quiver is being developed actively, and we hope people who are interested can build Quiver and make it a better GNN system.

![logo](./multi_medias/imgs/logo.png)

repository: mlhackergz/torch-quiver

branch_name: refs/heads/main

repo_name: mlhackergz/torch-quiver


text:
---
title: Service Plugin - Graphite
menu:
  influxdb_1_0:
    weight: 70
    parent: tools
---

InfluxDB provides an easy way to hook up Graphite as an input source.
See the [README on GitHub](https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md) for more information.


text:
# cra-antd-electron

> Template for quick develop desktop apps using Create React App, Ant Design and Electron.

### Development

```
  yarn install
  yarn develop
```

### Deployment

```
  yarn build
```

### Contribute

See [https://github.com/bkdev98/cra-antd-electron/issues](https://github.com/bkdev98/cra-antd-electron/issues).

### License

[MIT License](https://github.com/bkdev98/cra-antd-electron/blob/master/LICENSE.md)


text:
# What is Microsoft Azure?
_Microsoft Azure, formerly known as Windows Azure_

Microsoft Azure is a cloud computing service that works similarly to Amazon Web Services
(AWS) and the Google Cloud Platform.

You pay for as many computing resources as you need, and not a specific number of hardware
servers on a rack somewhere. Services you deploy in this way can either be public servers
available to everyone, or part of a “private cloud” that’s just used in an organization.

# What Can Microsoft Azure Do?

The Microsoft Azure website provides a directory of hundreds of different services you can
use, including full virtual machines, databases, file storage, backups, and services for
mobile and web apps.

# Azure Active Directory and Windows 10

Allows organizations to have centralized administration features without requiring them
to host their own Active Directory server (and set up the often complicated infrastructure
and access permissions needed to make it work remotely).

https://www.howtogeek.com/337961/what-is-microsoft-azure/

# Identity management

Most architectures have shared services that are hosted and accessed across networks. Those
services share common infrastructure and users need to access resources and data from anywhere.
For such architectures, relying on just network security controls isn't enough.

## How are you implementing Zero-Trust in your design?

* Support a single enterprise directory.
* Keep the cloud and on-premises directories synchronized, except for high-privilege accounts.
* Enforce and measure key security attributes when authenticating all users, especially for
  high-privilege accounts.
* Have a separate identity source for non-employees.
* Use passwordless and multi-factor authentication.
* Blocking legacy protocols and authentication methods.

# Use a dingle enterprise directory

_This single-source approach increases clarity and consistency for all roles in IT and Security teams._

# Use role-based access control (RBAC)

_Use Azure RBAC to control the level of access for users. Assign built-in roles to users, groups,
service principals, and managed identities._

# Synchronize the hybrid identity systems

_Keep your cloud identity synchronized with the existing identity systems to ensure consistency and
reduce human errors._

# Use a separate identity source for third-party accounts

Use cloud identity services that are designed to host third-party accounts. Grant the appropriate
level of access to those external entities instead of the default permissions that are given to
full-time employees.

* Azure AD – Employees and Enterprise Resources
* Azure AD B2B – Partners

# Use modern password protection

* Azure AD – Employees and Enterprise Resources
* Azure AD B2B – Partners
* Azure AD B2C – Customers, citizens

# Use modern password protection

Adopt passwordless authentication and multi-factor authentication (MFA) over time. You can
also reduce use of passwords through the use of Managed Identities.

* Password Guidance
* NIST guidance

# Disable legacy authentication methods

Don't use legacy protocols for internet-facing services. For Azure AD-based accounts,
configure Conditional Access to block legacy protocols.

# Use cross-platform credential management

Use a common identity provider:

* Azure Active Directory (AD), for authenticating all platforms
* (Windows, Linux, and others) and cloud services

Azure AD can authenticate on Windows:
* Linux, Azure
* Office 365
* Amazon Web Services (AWS)
* Google Services
* (remote access to) legacy on-premises applications
* Third-party Software as a Service providers

https://docs.microsoft.com/en-us/azure/architecture/framework/security/design-identity

path: /seattle-ops-301d1: Read11.md

filename: seattle-ops-301d1: Read11.md


text:
# Vulcand

[![](https://images.microbadger.com/badges/image/webhippie/vulcand.svg)](https://microbadger.com/images/webhippie/vulcand "Get your own image badge on microbadger.com")

These are docker images for [Vulcand](https://github.com/mailgun/vulcand) running on an [Alpine Linux container](https://registry.hub.docker.com/u/webhippie/alpine/).

## Versions

* [latest](https://github.com/dockhippie/vulcand/tree/master) available as ```webhippie/vulcand:latest``` at [Docker Hub](https://registry.hub.docker.com/u/webhippie/vulcand/)

## Volumes

* None

## Ports

* 8181
* 8182

## Available environment variables

```bash
ENV VULCAND_API_INTERFACE
ENV VULCAND_CERTPATH # As string or filename
ENV VULCAND_ENDPOINT_DIALTIMEOUT 5s
ENV VULCAND_ENDPOINT_READTIMEOUT 50s
ENV VULCAND_ETCD
ENV VULCAND_ETCD_CAFILE # As string or filename
ENV VULCAND_ETCD_CERTFILE # As string or filename
ENV VULCAND_ETCD_KEYFILE # As string or filename
ENV VULCAND_ETCD_CONSISTENCY STRONG
ENV VULCAND_ETCD_KEY vulcand
ENV VULCAND_INTERFACE
ENV VULCAND_LOG console
ENV VULCAND_LOG_SEVERITY WARN
ENV VULCAND_READTIMEOUT 1m0s
ENV VULCAND_SEALKEY
ENV VULCAND_SERVER_MAXHEADERBYTES 1048576
ENV VULCAND_SERVER_READTIMEOUT 1m0s
ENV VULCAND_SERVER_WRITETIMEOUT 1m0s
ENV VULCAND_STATSD_ADDR
ENV VULCAND_STATSD_PREFIX
```

## Inherited environment variables

```bash
ENV CRON_ENABLED false
```

## Contributing

Fork -> Patch -> Push -> Pull Request

## Authors

* [Thomas Boerger](https://github.com/tboerger)

## License

MIT

## Copyright

```
Copyright (c) 2015-2017 Thomas Boerger <http://www.webhippie.de>
```

repository: dockhippie/vulcand

gha_language: Shell

repo_name: dockhippie/vulcand


text:
# HttpIO

This package provides an IO type for reading large remote files that served through a server.
It will convert IO requests to http requests and use 'Range' header to request partial
content. Think of it as read-only memory map for remote resources.

Additionally, it provides support for files served with google cloud storage.

gha_language: Julia


text:
# Animations

* Default animations
* Custom animations
* Animations API (deprecated in >=v1.0.0)

Default animations

The project comes with some default animations located in [src/animation](./src/animation). It simply animates the placeholder with visual effects while waiting the real content to appear.

Adding an animation is possible using the [React Native Animated](https://facebook.github.io/react-native/docs/animations.html) component.

You can contribute by creating your own placeholder animations and submitting a pull request.

Custom animations

Recently, the project has allowed to use custom animations by using the HoC props : `customAnimate`. It accepts a `React.Component` representing an Animation.

From the [Example Folder](./Example/customAnimation.js), we have created a simple animation based on color transitions.

To use this in the code, simply use a `Placeholder` component with the `customAnimate` props :

```javascript
<Placeholder.Media onReady={this.state.isReadyMedia} customAnimate={CustomAnimation}>
  <Text>Media loaded</Text>
</Placeholder.Media>
```

Animations API

*This is deprecated since v1.0.0, use component instead of this API while working on version >=v1.0.0*

Here's the constraints needed to use Animations inside of [rn-placeholder](./README.md)

**start**

Type: `function`

Start the animation

**style**

Type: `object`

Style of the application

*Example*
```javascript

/**
 * Create a repetitive fadein / fadeout animation
 */
export default () => {
  const START_VALUE = 0.5;
  const animation = new Animated.Value(START_VALUE);

  function start() {
    return Animated.sequence([
      Animated.timing(animation, {
        toValue: 1,
        duration: 500,
      }),
      Animated.timing(animation, {
        toValue: START_VALUE,
        duration: 500,
      }),
    ]).start((e) => {
      if (e.finished) {
        start();
      }
    });
  }

  /**
   * The two mandatory properties to return
   */
  return {
    style: { opacity: animation },
    start,
  };
};
```

repository: tongxunlu/rn-placeholder

repo_name: tongxunlu/rn-placeholder


text:
# [Macbeth Comprehension Questions](http://alexa.amazon.com/#skills/amzn1.echo-sdk-ams.app.742a7c10-236a-4e39-b8b3-04b7fde8fc8d)
![0 stars](../../images/ic_star_border_black_18dp_1x.png)![0 stars](../../images/ic_star_border_black_18dp_1x.png)![0 stars](../../images/ic_star_border_black_18dp_1x.png)![0 stars](../../images/ic_star_border_black_18dp_1x.png)![0 stars](../../images/ic_star_border_black_18dp_1x.png) 0

To use the Macbeth Comprehension Questions skill, try saying...

* *Alexa ask macbeth comprehension questions*

* *my answer is one*

* *i don't know*

This skill is a short seven question comprehension quiz of Macbeth by Shakespeare. This skill will provide you with a question, and from four given answer choices, select the best answer to the question by saying "my answer is" and the number of your selected answer choice.
 No additional hardware is required. Initiate this skill by saying "Alexa ask macbeth comprehension questions"

***

### Skill Details

* **Invocation Name:** macbeth comprehension questions
* **Category:** null
* **ID:** amzn1.echo-sdk-ams.app.742a7c10-236a-4e39-b8b3-04b7fde8fc8d
* **ASIN:** B01EH8QK4U
* **Author:** Cyrus Vachha
* **Release Date:** April 25, 2016 @ 06:09:20
* **In-App Purchasing:** No


text:
This directory contains a copy of the MonoCurses binding from:

http://github.com/mono/mono-curses

The code has diverged from `mono-curses` a it's been leveraged for `Terminal.Gui`'s Curses driver.

gha_language: C#


text:
# ReleaseClusterHostGroup

Call the ReleaseClusterHostGroup operation to remove nodes from an EMR cluster.

## Debugging

[OpenAPI Explorer automatically calculates the signature value. For your convenience, we recommend that you call this operation in OpenAPI Explorer. OpenAPI Explorer dynamically generates the sample code of the operation for different SDKs.](https://api.aliyun.com/#product=Emr&api=ReleaseClusterHostGroup&type=RPC&version=2016-04-08)

## Request parameters

|Parameter|Type|Required|Example|Description|
|---------|----|--------|-------|-----------|
|Action|String|Yes|ReleaseClusterHostGroup|The operation that you want to perform. For API requests using the HTTP or HTTPS URL, this parameter is required. Valid values: ReleaseClusterHostGroup |
|ClusterId|String|Yes|C-D7958B72E59B\*\*\*\*|The ID of the PolarDB cluster. |
|HostGroupId|String|Yes|G-EF460256A55F\*\*\*\*|The ID of the machine Group, which can be obtained by using the ListClusterHostGroup. |
|RegionId|String|Yes|cn-hangzhou|The ID of the region. |
|InstanceIdList|String|No|\["i-bp1bm7y86rscdx1a\*\*\*\*"\]|The IDs of the ECS instances. |

## Response parameters

|Parameter|Command|Example|Description|
|---------|-------|-------|-----------|
|RequestId|String|991B3409-6C8D-48CB-903C-3B9C166E17A8|The ID of the request. |

## Examples

Sample requests

```
http(s)://[Endpoint]/? Action=ReleaseClusterHostGroup
&ClusterId=C-D7958B72E59B****
&HostGroupId=G-EF460256A55F****
&RegionId=cn-hangzhou
&<Common request parameters>
```

Sample success responses

`XML` format

```
<RequestId>991B3409-6C8D-48CB-903C-3B9C166E17A8</RequestId>
```

`JSON` format

```
{
    "RequestId":"991B3409-6C8D-48CB-903C-3B9C166E17A8"
}
```

## Error codes

|HttpCode|Error code|Error message|Description|
|--------|----------|-------------|-----------|
|403|Params.Illegal|The specified parameters are wrongly formed..|The error message returned because the format of the specified parameters is invalid.|
|500|InternalError|The request processing has failed due to some unknown error.|The error message returned because the request processing has failed due to an internal error. Submit a ticket.|
|404|ClusterId.NotFound|The ClusterId provided does not exist in our records.|The error message returned because the specified cluster ID does not exist. Enter a valid value.|
|500|InternalError|The request processing has failed due to some unknown error\(code:5100\)|The error message returned because the request processing has failed due to an internal error. Submit a ticket.|
|403|User.OtherUserResource.NotAllow|It is not allowed to operate other user's resource|The error message returned because you are not authorized to manage the resources of other users.|
|403|User.Account.Abnormal|The User Account maybe is out of service!|The error message returned because the Apsara Stack tenant account is out of service.|
|403|Cluster.Role.NotConfirm|The Role for EMR service to operate is Not confirmed!|The error message returned because the Resource Access Management \(RAM\) role you created for the EMR service is not granted the required permissions. Grant the required permissions and try again.|
|404|ClusterId.NotFound|ClusterId \[%s\] does not exist.|The error message returned because the specified cluster ID does not exist. Make sure that the cluster ID is valid.|
|400|Prepaid.Cluster.Cannot.Release|Prepaid cluster do not support release operation!|The error message returned because the nodes in a subscription cluster cannot be released.|
|400|Cluster.Cannot.Release|Cluster has been release or did not start normally!|The error message returned because the cluster is invalid. Select a valid cluster.|
|400|Cluster.RegionId.NotMatch|Cluster does not exist in region \[%s\]!|The error message returned because the cluster does not belong to the region. Switch to the region to which the cluster belongs.|

For a list of error codes, visit the [API Error Center](https://error-center.alibabacloud.com/status/product/Emr).

repository: waitinfuture/emapreduce

repo_name: waitinfuture/emapreduce


text:
## 1.0.0 (2020-02-14)

#### :rocket: Enhancement

- `git-log-scraper`, `git-temporal-react`, `git-temporal-vscode`
  - [#50](https://github.com/git-temporal/git-temporal/pull/50) Closes [#21](https://github.com/git-temporal/git-temporal/issues/21) - maintain scroll position ([@littlebee](https://github.com/littlebee))
- `git-temporal-react`
  - [#49](https://github.com/git-temporal/git-temporal/pull/49) Closes [#41](https://github.com/git-temporal/git-temporal/issues/41) - do not show files on commit card if file selected ([@littlebee](https://github.com/littlebee))

#### :bug: Bug Fix

- `git-temporal-react`
  - [#48](https://github.com/git-temporal/git-temporal/pull/48) Closes [#42](https://github.com/git-temporal/git-temporal/issues/42) - selected card in side panel hard to read ([@littlebee](https://github.com/littlebee))
  - [#46](https://github.com/git-temporal/git-temporal/pull/46) fixes several bugs (#44 & #32) and console warnings ([@littlebee](https://github.com/littlebee))
  - [#45](https://github.com/git-temporal/git-temporal/pull/45) Closes [#27](https://github.com/git-temporal/git-temporal/issues/27) - clicking in timeplot when zoomed should not reset zoom ([@littlebee](https://github.com/littlebee))

#### Committers: 1

- Bee Wilkerson ([@littlebee](https://github.com/littlebee))

## 0.5.2 (2020-02-01)

This is an internal version bump with automated testing enhancements only.

**NOTE**: no update published to the VSCode extension for this version bump.

#### :rocket: Enhancement

- `git-diff-scraper`, `git-log-scraper`, `git-temporal-react`
  - [#33](https://github.com/git-temporal/git-temporal/pull/33) Convert git-log-scraper and git-diff-scraper to Jest ([@littlebee](https://github.com/littlebee))
- `git-temporal-react`
  - [#29](https://github.com/git-temporal/git-temporal/pull/29) Fix unit tests in git-temporal-react ([@littlebee](https://github.com/littlebee))

#### Committers: 1

- Bee Wilkerson ([@littlebee](https://github.com/littlebee))

## 0.5.1 (2020-01-19)

#### :rocket: Enhancement

- [#20](https://github.com/git-temporal/git-temporal/pull/20) Add CHANGELOG.md generated by lerna-changelog ([@littlebee](https://github.com/littlebee))

#### Committers: 1

- Bee Wilkerson ([@littlebee](https://github.com/littlebee))

## 0.5.0 (2020-01-18)

#### :rocket: Enhancement

- `git-temporal-react`
  - [#18](https://github.com/git-temporal/git-temporal/pull/18) :star: Commit selection and display improvements ([@littlebee](https://github.com/littlebee))
- `git-temporal-vscode`
  - [#8](https://github.com/git-temporal/git-temporal/pull/8) Add publish scripts and update README with latest vsix download ([@littlebee](https://github.com/littlebee))
- Other
  - [#7](https://github.com/git-temporal/git-temporal/pull/7) Prepare for 0.4.1 release ([@littlebee](https://github.com/littlebee))

#### :bug: Bug Fix

- `git-temporal-vscode`
  - [#19](https://github.com/git-temporal/git-temporal/pull/19) GTVscode: Add user friendly display name and gallery banner ([@littlebee](https://github.com/littlebee))
- `git-temporal-react`
  - [#14](https://github.com/git-temporal/git-temporal/pull/14) Closes [#11](https://github.com/git-temporal/git-temporal/issues/11) - reset zoom when reseting date range ([@littlebee](https://github.com/littlebee))
  - [#12](https://github.com/git-temporal/git-temporal/pull/12) Closes [#10](https://github.com/git-temporal/git-temporal/issues/10) - Right diff header reports uncommited changes incorrectly ([@littlebee](https://github.com/littlebee))
- `git-diff-scraper`, `git-temporal-react`, `git-temporal-vscode`
  - [#9](https://github.com/git-temporal/git-temporal/pull/9) Fix issue with dir diff being too greedy ([@littlebee](https://github.com/littlebee))

#### Committers: 1

- Bee Wilkerson ([@littlebee](https://github.com/littlebee))

##

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [](#)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

##

repository: git-temporal/git-temporal

gha_language: TypeScript

repo_name: git-temporal/git-temporal


text:
---
layout: page
title: About Me
weight: 0
---

Hi! I am a fourth year PhD student in the Department of Computer Science at the  [University of California, Irvine](https://www.ics.uci.edu/). My Google Scholar

My research focuses on Inference in Probabilistic Graphical Models, advised by Prof.  [Erik Sudderth](https://www.ics.uci.edu/~sudderth/). You can visit my google scholar profile [here](https://scholar.google.com/citations?hl=en&user=uG2w1coAAAAJ). With the help of deep learning techniques, I am interested in developing models capable to reason or infer in images and texts.

Before graduate school, I was a Budding Scientist at [Conduent Labs](https://www.conduent.com/innovation/) (formerly known as Xerox Research Centre India). I received a Bachelors in Electrical Engineering (Instrumentation) with a minor in Computer Science Engineering from  [Indian Institute of Technology, Kharagpur]([http://www.iitkgp.ac.in/](http://www.ee.iitkgp.ac.in/)) in 2017. During that period, I was also a visiting researcher at [Duke University](https://hal.pratt.duke.edu/) and an intern at [TCS Innovation Labs](https://www.tcs.com/research-and-innovation).

When I'm not working, I like to spend my time cooking, dancing, learning about cultures and taking short trips outside the city!

You can reach me at<EMAIL_ADDRESS>or visit my [LinkedIn](https://www.linkedin.com/in/sakshi-agarwal-6a8b6186) profile. A short description of my works is present in my [CV](Sakshi.pdf)


text:
---
title: "Incremental Semantic Judgments"
collection: publications
permalink: /publication/2019-05-16-abstract-title-number-3
excerpt: 'We discuss vector semantics for Dynamic Syntax and present empirical results on incremental disambiguation experiments.'
type: abstract
date: 2019-05-16
year: 2019
venue: 'Third Dynamix Syntax Conference'
paperurl: 'https://drive.google.com/file/d/1hblEWmlc2CVu_tQ62vgKVcRh6PqHzgma/view'
citation: 'Purver, M, and Sadrzadeh, M, and Wijnholds, G, and Kempson, R, and Hough, J. "Incremental Semantic Judgments." <i>Third Dynamic Syntax Conference. 2019</i>.'
---
We discuss vector semantics for Dynamic Syntax and present empirical results on incremental disambiguation experiments.

gha_language: HTML


text:
CRUD app with Angular 7 and PrimeNG 7

    This example app shows how to build a basic CRUD app with Angular 7.0, PrimeNG and localstorage.

## Contents

- [Quick start](#quick-start)
- [What's included](#whats-included)
- [Authors](#authors)
- [License](#license)

## Quick start

**Warning**

> Verify that you are running at least node 8.9.x and npm 5.x.x by running node -v and npm -v in a terminal/console window. Older versions produce errors, but newer versions are fine.

1. Go to project folder and install dependencies.
 ```bash
 npm install
 ```

2. Launch development server:
 ```bash
 npm start
 ```

 3. Open aplication in browser:
 ```bash
 http://localhost:4200/
 ```

## What's included

* CRUD: create, update and remove items with localstorage!
* Search bar, to look for items
* Modal Confirm (PrimeNG ConfirmDialog)
* Toasts (PrimeNG Toast)
* Responsive layout (PrimeNG Grid)
* Datatable (PrimeNG TurboTable)

## Authors

* **Wilson Filho**  - [Linkedin](https://www.linkedin.com/in/wilson-filho-4424b5bb)

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details


text:
Original docs: [https://github.com/stffn/declarative_authorization/blob/master/README.rdoc](https://github.com/stffn/declarative_authorization/blob/master/README.rdoc)

Extends Declarative Authorization with Rails 5.2 and Ruby 2.5 support, aiming for backwards compatibility as far as Rails 4.2 and Ruby 2.2 on a best-effort basis.

Branch r5: [![Build Status](https://travis-ci.org/Xymist/declarative_authorization.svg?branch=r5)](https://travis-ci.org/Xymist/declarative_authorization)

### Running tests for DA

```
cp gemfiles/{RAILS_VERSION}.gemfile Gemfile
bundle

bundle exec rake test
```

gha_language: Ruby


text:
# Outline Logic Tier

This is the logic tier for the Outline Braiiin application.

# Getting Started

Here is how to setup a local instance.

2. Check that needed commands are accessible `source check.sh`.
3. Install `source install.sh`.
4. Run server `source activate.sh`.
5. Browse[Logic Core Documentation](http: // logic - braiiin.readthedocs.org). (coming soon)
6. Read the[Guidelines](https: // github.com / Braiiin / template - client  # guidelines).
7. If you are uncertain of where to start, see[How to get Working](https: // github.com / Braiiin / template - client  # how-it-works) below.

> To update your installation of the logic core, use:
 1. `git submodule foreach git reset head - -hard` to remove any inadvertent changes.
 2. `git submodule foreach git pull origin master` to update the submodules.

repository: Braiiin/outline-logic

repo_name: Braiiin/outline-logic


text:
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.2.1] - 2020-08-18
### Security
- Updated and audited npm dependencies.

## [1.2.0] - 2020-01-18
### Added
- Example page for viewing the component and its events in action.
- Instructions in readme for launching the demo locally.
### Changed
- Switched package entry point directly to Vue component, removing the need for a
build process.

## [1.1.1] - 2020-01-09
### Security
- Updated Vue CLI to resolve several vulnerabilities in older dependencies.

## [1.1.0] - 2020-01-09
### Added
- Documentation about handling measurement events via the containing map object.
### Changed
- Updated Leaflet.PolylineMeasure to version 2019-11-15, which includes additional
events as well as various bug fixes.

## [1.0.0] - 2019-04-25
### Added
- This changelog.
### Changed
- Updated the README.
### Removed
- No longer emits a `ready` event on mount.

## [0.0.3] - 2019-04-24
### Added
- CSS import from Leaflet.PolylineMeasure to correctly style buttons and tooltips.

gha_language: Vue


text:
# CHANGELOG

# 0.12.0 beta 1

TL;DR Support for binding with TableView was added. Some important features are missing and will be added in beta 2.

## Added

- [Collection] DataSource protocol for table views. It is very similar to the one provided by Apple while emphasis it
- [Collection] View Models protocols to interact with a collection dataset.
- [Collection] TableView templates to register ```ComponentViewModel``` instances on ```UITableView```
- [Collection] TableView layout system
- [Binding] Added `bindTo` on optional `Command`

## Enhancements

- Renamed ```ViewModel``` as ```ComponentViewModel``` for consistency with other related protocols (```ComponentView``` and ```ComponentController```)

# 0.12.0 alpha 2

Akane is now compatible with Xcode 7.1+.

## Added

## Enhancements

- All protocols were re-written in Swift
- ViewComponent was renamed ComponentView
- Presenter was renamed ComponentController

# 0.12.0 alpha

0.12.0 requires iOS8+ and Swift 2.0+.

Also note that due to a bug into Cocoapods 0.39, Akane is not compatible with Xcode 7.1+.

## Added

- "Native" binding. Use the new method ```bindings:viewModel``` from ```ViewComponent``` protocol.
- Command pattern support to communicate between View and ViewModel. Use ```RelayCommand``` or a custom one with ```Command``` protocol.
- Binding converters.

## Enhancements

- Removed deprecated methods. Some are still available and will be removed in future beta releases.
- Made Akane compatible with Swift only.

repository: smallkot/Akane

repo_name: smallkot/Akane


text:
# Pandoc Citer Extension for Visual Studio Code

This extension provides autocompletion of citations stored in a bibtex file, for use with [pandoc-markdown](https://pandoc.org/) documents, along with auto-completion for [pandoc-crossref](https://github.com/lierdakil/pandoc-crossref) cross-references.

## Features

### Autocomplete citations

![Example of citation autocomplete](imgs/exmplCiter.gif)

- Citations follow the standard syntax recognized by [pandoc-citeproc](https://github.com/jgm/pandoc-citeproc), i.e.:
    - Citation syntax: `@BibKey` or `[@bibKey]`
        + Typing '@' should automatically trigger the suggestion window to show up. If not, try `Ctrl + Space`.
        + The suggestion window (probably) won't trigger if the `@` is right after some text without a space, i.e. `text@` won't trigger the suggestions but `text @` will. This restriction was made so that typing an email address won't trigger the citation suggestions.
    - The YAML should contain the line
    ```
    bibliography: [path/to/.bib]
    ```
    - The path/to/.bib can either be a single path, a list of paths enclosed by square brackets, or a bullet list of paths
    - The path/to/.bib can be either a relative or absolute path
    - Multiple bib files can be included if separated by comma, e.g. `[path/to/refsA.bib, path/to/refsB.bib]`

## Configuration

- `PandocCiter.CrossRefMode`
    + Type: String, either "full" (default), "minimal", or "none"
    + Function: This changes the detail shown during auto-completion of cross-references. Setting this to "none" will disable auto-completion of pandoc-crossref references.
- `PandocCiter.ViewType`
    + Type: String, either "inline" (default) or "browser"
    + Function: Changes how the suggested citations are shown
- `PandocCiter.RootFile`
    + Type: String, path to markdown file
    + Function: This is useful if there are multiple markdown files with one external master file containing the YAML header with the `bibliography: [path/to/bib]` entry. This avoids having to insert the bibliography YAML header into each individual file.
- `PandocCiter.DefaultBibs`
    + Type: Array of Strings, absolute or relative path to bib file
        - Note: For relative paths, it will be assumed that the path is relative to the workspace folder
    + Function: This is useful if there is a default bib file that is preferred for all projects. This bib file will be loaded regardless of whether or not a YAML entry is included.
- `PandocCiter.DefaultBib`
    + Same as `PandocCiter.DefaultBibs`, except this is older and only accepts a single path as a string. Not recommended for new users. This only exists to avoid breaking older setups.
- `PandocCiter.UseDefaultBib`
    + Type: Boolean, default is `true`
    + Function: This should be a project-specific setting. It is useful when a DefaultBib is defined but you don't want it to be used for a specific project.
- `PandocCiter.ForgetUnusedBib`
    + Type: Boolean, default is `true`
    + Function: This will check if the bibliography YAML matches the watched bib files, and if any watched files are not in the YAML, then their suggestions will be removed.
        + For example, suppose two markdown files are opened but only one of them has a bibliography entry in their YAML header. If this config is set to `false`, then both markdown files will show citation suggestions. If this config is set to `true`, then citation suggestions will only be shown in the file with the bibliography YAML.
- `PandocCiter.ShowLog`
    + Type: Boolean, default is `false`
    + Function: This will show a log in the Output panel, useful for answering questions like "Why isn't the extension finding my bib file?". The Output panel can be accessed via `VSCode Top Menubar -> View -> Output` which will open a panel at the bottom. Select `PandocCiter` from the drop down list, and the extension's log should show up.
- `PandocCiter.CitationFormat`
    + Type: String array
    + Function: Controls which bibtex fields will be shown in the suggestions window. These items will also be used to filter the suggestions during autocompletion.

## Acknowledgements

- This extension started out as a a stripped down version of the [LaTeX-Workshop extension](https://github.com/James-Yu/LaTeX-Workshop) that has been adjusted for markdown/pandoc
    + The structure of the `src` directory was preserved, so that any updates/fixes here could also be ported to LaTex-Workshop, and vice versa
- Contributors:
    - [@smartens](https://github.com/smartens) - fixing bugs
    - [@yarray](https://github.com/yarray) - adding pandoc-crossref support
    - [@Dominic-DallOsto](https://github.com/Dominic-DallOsto) - adding hover and go-to-definition support
    - [@MaxChang3](https://github.com/MaxChang3) - improving hover support

repository: notZaki/PandocCiter

repo_name: notZaki/PandocCiter


text:
# Creating a MongoDb account

## Prerequisites

Gmail account.

## Sign up

Visit https://account.mongodb.com/account/register .

Choose Sign up with Google

![](https://i.imgur.com/aO7RQjP.png)

Choose the Gmail account you want to use for signing up.

Accept privacy policy and terms of service

![](https://i.imgur.com/O4GxgLS.png)

You will be asked to name your organization and project ( in lack of better idea it is safe to accept the defaults )

![](https://i.imgur.com/hSADaqk.png)

Select your preferred programming language, then press Continue

![](https://i.imgur.com/vb9fRbA.png)

Create a FREE cluster

![](https://i.imgur.com/qr6Wxcv.png)

Edit your account details ( it is safe to accept defaults ), then press Create Cluster

![](https://i.imgur.com/j7vWBUi.png)

## Setting up database

From the Get Started guide select Create your first database user

![](https://i.imgur.com/FmjY0L4.png)

Click on Database Access

![](https://i.imgur.com/v0K5h71.png)

Click on Add New Database User

![](https://i.imgur.com/tSvDTwq.png)

For Authentication Method select "Password", and enter a username ( consisting of only small cap letters, don't use any special characters, **`YoBot_v2`** not good, **`yobotversiontwo`** good ) and password ( in the image a password containing special characters was typed in manually, which is not a good idea, because this needs url encoding due to special characters, so it is safest to press **Autogenerate Secure Password** and SHOW / Copy this auto generated password )

![](https://i.imgur.com/F5WigqF.png)

From Database User Privileges select Atlas admin, then press Add User

![](https://i.imgur.com/0n1zVVi.png)

You can add further users to your database by pressing ADD NEW DATABASE USER, but we will be content with one user

![](https://i.imgur.com/pjcDowD.png)

Click on Network Access

![](https://i.imgur.com/UPaLOOK.png)

Click on Add IP Address

![](https://i.imgur.com/8KbHrXM.png)

Select Allow access from anywhere then press Confirm

![](https://i.imgur.com/NEAYcuj.png)

From the Get Started guide select Connect to your cluster, then press Connect

![](https://i.imgur.com/a4kIDHq.png)

## Obtaining MONGODB_URI

Select Connect your application

![](https://i.imgur.com/Uflg9Y2.png)

Copy your connection string and replace the password with your database user's password, delete everything from /dbname onward, you won't need this.

![](https://i.imgur.com/6sPNvSA.png)

If you copied

```
mongodb+srv://hyperchessbot:<password>@cluster0.bv7tb.mongodb.net/<dbname>?retryWrites=true&w=majority
```

edit it to

```
<EMAIL_ADDRESS>```

and save this as a Heroku config var under the key `MONGODB_URI` .

Of course instead of hyperchessbot you need your actual database user name ( but this will be filled in for you, when you copy ).

**There should be no forward slash at the end of `MONGODB_URI`!**

repository: Ganiduyapa/hyperbot

repo_name: Ganiduyapa/hyperbot


text:
#The arraignment, tryal, conviction and condemnation of Henry Harrison, Gent. for the barbarous murther of Andrew Clenche, of Brownelowe-Street, in the parish of St. Andrew's Holborne, in the county of Middlesex, Doctor of physick at the sessions of the goal-delivery of Newgate, holden for the city of London, at Justice-Hall in the Old-Baily, on the 6th, 7th, and 9th days of April, 1692, in the fourth year of Their Majesties reign : and also the examination of the said Henry Harrison, taken the 6th day of January, 1691, before the Right Honourable Sir John Holt, Knight, Lord Chief Justice of Their Majesties Court of King's and Queen's Bench at Westminster : to which is also added, The tryal of John Cole, for the murther of the said Doctor Clenche.#

##Harrison, Henry, d. 1692, defendant.##
The arraignment, tryal, conviction and condemnation of Henry Harrison, Gent. for the barbarous murther of Andrew Clenche, of Brownelowe-Street, in the parish of St. Andrew's Holborne, in the county of Middlesex, Doctor of physick at the sessions of the goal-delivery of Newgate, holden for the city of London, at Justice-Hall in the Old-Baily, on the 6th, 7th, and 9th days of April, 1692, in the fourth year of Their Majesties reign : and also the examination of the said Henry Harrison, taken the 6th day of January, 1691, before the Right Honourable Sir John Holt, Knight, Lord Chief Justice of Their Majesties Court of King's and Queen's Bench at Westminster : to which is also added, The tryal of John Cole, for the murther of the said Doctor Clenche.
Harrison, Henry, d. 1692, defendant.

##General Summary##

**Links**

[TCP catalogue](http://www.ota.ox.ac.uk/tcp/)  •
[HTML](http://tei.it.ox.ac.uk/tcp/Texts-HTML/free/A25/A25881.html)  •
[EPUB](http://tei.it.ox.ac.uk/tcp/Texts-EPUB/free/A25/A25881.epub) •
[Page images (Historical Texts)](https://data.historicaltexts.jisc.ac.uk/view?pubId=eebo-12937056e&pageId=eebo-12937056e-95781-1)

**Availability**

This keyboarded and encoded edition of the
	       work described above is co-owned by the institutions
	       providing financial support to the Early English Books
	       Online Text Creation Partnership. This Phase I text is
	       available for reuse, according to the terms of Creative
	       Commons 0 1.0 Universal. The text can be copied,
	       modified, distributed and performed, even for
	       commercial purposes, all without asking permission.

**Major revisions**

1. __2006-01__ __TCP__ *Assigned for keying and markup*
1. __2006-02__ __Apex CoVantage__ *Keyed and coded from ProQuest page images*
1. __2006-03__ __Emma (Leeson) Huber__ *Sampled and proofread*
1. __2006-03__ __Emma (Leeson) Huber__ *Text and markup reviewed and edited*
1. __2006-04__ __pfs__ *Batch review (QC) and XML conversion*

##Content Summary##

#####Front#####
I Do Appoint William Battersby and Thomas Braddyll to Print the ſeveral Trials of Henry Harriſon, Ge
1. TO THE READER.

**Types of content**

  * There are 546 **drama** parts! This is **prose drama**.
  * Oh, Mr. Jourdain, there is **prose** in there!

There are 11 **ommitted** fragments!
 @__reason__ (11) : illegible (10), blank (1)  •  @__resp__ (10) : #OXF (10)  •  @__extent__ (11) : 1 letter (8), 2 letters (1), 1 word (1), 3 letters (1)

**Character listing**

|Text|string(s)|codepoint(s)|
|---|---|---|
|Latin Extended-A|ſ|383|
|Latin Extended-B|Ʋ|434|
|General Punctuation|•—|8226 8212|
|Geometric Shapes|▪|9642|

##Tag Usage Summary##

###Header Tag Usage###

|No|element name|occ|attributes|
|---|---|---|---|
|1.|__author__|3||
|2.|__availability__|1||
|3.|__biblFull__|1||
|4.|__change__|5||
|5.|__date__|8| @__when__ (1) : 2006-06 (1)|
|6.|__edition__|1||
|7.|__editionStmt__|1||
|8.|__editorialDecl__|1||
|9.|__extent__|2||
|10.|__idno__|6| @__type__ (6) : DLPS (1), STC (2), EEBO-CITATION (1), OCLC (1), VID (1)|
|11.|__keywords__|1| @__scheme__ (1) : http://authorities.loc.gov/ (1)|
|12.|__label__|5||
|13.|__langUsage__|1||
|14.|__language__|1| @__ident__ (1) : eng (1)|
|15.|__listPrefixDef__|1||
|16.|__note__|5||
|17.|__notesStmt__|2||
|18.|__p__|11||
|19.|__prefixDef__|2| @__ident__ (2) : tcp (1), char (1)  •  @__matchPattern__ (2) : ([0-9\-]+):([0-9IVX]+) (1), (.+) (1)  •  @__replacementPattern__ (2) : http://eebo.chadwyck.com/downloadtiff?vid=$1&page=$2 (1), https://raw.githubusercontent.com/textcreationpartnership/Texts/master/tcpchars.xml#$1 (1)|
|20.|__projectDesc__|1||
|21.|__pubPlace__|2||
|22.|__publicationStmt__|2||
|23.|__publisher__|2||
|24.|__ref__|2| @__target__ (2) : https://creativecommons.org/publicdomain/zero/1.0/ (1), http://www.textcreationpartnership.org/docs/. (1)|
|25.|__seriesStmt__|1||
|26.|__sourceDesc__|1||
|27.|__term__|4||
|28.|__textClass__|1||
|29.|__title__|3||
|30.|__titleStmt__|2||

###Text Tag Usage###

|No|element name|occ|attributes|
|---|---|---|---|
|1.|__body__|5||
|2.|__closer__|5||
|3.|__date__|6||
|4.|__dateline__|5||
|5.|__desc__|11||
|6.|__div__|7| @__type__ (7) : privilege (1), title_page (2), to_the_reader (1), trial (1), examination (1), text (1)|
|7.|__floatingText__|3| @__type__ (3) : letter (3)|
|8.|__front__|1||
|9.|__g__|224| @__ref__ (224) : char:EOLhyphen (216), char:punc (7), char:V (1)|
|10.|__gap__|11| @__reason__ (11) : illegible (10), blank (1)  •  @__resp__ (10) : #OXF (10)  •  @__extent__ (11) : 1 letter (8), 2 letters (1), 1 word (1), 3 letters (1)|
|11.|__head__|10||
|12.|__hi__|1413| @__rend__ (2) : sup (2)|
|13.|__item__|29||
|14.|__list__|4||
|15.|__opener__|5||
|16.|__p__|676||
|17.|__pb__|49| @__facs__ (49) : tcp:95781:1 (2), tcp:95781:2 (2), tcp:95781:3 (2), tcp:95781:4 (2), tcp:95781:5 (2), tcp:95781:6 (2), tcp:95781:7 (2), tcp:95781:8 (2), tcp:95781:9 (2), tcp:95781:10 (2), tcp:95781:11 (2), tcp:95781:12 (2), tcp:95781:13 (2), tcp:95781:14 (2), tcp:95781:15 (2), tcp:95781:16 (2), tcp:95781:17 (2), tcp:95781:18 (2), tcp:95781:19 (2), tcp:95781:20 (2), tcp:95781:21 (2), tcp:95781:22 (2), tcp:95781:23 (2), tcp:95781:24 (2), tcp:95781:25 (1)  •  @__n__ (44) : 1 (1), 2 (1), 3 (1), 4 (1), 5 (1), 6 (1), 7 (1), 8 (1), 9 (1), 10 (1), 11 (1), 12 (1), 13 (1), 14 (1), 15 (1), 16 (1), 17 (1), 18 (1), 19 (1), 20 (1), 21 (1), 22 (1), 23 (1), 24 (1), 25 (1), 26 (1), 27 (1), 28 (1), 29 (1), 30 (1), 31 (1), 32 (1), 33 (1), 34 (1), 37 (1), 38 (1), 39 (1), 40 (1), 41 (1), 42 (1), 43 (1), 44 (1), 45 (1), 46 (1)|
|18.|__salute__|3||
|19.|__signed__|5||
|20.|__sp__|546||
|21.|__speaker__|537||
|22.|__text__|2| @__xml:lang__ (2) : eng (0)|
|23.|__trailer__|1||