text:
*This codelab was made in collaboration with [Google Genomics](https://github.com/googlegenomics).
New [Standard SQL](https://cloud.google.com/bigquery/docs/reference/standard-sql/)
versions of many of these queries can be found
[here](https://github.com/googlegenomics/codelabs/tree/master/R/PlatinumGenomes-QC/sql).*

# Part 2: Sample-Level QC

In Parts 2 & 3 of the codelab we describe the details of each quality control step implemented on our population of genomes.  In Part 2 we perform some quality control analyses that could help to identify any problematic genomes that should be removed from the cohort before proceeding with further analysis.  The appropriate cut off thresholds will depend upon the input dataset and/or other factors.

To skip the details and jump to the implementation of the quality control methods skip to [Part 4: QC Implementation](./QC-Implementation.md).
* [Setup](#setup)
* [Missingness Rate](#missingness-rate)
* [Singleton Rate](#singleton-rate)
* [Inbreeding Coefficient](#inbreeding-coefficient)
* [Heterozygosity Rate](#heterozygosity-rate)
* [Sex Inference](#sex-inference)
* [Genotyping Concordance](#genotyping-concordance)
* [Batch Effect](#batch-effect)
* [Ethnicity Inference](#ethnicity-inference)
* [Genome Similarity](#genome-similarity)

## Setup

```r
tableReplacement <- list("_THE_TABLE_"="va_aaa_pilot_data.genome_calls_seq_qc",
                          "_THE_EXPANDED_TABLE_"="va_aaa_pilot_data.multi_sample_variants_seq_qc",
                          "_GENOTYPING_TABLE_"="va_aaa_pilot_data.genotyping_data")

ibs <- read.table("./data/all-genomes-ibs.tsv",
                  col.names=c("sample1", "sample2", "ibsScore", "similar", "observed"))

sampleData <- read.csv("./data/patient_info.csv")
sampleInfo <- select(sampleData, call_call_set_name=Catalog.ID, gender=Gender)
```

ggplot2 themes to use throughout this page.

```r
plot_theme = theme_minimal(base_size = 14, base_family = "Helvetica") +
  theme(axis.line = element_line(colour = "black"),
        panel.grid = element_blank())

boxPlotTheme = theme_minimal(base_size=14, base_family = "Helvetica") +
  theme(panel.grid = element_blank())
```

## Missingness Rate

Missingess is defined as the proportion of sites found in the reference genome that are not called in a given genome. We calculate the missingness rate of each genome in our cohort in order to identify samples that are potentially low quality.  If a sample has a high missingness rate it may be indicative of issues with sample preparation or sequencing.  Genomes with a missingness rate greater than 0.1 are removed from the cohort.

```r
result <- DisplayAndDispatchQuery("./sql/missingness-sample-level.sql",
                                  project=project,
                                  replacements=tableReplacement)
```

```
# Determine the level of missingness for each sample when compared to the hg19 reference genome
SELECT
  g.sample_id AS sample_id,
  ROUND(((hg19.count - g.all_calls_count)/hg19.count), 3) AS missingness
FROM (
  SELECT
    call.call_set_name AS sample_id,
    ref_count + alt_count AS all_calls_count,
  FROM (
    SELECT
      call.call_set_name,
      SUM(IF(genotype = '0,0',
        (end - start),
        0)) AS ref_count,
      SUM(IF(genotype NOT IN ('0,0', '-1,-1'),
        1,
        0)) AS alt_count,
    FROM (
    SELECT
      call.call_set_name,
      start,
      end,
      GROUP_CONCAT(STRING(call.genotype)) WITHIN call AS genotype
    FROM
      [va_aaa_pilot_data.genome_calls_seq_qc])
    OMIT call IF call.call_set_name = 'LP6005243-DNA_A08'
    GROUP BY
      call.call_set_name)) AS g
  CROSS JOIN (
    SELECT
      COUNT(Chr) AS count
    FROM
      [google.com:biggene:test.hg19]) AS hg19
ORDER BY
  missingness DESC
```
Number of rows returned by this query: 459.

Displaying the first few results:

  sample_id 	  missingness

  LP6005038-DNA_C02 	  0.06

  LP6005692-DNA_D09 	  0.05

  LP6005038-DNA_D02 	  0.05

  LP6005051-DNA_D07 	  0.05

  LP6005243-DNA_D01 	  0.05

  LP6005243-DNA_C01 	  0.05

And visualizing the results:

```r
ggplot(result) +
  geom_point(aes(x=sample_id, y=missingness)) +
  xlab("Sample") +
  ylab("Missingness Rate") +
  ggtitle("Genome-Specific Missingness") +
  plot_theme +
  theme(axis.ticks = element_blank(),
        axis.text.x = element_blank())
```

## Singleton Rate

Singleton rate is defined as the number of variants that are unique to a genome.  If a variant is found in only one genome in the cohort it is considered a singleton.  Genomes with singleton rates more than 3 standard deviations away from the mean are removed from the cohort.

```r
result <- DisplayAndDispatchQuery("./sql/private-variants.sql",
                                  project=project,
                                  replacements=tableReplacement)
```

```
# Compute private variants counts for each sample.
SELECT
  call.call_set_name,
  COUNT(call.call_set_name) AS private_variant_count,
FROM (
  SELECT
    reference_name,
    start,
    GROUP_CONCAT(CASE WHEN cnt = 1 THEN 'S'
      WHEN cnt = 2 THEN 'D'
      ELSE STRING(cnt) END) AS SINGLETON_DOUBLETON,
    reference_bases,
    alternate_bases,
    GROUP_CONCAT(call.call_set_name) AS call.call_set_name,
    GROUP_CONCAT(genotype) AS genotype,
    SUM(num_samples_with_variant) AS num_samples_with_variant
  FROM (
    SELECT
      reference_name,
      start,
      reference_bases,
      alternate_bases,
      alt_num,
      call.call_set_name,
      GROUP_CONCAT(STRING(call.genotype)) WITHIN call AS genotype,
      SUM(call.genotype == alt_num) WITHIN call AS cnt,
      COUNT(call.call_set_name) WITHIN RECORD AS num_samples_with_variant
    FROM (
        FLATTEN((
          SELECT
            reference_name,
            start,
            reference_bases,
            alternate_bases,
            POSITION(alternate_bases) AS alt_num,
            call.call_set_name,
            call.genotype,
          FROM
            [va_aaa_pilot_data.genome_calls_seq_qc]
          # Optionally add a clause here to limit the query to a particular
          # region of the genome.
          #_WHERE_
          OMIT call IF EVERY(call.genotype = -1)
        ), alternate_bases)
        )
    OMIT RECORD IF alternate_bases IS NULL
    HAVING
      cnt > 0
      )
    GROUP EACH BY
      reference_name,
      start,
      reference_bases,
      alternate_bases
  HAVING
    num_samples_with_variant = 1
    )
GROUP BY
  call.call_set_name
#_ORDER_BY_
```
Number of rows returned by this query: 460.

Displaying the first few results:

  call_call_set_name 	  private_variant_count

  LP6005038-DNA_C05 	  67704

  LP6005692-DNA_G01 	  26217

  LP6005144-DNA_D08 	  23597

  LP6005692-DNA_F11 	  23547

  LP6005051-DNA_E04 	  47140

  LP6005144-DNA_G08 	  26134

And visualizing the results:

```r
ggplot(result) +
  geom_point(aes(x=call_call_set_name, y=private_variant_count)) +
  xlab("Sample") +
  ylab("Number of Singletons") +
  ggtitle("Count of Singletons Per Genome") +
  scale_y_continuous(labels=comma) +
  plot_theme +
  theme(axis.ticks = element_blank(),
        axis.text.x = element_blank())
```

## Inbreeding Coefficient

The inbreeding coefficient (F) is a measure of expected homozygosity rates vs observed homozygosity rates for individual genomes.  Here, we calculate the inbreeding coefficient using the method-of-moments estimator.  Genomes with an inbreeding coefficient more than 3 standard deviations away from the mean are removed from the cohort.

```r
result <- DisplayAndDispatchQuery("./sql/homozygous-variants.sql",
                                  project=project,
                                  replacements=tableReplacement)
```

```
# Compute the expected and observed homozygosity rate for each individual.
SELECT
  call.call_set_name,
  O_HOM,
  ROUND(E_HOM, 2) as E_HOM,
  N_SITES,
  ROUND((O_HOM - E_HOM) / (N_SITES - E_HOM), 5) AS F
FROM (
  SELECT
    call.call_set_name,
    SUM(first_allele = second_allele) AS O_HOM,
    SUM(1.0 - (2.0 * freq * (1.0 - freq) * (called_allele_count / (called_allele_count - 1.0)))) AS E_HOM,
    COUNT(call.call_set_name) AS N_SITES,
  FROM (
    SELECT
      reference_name,
      start,
      reference_bases,
      GROUP_CONCAT(alternate_bases) WITHIN RECORD AS alternate_bases,
      call.call_set_name,
      NTH(1, call.genotype) WITHIN call AS first_allele,
      NTH(2, call.genotype) WITHIN call AS second_allele,
      COUNT(alternate_bases) WITHIN RECORD AS num_alts,
      SUM(call.genotype >= 0) WITHIN RECORD AS called_allele_count,
      IF((SUM(1 = call.genotype) > 0),
        SUM(call.genotype = 1)/SUM(call.genotype >= 0),
        -1)  WITHIN RECORD AS freq
    FROM
      [va_aaa_pilot_data.multi_sample_variants_seq_qc]
    # Optionally add a clause here to limit the query to a particular
    # region of the genome.
    #_WHERE_
    # Skip no calls and haploid sites
    OMIT call IF SOME(call.genotype < 0) OR (2 > COUNT(call.genotype))
    HAVING
      # Skip 1/2 genotypes _and non-SNP variants
      num_alts = 1
      AND reference_bases IN ('A','C','G','T')
      AND alternate_bases IN ('A','C','G','T')
      )
  WHERE
    freq > 0
  GROUP BY
    call.call_set_name
    )
#_ORDER_BY_
```
Number of rows returned by this query: 460.

Displaying the first few results:

  call_call_set_name 	  O_HOM 	  E_HOM 	  N_SITES 	  F

  LP6005051-DNA_E03 	  24036439 	  23978412.34 	  26185742 	  0.03

  LP6005243-DNA_E08 	  24085065 	  24081483.69 	  26310910 	  0.00

  LP6005692-DNA_F04 	  24006207 	  24002414.10 	  26224203 	  0.00

  LP6005038-DNA_A05 	  23925478 	  23955837.74 	  26170707 	  -0.01

  LP6005144-DNA_A10 	  24036795 	  23965529.58 	  26182425 	  0.03

  LP6005144-DNA_C11 	  23953076 	  23958021.31 	  26172270 	  -0.00

And visualizing the results:

```r
limits <- c(min(result$O_HOM, result$E_HOM),
            max(result$O_HOM, result$E_HOM))

ggplot(result) +
  geom_point(aes(x=O_HOM, y=E_HOM, label=call_call_set_name), alpha=1/1.5) +
  geom_abline(color="darkslateblue") +
  scale_x_continuous(limits=limits, labels=comma) +
  scale_y_continuous(limits=limits, labels=comma) +
  xlab("Observed Homozygous Variants") +
  ylab("Expected Homozygous Variants") +
  ggtitle("Homozygosity") +
  plot_theme
```

## Heterozygosity Rate

Heterozygosity rate is simply the the number of heterozygous calls in a genome.  Genomes with a heterozygosity rate more than 3 standard deviations away from the mean are removed from the cohort.

```r
result <- DisplayAndDispatchQuery("./sql/heterozygous-calls-count.sql",
                                  project=project,
                                  replacements=tableReplacement)
```

```
  # Count the number of heterozygous calls for each genome in the multi sample variants table.

  SELECT
    call.call_set_name,
    SUM(first_allele != second_allele) AS O_HET
  FROM (
    SELECT
      reference_name,
      start,
      reference_bases,
      GROUP_CONCAT(alternate_bases) WITHIN RECORD AS alternate_bases,
      call.call_set_name,
      NTH(1, call.genotype) WITHIN call AS first_allele,
      NTH(2, call.genotype) WITHIN call AS second_allele,
      COUNT(alternate_bases) WITHIN RECORD AS num_alts
    FROM
      [va_aaa_pilot_data.multi_sample_variants_seq_qc]
    OMIT call IF SOME(call.genotype < 0) OR (2 > COUNT(call.genotype))
    HAVING
      num_alts = 1
      AND reference_bases IN ('A','C','G','T')
      AND alternate_bases IN ('A','C','G','T')
    )
  GROUP BY
    call.call_set_name
  #_ORDER_BY
```
Number of rows returned by this query: 460.

Displaying the first few results:

  call_call_set_name 	  O_HET

  LP6005051-DNA_H06 	  2142264

  LP6005144-DNA_A11 	  2160295

  LP6005144-DNA_H05 	  2130613

  LP6005038-DNA_E02 	  2144217

  LP6005144-DNA_G11 	  2113643

  LP6005692-DNA_D11 	  2111111

And visualizing the results:

```r
ggplot(result) +
  geom_point(aes(y=O_HET, x=call_call_set_name, label=call_call_set_name), alpha=1/1.5) +
  xlab("Sample") +
  ylab("Observed Heterozygous Call Counts") +
  ggtitle("Hetrozygosity Rates") +
  scale_y_continuous(labels=comma) +
  plot_theme +
  theme(axis.ticks = element_blank(),
        axis.text.x = element_blank())
```

## Sex Inference

Gender is inferred for each genome by calculating the heterozygosity rate on the X chromosome.  Genomes who's inferred sex is different from that of the reported sex are removed from the cohort.  Although it is possible for people to be genotypically male and phenotypically female, it is more likely that samples or phenotypic records were mislabeled.

```r
result <- DisplayAndDispatchQuery("./sql/gender-check.sql",
                                  project=project,
                                  replacements=tableReplacement)
```

```
# Compute the the homozygous and heterozygous variant counts for each individual
# within chromosome X to help determine whether the gender phenotype value is
# correct for each individual.
SELECT
  call.call_set_name,
  ROUND((het_RA_count/(hom_AA_count + het_RA_count))*1000)/1000 AS perct_het_alt_in_snvs,
  ROUND((hom_AA_count/(hom_AA_count + het_RA_count))*1000)/1000 AS perct_hom_alt_in_snvs,
  (hom_AA_count + het_RA_count + hom_RR_count) AS all_callable_sites,
  hom_AA_count,
  het_RA_count,
  hom_RR_count,
  (hom_AA_count + het_RA_count) AS all_snvs,
FROM
  (
  SELECT
    call.call_set_name,
    SUM(0 = first_allele
      AND 0 = second_allele) AS hom_RR_count,
    SUM(first_allele = second_allele AND first_allele > 0) AS hom_AA_count,
    SUM((first_allele != second_allele OR second_allele IS NULL)
      AND (first_allele > 0 OR second_allele > 0)) AS het_RA_count
  FROM (
    SELECT
      reference_bases,
      GROUP_CONCAT(alternate_bases) WITHIN RECORD AS alternate_bases,
      COUNT(alternate_bases) WITHIN RECORD AS num_alts,
      call.call_set_name,
      NTH(1, call.genotype) WITHIN call AS first_allele,
      NTH(2, call.genotype) WITHIN call AS second_allele,
    FROM
      [va_aaa_pilot_data.multi_sample_variants_seq_qc]
    WHERE
      reference_name = 'chrX'
      AND start NOT BETWEEN 59999 AND 2699519
      AND start NOT BETWEEN 154931042 AND 155260559
    HAVING
      # Skip 1/2 genotypes _and non-SNP variants
      num_alts = 1
      AND reference_bases IN ('A','C','G','T')
      AND alternate_bases IN ('A','C','G','T')
      )
  GROUP BY
    call.call_set_name)
ORDER BY
  call.call_set_name
```
Number of rows returned by this query: 460.

Displaying the first few results:

  call_call_set_name 	  perct_het_alt_in_snvs 	  perct_hom_alt_in_snvs 	  all_callable_sites 	  hom_AA_count 	  het_RA_count 	  hom_RR_count 	  all_snvs

  LP6005038-DNA_A01 	  0.03 	  0.97 	  872243 	  71719 	  2090 	  798434 	  73809

  LP6005038-DNA_A02 	  0.03 	  0.97 	  866001 	  72907 	  1949 	  791145 	  74856

  LP6005038-DNA_A03 	  0.59 	  0.41 	  870230 	  44797 	  64482 	  760951 	  109279

  LP6005038-DNA_A04 	  0.03 	  0.97 	  868767 	  71427 	  2215 	  795125 	  73642

  LP6005038-DNA_A05 	  0.58 	  0.42 	  868743 	  44537 	  61520 	  762686 	  106057

  LP6005038-DNA_A06 	  0.03 	  0.97 	  867771 	  76007 	  2232 	  789532 	  78239

Let's join this with the sample information:

```r
joinedResult <- inner_join(result, sampleInfo)
```

And visualize the results:

```r
ggplot(joinedResult) +
  geom_point(aes(x=call_call_set_name, y=perct_het_alt_in_snvs, color=gender), size=3) +
  xlab("Sample") +
  ylab("Heterozygosity Rate") +
  ggtitle("Heterozygosity Rate on the X Chromosome") +
  scale_colour_brewer(palette="Set1", name="Gender") +
  plot_theme +
  theme(axis.ticks = element_blank(),
    axis.text.x = element_blank())
```

## Genotyping Concordance

We next want to look at the concordance between SNPs called from the sequencing data and those called through the use genotyping.  This allows us to identify samples that may have been mixed up in the laboratory.  Samples with low concordance (>99%) should be removed from the cohort.

```r
concordanceResult <- DisplayAndDispatchQuery("./sql/genotyping-concordance.sql",
                                  project=project,
                                  replacements=tableReplacement)
```

```
# Calculate the concordance between sequencing data and genotyping data.

Displaying the first few results:
<!-- html table generated in R 3.1.2 by xtable 1.7-4 package -->
<!-- Tue Sep 22 21:55:25 2015 -->
<table border=1>
<tr> <th> sample_id </th> <th> calls_in_common </th> <th> identical_calls </th> <th> concordance </th>  </tr>
  <tr> <td> LP6005038-DNA_D09 </td> <td align="right"> 2155898 </td> <td align="right"> 2134125 </td> <td align="right"> 0.99 </td> </tr>
  <tr> <td> LP6005693-DNA_A01 </td> <td align="right"> 2196713 </td> <td align="right"> 2184674 </td> <td align="right"> 0.99 </td> </tr>
  <tr> <td> LP6005243-DNA_E08 </td> <td align="right"> 2151013 </td> <td align="right"> 2129531 </td> <td align="right"> 0.99 </td> </tr>
  <tr> <td> LP6005038-DNA_G05 </td> <td align="right"> 2154290 </td> <td align="right"> 2132563 </td> <td align="right"> 0.99 </td> </tr>
  <tr> <td> LP6005692-DNA_D11 </td> <td align="right"> 2193881 </td> <td align="right"> 2181898 </td> <td align="right"> 0.99 </td> </tr>
  <tr> <td> LP6005692-DNA_E09 </td> <td align="right"> 2195790 </td> <td align="right"> 2183423 </td> <td align="right"> 0.99 </td> </tr>
   </table>

Get the sample preparation plate for each sample

```r
plate = substr(concordanceResult$sample_id, 1, 9)
concordanceResult = cbind(concordanceResult, plate)
```

Visualizing the results:

```r
ggplot(concordanceResult) +
  geom_point(aes(x=sample_id, y=concordance, color=plate), size=3) +
  xlab("Sample") +
  ylab("Concordance") +
  ggtitle("Concordance with Genotyping Data") +
  scale_colour_brewer(name="Sample Prep Plate", palette="Set1") +
  plot_theme +
  theme(axis.ticks = element_blank(),
        axis.text.x = element_blank())
```

<img src="figure/concordance-1.png" title="plot of chunk concordance" alt="plot of chunk concordance" style="display: block; margin: auto;" />

## Batch Effect

We next want to determine if there are any batch effects that were introduced during sequencing.  Since the genomes in our study were sequenced over time it is important to check if we can identify sequencing batches by variant data alone.  We can check for batch effects by running principal component analysis on all of the variants from our dataset.  If any of the batches are outliers in this part of the analysis we may want to remove them from the cohort.

Note: This analysis is run on a Google Compute Engine rather than in BigQuery.  For full instructions on how to run PCA see the [Google Genomics Cookbook](https://googlegenomics.readthedocs.org/en/latest/use_cases/compute_principal_coordinate_analysis/1-way-pca.html?highlight=pca).

Create cluster with Apache Spark installed
```
./bdutil -e extensions/spark/spark_env.sh deploy
```

Copy client_secrets to head node
```
gcloud compute copy-files client_secrets.json hadoop-m:~/
```

SSH into the head node
```
gcloud compute ssh hadoop-m
```

Install SBT
```
echo "deb http://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-get update
sudo apt-get install sbt
```

Install git and clone spark-examples repository.
```
sudo apt-get install git
git clone https://github.com/googlegenomics/spark-examples.git
```

Compile the jar
```
cd spark-examples
sbt assembly
cp target/scala-2.10/googlegenomics-spark-examples-assembly-*.jar ~/
cd ~/
```

Running the job.

Variant set definitions:
2442214609072563359: Full variant set for AAA dataset.

```
spark-submit \
  --class com.google.cloud.genomics.spark.examples.VariantsPcaDriver \
  --conf spark.shuffle.spill=true \
  --master spark://hadoop-m:7077 \
  /home/gmcinnes/spark-examples/target/scala-2.10/googlegenomics-spark-examples-assembly-1.0.jar \
  --client-secrets /home/gmcinnes/client_secrets.json \
  --variant-set-id<PHONE_NUMBER>072563359 \
  --all-references \
  --num-reduce-partitions 15 \
  --output-path gs://gbsc-gcp-project-mvp-va_aaa_hadoop/spark/output/aaa-batch-effect-pca.tsv
```

Next, we need to download the files that were output from the job to our local machine.

```
gsutil cat gs://gbsc-gcp-project-mvp-va_aaa_hadoop/spark/output/aaa-batch-effect-pca.tsv* > aaa-batch-effect-pca.tsv
```

Now let's visualize the results.

```r
pcaFile = './data/aaa-batch-effect-pca.tsv'
pcaResult = read.table(pcaFile)
names(pcaResult) = c('sample_id','pc1', 'pc2', 'extra')
plate = substr(pcaResult$sample_id, 1, 9)
pcaResult = cbind(pcaResult, plate)
```

```r
ggplot(pcaResult, aes(pc1, pc2, color=plate)) +
  geom_point(size=4, alpha=0.5) +
  ggtitle("Batch Effect PCA") +
  xlab("Principal component 1") +
  ylab("Principal component 2") +
  scale_colour_brewer(name="Library Prep Plate", palette="Set1") +
  plot_theme +
  theme(axis.text.x=element_blank(),
        axis.text.y=element_blank(),
        axis.ticks=element_blank(),
        legend.position = c(0.85,0.75))
```

<img src="figure/pca-batch-effect-publication-1.png" title="plot of chunk pca-batch-effect-publication" alt="plot of chunk pca-batch-effect-publication" style="display: block; margin: auto;" />

## Ethnicity Inference

Another application of principle component analysis is inferring ethnicity.  We can do so by performing PCA with our data merged with data from the 1,000 Genomes Project.  When we do this we expect to see that genomes from the 1,000 Genome Project and genomes from our dataset will cluster by ethnicity.  Since almost all of the genomes from our dataset are of European descent the European subpopulations from 1,000 Genomes and our genomes should cluster together.  Any genomes that cluster with an unexpected ethnic group should be removed from the cohort.

Full instructions for how to compute principal component analysis on the intersection of two datasets using Google Cloud can be found [here](http://googlegenomics.readthedocs.org/en/latest/use_cases/compute_principal_coordinate_analysis/2-way-pca.html).

#### Step by step instructions for running 2-way PCA

These instructions are identical to the Batch Effect step except that we run 2-way PCA instead of 1-way PCA.

Create cluster with Apache Spark installed
```
./bdutil -e extensions/spark/spark_env.sh deploy
```

Copy client_secrets to head node
```
gcloud compute copy-files client_secrets.json hadoop-m:~/
```

SSH into the head node
```
gcloud compute ssh hadoop-m
```

Install SBT
```
echo "deb http://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-get update
sudo apt-get install sbt
```

Install git and clone spark-examples repository.
```
sudo apt-get install git
git clone https://github.com/googlegenomics/spark-examples.git
```

Compile the jar
```
cd spark-examples
sbt assembly
cp target/scala-2.10/googlegenomics-spark-examples-assembly-*.jar ~/
cd ~/
```

#### Run the job

Variant set definitions:
18444522614861607145: SNPs from the AAA dataset to use for ethnicity inference.
12511568612726359419: 1000 genomes variant set.

```
spark-submit   \
  --class com.google.cloud.genomics.spark.examples.VariantsPcaDriver \
  --conf spark.shuffle.spill=true \
  --master spark://hadoop-m:7077 \
  /home/gmcinnes/spark-examples/target/scala-2.10/googlegenomics-spark-examples-assembly-1.0.jar \
  --client-secrets /home/gmcinnes/client_secrets.json \
  --variant-set-id<PHONE_NUMBER>4861607145<PHONE_NUMBER>2726359419 \
  --all-references \
  --num-reduce-partitions 15 \
  --output-path gs://gbsc-gcp-project-mvp-va_aaa_hadoop/spark/output/aaa-vs-1kg-pca.txt
```

Next, we need to download the files that were output from the job to our local machine.

```
gsutil cat gs://gbsc-gcp-project-mvp-va_aaa_hadoop/spark/output/aaa-vs-1kg-pca.txt* > aaa-vs-1kg-pca.tsv
```

Now let's visualize the results.

```r
pca = read.table('./data/aaa-vs-1kg-pca.tsv')
names(pca) <- c("sample_id","pc1","pc2")
```

```r
populations = read.csv('./data/1kg_info.csv')
pca = join(pca, populations, by='sample_id')
```

```r
ggplot(pca) +
  geom_point(aes(pc1,pc2, color=Population)) +
  ggtitle("Ethnicity inference") +
  scale_colour_brewer(name="1KG super population", palette="Set1") +
  xlab("Principal component 1") +
  ylab("Principal component 2") +
  plot_theme +
  theme(axis.ticks=element_blank(),
        axis.text=element_blank(),
        legend.position = c(0.85, 0.7))
```

## Genome Similarity

The final step in the Sample QC portion of our workflow is a check for unexpected relatedness among our genomes.  To do this we perfrom an analysis called Identity By State (IBS).  This analysis checks the similarity between all positions in every genome against every other genome.  Any unexpected similarity among our genomes may be the result of unknown family relationships or the result of cross-sample contamination.  Any samples with unexpected relatedness should be removed from the cohort.

Full instructions for setting up and running this job can be found in the [Google Genomics Cookbook](http://googlegenomics.readthedocs.org/en/latest/use_cases/compute_identity_by_state/).

Note that this `n^2` analysis is a cluster compute job instead of a BigQuery query.

 Tue Sep 22 21:55:29 2015 -->

  sample1 	  sample2 	  ibsScore

  LP6005243-DNA_G12 	  LP6005243-DNA_H12 	  0.10

  LP6005243-DNA_B12 	  LP6005243-DNA_D12 	  0.10

  LP6005243-DNA_A12 	  LP6005243-DNA_C12 	  0.10

  LP6005692-DNA_H12 	  LP6005693-DNA_C03 	  0.08

  LP6005243-DNA_C12 	  LP6005243-DNA_D12 	  0.10

  LP6005243-DNA_A12 	  LP6005243-DNA_F12 	  0.10

--------------------------------------------------------
_Next_: [Part 3: Variant-Level QC](./Variant-Level-QC.md)


text:
# Refactoring & AJAX - Ruby on Rails - Week 3

#### _By Maggie Summers_

## Description

_This repo was cloned down from Epicodus Lessons and given a facelift via AJAX functionality and refactoring to make a more seamless web experience._

##  Specifications
  * AJAX Objectives

    * Users should be able to add products to their shopping cart from the index page with AJAX. The item should be added to the shopping cart and the number of items in the cart (shown in the navbar) should update.
    * Users should be able to click on a product and show/hide the product detail using AJAX. The product detail should include an image (either Paperclip or an image link), the description, and any other fields you choose to add.
    * Users should be able to remove items from the shopping cart without a page reload. The "delete" link should result in the item being removed from the shopping cart and the total price being updated.

  * Refactoring Objectives

    * Ensure that users can't order a negative number of items.
    * Add product validations.
    * Add flash messages for signing up, signing in and signing out.
    * Add product update and delete functionality for admins.
    * Add admin flash messages for adding, updating and deleting products.
    * Add password validations to ensure a user's password is sufficiently complex.

## Installation

  1. _`$ git clone https://github.com/summersmaggie/marios-whole-foods`_

  2. _`$ cd refactor-ajax-code-review`_

  3. _`$ bundle install`_

  4. _`$ rake db:create`_

  5. _`$ rake db:migrate db:test:prepare`_

  6. _`$ rake db:seed`_

  7. _`$ rails server`_

#### Development server

Run `rails server` for a dev server. Navigate to `http://localhost:3000/`. The app will automatically reload if you change any of the source files.

## Known Bugs

  _No known bugs at this time._

## Support and contact details

  _To suggest changes, submit a pull request in the GitHub repository._

## Technologies Used

  * HTML
  * CSS/Bootstrap
  * Ruby
  * Rails
  * ActiveRecord

### License

  *MIT License*

Copyright (c) 2018 **Maggie Summers**

license: MIT

repository: summersmaggie/refactor-ajax-code-review

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: summersmaggie/refactor-ajax-code-review


text:
# WebCorrector-Ext
script manager extension

## How to join to development
1. Clone the repository:
2. Load a dependencies of the build system:
```
npm install
```

## Dev-server
To test **pop-up** window use the webpack-dev-server.
To run dev-server, type:
```
npm run dev
```
A pop-up window will be available at http://localhost:8080

## Production build
To build extension, type:
```
npm run build
```
A ready-to-use extension will be available in the **extension** directory.


text:
---
layout: post
title: "Episode 106: Working From Home Without Rotting and Meetup Etiquette"
date: 2018-04-22 12:00:00 -0700
guid: 580b8daa-6793-42c5-9d00-8943d4adaeeb
duration: "27:19"
length: 26231590
file: "https://dts.podtrac.com/redirect.mp3/download.softskills.audio/sse-106.mp3"
categories: episode
enable_comments: true
---

[Dave](https://twitter.com/djsmith42) and [Jamison](https://twitter.com/jamison_dance) answer these questions:

1. Since working remotely I've noticed a trend to do things like not leaving the house, growing my beard out to above average length, or not wearing (real) pants. What should I do to keep from losing any/all interpersonal skills?
2. Is there such a thing as meetup etiquette? When I attend meetups and attempt to initiate conversion with people, I'm hesitant to interrupt people who are in discussion with others. Should I wait, try to join the discussion or just barge in on the conversion?

repository: soft-skills-engineering/website

branch_name: refs/heads/gh-pages

gha_language: JavaScript

gha_license_id: MIT

repo_name: soft-skills-engineering/website


text:
# val.cr [![Build Status](https://travis-ci.org/maiha/val.cr.svg?branch=master)](https://travis-ci.org/maiha/val.cr)

Scala-ish lazy `val`, `var` for [Crystal](http://crystal-lang.org/).

- crystal: 0.20.4

```yaml
dependencies:
  val:
    github: maiha/val.cr
    version: 0.2.0
```

## Usage

```crystal
require "val"

class Foo
  val x = 1
  var y = "123"
  val z : Int32 = (a = [1]; a << 2; a.sum)

  val cached? = true
end

foo = Foo.new
foo.x         # => 1
foo.y = "xxx" # => "xxx"
foo.z         # => 3
foo.cached?   # => true
```

## NOTE

`val` is almost same as `getter` except two points.
- 1. `getter` with `=` evaulates code immediately, but `val` runs lazily.
- 2. `getter` with `=` fails with multiple lines, but `val` accepts it.

Although `getter` with `block` runs lazily, block format has less readability than `=`.

## Contributing

1. Fork it ( https://github.com/maiha/val.cr/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

## Contributors

- [maiha](https://github.com/maiha) maiha - creator, maintainer


text:
# Application name

 ## Neighbourhoods

# Description
a web application that allows you to be in the loop about everything happening in your neighborhood. From contact information of different handyman to meeting announcements or even alerts.

## Link to Live Site

## Installation Requirements
  Clone this repository and navigate to the folder.
  Run the following commands to allow functionality of the app:-
  * sudo add-apt-repository ppa:jonathonf/python-3.6
  * sudo apt-get update
  * sudo apt-get install python3.6
  * sudo apt-get install python3-pip
  * sudo apt-get install python3.6-venv
  * python3.6 -m venv virtual
  * source virtual/bin/activate

## Dependencies
* config==0.3.9
* beautifulsoup4==4.8.1
* gunicorn==19.9.0
* confusable-homoglyphs==3.2.0
* dj-database-url==0.5.0
* Django==1.11.6
* django-bootstrap3==11.1.0
* django-bootstrap4==1.0.1
* django-heroku==0.3.1
* django-registration==2.4.1
* django-tinymce==2.8.0
* djangorestframework==3.10.3
* Pillow==6.2.1
* psycopg2==2.8.4
* python-decouple==3.1
* pytz==2019.3
* soupsieve==1.9.4
* sqlparse==0.3.0
* whitenoise==4.1.4
## Technical Requirements
This can be found in the specs.md file.
## User Stories
* Sign in with the application to start using.
* Set up a profile about me and a general location and my neighborhood name.
* Find a list of different businesses in my neighborhood.
* Find Contact Information for the health department and Police authorities near my neighborhood.
* Create Posts that will be visible to everyone in my neighborhood.
* Change My neighborhood when I decide to move out.
* Only view details of a single neighborhood.

## Technologies Used
  * Python 3.6.5
  * Bootstrap
  * Django Framework
  * Postgressql
  * Heroku
## License
MIT License
Licensed under MIT License
Ⓒ Made in November 2020 by Eugene

gha_language: Python


text:
# The USDF cellular automata
Two-dimensional United we stand, divided we fall (USDF) model (or Sznajd model).

## The rules
* if two randomly selected neighbors agree - people from their surrounding change their opinion
* if they do not agree - nothing change or the surrounding argue with them - based on user's choice

## Used technologies
* js + HTML + CSS
* Canvas
* Bootstrap

## Input parameters
* density of positive decisions
* parameter if neighbours should argue
* size of the automata's matrix


text:
fdk
=============

The [Serverless](https://serverless.com) Function Development Kit (FDK)

The Serverless FDK is designed to ease the adoption of serverless technology. It provides a simple middleware abstraction for adding functionality and migrating existing solutions (like Express) without having to rewrite your code. It further enables runtime interaction with Serverless Services including invoking functions and dispatching events to the target Service.

## Build Status
[![npm version](https://badge.fury.io/js/%40serverless%2Ffdk.svg)](https://badge.fury.io/js/%40serverless%2Ffdk)

[![Build Status](https://travis-ci.org/serverless/fdk.svg)](https://travis-ci.org/serverless/fdk)

[![NPM](https://nodei.co/npm/@serverless/fdk.png?downloads=true&downloadRank=true&stars=true)](https://nodei.co/npm/@serverless/fdk/)

## Usage

```js
// index.js
const fdk = require('@serverless/fdk')

const myHandler = fdk()
  .handler(() => 'Hello World')

module.exports.myHandler = myHandler
```

```yaml
# serverless.yml

service:
  name: my-service

functions:
  myHandler:
    handler: index.myHandler
```

## Documentation

[Full API documentation](./docs) - Learn about available methods and concepts.

## Examples

[Plenty of examples](./examples) - Examples for various cloud providers and uses of functionality.

## Install

```js
npm install --save @serverless/fdk
```

repository: serverless/fdk-tmp

repo_name: serverless/fdk-tmp


text:
# 3D forward modeling of gravity and magnetic data under the Cartesian coordinates.

## Program Propose

3D model construction and forward calculation of gravity and magnetic data using the Cartesian coordinates. density or magnetic models are built using elements of rectangular blocks. The program can either build 3D models or forward calculating gravity and magnetic data from a input model file. Some typical source types are supported by the program for fast model construction. File format of the 3D model used in this program is the 2.0 .msh file of the [Gmsh](https://gmsh.info) software.

## Installation

You need to compile the executable file `gm3d` using the `makefile`. Please change the variable `CC` in the `makefile` to the compiler you use.

If you don't have `gcc` installed.
+ For Mac OS users, install `gcc` using [Homebrew](https://brew.sh) by

```bash
brew install gcc
```

+ For Linux users, install `gcc` by

```bash
apt-get install gcc
```

After the `gcc` is installed. Use `make test` the compile the program. Then move or symlink the executable file `gm3d` to any directory included in your $PATH, for instance, `/usr/local/bin`.

## Usage

```bash
Usage: gm3d [-i<input-msh-file>] [-o<output-msh-file>] [-f<output-data-file>] [-d<xs>/<dx>/<xe>/<ys>/<dy>/<ye>/<zs>/<dz>/<ze>] [-m<model-parameters>] [-e<element-data-name>] [-p<observation-file>|<xs>/<dx>/<xe>/<ys>/<dy>/<ye>/<elevation>] [-tVz|Vzx|Vzy|Vzz|DT|DTx|DTy|DTz|Hax|Hay|Za] [-v<I0>/<D0>/<I>/<D>] [-n<noise-mean>/<noise-dev>] [-r] [-h]
```

## Options

+ __-i__: Filename of the input Gmsh(.msh) model file for forward calculation.
+ __-o__: Filename of the output Gmsh(.msh) model file built with given parameters.
+ __-f__: Filename of the output observation file of gravity or magnetic data.
+ __-d__: 3D dimensions of the model space. the suffix 's' means the starting coordinate and 'e' represents the ending coordinate in axial directions. 'dx', 'dy' and 'dz' are step lengths. The default value is 10/20/990/10/20/990/10/20/490. The axial orientation adopted by the program is a right-hand Cartesian system with the z-axis point vertical downward.
+ __-m__: Model file that contains different types of model parameter. See instructions for formats of different model types.
+ __-e__: Element data name of the input/output Gmsh(.msh) file. Note that the name must be around by "".
+ __-p__: Observation locations. You can either initialize the observation points from parameters or a file. Each line of the file contain coordinates y(easting), x(northing) and z(elevation) of an observation point.
+ __-t__: Forward component Vz, Vzx, Vzy or Vzz for gravitational data and DeltaT, DeltaTx, DeltaTy, DeltaTz, Hax, Hay and Za for magnetic data.
+ __-v__: Inclination and declination of the geomagnetic field and magnetization.
+ __-n__: Add noise to the forward calculated data
+ __-r__: Remove model elements with no data in the output Gmsh(.msh) file.
+ __-h__: Display help information.

## Input File Formats

### model parameters

There are four model types currently supported by the program, which are __regular block__, __tilted block__, __sphere__ and __interface__. The program use key words to specify a model's type and associated geometric and physical properties. Therefore, an entrance of model parameters has following parts:

<model_type> <value_type> <physical_property> <geometric_property>

Formats of these key words for different models will be discussed below. Subsequently, the model file are made up by individual entrances of model parameters. Note that any line starts with "#" will be skipped.

#### regular block

The <model_type> for a regular block is simply "regular_block". You can choose either "replace", "add" or "erase" for the block's value type. The "replace" type will replace the physical values of model cells within the block's dimensions. Correspondingly, the "add" and "erase" type will add or erase values of the model cells. The <physical_property> is a float value that represents the density or magnetism within the block's dimension. The <geometric_property> of a regular block has the format of <xmin>/<xmax>/<ymin>/<ymax>/<zmin>/<zmax> which limits the dimensions of the block.

regular_block replace|add|erase float-number <xmin>/<xmax>/<ymin>/<ymax>/<zmin>/<zmax>

#### tilted block

The <model_type> for a tilted block is simply "tilted_block". You can choose either "replace", "add" or "erase" for the block's value type. The "replace" type will replace the physical values of model cells within the block's dimensions. Correspondingly, the "add" and "erase" type will add or erase values of the model cells. The <physical_property> is a float value that represents the density or magnetism within the block's dimension. The <geometric_property> of a tilted block has the format of <xmin_z>/<xmax_z>/<ymin_z>/<ymax_z>/<zmin>/<xmin_Z>/<xmax_Z>/<ymin_Z>/<ymax_Z>/<zmax> which limits the dimensions of the block. The suffix z and Z indicate parameters of the shallow and deep facets of the tilted block.

tilted_block replace|add|erase float-number <xmin_z>/<xmax_z>/<ymin_z>/<ymax_z>/<zmin>/<xmin_Z>/<xmax_Z>/<ymin_Z>/<ymax_Z>/<zmax>

#### sphere

The <model_type> for a sphere or ellipsoid is simply "sphere". You can choose either "replace", "add" or "erase" for the block's value type. The "replace" type will replace the physical values of model cells within the block's dimensions. Correspondingly, the "add" and "erase" type will add or erase values of the model cells. The <physical_property> is a float value that represents the density or magnetism within the block's dimension. The <geometric_property> of a sphere has the format of <x_c>/<y_c>/<z_c>/<x_radius>/<y_radius>/<z_radius> which limits the dimensions of the sphere. The suffix c represents the center coordinates of the model and radius specify radii of the sphere or ellipsoid in corresponding axises.

sphere replace|add|erase float-number <x_c>/<y_c>/<z_c>/<x_radius>/<y_radius>/<z_radius>

#### interface

The <model_type> for a interface is simply "interface". You can choose either "replace", "add" or "erase" for the block's value type. The "replace" type will replace the physical values of model cells within the block's dimensions. Correspondingly, the "add" and "erase" type will add or erase values of the model cells. The <physical_property> is a float value that represents the density or magnetism within the block's dimension. The <geometric_property> of a interface is a filename of a regular grid that are used to specify the undulation of the interface. A line consist of "# range=<xmin>/<dx>/<xmax>/<ymin>/<dy>/<ymax>" must be given in the file head to provide necessary information about the grid data.

regular_block replace|add|erase float-number grid-filename

### text file format

#### Input observation locations

Input locations of the observation (see command -p) specify the coordinates of observation point. Each line of the input text file contains the y (easting), x (northing) and z (elevation) positions of a point. Note that any line starts with '#' will be skipped.

#### Output observations

Output files of the observation contain positions and forward calculated data and their uncertainties. Each line of the input text file contains the y (easting), x (northing) and z (elevation) positions of a point and forward calculated data (mGal or nT) and associated uncertainties (mGal and nT).

repository: YiZhangCUG/Toolbox

gha_language: HTML

repo_name: YiZhangCUG/Toolbox


text:
# Release Notes

## Version 4.0.0

https://github.com/VincentWei/mg-samples

### What's new in this version

  * New sample `complexscripts` to show how to layout, shape, and render text
    in complex scripts.

## Version 3.2.2

https://github.com/VincentWei/mg-samples

### What's new in this version

  * New sample `commlcd` to show how to define operators needs by
    the CommLCD NEWGAL engine and the COMM IAL engine in your app code.
  * New sample `incore-data` to show how to load images and/or fonts
    from incore memory data instead of files.

repository: chaocfb/mg-samples

repo_name: chaocfb/mg-samples


text:
# serverless-aws-static-websites
Host your static website on AWS via S3, with global CDN via CloudFront, using SSL certificates provided by ACM using your own domain name!

All set up via one [Serverless](https://www.serverless.com) command and minimum manual configuration!

## Architecture

![Serverless static websites on AWS](docs/architecture.png)[]()

### What is provisioned in your AWS account?
* A S3 bucket containing your static website
* A CloudFront distribution for global hosting via CDN
* A HostedZone on Route53 with A records for your domain name
* A Lambda function for automatic SSL certificate generation via ACM for your domain name (run once upon deployment)

## Preconditions
This guide assumes that you have a pre-existing domain which you want to use for hosting your static website. Furthermore, you need to have access to the domain's DNS configuration.

Also, you need to have an install of [Serverless](https://www.serverless.com) on your machine.

## How-to
To use this blueprint with your own static websites, you can follow the steps below to get started.

### Set up a Serverless project for your static website
There are basically two ways to get started, either use [Serverless](https://www.serverless.com) to generate a basic project structure, or use the "traditional" fork and clone mechanisms.

#### Use Serverless templates
The following command will create a local project structure you can use to deploy your static website in the `mystaticwebsite` folder relative to your current working directory:

```bash
$ sls create --template-url https://github.com/tobilg/serverless-aws-static-websites --path mystaticwebsite
Serverless: Generating boilerplate...
Serverless: Downloading and installing "serverless-aws-static-websites"...
Serverless: Successfully installed "serverless-aws-static-websites"
```

**Hint**
When using this method, Serverless is replacing the `service.name` in the `serverless.yml` file automatically with `mystaticwebiste`. If you want to use a different stack name, you have to replace it manually. You also need to take care of that the stack name is using only allowed characters. When using the "Fork and clone" method below, the stack name is automatically derived from the domain name and sanitized regarding the allowed characters.

#### Fork and clone
Once you forked the repo on GitHub, you can clone it locally via

```bash
$ git clone git@github.com:youraccount/yourrepo.git
```

where `youraccount/yourrepo` needs to be replaced with the actual repository name of your forked repo.

### Install dependencies
To install the dependencies, do a

```bash
$ npm i
```

After that, the project is usable.

### Create your static website
You can now create your static website in the `src` folder of your cloned repo.

### Deploy
You can deploy your static website with the following command:

```bash
$ sls deploy --domain yourdomain.yourtld
```

where `yourdomain.yourtld` needs to be replaced with your actual domain name. You can also specify a AWS region via the `--region` flag, otherwise `us-east-1` will be used.

#### Manual update of DNS records on first deploy
On the first deploy, it is necessary to update the DNS setting for the domain manually, otherwise the hosted zone will not be able to be established.

Therefore, once you triggered the `sls deploy` command, you need to log into the AWS console, go to the [Hosted Zones](https://console.aws.amazon.com/route53/home?region=eu-central-1#hosted-zones:) menu and select the corresponding domain name you used for the deployment.

The nameservers you have to configure your domain DNS to can be found under the `NS` record and will look similar to this:

```bash
ns-1807.awsdns-33.co.uk.
ns-977.awsdns-58.net.
ns-1351.awsdns-40.org.
ns-32.awsdns-04.com.
```

You should then update your DNS settings for your domain with those values, **otherwise the stack creation process will fail**.

This is a bit misfortunate, but to the best of knowledge there's currently no other way possible if you use AWS external (non-Route53) domains. During my tests with [namecheap.com](https://www.namecheap.com) domains the DNS records were always updated fast enough, so that the stack creation didn't fail.

#### Deployment process duration
As a new CloudFront distribution is created (which is pretty slow), it can take **up to 45min** for the initial deploy to finish. This is normal and expected behavior.

### Post-deploy
If the deployment finished successfully, you will be able to access your domain via `https://www.yourdomain.yourtld` and `https://yourdomain.yourtld`.

This setup should give you some good [PageSpeed Insights](https://developers.google.com/speed/pagespeed/insights/?hl=en) results.

### Updates
For every update of your website, you can trigger a deploy as stated above. This will effectively just do s S3 sync of the `src` folder.

To do a manual sync, your can also use `sls s3sync`. There's also the possibility to customize the caching behavior for individual files or file types via the `serverless.yml`, see the [s3sync plugin's documentation](https://www.npmjs.com/package/serverless-s3-sync#setup).

As **CloudFront caches the contents of the website**, a [Serverless plugin](https://github.com/aghadiry/serverless-cloudfront-invalidate) is used to invalidate files. This [may incur costs](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#PayingForInvalidation), see the [docs](https://aws.amazon.com/de/premiumsupport/knowledge-center/cloudfront-serving-outdated-content-s3/) for more info.

You can run `sls cloudfrontInvalidate` to do a standalone invalidation of the defined files in the `serverless.yml`.

## Removal
If you want to remove the created stack, you will have to delete all records of the Hosted Zone of the respective domain except the `SOA` and `NS` records, otherwise the stack deletion via

```bash
$ sls remove --domain yourdomain.yourtld
```

will fail.

repository: MysteryDrop/mystery-drop

repo_name: MysteryDrop/mystery-drop


text:
# noobkube

The code in this repository will allow you to test out
[kubernetes](https://kubernetes.io) in the context of deploying and managing
some very simple applications.

The intention here is to provide a bridge between the [kubernetes interactive
tutorials](https://kubernetes.io/docs/tutorials/kubernetes-basics/) and the
rather dense [concept overview](https://kubernetes.io/docs/concepts/), the
rather disparate [tutorials](https://kubernetes.io/docs/tutorials/), and the
rather encyclopaedic [API reference](https://kubernetes.io/docs/reference/).

We will start with a very simple service, which allows its caller to sample from
a uniform distribution, and build out a family of services that build upon it.
In the process, we will experiment with different configurations of the
underlying [Docker](https://www.docker.com/) containers on our kubernetes
clusters.

## Prerequisites

1. [Install `kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
-- kubernetes CLI.

1. [Install `minikube`](https://kubernetes.io/docs/tasks/tools/install-minikube/)
-- Sets up a single-node kubernetes cluster on your local machine (inside a VM).

1. (Optional) [Install `docker`](https://www.docker.com/community-edition) --
Makes it easier for you to test the services locally, especially if you want to
make any modifications of your own.

1. (Optional) [Sign up for an account on DockerHub](https://hub.docker.com/) --
Makes it easy to deploy your containers to your kubernetes cluster.

1. (Optional) [Install node.js `v8.9.3`](https://nodejs.org/en/blog/release/v8.9.3/)
-- Useful if you want to test the services outside of the provided containers,
but certainly not necessary.

## Core service

The `random-server` is the service that lies at the core of our experiments. It
is a very simple service which returns a random number between 0 and 1. If you
would like to see it run and have Docker installed, you can pull the
[image from DockerHub](https://hub.docker.com/r/fuzzyfrog/random-server/):

```bash
docker pull fuzzyfrog/random-server
```

and then run it on port `<PORT>` (replace this with the port of your choice in the
commands below):

```bash
docker run -d -p <PORT>:8080 fuzzyfrog/random-server
```

Now you can test the service by calling `curl`:

```bash
curl localhost:<PORT>
```

This should display a random number between 0 and 1 in your terminal.

If you are curious, you can see the [service implementation](./random-server/index.js)
or its [Dockerfile](./random-server/Dockerfile).

## minikube cluster

You can bring up your `minikube` cluster by
simply:

```bash
minikube start
```

This will automatically make the `minikube` cluster available to `kubectl`. To
check this, try running

```bash
kubectl cluster-info
```

This should tell you that you have `Kubernetes master` running at a given IP
address. We will make use of this later.

#### Aside

It may seem strange to you that `kubectl` automatically picks up and allows you
to interact with the `minikube` cluster. What if you wanted to interact with some
other cluster instead? All this information can be specified to `kubectl` through
the commands under `kubectl config`. You likely won't have to worry about this
now, but this tidbit of information will come in handy if your kubernetes voyage
takes you past this repo. When you need to work with more than one cluster,
start out with a simple

```bash
kubectl config --help
```

## Deploying `random-server` to the minikube cluster

We can deploy our `random-server` application to the minikube cluster using the
`kubectl run` command. We will pull the image from DockerHub:

```bash
kubectl run random --image=fuzzyfrog/random-server:v1
```

Now, you can see that the `random` server has been deployed to your cluster with

```bash
kubectl get deployments
```

Moreover, you should be able to see the pod corresponding to that deployment

```bash
kubectl get pods
```

Although a pod is up and running our `random-server`, this server is not currently
exposed to the world outside minikube cluster. To make it generally available,
we use

```bash
kubectl expose deployment random --type=NodePort --name=random-service --port=8080
```

This exposes the `deployment` as a `service`, which we can see now with

```bash
kubectl get services
```

That tells us which port our containers' ports `8080` are exposed as. Under the
ports column, you should see something like

```
8080:<RANDOM_SERVICE_PORT>/TCP
```

If you do `kubectl cluster-info`, you should also see the IP address at which
your minikube cluster is runnning as

```
https://<CLUSTER_IP>:<IGNORE_THIS_PORT>
```

To make a GET request against our `random-service`, all you have to do now is

```bash
curl ${CLUSTER_IP}:${RANDOM_SERVICE_PORT}
```

You can also scale the deployment of the `random-server`, which was called
`random` (to confirm, try `kubectl get deployment`), using

```bash
kubectl scale deployment/random --replicas=${NUM_REPLICAS}
```

where `${NUM_REPLICAS}` is the number of pods running the `random-server` you
would like to have running on the cluster.

#### Aside

The `-o` (or `--output`) argument to `kubectl get pods` allows you to
meaningfully interact with the table listing pod information. For example,
we can set an environment variable to be the name of pod running our random
service as follows:

```bash
POD_NAME=$(kubectl get pods -o jsonpath="{.items[*].metadata.name}" | grep "random-")
```

There is more information on `jsonpath` formatting [here](https://kubernetes.io/docs/reference/kubectl/jsonpath/).

## Downstream services

With at least one `random-server` instance running on the minikube cluster, let
us define a downstream service that makes use of it.

[`randint-server`](./randint-server/index.js) is an application that accepts
GET requests along the route `/${N}` for a positive integer `N` and, in the
response body, returns a random integer from 0 to N-1 (inclusive). It relies on
a `random-server` instance to generate this integer. I have put an [image for
this server on DockerHub](https://hub.docker.com/r/fuzzyfrog/randint-server/).
You can also see use the [Dockerfile](./randint-server/Dockerfile) directly if
you prefer.

Let us deploy `randint-server` pods to the minikube cluster. Note that
`randint-server` requires the injection of a `random-server` URL using the
`RANDOM_SERVER_URL` environment variable. `kubectl run` allows us to do this
using the `--env` parameter (try `kubectl run --help` for more information).

```bash
kubectl run randint --image=fuzzyfrog/randint-server:v1 --env "RANDOM_SERVER_URL=http://${CLUSTER_IP}:${RANDOM_SERVICE_PORT}"
```

where `${CLUSTER_IP}` and `${RANDOM_SERVICE_PORT}` have the same values as above.

Next, expose the `randint` deployment as a service using

```bash
kubectl expose deployment randint --type=NodePort --name=randint-service --port=8081
```

Now

```bash
kubectl get service randint-service
```

will tell you the port on your minikube cluster that connects to ports `8081` on
your `randint` pods, let us call it `${RANDINT_SERVICE_PORT}`. You can now make
calls against the `randint-service` as follows

```bash
curl ${CLUSTER_IP}:${RANDINT_SERVICE_PORT}/42
```

This will return a random integer between 0 and 41 (including possibly 0 or 41).

Just as with `random`, you can scale up your `randint` deployment using

```bash
kubectl scale deployment/randint --replicas=${NUM_RANDINT_REPLICAS}
```

where `${NUM_RANDINT_REPLICAS}` specifies the number of pods you want to bring
up for this deployment.

Note that you can also see the logs for all the pods for your `randint`
deployment (even if you scaled them up using `kubectl scale` as demonstrated
above) using

```bash
kubectl logs -l run=randint
```

These logs aren't particularly meaningful because the version of the application
packaged in the `fuzzyfrog/randint-server:v1` image on DockerHub that we
deployed is particularly poorly implemented. The `fuzzyfrog/randint-server:v2`
image improves upon the logging situation. To update our `randint` deployment
to use this image, we can simply

```bash
kubectl set image deployment/randint randint=fuzzyfrog/randint-server:v2
```

If you had scaled up the number of replicas on your `randint` deployment
earlier, this update will hit all the replicas. The `randint-service` now
spreads its load over the new `randint` pods. To test this, make a few calls
to the service using

```bash
curl ${CLUSTER_IP}:${RANDINT_SERVICE_PORT}/42
```

Change up the `42` if you're feeling particularly wild.

Now run

```bash
kubectl logs -l run=randint
```

The different calls should now be logged in the resulting output. This will
also show you how `randint-service` distributes calls over your pods.

## Cleaning up

You can erase all traces of what we've done so far from your kubernetes cluster using:

```bash
kubectl delete service random-service randint-service
```

and

```bash
kubectl delete deployment random randint
```

## Doing it all in one go

repository: zomglings/noobkube

repo_name: zomglings/noobkube


text:
### Abstract Factory

#### Intent
"Provide an interface for creating families of related or dependent objects without
specifying their concrete classes." (GoF)

#### Also Known As
Kit


text:
# Changelog

## v0.4.1

* Improvements
  * Move port process build products under `_build`. This fixes an issue where
    changes in MIX_TARGET settings would not be picked up.
  * Improved some specs to remove Dialyzer warnings in some cases

## v0.4.0

* New features
  * MuonTrap.Daemon no longer sends all of the output from the process to the
    logger by default. If you want it logged, pass in a `{:log_output, level}`
    option. This also slightly improves the logged message to make it easier
    to read.

## v0.3.1

* Bug fixes
  * Make MuonTrap.Daemon usable (childspecs, options)

## v0.3.0

* Bug fixes
  * Make MuonTrap.cmd/3 pass the System.cmd/3 tests
  * Add a few more specs and fix Dialyzer errors

## v0.2.2

* Bug fixes
  * Add missing dependency on `:logger`

## v0.2.1

* Bug fixes
  * Fix hex package contents

## v0.2.0

* Bug fixes
  * Fix shutdown timeout and issues with getting EINTR
  * More progress on cgroup testing; docs

## v0.1.0

* Initial release

repository: tverlaan/muontrap

repo_name: tverlaan/muontrap


text:
Purpose
--------------

Useful macro's for Objective-C projects.

Supported OS & SDK Versions
-----------------------------

* Supported build target - iOS 7.1
* Earliest supported deployment target - iOS 2.0

ARC Compatibility
------------------

The SPXDefines macro's will work correctly with or without ARC enabled.

Installation
--------------

To use the SPXDefines in an app, just drag the files into your project and import the header file into any class where you wish to make use of the SPXDefines functionality.

If you're using PODs, simply include it in your Podfile: `pod 'SPXDefines'`

Macro's
-------------

**SPXAssertionDefines**

Provides better assertion handling in an iOS project. It will NEVER crash in a RELEASE build, but will assert and break on the offending line of code in DEBUG builds. There are equivalent `CAssertTrueOr...` methods for usage inside C functions.

The following code will assert 'condition', if it fails, write the assertion to the console and break on the offending line of code. In a release build it will simply return.

		AssertTrueOrReturnNo(condition);
		AssertTrueOrReturnNil(condition);
		AssertTrueOrReturnError(condition);
		AssertTrueOrReturn(condition);

The following code will also assert 'condition', but instead of returning, it will perform the specified action.

		AssertTrueOrPerformAction(condition, NSLog(@"Help!"));

**SPXEncodingDefines**

The following code will encode or decode variables using NSCoding. It uses `encode()` and `decode()` which expand to `[aEncoder encodeObject:name forKey:@"name"]` and `[aDecoder decodeObjectForKey:@"name"]` respectively.

	// defined in header or private interface
	@property (nonatomic, copy) NSString *name;
	@property (nonatomic, copy) NSInteger age;

	...

	- (instancetype)initWithCoder:(NSCoder *)aDecoder
	{
		self = [super init];
		if (self) {
			decode(name);
			decode(age);
		}
		return self;
	}

	- (void)encodeWithCoder:(NSCoder *)aCoder
	{
		encode(name);
		encode(age);
	}

**SPXLoggingDefines**

The following methods will first look for CocoaLumberjack and if it exists, use that for logging. If not it will fall back to NSLog but with much improved logging ;)

The pretty format
`YYYY-MM-DD HH:MM:SS | LINE # | CLASS | SELECTOR | MESSAGE`

Example
`2014-03-21 14:31:22 | 23 | AppDelegate | applicationDidFinishLaunchingWithOptions: | Hello World!`

	logMethod; // simply logs the current class and selector
	SPXLog(@"Hello World!");

repository: samdods/SPXDefines

repo_name: samdods/SPXDefines


text:
---
title: How to configure servos (Penny K Pinball PKONE)
---

# How to configure servos (Penny K Pinball PKONE)

Related Config File Sections:

* [servos:](../../config/servos.md)

You can drive up to four servos from any PKONE Extension board.

Overview video about [servos](../../mechs/servos/index.md):

## number:

When you're using PKONE Extension boards, coils plug into individual
Extension boards. Then the Extension boards are connected together in a
chain to the controller.

![image](/hardware/images/pkone-extension.png)

The `number:` setting for each servo is its board's Address ID number
in the PKONE chain, then the dash, then the servo output number (11-14).

``` mpf-config
servos:
  servo_1:
    number: 0-11    # Extension board with Address ID 0, servo 11 (the first one)
  some_other_servo:
    number: 2-14    # Extension board with Address ID 2, servo 14
```

Notes:

* The PKONE Extension board Address ID switches can be set from 0 to
    7.
* Servos are numbered from 11 to 14 on the PKONE Extension board and
    not from 1 to 4.

All the servo config options are explained in-depth in the
[servos: section](../../config/servos.md) of the
config file reference.

## What if it did not work?

Have a look at our
[PKONE troubleshooting guide](../../troubleshooting/index.md).

repository: missionpinball/mpf-docs

branch_name: refs/heads/main

repo_name: missionpinball/mpf-docs


text:
### [Pygments](http://pygments.org/)

#### Install using Git

If you are a git user, you can install the theme and keep up to date by cloning the repo:

    git clone https://github.com/dracula/pygments.git

#### Install manually

Download using the [GitHub .zip download](https://github.com/dracula/pygments/archive/master.zip) option and unzip them.

#### Activating theme

You can generate the CSS stylesheet (also included in this repository) by using the included `dracula.py` file and the [Pygments Command Line Interface](http://pygments.org/docs/cmdline/):

1.  [Download Pygments](http://pygments.org/download/) on your machine
2.  Check install location by attempting to install again (may vary depending on Python setup, e.g., Anaconda) by running.

    ```shell
    pip install Pygments
    ```

    Which will return:

    ```
    Requirement already satisfied: Pygments in /path/to/pygments
    ```

    Therefore:

    - For an Anaconda setup it should be in `~/anaconda3/lib/python3.x/site-packages/pygments`
    - For a non-Anaconda setup it should be in `/usr/local/lib/python3.x/dist-packages/pygments`

3.  Copy `dracula.py` into the `styles` folder.

4.  Run the following command on your terminal:

    `pygmentize -S dracula -f html > dracula.css`

#### Activating theme in iPython

To use with iPython:

1. Run iPython on your Terminal of choice.

   `ipython`

2. To create the blank configuration files, run:

   ```
   ipython profile create [profilename]
   ```

   If you leave out the profile name, the files will be created for the `default` profile. These will typically be located in `~/.ipython/profile_default/`, and will be named `ipython_config.py`.

3. Open `ipython_config.py` file and append the following:

   ```python
   c = get_config()

   c.TerminalInteractiveShell.highlighting_style = "dracula"
   c.TerminalInteractiveShell.true_color = True
   ```

![](ipython.png)

repository: dracula/pygments

gha_language: CSS

repo_name: dracula/pygments


text:
migrated: https://gitlab.com/l1905/maven-business/lookio-legacy/heartbleed

heartbleed
==========

To build the server, use Docker and run

```
sudo docker build .
```

Then run the server with:

```
sudo docker run -t -i -p 8000:8000 -v `pwd`:/code <image_id> /bin/bash
```

Inside the Docker container:

```
cd /code
go build .
./code -v
```

This is for running a dev build, running in prod is different!

repository: lookio/heartbleed

repo_name: lookio/heartbleed


text:
---
date: 2023-03-26
time: 22h:00min
duration: "2:30:14"
title: "Tech News & AMA #23"
tags: ["ama", "ai", "chat-gpt"]
category: "AMA"
isNext: false
youtube: https://www.youtube.com/watch?v=0x1a-7JK_co
published: true
featured: false
---

Tech News & AMA #22 with our community members Yousssouf, Kawtar, Abderahim, Otmane and Mohammed. During this episode, we discuss last tech news related to AI, GPT-3, and much more.

## Guests

- [Kawtar Choubari](https://www.linkedin.com/in/choubari/)

- [Mohammed Daoudi](https://twitter.com/MIduoad)

- [Soubai Abderahim](https://soubai.me)

- [Otmane Fettal](https://twitter.com/ofettal)

## Notes

0:00:00 - Introduction and welcoming

0:04:00 - Guests learning during the last period and Productivity in Ramadan.

0:42:50 - ChatGpt Plugins, how is will be used in the future? and the launch of Bard Google assistant.

1:11:00 - How nervous vs excited our guests about AI revolution we are living in?

1:27:00 - Spark of AGI with GPT-4.

1:47:00 - QA

2:10:00 - Future of education with AI revolution.

2:23:00 - WrapUp & Goodbye.

## Links

- [T3](https://create.t3.gg/)

- [FuturePedia](https://www.futurepedia.io/)

- [Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast](https://www.youtube.com/watch?v=L_Guz73e6fw&ab_channel=LexFridman)

- [Bard](https://bard.google.com/)

- [Authjs](https://authjs.dev/)

- ['Sparks of AGI' - Bombshell GPT-4 Paper: Fully Read w/ 15 Revelations](https://www.youtube.com/watch?v=Mqg3aTGNxZ0&ab_channel=AIExplained)

- [Menara Programme](https://manara.tech/our-program)

- [Google Summer code](https://summerofcode.withgoogle.com/)

- [Udacity scholarships](https://www.udacity.com/scholarships/suse-cloud-native-foundations-scholarship)

- [Google Developer Student Clubs](https://developers.google.com/community/dsc)

- [markprompt](https://markprompt.com)

- [Embeddings open ai docs](https://platform.openai.com/docs/guides/embeddings)

- [Humata](https://www.humata.ai/pricing)

## Prepared and Presented by

- [Youssouf EL Azizi](https://elazizi.com/)


text:
#A Logic Playground

##Where I will learn to build a simple connect four game

###Tasks at hand

- build logic for computer moves
- move/overlay top table to be every row of other table
- add image to top table that is yellow squares with alpha holes
- add css to style page
- add play again button
- add delay to computer move
- add animation of block falling into place

###resources
[Wikipedia's Information](https://en.wikipedia.org/wiki/Connect_Four)

repository: W-Ely/Four-Across

repo_name: W-Ely/Four-Across


text:
      This project is no longer actively maintained by the Google Creative Lab but remains here in a read-only Archive mode so that it can continue to assist developers that may find the examples helpful. We aren’t able to address all pull requests or bug reports but outstanding issues will remain in read-only mode for reference purposes. Also, please note that some of the dependencies may not be up to date and there hasn’t been any QA done in a while so your mileage may vary.

      For more details on how Archiving affects Github repositories see this documentation .

      We welcome users to fork this repository should there be more useful, community-driven efforts that can help continue what this project began.

Landmarker
=========

Landmarker turns your orientation into an interface, revealing key destinations around you. Click each result to get directions.

Written by [Anthony Tripaldi](https://github.com/trippedout) for Google Creative Lab.

Download it in the [Play Store](https://play.google.com/store/apps/details?id=com.androidexperiments.landmarker)

More Info
---------

Check out more [Android Experiments](http://www.androidexperiments.com) and submit your own experiment!

Report any issues [here](https://github.com/googlecreativelab/landmarker/issues) - we love pull requests!

License
-------

```
Copyright 2015 Google Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```

repository: googlecreativelab/landmarker

gha_language: Java

repo_name: googlecreativelab/landmarker


text:
# Unique  {#openvino_docs_ops_movement_Unique_10}

@sphinxdirective

.. meta::
  :description: Learn about Unique-10 - a data movement operation, which can be
                performed on one required and one optional input tensor.

**Versioned name**: *Unique-10*

**Category**: *Data movement*

**Short description**: *Unique* finds unique elements of a given input tensor. Depending on the operator's attributes it either looks for unique values globally in the whole (flattened) tensor or performs the search along the specified axis.

**Detailed description**
The operator can either work in elementwise mode searching for unique values in the whole tensor or it can consider slices of the input tensor along the specified axis. This way the op is able to find unique subtensors in the input data. Except for the unique elements the operator also produces the indices of the unique elements in the input tensor and the number of occurrences of unique elements in the input data.

**Attributes**:

* *sorted*

  * **Description**: controls whether the unique elements in the output tensor are sorted in ascending order.
  * **Range of values**:

    * false - output tensor's elements are not sorted
    * true - output tensor's elements are sorted
  * **Type**: boolean
  * **Default value**: true
  * **Required**: *no*

* *index_element_type*

  * **Description**: controls the data type of the output tensors containing indices.
  * **Range of values**: "i64" or "i32"
  * **Type**: string
  * **Default value**: "i64"
  * **Required**: *no*

* *count_element_type*

  * **Description**: controls the data type of the last output tensor.
  * **Range of values**: "i64" or "i32"
  * **Type**: string
  * **Default value**: "i64"
  * **Required**: *no*

**Inputs**

* **1**: A tensor of type *T* and arbitrary shape. **Required.**
* **2**: A tensor of type *T_AXIS*. The allowed tensor shape is 1D with a single element or a scalar. If provided, this input has to be connected to a Constant. **Optional**
  When provided this input is used to "divide" the input tensor into slices along the specified axis before unique elements processing starts. When this input is not provided the operator works on a flattened version of the input tensor (elementwise processing). The range of allowed values is ``[-r; r-1]`` where ``r`` is the rank of the input tensor.

**Outputs**

* **1**: The output tensor containing unique elements (individual values or subtensors). This tensor's type matches the type of the first input tensor: *T*. The values in this tensor are either sorted ascendingly or maintain the same order as in the input tensor. The shape of this output depends on the values of the input tensor and will very often be dynamic. Please refer to the article describing how :doc:`Dynamic Shapes <openvino_docs_OV_UG_DynamicShapes>` are handled in OpenVINO.
* **2**: The output tensor containing indices of the locations of unique elements. The indices map the elements in the first output tensor to their locations in the input tensor. The index always points to the first occurrence of a given unique output element in the input tensor. This is a 1D tensor with type controlled by the ``index_element_type`` attribute.
* **3**: The output tensor containing indices of the locations of elements of the input tensor in the first output tensor. This means that for each element of the input tensor this output will point to the unique value in the first output tensor of this operator. This is a 1D tensor with type controlled by the ``index_element_type`` attribute.
* **4**: The output tensor containing the number of occurrences of each unique value produced by this operator in the first output tensor. This is a 1D tensor with type controlled by the ``count_element_type`` attribute.

**Types**

* *T*: any supported data type.
* *T_AXIS*: ``int64`` or ``int32``.

repository: openvinotoolkit/openvino

repo_name: openvinotoolkit/openvino


text:
# **Penguin Sledding Adventure**

## IT 276: 2D Game Development

### 1. Description

*   **Deliverable:** The goal is to deliver a 2D top-down scrolling video game about a penguin sledding down a slope. Avoid obstacles, collect fish, and unlock playable characters along the way!
*   **Motivation:** I think a penguin sledding game is a pretty fun and simple idea. I wanted something more wholesome in contrast to the action-packed video games that make up a large part of the market.
*   **Technical Approach:** The game will be built entirely in C and will use the SDL2 library. It will be built on the 2D game engine framework we’ve been learning in class. [https://github.com/engineerOfLies/gameframework2d](https://github.com/engineerOfLies/gameframework2d) The game will feature open-source art assets from online sources such as [https://opengameart.org/](https://opengameart.org/)

### 2. Dependencies & Risks

*   I’ve worked with C before, but have never built something on this scale with it. I’m expecting that it will take some time to get comfortable with the language, and that it will be a learning process all the way through to the end.
*   After a cursory search, there are a limited number of open-source art assets that fit the game’s theme. I’m expecting to spend some time getting all the art assets that fit well, and in the end may have to make some compromises.
*   Since C is a very flexible and powerful language, there are many ways to accomplish the same goal. I’ll have to be mindful when choosing my techniques to avoid potential bugs or performance issues later down the line.
*   The “Infinite Level” will likely prove to be a challenge. I should allocate enough time to make sure I can ask the professor for help when developing it and still have enough time to complete the rest of the game.

### 3. Sprint Milestones

*   **Sprint 1 - Prototype**: The base of the app is the core game functionality will be implemented. This version will be unpolished and will function more of a prototype and game’s core mechanics.
*   **Sprint 2 - MVP (Goal: Finish by May 5th):** Building on the Sprint 1, this version will feature a more fleshed out version of the game. The levels will feel more complete, and the game will have features implemented such as unlockable characters, a level selector, secret levels, different interactions with obstacles, and an in-game level creator.
*   **Sprint 3 - Polish (Due date: May 12th):** This version of the game will be a more polished version of the MVP. Any unused code or libraries will be removed, art assets will be finalized, music and sound effects will be finalized, and a level of beautification to the game will be applied. These will be simple finishing touches.

### 4. Sprint 2 User Stories

1. **As a regular user, I should be able to make use of a pause button in game**
    1. The pause button should have options such as resume and exit to main menu
    2. The game will automatically save the player’s current score upon exit
2. **As a regular user, I should be able to interact with obstacles in interesting ways**
    1. Brainstorm w/ Professor about this
3. **As a regular user, I should be able to select a level from the level selector**
    1. The level selector should be an option in the main menu
    2. Once selected, I’m taken to a screen where I can choose a level
    3. There should be two columns, one column displaying prebuilt and levels and the other column showing custom levels
    4. When selecting a level, I should see an option to load the level along with the current high score for that level
    5. There should also be a load level option which will load the level when selected
4. **As a regular user, I should be able to unlock secret levels**
    1. One rare collectable is present in some of the prebuilt non-secret levels for me to find
    2. Upon collecting one, the new secret level will be available in the main menu screen
5. **As a regular user, I should be able to create a custom level using the level editor**
    1. The level editor should be an option in the main menu
    2. When selected, I’ll be taken to a blank level with the default backdrop
    3. I’ll be able to pick from a list of obstacles and power ups.
    4. Once I select one, I can choose where to place it in the level
    5. Once I save the level, the level will be saved in json format in the game’s level directory.
6. **As a regular user, I should be able to unlock more penguins**
    1. After completing a secret level, one of the unlockable penguins will be available to play
    2. The penguin can be selected from the penguin selector
    3. There should be at least three unlockable penguins
7. **As a regular user, I should be able to select which penguin I want to play as**
    1. The penguin selector should be an option on the main menu
    2. Once selected, I’m presented with a list of penguins that I currently have unlocked
    3. Each option will show the art for each penguin along with a description of its unique ability
    4. Upon selecting a penguin, I will play as that penguin in all levels until another penguin is selected.
8. **As a regular user, I should be able to select the infinite level from the level selector**
    1. The level will load in “chunks”. The game will load these chunks as the player progresses through the level
    2. There will be a list of pre-generated level “chunks” available to choose from.
    3. The player must exit via the pause menu to leave the level
9. **Persistent Upgrades (per profile)**
    1. Clarify w/ professor for more details about this
10. **As a regular user, I should be able to hear music and audio while playing**
    1. Music during the level and main menu
    2. Sound effects for sledding
    3. Sound effects for collecting items
    4. Sound effects for interacting with obstacles
11. **As a regular user, I should be able to create and select different profiles**
    1. Profile selector is an option on the main menu
    2. There should an option to create a profile
    3. There should be a list of currently existing profiles
    4. Upon selecting one, there should be an option to select or delete the profile

### 6. Design Mocks

1. **Main menu**

![alt_text](docs/images/main_menu_mock.png "Main Menu Mock")

2. **Level Creator**

![alt_text](docs/images/level_selector_mock.png "Level Selector Mock")

gha_language: C

gha_license_id: NOASSERTION


text:
Marshmallow Patrons
===================

The following is a list of friends and patrons of the project.

Thanks for your support!
------------------------

### Individuals

	Romain Pokrzywka
	Ingolf Schäfer

### Organizations

	Adafruit Industries
	The Raspberry Pi Foundation
	Research in Motion

branch_name: refs/heads/staging


text:
# Scope Command Reference

Below is a detailed list outlining all of the available scope commands, including what they do, how to use them, and information on how they fit into the workflow of an air traffic controller.

Other than moving the data block or accepting handoffs, most commands require that the radar target be under your scope's control.

Please note that the bracketed words (and brackets themselves) used in the syntax examples should be replaced with the appropriate data. Note that in lieu of having to manually type the `callsign` into the command input, it is also possible to click on the aircraft on the radar scope; doing so will automatically append its callsign to the command input and execute the completed command.

While `callsign` is used in the examples to reference individual aircraft, you can also substitute it with their squawk code. It is envisioned that in future it will be possible for aircraft to also be referenced by their CID.

## Table of Contents

- [~~Accept Handoff~~](#accept-handoff)
- [~~Amend Altitude~~](#amend-altitude)
- [~~Initiate Handoff~~](#initiate-handoff)
- [Move Data Block](#move-data-block)
- [~~Propogate Data Block (and Point Out)~~](#propogate-data-block-and-point-out)
- [~~Route~~](#route)
- [Set Scratchpad](#set-scratchpad)
- [Toggle Halo](#toggle-halo)

---

### ~~Accept Handoff~~

~~_Syntax -_ `[callsign]`~~

~~_Description -_ This accepts a _pending, incoming_ handoff from another sector, thus giving the receiving sector control of the track.~~

**THIS COMMAND IS NOT YET AVAILABLE**

### ~~Amend Altitude~~

~~_Syntax -_ `QZ [FL] [callsign]`~~

~~_Shortcut -_ `[F8] [FL] [callsign]`~~

~~_Description -_ This accepts a _pending, incoming_ handoff from another sector, thus giving the receiving sector control of the track.~~

**THIS COMMAND IS NOT YET AVAILABLE**

### ~~Initiate Handoff~~

~~_Syntax -_ `[sector handoff code] [callsign]`~~

~~_Description -_ This initiates a handoff to another sector, which they must accept to transfer control of the track.~~

**THIS COMMAND IS NOT YET AVAILABLE**

### Move Data Block

_Syntax -_ `[direction] [callsign]` or `[direction]/[length] [callsign]` or `/[length] [callsign]`

_Description -_ This command will move the leader line and data block for the specified radar target to the specified direction and/or length. Note that the direction is based on the physical placement of the buttons on a standard computer's numberpad (shown below). So position `8` is directly _up_, position `1` is down and to the left, etc. Position `5` is also available as another way to "shortstem" the data block, so it is centered directly over the target. Position `0` is not used. Leader line lengths 0 through 6 are permitted only.

```text
7 8 9
4 5 6
1 2 3
```

_More Info -_ On real ATC systems, moving the data block is sometimes used by controllers to indicate the status of the aircraft, in reference to whether or not they have been told to do something yet (for instance, approach might move all the blocks down for a/c that have been switched to tower frequency).

### ~~Propogate Data Block (and Point Out)~~

~~_Syntax -_ `QP [callsign]` or `QP [sector handoff code] [callsign]`~~

~~_Shortcut -_ `[F10] [callsign]` or `[F10] [sector handoff code] [callsign]`~~

~~_Description -_ This controls whether a data block for a given target is suppressed on the specified scope. When no sector code is specified, this will toggle suppression of the data block on _your_ scope. You can also force a data block to appear (but not disappear) on another controller's scope by including the sector's handoff code. _Note that targets may not be suppressed on the scope who owns the track._~~

**THIS COMMAND IS NOT YET AVAILABLE**

### ~~Route~~

~~_Syntax -_ `QR [callsign]` or `QR [new partial/full route string] [callsign]`~~

~~_Shortcut -_ `[F6] [callsign]` or `[F6] [new partial/full route string] [callsign]`~~

~~_Description -_ This command can perform two unique functions. When only the callsign is provided, the last known route for the specified target will be toggled on the scope. And when a route string (with or without continuity with previous route) is provided, the route stored in the scope for that flight will be combined with or changed to the route provided.~~

**THIS COMMAND IS NOT YET AVAILABLE**

### Set Scratchpad

_Syntax -_ `[scratchpad text] [callsign]`

_Description -_ This will amend the value stored in the target's data block's scratchpad. This value can be either two or three alphanumeric characters.

_More Info_ - To reset the scratchpad, set it to a period symbol `. [callsign]`

### Toggle Halo

_Syntax -_ `QP_J [callsign]` or `QP_J [radius] [callsign]`

_Shortcut -_ `[F7] [callsign]` or `[F7] [radius] [callsign]`

_Description -_ This toggles a circle around the center of the radar target, with a radius you can specify in the command. If the radius is not specified, the default halo size will be used (generally 3nm for terminal, or 5nm for en route).

_More Info -_ Note that if a resized halo exists, entering `[F7] [callsign]` will actually be interpreted by the system as `[F7] 3 [callsign]`-- meaning the halo will be resized to 3nm, not removed. With the halo back to 3nm, entering `[F7] [callsign]` again will remove the halo. If you add a 10nm halo with `[F7] 10 [callsign]`, it can be removed instead of resized by specifying `[F7] 10 [callsign]`, or use the above method of entering `[F7] [callsign]` twice.

repository: openscope/openscope

branch_name: refs/heads/develop

repo_name: openscope/openscope


text:
[//]: # (title: Internal Actions Menu)

<!-- Copyright 2000-2020 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->

The Internal Actions menu provides plugin developers with a suite of tools to help develop, debug, and test their IntelliJ Platform plugins.

 >  If the menu item **Tools \| Internal Actions** is not available in IntelliJ IDEA, then the first step is to [Enabling Internal Mode](enabling_internal.md)
 >
 {type="tip"}

Click on the following topics to learn more about the _Internal Actions_ menu.
* [Enabling Internal Mode](enabling_internal.md) provides instructions for enabling the Internal Actions menu in IntelliJ IDEA.
* [UI Tools](internal_ui_sub.md) has information about some of the Internal Actions menu tools for inspecting and testing plugin UI.

repository: renuacpro/intellij-sdk-docs

repo_name: renuacpro/intellij-sdk-docs


text:
# Security Policy

## Supported Versions

The "current" version of spotify-playlist-sampler is always the supported version. If you find a vulnerability affecting the latest version of this service, please report it.

## Reporting a Vulnerability

Security issues can be reported publicly using the GitHub Issues tracker above. However, please do not include sensitive data (data from other users, keys, etc.)

Reports should clearly describe the vulnerability *and* impact.

repository: JosiahOne/spotify-playlist-sampler

repo_name: JosiahOne/spotify-playlist-sampler


text:
---
layout: post
title: "Hide Those ENV VARS"
date: 2014-12-11 11:40:03 -0800
---

I was recently putting more CSS touches on my Danebook clone when I got an unexpected e-mail from Amazon. It said that my s3 account had been compromised somehow and that I racked up $2200 dollars in charges in the process!

>Dear AWS Customer,
>
>Your AWS Account is compromised! Please review the following notice and take >immediate action to secure your account.

As I scanned through the e-mail I quickly realized that it was I that had compromised my own account. When deploying to Heroku, I didn't take into account the publication of environment variables, often referred to as ENV vars. There is a handy little gem called Figaro which is extremely easy to install and use, and even ships your variables off to Heroku if you'd like!

Upon further examination I noticed that in my haste to get this clone onto the interwebs, I installed Figaro but failed to run the CLI scripts to get it set up properly. Consequently, the default application.yml was generated but the .gitignore file was not updated to ignore tracking. I loaded it up with variables and up went the application.yml, exposed to the bone.

For future reference, I'll remember that I need to do the following(per laserlemon's instructions):

Add Figaro to the Gemfile and bundle install

{% highlight ruby linenos %}
gem 'figaro'
{% endhighlight %}

Install Figaro

{% highlight ruby linenos %}
figaro install
{% endhighlight %}

This will create a commented config/application.yml file and add it to the .gitignore file. That's it! How did I miss this?! Now you can add your variables to your application.yml file and use them where you'd like.

In my case, I had them called to the secrets.yml file for setting up my s3 account for image uploading.

With a deploy to Heroku, the following command prepared and shipped off my variables to the app sans exploitation:

{% highlight ruby linenos %}
figaro heroku:set -e production
{% endhighlight %}

Problem solved. Just had to remember to follow further instructions in the e-mail for rotating my access keys and deactivating the exposed one. Bada-bing, bada-boom! Now about that $2200...

license: Unlicense

repository: chewieee/old-liquid-site

extension: markdown

repo_name: chewieee/old-liquid-site


text:
# Clojure dependencies in private Git repositories in Docker

## How to use

Create a Docker image:
```bash
docker build --build-arg ssh_prv_key="$(cat ~/.ssh/id_rsa)" --build-arg ssh_pub_key="$(cat ~/.ssh/id_rsa.pub)" -t clojure-private-git-deps:test  .
```

Run the docker container with an interactive bash shell:
```bash
docker run -it clojure-private-git-deps:test bash
```

Start a Clojure REPL with your private Clojure module:
```bash
clojure -Sdeps '{:deps {me/module {:git/url "git@gitlab.com:me/module.git" :sha "2aa1c686760cff280b7b65b825a4321bdb8de53e"}}}'
```

Voila!

## Notes on implementation

As of now, GitHub, GitLab, and BitBucket hosts are supported as sources of private Git repositories.

To configure another host for git repositories add a line to Dockerfile:
```dockerfile
RUN ssh-keyscan your-git.host >> /root/.ssh/known_hosts
```

## Thanks!

The solution is based on an infinite amount of Googling and these online resources:
- https://codehopper.nl/2018/09/27/clojure-gitlab-deps/
- https://stackoverflow.com/questions/18136389/using-ssh-keys-inside-docker-container/42125241
- https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-91?filter=allopenissues

repository: tokenmill/docker-images

gha_language: Dockerfile

repo_name: tokenmill/docker-images


text:
# Ochrolechia alboflavescens (Wulfen) Zahlbr. SPECIES

#### Status
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Cat. Lich. Univers. 4: 676 (1927)

#### Original name
Lichen alboflavescens Wulfen

### Remarks
null

repository: mdoering/backbone

repo_name: mdoering/backbone


text:
# pokemonblueAI
AI that plays through Pokemon Blue

Made a lot of radical changes to my previous project. Basically starting over.

This runs via VBA-RR and a pokemon blue rom. Simply load up a lua script in VBA-RR and select 'main.lua' or 'test.lua' (currently does nothing but debug stuff, this is a work in progress)

Runs on Windows but I'm using a WINE emulator to code this on a Mac.

Dependencies:
busted lua test framework


text:
2.0
- Upgrade bundle to EventSauce 1.x

1.3
- Add configuration value for json encode options for doctrine message repository.

1.2
- Add snapshotting configuration.

1.1
- Breaking changes in service configuration because of changes to serialization in EventSaucse 0.6.0 (https://eventsauce.io/docs/upgrading/to-0-6-0).
- Breaking changes in database schema because of storing aggregate version in EventSaucse 0.7.0 (https://eventsauce.io/docs/upgrading/to-0-7-0)
- Autoconfiguration of aggregate repositories is no longer possible due to private constructor in AggregateRootBehaviour trait.
You must specify a list of aggregates that you want to create a default repository for because the symfony service container can no longer find them.

1.0
- First release based on EventSaucse 0.5.0

repository: jphooiveld/EventSauceBundle

gha_language: PHP

repo_name: jphooiveld/EventSauceBundle


text:
Extensions.Map<TResult> Method (DbConnection, SqlBuilder, TextWriter)
========================================================================
Maps the results of the *query* to TResult objects. The query is deferred-executed.

**Namespace:** [DbExtensions][1]
**Assembly:** DbExtensions (in DbExtensions.dll)

Syntax
------

```csharp
public static IEnumerable<TResult> Map<TResult>(
	this DbConnection connection,
	SqlBuilder query,
	TextWriter logger
)
```

### Parameters

#### *connection*
Type: [System.Data.Common.DbConnection][2]
The connection.

#### *query*
Type: [DbExtensions.SqlBuilder][3]
The query.

#### *logger*
Type: [System.IO.TextWriter][4]
A [TextWriter][4] used to log when the command is executed.

Type Parameters
---------------

#### *TResult*
The type of objects to map the results to.

### Return Value
Type: [IEnumerable][5]<**TResult**>
The results of the query as TResult objects.
### Usage Note
In Visual Basic and C#, you can call this method as an instance method on any object of type [DbConnection][2]. When you use instance method syntax to call this method, omit the first parameter. For more information, see [Extension Methods (Visual Basic)][6] or [Extension Methods (C# Programming Guide)][7].

See Also
--------
[Extensions Class][8]
[DbExtensions Namespace][1]
[Extensions.Map<TResult>(IDbCommand, TextWriter)][9]

[1]: ../README.md
[2]: http://msdn.microsoft.com/en-us/library/c790zwhc
[3]: ../SqlBuilder/README.md
[4]: http://msdn.microsoft.com/en-us/library/ywxh2328
[5]: http://msdn.microsoft.com/en-us/library/9eekhta0
[6]: http://msdn.microsoft.com/en-us/library/bb384936.aspx
[7]: http://msdn.microsoft.com/en-us/library/bb383977.aspx
[8]: README.md
[9]: Map__1_7.md

repository: shalves/DbExtensions

repo_name: shalves/DbExtensions