text:
# Docker

### Summary

In this section you will learn about:
* Running Docker locally
* Basics of Docker
* Containerizing a simple application
* Building and Pushing an Image

### Basics of Docker and Containers

Docker has a very well structured six-part tutorial.
While for this workshop you don't need to go through all of them, part 1 and 2 are required:
* [Get Started, Part 1: Orientation and setup](https://docs.docker.com/get-started)
* [Get Started, Part 2: Containers](https://docs.docker.com/get-started/part2/)

By the end of Part 2, you should have a simple container up and running, and understand the basic concepts of a container.

#### Additional Important Docker Command

Here a few other docker commands that are important to be aware of for the rest of this workshop:

1. `docker ps`

    The docker `ps` command allows to list the status of the containers.

    A container could be either stopped or running. When it finishes to execute the process, it will stop.

    For example if you run the command `docker run -it ubuntu hostname` this will :
    - Pull the official ubuntu image from the registry
    - Start the container in the interactive mode `-it`
    - Execute the command : `hostname`
    - Stop

    ```
    $ docker run -it ubuntu hostname
    0d0af5005fc7
    ```

    If you run the command `docker ps -a` you should see :
    ```
     $ docker ps -a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                          PORTS               NAMES
    0d0af5005fc7        ubuntu              "hostname"          58 seconds ago      Exited (0) About a minute ago                       gifted_darwin
    ```

    > The `-a` allows you to list all the containers, not just the one that is running

    We can notice a few things here such as :
    - The status `Exited...` for our container
    - The name `gifted_darwin` randomly generated for our container, we can specify a custom one using the command `--name` explained in the previous section.
    - We can re-execute our container using the command `docker start gifted_darwin`
    - We can run the command `docker run -it ubuntu hostname` again and do a `docker ps -a`; we should see how two containers exited.

1. `docker logs`

    The docker `logs` allow to fetch the console output from inside the container

    From our previous example, we can run `docker logs gifted_darwin`

    ```
    $ docker logs gifted_darwin
    0d0af5005fc7
    ```

    You can also stream the logs using the command `-f` and print in real time in your console the stdout of your container.

1. `docker rm`

    The docker `rm` command allows to remove a container.

    From the previous example, we can see that we have a container listed as exited in our environment, or maybe more if we run the same command `docker run -it ubuntu hostname` multiple times. If we want to do some cleaning and remove those executions from our environment we can use the command `docker rm`

    ```
    $ docker rm gifted_darwin
    gifted_darwin
    ```

    > You can either specify the **CONTAINER ID** or the **NAME** of the container to refer to it

1. `docker images`

    This command allows us to list all the base images available in the environment.

    ```
    $ docker images
    REPOSITORY                                      TAG                 IMAGE ID            CREATED             SIZE
    ubuntu                                          latest              20c44cd7596f        2 days ago          123MB
    example-scratch                                 latest              32ff7b65f567        5 days ago          30.7MB
    node                                            8.9.1-slim          a6bb2cc1118f        11 days ago         230MB
    buildpack-deps                                  xenial              a27b6a8abd1c        2 weeks ago         644MB
    ```

    > You can manage your images by removing them using `docker rmi IMAGENAME` or pulling a new one with `docker pull IMAGENAME`

### Containerizing a TensorFlow model

Now that we understand the basics of Docker, let's containerize our first TensorFlow model that we will reuse in the following modules.
Our first model will be a very simple MNIST classifier. You can see the source code in [`./src/main.py`](./src/main.py).
As you can see there is nothing specific to containers in this code, you can run this script directly on your laptop or on a VM.

Now, to have this run in a container, we need to build an image containing this code and it's dependencies.
As you saw in the tutorial, we will use a `Dockerfile` to do this.

Here is the (very simple) `Dockerfile` that we are going to use for this model (located in [`./src/Dockerfile`](./src/Dockerfile)):

```dockerfile
FROM tensorflow/tensorflow:1.4.0
COPY main.py /app/main.py

ENTRYPOINT ["python", "/app/main.py"]
```

As you can see, we are not building a new image from scratch, instead we are using a base image from TensorFlow. Indeed, TensorFlow has a bunch of base images that you can start with.
You can see the full list here: https://hub.docker.com/r/tensorflow/tensorflow/tags/.

What is important to note is that different tags need to be used depending on if you want to use GPU or not.
For example, if you wanted to run your model with TensorFlow 1.4.0 and CPU only, you would use `tensorflow/tensorflow:1.4.0`.
If instead you wanted to use GPU, you would start from `tensorflow/tensorflow:1.4.0-gpu`.

The two other instructions are pretty straightforward, first we copy our script into the container, and then we set this script as the entry point for our container, so that any argument passed to our container would actually get passed to our script.

#### Building the image

> If you don't already have a Docker account, see [Log in with your Docker ID](https://docs.docker.com/get-started/part2/#log-in-with-your-docker-id).

The next step is to build our image to be able to run it using docker. For that, we will use the command `docker build`.

From the [`./src`](./src) repository, we can build the image with

```console
docker build -t ${DOCKER_USERNAME}/tf-mnist .
```
> Reminder: the `-t` argument allows to **tag** the image with a specific name.

`${DOCKER_USERNAME}` should be your Docker username that you use to connect to Docker Hub.

The output from this command should look like this:

```
Sending build context to Docker daemon  11.26kB
Step 1/3 : FROM tensorflow/tensorflow:1.4.0
 ---> a61a91cc0d1b
Step 2/3 : COPY main.py /app/main.py
 ---> b264d6e9a5ef
Removing intermediate container fe8128425296
Step 3/3 : ENTRYPOINT python /app/main.py
 ---> Running in 7acb7aac7a9f
 ---> 92c7ed17916b
Removing intermediate container 7acb7aac7a9f
Successfully built 92c7ed17916b
Successfully tagged wbuchwalter/tf-mnist:latest
```
Let's analyse this image full name (`wbuchwalter/tf-mnist:latest`):
* `wbuchwalter` is the name of my repository, this is where we can find the image. This will be different for you (same as your docker hub username).
* `tf-mnist` is the name of the image itself
* `latest` is the tag. `latest` is the default tag if you don't specify any. Tags are usually used to denote different versions or flavors of a same image. For example you could have a tag `v1` and `v2` to denote different versions, or `cpu` and `gpu` to denote what hardware it can run on.

When you have the successfully built message, you should now be able to see if your image is locally available with the command `docker images` described earlier.

#### Running the image

Now we can try to run it locally using the `docker run` command.
By default the model will run 1000 training steps which can take a few minutes on a laptop. Let's reduce this number to 100 with the `--max_steps` argument.

```console
docker run -it ${DOCKER_USERNAME}/tf-mnist --max_steps 100
```

If everything is okay you should see the model training:

```
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting /tmp/tensorflow/input_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting /tmp/tensorflow/input_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting /tmp/tensorflow/input_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting /tmp/tensorflow/input_data/t10k-labels-idx1-ubyte.gz
2017-11-29 18:32:41.992194: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Accuracy at step 0: 0.1292
Accuracy at step 10: 0.7198
Accuracy at step 20: 0.834
Accuracy at step 30: 0.8698
Accuracy at step 40: 0.8783
Accuracy at step 50: 0.8968
Accuracy at step 60: 0.9023
Accuracy at step 70: 0.9059
Accuracy at step 80: 0.9084
Accuracy at step 90: 0.9154
Adding run metadata for 99
```

You can kill the process and exit the container at any time with `ctrl + c`.

### Running the image with the NVIDIA GPU of your machine (If you have one)

**Currently, running docker containers with GPU is only supported on Linux.**

First install [nvidia-docker](https://github.com/NVIDIA/nvidia-docker).

You also need to make sure the image you are going to use is optimized for GPU.
In our example you need to modify the `Dockerfile` to use a TensorFlow image built for GPU:

```dockerfile
FROM tensorflow/tensorflow:1.4.0-gpu
COPY main.py /app/main.py

ENTRYPOINT ["python", "/app/main.py"]
```

Then simply rebuild the image with a new tag (you can use docker or nvidia-docker interchangeably for any command except run):

```console
docker build -t ${DOCKER_USERNAME}/tf-mnist:gpu .
```

Finally run the container with nvidia-docker:

```console
nvidia-docker run -it ${DOCKER_USERNAME}/tf-mnist:gpu
```

> Note: If the command fails with `Unknown runtime specified nvidia`, follow the steps described here (Systemd drop-in file): https://github.com/nvidia/nvidia-container-runtime#docker-engine-setup

#### Publish the Image

Our image is now built and running locally, but what about sharing it to be able to use it from anywhere by anyone?
Most importantly we want to be able to reuse this image on the Kubernetes cluster we are going to create in module 2.
So let's push our image to Docker Hub:

```console
docker push ${DOCKER_USERNAME}/tf-mnist
```

If this command doesn't look familiar to you, make sure you went through part 1 and 2 of Docker's tutorial, and more precisely: [Tutorial - Share your image](https://docs.docker.com/get-started/part2/#share-your-image)

### Useful Links
* [What is Docker ?](https://www.docker.com/what-docker)
* [Docker for beginner](https://github.com/docker/labs/blob/master/beginner/readme.md)

## Next Step
[2 - Kubernetes](../2-kubernetes/README.md)


text:
# 8.7. 列舉型別

Enumerated \(enum\) types are data types that comprise a static, ordered set of values. They are equivalent to the `enum` types supported in a number of programming languages. An example of an enum type might be the days of the week, or a set of status values for a piece of data.

## 8.7.1. Declaration of Enumerated Types

Enum types are created using the [CREATE TYPE](https://www.postgresql.org/docs/12/sql-createtype.html) command, for example:

```text
CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
```

Once created, the enum type can be used in table and function definitions much like any other type:

```text
CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
CREATE TABLE person (
    name text,
    current_mood mood
);
INSERT INTO person VALUES ('Moe', 'happy');
SELECT * FROM person WHERE current_mood = 'happy';
 name | current_mood
------+--------------
 Moe  | happy
(1 row)
```

## 8.7.2. Ordering

The ordering of the values in an enum type is the order in which the values were listed when the type was created. All standard comparison operators and related aggregate functions are supported for enums. For example:

```text
INSERT INTO person VALUES ('Larry', 'sad');
INSERT INTO person VALUES ('Curly', 'ok');
SELECT * FROM person WHERE current_mood > 'sad';
 name  | current_mood
-------+--------------
 Moe   | happy
 Curly | ok
(2 rows)

SELECT * FROM person WHERE current_mood > 'sad' ORDER BY current_mood;
 name  | current_mood
-------+--------------
 Curly | ok
 Moe   | happy
(2 rows)

SELECT name
FROM person
WHERE current_mood = (SELECT MIN(current_mood) FROM person);
 name
-------
 Larry
(1 row)
```

## 8.7.3. Type Safety

Each enumerated data type is separate and cannot be compared with other enumerated types. See this example:

```text
CREATE TYPE happiness AS ENUM ('happy', 'very happy', 'ecstatic');
CREATE TABLE holidays (
    num_weeks integer,
    happiness happiness
);
INSERT INTO holidays(num_weeks,happiness) VALUES (4, 'happy');
INSERT INTO holidays(num_weeks,happiness) VALUES (6, 'very happy');
INSERT INTO holidays(num_weeks,happiness) VALUES (8, 'ecstatic');
INSERT INTO holidays(num_weeks,happiness) VALUES (2, 'sad');
ERROR:  invalid input value for enum happiness: "sad"
SELECT person.name, holidays.num_weeks FROM person, holidays
  WHERE person.current_mood = holidays.happiness;
ERROR:  operator does not exist: mood = happiness
```

If you really need to do something like that, you can either write a custom operator or add explicit casts to your query:

```text
SELECT person.name, holidays.num_weeks FROM person, holidays
  WHERE person.current_mood::text = holidays.happiness::text;
 name | num_weeks
------+-----------
 Moe  |         4
(1 row)
```

## 8.7.4. Implementation Details

Enum labels are case sensitive, so `'happy'` is not the same as `'HAPPY'`. White space in the labels is significant too.

Although enum types are primarily intended for static sets of values, there is support for adding new values to an existing enum type, and for renaming values \(see [ALTER TYPE](https://www.postgresql.org/docs/12/sql-altertype.html)\). Existing values cannot be removed from an enum type, nor can the sort ordering of such values be changed, short of dropping and re-creating the enum type.

An enum value occupies four bytes on disk. The length of an enum value's textual label is limited by the `NAMEDATALEN` setting compiled into PostgreSQL; in standard builds this means at most 63 bytes.

The translations from internal enum values to textual labels are kept in the system catalog [`pg_enum`](https://www.postgresql.org/docs/12/catalog-pg-enum.html). Querying this catalog directly can be useful.

license: MIT

repository: pgsql-tw/gitbook-docs

branch_name: refs/heads/master

extension: md

gha_language: HTML

gha_license_id: MIT

license_type: permissive

repo_name: pgsql-tw/gitbook-docs


text:
# Content-Type Negotiation

A compiled list and resources to figure out content-type, and extensions for files. Each list is JSON format and comes with multiple variations. Feel free to transpile the list if it doesn't fit your needs.

## JSON Files
There are 4 different json files located in the `/bin/` folder.
 * `mime-extensions.json` - Array of objects each containing a MIME string and the corresponding extensions associated with it. The extensions property is an array as there could be multiple extensions; ie: image/jpeg could be jpg, jpeg, or even jpe
 * `mime2ext.json` - Dictionary object containing the mime as a key and a single extension as the value. _NOTE: The value is automatically pulled during conversion so it may not be the most popular extension_
 * `ext2mime.json` - Dictionary object containing the extension as the key, and the common MIME type as the value
 * `complete.json` - Array of objects each containing a mime type, extensions, and byte-sginatures. The byte-signatures are the known "magic numbers" found in the file's header.

The files directly in `/bin/` are pretty-formatted JSON files, the original condensed versions are located in `/converter/output/`

## Data Type Description
The files `mime2ext.json` and `ext2mime.json` are pretty self-explanitory, the complicated one is `complete.json`. The novice JSON/JavaScript user should be able to figure it out instantly but in case not, heres the layout:

```
{
    "mime": "image/jpeg",
    "extension": ["jpe", "jpeg", "jpg"],
    "signature": [255, 216, 255]
}
```
 * mime - Always a string, every object has at least the mime-type. It is one of the requirements when converting
 * extension - Either a string, or an array of strings, there's no order to extensions (the converter assembles this arbitrarily) so the most "popular" may not be first.
 * signature - Always an array, but may be an array-of-arrays. The signatures represent the "magic numbers" found in a file's header. The values of the signature are base-10 and NOT hex. In the case of array-of-arrays value it's because multiple signatures could match that type.

## In Progress
 * Small JavaScript library to make sense of the lists. Should be useful for anyone having to deal with uploads/downloads and cloud systems.
 * Considering a Java library for server-side parsing

## Data Sources
Information used to generate these files was gatherered from
 * http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/docs/conf/mime.types?revision=1750837&view=co
 * http://www.garykessler.net/library/file_sigs.html

repository: chris-pikul/content-type-negotiation

repo_name: chris-pikul/content-type-negotiation


text:
# AWS API Gateway Example

Greeting API using AWS Gateway and Lambda.

## Pre-requisites

- Docker
- Docker Compose
- AWS credentials in ~/.aws or environment variables
  > Environment variables can be defined inside your shell session using `export VAR=value` or setting them in .env file. See `.env.template` for more information.

## Installation

```bash
$ git clone https://github.com/amaysim-au/docker-serverless.git
$ cd docker-serverless/example/apigw/
```

## Usage

```bash
# Create .env file based on env.example
$ make envfile ENVFILE=env.example
# Install dependencies
$ make deps

# Test
$ make test
# Build
$ make build

# Deploy to AWS
$ make deploy
# You should see something like:
#   endpoints:
#     GET - https://xyz.execute-api.ap-southeast-2.amazonaws.com/dev/greet
$ curl https://xyz.execute-api.ap-southeast-2.amazonaws.com/dev/greet
# <html>
#   <body>
#     <h1>"Welcome to Amaysim Serverless"</h1>
#   </body>
# </html>

# Remove the api gateway
$ make remove
# Clean your folder
$ make clean
```

repository: enouvoci/serverless-docker-alpine

repo_name: enouvoci/serverless-docker-alpine


text:
Pattern: Explicit `TreeSet` instantiation

Issue: -

## Description

This rule checks for explicit calls to the no-argument constructor of `TreeSet`. In Groovy, it is best to replace `new TreeSet()` with `[] as SortedSet`, which creates the same object.

## Further Reading

* [CodeNarc - ExplicitTreeSetInstantiation](https://codenarc.github.io/CodeNarc/codenarc-rules-groovyism.html#explicittreesetinstantiation-rule)

repository: Adroiti/docs-for-code-review-tools

repo_name: Adroiti/docs-for-code-review-tools


text:
---
layout: post
title: "Blue People v. City of Ney"
short-summary: "We show that when ML
models use noisy features, they're incentivized to devalue a good score
when the candidate is from a lower-performing group."
summary: "Discriminatory behavior towards certain groups by machine learning (ML)
models is concerning in critical applications such as hiring.
We show that when ML
models use noisy features, they're incentivized to devalue a good score
when the candidate is from a lower-performing group."
feature-img: "assets/img/posts/2020-12-20-Bluepeoplevs.Neycity/image7.png"
thumbnail: "assets/img/posts/2020-12-20-Bluepeoplevs.Neycity/image7.png"
author: Fereshte Khani
tags: [ML]
---

{% figure %}

{% endfigure %}

Introduction
============

Discriminatory behavior towards certain groups by machine learning (ML) models is especially concerning in critical applications such as hiring. This blog post explains one source of discrimination: the reliance of ML models on different groups' data distributions. We will show that when ML models use noisy features (which are pervasive in the real world, e.g., exam scores), they're incentivized to devalue a good candidate from a lower-performing group. This blog post is based on:

*Fereshte Khani and Percy Liang, "Feature Noise Induces Loss Discrepancy
Across Groups." International Conference on Machine Learning. PMLR, 2020*

The findings are illustrated by reviewing the hiring process in the
fictitious city of Ney, where recently a group of people has accused the
government of discrimination.

Hiring people in Ney
====================

The government of Ney wants to hire qualified people. Each person in Ney has a skill level that is normally distributed with a mean  \\(\mu\\) and a standard deviation
of  \\(\sigma_\text{skill}\\). A person is qualified if their skill level is greater than 0 and non-qualified
otherwise. The government wants to hire qualified people (all people
with skills greater than 0). For example, Alice with skill level 2, is
qualified, but Bob with the skill level of -1 is not qualified.

{% figure %}

*The skills level of the people in Ney is normally distributed with a mean of  \\(\mu\\) and a standard deviation of  \\(\sigma_\text{skill}\\).*
{% endfigure %}

To assess people's skills, the government created an exam. The exam score is a noisy indicator of the applicant's skill since it cannot capture the true skill of a person (e.g., the same applicant would score differently on different versions of SAT). In the city of Ney, exam noise is nice and simple: If an individual has skill  \\(z\\), then their
score is distributed as  \\(\mathcal{N} (z,
\sigma_\text{noise}^2)\\),
where  \\(\sigma_\text{noise}^2\\) indicates the variance of noise
on the exam.

{% figure %}

*The exam score of an individual with a skill of \\(z\\) is a random variable normally distributed with a mean of  \\(z\\) and a standard deviation of  \\(\sigma_\text{noise}\\).*
{% endfigure %}

The government wants to choose a threshold  \\(\tau\\), and hire all
people whose exam scores are greater than  \\(\tau\\). There are two
kinds of errors that the government can make:

1.  Not hiring a qualified person  (\\(z > 0 \land x \le \tau\\))
2.  Hiring a non-qualified person (\\(z \le 0 \land x > \tau\\))

For simplicity, let's assume the government cares about these two types
of errors equally and wants to minimize the overall error, i.e., the
number of non-qualified hired people plus the number of qualified
non-hired people.

$$
\begin{align}
\text{Error} = \mathbb{E}\left[[z>0] \neq [x > \tau]\right]\\
\end{align}
$$

{% figure %}

*The government’s goal is to find a cut-off threshold such that it minimizes the error.*
{% endfigure %}

Given all exam scores and knowledge of the skill distribution of the people,
what cut-off threshold should the government use to minimize the error (the above equation)?
Is it a good strategy for the government to simply use 0 as the
threshold and hire all individuals with scores greater than zero?

Let’s consider an example where the skill distribution
is  \\(\mathcal{N}(-1,1)\\), and the exam noise
has a standard deviation of  \\(\sigma_\text{noise}=1\\).  The following lines of code plot
the average error for various thresholds for this example. As
illustrated, 0 is not the best threshold to use. In fact, in this
example, a threshold of  \\(\tau=1\\) leads to minimum error.

{% figure %}

*A simple example with  \\(\mu=-1\\) and  \\(\sigma_\text{skill}=\sigma_\text{noise}=1\\). As shown on the right, accepting individuals with a score higher than  \\(0\\) does not result in the minimum error.*
{% endfigure %}

>> #### The government wants to minimize the number of hired people with negative skill levels + the number of non-hired people with positive skill levels. Hiring all people with positive exam scores (a noisy indicator of the skill) is not optimal.

If 0 is not always the optimal threshold, then what is the optimal
threshold for minimizing error for different values of  \\(\mu,
\sigma_\text{skill}\\) and  \\(\sigma_\text{noise}\\)?
Generally, given a person's exam score (\\(x\\)) and the  skill level distribution (\\(\mathbb{P}(z)\\)), what can we infer
about their real skill (\\(z\\))? Here is where Bayesian inference
comes in.

Bayesian inference
====================

Let's see what we can infer about a person's skill given their exam score and knowing the skill level distribution
 \\(\mathbb{P} (z)\\) (known as the *prior distribution* since it shows the prior over a person's skill). Using Bayes rule, we can calculate  \\(\mathbb{P} (z|x)\\) (known as the *posterior distribution* since it shows the distribution over a person's skill after observing their score).

Let's first consider two extreme cases:

1.  If the exam is completely precise
    (i.e.,  \\(\sigma_\text{noise}=0\\)), then the exam score is
    the exact indicator of a person's skill (irrespective of the prior
    distribution).
2.  If the exam is pure noise (i.e.,  \\(\sigma_\text{noise}
    \rightarrow \infty\\)), then the exam score is meaningless, and
    the best estimate for a person's skill is the average
    skill  \\(\mu\\) (irrespective of the exam score).

Intuitively, when the noise variance has a value between  \\(0\\) and  \\(\infty\\), the best estimate of a person's skill is a number
between their exam score (\\(x\\)) and the average skill
(\\(\mu\\)). The figure below shows the standard formulation of the
posterior distribution  \\(\mathbb{P} (z \mid x)\\) after observing
an exam score (\\(x_0\\)). For more details on how to derive this
formula, see
[this](https://www.google.com/url?q=https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf&sa=D&ust=1608704068777000&usg=AOvVaw1E_EmGAxQ8A_gOtp6_dTHk).

{% figure %}

*Posterior distribution of a person's skill after observing their exam score (\\(x_0\\)).*
{% endfigure %}

Based on this formula (and as hypothesized), depending on the amount of noise,  \\(\mathbb{E} [z\mid x]\\) is a number between  \\(x\\) and   \\(\mu\\).

>> #### An applicant’s expected skill level is between their exam score and the average skill among Ney people. If the exam is noisier, it is closer to the average skill; if the exam is more precise, it is closer to the exam score.

Optimal threshold
=================
Now that we have exactly characterized the posterior distribution
(\\(\mathbb{P} (z \mid x)\\)), the government can find the optimal
threshold. For any exam score  \\(x\\), if the government hires people
with score   \\(x\\), it incurs   \\(\mathbb{P}(z \le 0 \mid x) \\)
error (probability of hiring non-qualified people). On the other hand,
if it does not hire people with score   \\(x\\), it
incurs   \\(\mathbb{P}(z > 0 \mid x)\\) error (probability of
non-hiring qualified people). Thus, in order to minimize the error, the
government should hire a person iff   \\(\mathbb{P} (z > 0 \mid x) >
\mathbb{P}(z \le 0 \mid x)\\). Since the posterior distribution is a
normal distribution, the government must hire an applicant
iff   \\(\mathbb{E}[z \mid x] > 0\\).

Using the formulation in the previous section, we have:

$$
\begin{align}\mu \frac{\sigma_\text{noise}^2}{\sigma_\text{noise}^2 +
\sigma_\text{skill}^2} + x
\frac{\sigma_\text{skill}^2}{\sigma_\text{skill}^2 +
\sigma_\text{noise}^2} > 0 \iff x > -\mu
\frac{\sigma_\text{noise}^2}{\sigma_\text{skill}^2}
\end{align}
$$

Therefore, the optimal threshold is:

$$
\bbox[5px, border: 2px solid grey]{
\text{optimal threshold} = -\mu\frac{\sigma_\text{noise}^2}{\sigma_\text{skill}^2}
}
$$

In our running example with average skill   \\(\mu=-1\\)
and   \\(\sigma_\text{skill} = \sigma_\text{noise}=1\\), the optimal threshold is 1.
The figure below shows how the optimal threshold varies according
to   \\(\mu\\) and   \\(\sigma_\text{noise}\\).
As   \\(\sigma_\text{noise}\\) increases or   \\(\mu\\) decreases,
the optimal threshold moves farther away from   \\(0\\).

{% figure %}

*(left) The optimal threshold increases as  the average of the prior distribution decreases (with a fixed exam noise    \\(\sigma_\text{noise} > 0\\)). (right) The optimal threshold increases if the exam noise increases (with a fixed average skill   \\(\mu < 0\\)). Note that, if exam scores are not noisy or the average skill is zero, then the optimal threshold is zero.*
{% endfigure %}

>> #### As exams become more noisy or the average skill becomes more negative, the optimal threshold moves further away from 0.

What does machine learning have to do with all of this?
=======================================================

So far, we precisely identified the optimal cut-off threshold given the
exact knowledge of   \\(\mu, \sigma_\text{skill}\\),
and   \\(\sigma_\text{noise}\\). But how can the government find the
optimal threshold using observational data? This is where machine
learning (ML) comes into the picture.
Let's imagine very favorable conditions. Let's assume everyone (an infinite number of them!) takes the exam, the government hires all of them and observes their true skills. Further, assume the modeling assumption is perfectly correct (i.e., both the true prior distribution and conditional distribution are normal). What would happen if the government trains a model with an infinite number of \\((x,z)\\)
pairs?

{% figure %}

*The government has collected lots of data and now wants to use ML models to predict the best threshold that minimizes the error.*
{% endfigure %}

Before delving into this, we would like to note that in real-world
scenarios, we do not have infinite data (finite data issues); the
government does not hire everyone (selection bias issues), and the true
skill is not perfectly observable (target noise/biases issues).
Furthermore, the modeling assumptions are often incorrect (model
misspecification issues). Each of these issues may affect the model
adversely; however, in this blog post our goal is to analyze the model
decisions when none of these issues exist. In the next section, we will show that discrimination occurs even under these ideal conditions.

Under these very favorable conditions and the right loss function,
machine learning algorithms can perfectly predict   \\(\mathbb{E} [z
\mid x]\\) from   \\(x\\); therefore, can find the optimal threshold
that minimizes the error.  The following few lines of Python code show
how linear regression and logistic regression fit the data. In this
example, we set   \\(\mu = -1,
\sigma_\text{skill}=\sigma_\text{noise}=1\\), and as shown in
the figure on the right, the cut-off threshold predicted by the model is
one, which matches the optimal threshold as we observed previously.
{% figure %}

A simple example along with the predicted cut-off
threshold for linear and logistic regression. The predicted cut-off
threshold results in the minimum error, as previously discussed.
{% endfigure %}

>> #### Under very favorable conditions, machine learning models find the optimal threshold, which is a function of average skill, exam noise, and skill variance among people.

Optimal thresholds for different groups
=======================================

So far, we have shown how to calculate the optimal threshold and
illustrated that ML models also recover this threshold. Let's now
analyze the optimal threshold when different groups exist in the
population. There are two kinds of people in the city of Ney: blue and red. The
blue people's skills are normally distributed centered
on   \\(\mu_\text{blue}\\), and the red people's skills are normally
distributed centered on   \\(\mu_\text{red}\\). The standard deviation for
both groups is   \\(\sigma_\text{skill}\\). There can be various
reasons for disparities between groups, for example historically blue
people might not have been allowed to attend school.

{% figure %}

*In Ney, people are divided into two groups: blue and red. The blue people have a lower average skill level than the red people.*
{% endfigure %}

First of all, let's see what happens if the exam is completely precise. As
previously discussed in this case, the optimal threshold to use is 0 for
both groups independent of their distribution. Thus, both groups are
held to the same standard, and the error for the government is 0.

>> #### If there is no noise in the exam, then zero is the optimal threshold for both groups and leads to zero error.

Now let's analyze the case where the exam is noisy
(   \\(\sigma_\text{noise} > 0\\)). As discussed in the prior
sections, the optimal threshold depends on the average of the prior
distribution, thus the optimal threshold differs between blue and red
groups. Therefore, if the government knows the demographic information,
then it's a better strategy for the government to classify different
groups separately (in order to minimize the error). In particular, the
government can calculate the optimal threshold for blue and red people
using Bayesian inference.

$$
\begin{align}
\text{Red Threshold} = -\mu_\text{red} \frac{\sigma_\text{noise}^2}{\sigma_\text{skill}^2} \quad \quad \text{Blue Threshold} = -\mu_\text{blue}\frac{\sigma_\text{noise}^2}{\sigma_\text{skill}^2}
\end{align}
$$

>> #### People in a group that has lower average skills need to pass a higher bar for hiring! Not only do blue people need to overcome other associated effects of being in a group with lower average skills, they also need to pass a higher bar to get hired.

{% figure %}

The cut-off threshold for hiring is higher for blue people in comparison to the red people.
{% endfigure %}

As stated, the government uses a higher threshold for people in a group
with a lower average skill! Consider two individuals with the same skill
level but from different groups. The blue person is less likely to get
hired by the government than the red person. Surprisingly, blue people
who are already in a group with a lower average skill (which probably
affects their confidence and society’s view of them) need to also pass a
higher bar to get hired!

Finally, note that the gap between thresholds for the different groups
grows as the noise increases.

{% figure %}

As the exam noise increases, the gap between the optimal thresholds among different groups widens. Blue people need to get a better score than red people on the exam to get hired.
{% endfigure %}

>> #### A blue person has a lower chance of getting hired in comparison with a red person with the same skill.

Conclusion
=========

We examined the discriminatory effect of relying on noisy features. When ML models use noisy features, they're naturally incentivized to devalue a good score when the candidate in question comes from an overall lower-performing group. Note that noisy features are prevalent in any real-world application (here, we assumed that noise is the same among all individuals, but it's usually worse for disadvantaged groups). Ideally, we would like to improve the features to better reflect a candidate’s skill/potential or make the features more closely approximate the job requirements. If that’s not possible,  it's important to be conscious that the "optimal decision" is to discriminate, and we should adjust our process (e.g., hiring) in acknowledgment that group membership can shade an individual's evaluation.

***

Frequently asked questions
================

##### **Can we just remove the group membership information, so the model treats individuals from both groups similarly?**

Unlike this example where group membership is a removable feature,
real-world datasets are more complex. Usually, datasets contain many
features such that the group membership can be predicted from them
(recall that ML models benefit from predicting group membership since it
lowers error). Thus, it is not obvious how to remove group membership in
these datasets. See
[[1](http://proceedings.mlr.press/v28/zemel13.pdf),[2](https://arxiv.org/pdf/1707.00075.pdf),[3](https://arxiv.org/abs/1907.00020)]
for some efforts on removing group information.

##### **Why should we treat these two groups similarly when their distributions are inherently different? Utilizing group membership information reduces error overall and for both groups!**

Fairness in machine learning usually studies the impact of ML algorithms
on groups according to protected attributes such as sex, sexual
orientation, race, etc. Usually, there has been some discrimination
towards these groups throughout history, which leads to huge disparities
among their distributions. For example, women (because of their sex)
were not allowed to go to universities. Thus, these disparities are not
inherent and could (and probably should!) change over time. For
instance, see women in the labor force
[[4](https://www.dol.gov/agencies/wb/data/facts-over-time/women-in-the-labor-force%23civilian-labor-force-by-sex)].

Another reason to avoid relying on disparities among protected groups in
models is feedback loops. Feedback loops might exacerbate distributional
disparities among protected groups over time. (e.g., few women get
accepted → the self-doubt between women increases → women perform
worse in the exam → fewer women get accepted and so on). For
instance, see
[[5](https://arxiv.org/abs/1806.08010)]
and
[[6](https://arxiv.org/abs/1706.09847)].

Finally, note that although the government objective may be to minimize the
error by weighting the costs of hiring non-qualified and non-hiring
qualified candidates similarly, it is not clear whether the group
objectives should be the same. For example, a group might be worse off
as a result of the government not hiring its qualified members than if
the government had hired its non-qualified members (for example, in
settings where the lack of minority role models in higher-level
positions leads to a lower perceived sense of belonging in other members
of a group). Thus, using group membership to minimize the error is not
necessarily the most beneficial outcome for a group; and depending on
the context we might need to minimize other objectives.

##### **What about other notions of fairness in machine learning?**

In this blog post, we studied the ML model's prediction for two similar individuals (here same z) but from different groups (blue vs. red). This is referred to as the counterfactual notion of fairness. There is another common notion of fairness known as the statistical notion of fairness, which looks at the groups as a whole and compares their incurred error (it is also common to compare the error incurred by qualified members of different groups known as the equal opportunity [[7](https://arxiv.org/pdf/1610.02413.pdf)]). Statistical and counterfactual notions of fairness are independent of each other, and satisfying one does not guarantee satisfying the other. Another consequence of feature noise is causing a trade-off between these two notions of fairness, which is beyond this blog post's scope. See our paper [[8](https://arxiv.org/abs/1911.09876)] for critiques regarding these two notions and the effect of feature noise on statistical notions of fairness.

Acknowledgement
===========
I would like to thank Percy Liang, Megha Srivastava, Frieda Rong, and Rishi Bommasani, Yeganeh Alimohammadi, and Michelle Lee for their useful comments.

branch_name: refs/heads/main


text:
## Requirements

python=3.6 tensorflow-gpu=1.12 opencv matplotlib

## Data Prepare
   see ./prepare
   datatree: dataset/class/video.ext
   download [crop_mean.npy , sports1m_finetuning_ucf101.model][1] to ./

## Thanks

- [C3D-tensorflow][7] code.

[1]: https://github.com/hx173149/C3D-tensorflow
[7]: https://github.com/hx173149/C3D-tensorflow

repository: zbhoscar/submax

repo_name: zbhoscar/submax


text:
React-Double Scrollbar
======================
Adds a horizontal scrollbar to the top of an element.

## Usage
Example in a React.Component render method:

```javascript
render() {
    return (
      <div>
        <DoubleScrollbar>
          <table>...</table>
        </DoubleScrollbar>
      </div>
    );
  }
```

## Installation

The easiest way to use React-Double Scrollbar is to install it from NPM and include it in your own build.

```javascript
npm install react-double-scrollbar --save
```

You can also use the standalone build by including `dist/DoubleScrollbar.js` in your page.

# License

MIT

# Change log
v0.0.15
- Recalculate the width of the children in componentDidUpdate and if the width has changed update the width of the top scrollbar div.

v0.0.11
- Add onresize to update the scrollbar when the window size changes

v0.0.10
- Dependency fix, added peerDependency and allow for react 0.14.7 or newer

v0.0.9
- Added tests and documentation

# Contributing

All help is appreciated. To contribute please create a branch, make changes, add or update tests then create a pull request.

# Development

Clone repo then run

`npm install`

To build run

`npm run build`

To test

`npm test`

Test in watch mode

`npm test -- --watch`

# To create and publish a new version

Make sure to update version in package.json

To create a new tag run

`git tag <new version>`

`git push origin --tags`

To publish to npm

`npm publish`


text:
# Lion SoC

[System-On-Chip](https://en.wikipedia.org/wiki/System_on_a_chip) using Lion and targeting the [VELDT FPGA development board](https://standardsemiconductor.com).

## Setup
* [Project IceStorm](https://github.com/standardsemiconductor/VELDT-info#project-icestorm)
* [riscv-gnu-toolchain](https://github.com/riscv/riscv-gnu-toolchain)
  * Need `riscv64-unknown-*` binaries e.g.:
    ```console
    foo@bar:~$ git clone https://github.com/riscv/riscv-gnu-toolchain.git
    foo@bar:~$ cd riscv-gnu-toolchain
    foo@bar:~/riscv-gnu-toolchain$ git submodule update --init --recursive
    foo@bar:~/riscv-gnu-toolchain$ ./configure --prefix=/opt/riscv/
    foo@bar:~/rsicv-gnu-toolchain$ sudo make
    foo@bar:~/riscv-gnu-toolchain$ export PATH=$PATH:/opt/riscv/bin
    ```

## Usage
1. `cabal build`
2. Ensure the VELDT is ON and in the FLASH mode.
3. `cabal run soc -- prog`
4. Cycle power switch, set mode switch to FPGA.
5. `cabal run com` to open serial port, then press any key to display the Lion SoC name. The Lion Soc will echo any further input. Press <kbd>Ctrl-c</kbd> to end the program. The default port for `com` is `/dev/ttyUSB0`. If you require a different port, specify it as an argument: `cabal run com -- path/to/port`. Other serial port programs will work as well; be sure to specify 8 data bits, 1 stop bit, no flow control, and a baud rate of 19200.
```
   __   _             ____     _____
  / /  (_)__  ___    / __/__  / ___/
 / /__/ / _ \/ _ \  _\ \/ _ \/ /__
/____/_/\___/_//_/ /___/\___/\___/

Standard Semiconductor (c) 2021

Checking FLASH...SUCCESS
```
To compile, synthesize, and route Lion SoC without programming: `cabal run soc`

### Clean
`cabal run soc -- clean`

## Metrics as of Dec 30 2021 for iCE40
### Device utilisation
```
Device utilisation:
   ICESTORM_LC:  2600/ 5280    49%
  ICESTORM_RAM:     8/   30    26%
         SB_IO:     6/   96     6%
         SB_GB:     8/    8   100%
  ICESTORM_DSP:     1/    8    12%
ICESTORM_HFOSC:     1/    1   100%
        SB_SPI:     1/    2    50%
   SB_LEDDA_IP:     1/    1   100%
   SB_RGBA_DRV:     1/    1   100%
ICESTORM_SPRAM:     4/    4   100%
```
### Clock frequency
```
Max frequency for clock: 14.17 MHz (PASS @ 12Mhz)
```

## Memory Map
| Peripheral | Start Address | End Address |
|------------|---------------|-------------|
| LED        |  0x00000000   | 0x00000000  |
| UART       |  0x00000004   | 0x00000004  |
| SPI Flash  |  0x00000008   | 0x00000008  |
| ROM        |  0x00000400   | 0x000007FF  |
| SPRAM      |  0x00020000   | 0x0003FFFF  |

## Peripherals
### Led
| Byte 3   | Byte 2   | Byte 1           | Byte 0        |
|----------|----------|------------------|---------------|
| Reserved | Reserved | Register Address | Register Data |

See [Appendix D of the iCE40 LED Driver Usage Guide](https://github.com/standardsemiconductor/VELDT-info/blob/master/ICE40LEDDriverUsageGuide.pdf) for information on LED control bus addressable registers and register field descriptions.

### UART
| Byte 3   | Byte 2 | Byte 1    | Byte 0    |
|----------|--------|-----------|-----------|
| Reserved | Status | RX Buffer | TX Buffer |

Status Byte:
```
76543210
......**
      ||__Transmitter Status: 0 = Empty (Idle), 1 = Full (Busy)
      |___Receiver Status:    0 = Empty, 1 = Full
```

#### UART Usage Examples
##### UART Write Byte
```assembly
   li   a0, 0xAB    # load desired byte to send
   li   a1, 0x4     # set pointer to UART peripheral memory location
1: lbu  a2, 0x2(a1) # read status
   andi a2, a2, 0x1 # mask transmitter status
   bnez a2, 1b      # wait until transmitter empty
   sb   a0, (a1)    # transmit byte
```

##### UART Read Byte
```assembly
   li   a0, 0x4     # set pointer to UART peripheral memory location
1: lbu  a1, 0x2(a0) # read status register
   andi a1, a1, 0x2 # mask receiver status
   beqz a1, 1b      # wait until receiver full
   lbu  a1, 0x1(a0) # read receiver buffer
```
### SPI Flash

See [VELDT-info](https://github.com/standardsemiconductor/VELDT-info#veldt-info) for more information about the on-board [SPI flash chip](https://github.com/standardsemiconductor/VELDT-info/blob/master/AT25SF081.pdf) and the [Lattice SPI and SysBus interface](https://github.com/standardsemiconductor/VELDT-info/blob/master/AdvancediCE40SPII2CHardenedIPUsageGuide.pdf).

#### Peripheral Register
| Byte 3        | Byte 2                                | Byte 1                 | Byte 0              |
|---------------|---------------------------------------|------------------------|---------------------|
| SysBus Status | SysBus Read/Write AND SysBus Received | SysBus Command Address | SysBus Command Data |

##### SysBus Status
```
76543210
.......*
       |__ 0 = Idle, 1 = Busy
```

##### SysBus Read/Write AND SysBus Received
When writing to this memory location: 0 indicates READ, 1 indicates WRITE.

When reading from this memory location: most recently received byte from SysBus.

SysBus Read/Write:
```
76543210
.......*
       |__ 0 = READ, 1 = WRITE
```

#### SPI Usage Examples

##### SPI Enable
```assembly
   li   a0, 0x8        # set pointer to SPI peripheral memory address
1: lbu  a1, 3(a0)      # read bus status
   bnez a1, 1b         # wait until bus idle
   li   a1, 0x00010000 # set WRITE mode
   li   a2, 0x00000900 # set CR1 address
   or   a1, a1, a2     # set CR1 address
   ori  a1, a1, 0x80   # set SPI enable
   sw   a1, (a0)       # send SPI enable command
```
##### SPI Read Status Register
```assembly
   li   a0, 0x8        # set pointer to SPI peripheral memory address
1: lbu  a1, 3(a0)      # read bus status
   bnez a1, 1b         # wait until bus idle
   li   a1, 0x00000C00 # set READ mode and SR address
   sw   a1, (a0)       # send SPI read status command
2: lbu  a1, 3(a0)      # read bus status
   bnez a1, 2b         # wait until bus idle
   lbu  a1, 2(a0)      # read SPI status from bus received
```

repository: standardsemiconductor/lion

gha_language: Haskell

repo_name: standardsemiconductor/lion


text:
---
title: date.time() function
description: >
  `date.time()` returns the time value of a specified relative duration or time.
menu:
  flux_0_x_ref:
    name: date.time
    parent: date
    identifier: date/time
weight: 101

introduced: 0.172.0
---

<!------------------------------------------------------------------------------

IMPORTANT: This page was generated from comments in the Flux source code. Any
edits made directly to this page will be overwritten the next time the
documentation is generated.

To make updates to this documentation, update the function comments above the
function definition in the Flux source code:

https://github.com/influxdata/flux/blob/master/stdlib/date/date.flux#L93-L93

Contributing to Flux: https://github.com/influxdata/flux#contributing
Fluxdoc syntax: https://github.com/influxdata/flux/blob/master/docs/fluxdoc.md

------------------------------------------------------------------------------->

`date.time()` returns the time value of a specified relative duration or time.

`date.time` assumes duration values are relative to `now()`.

##### Function type signature

```js
(t: A, ?location: {zone: string, offset: duration}) => time where A: Timeable
```

{{% caption %}}For more information, see [Function type signatures](/flux/v0.x/function-type-signatures/).{{% /caption %}}

## Parameters

### t
({{< req >}})
Duration or time value.

Use an absolute time or relative duration.
Durations are relative to `now()`.

### location

Location used to determine timezone.
Default is the `location` option.

## Examples

- [Return the time for a given time](#return-the-time-for-a-given-time)
- [Return the time for a given relative duration](#return-the-time-for-a-given-relative-duration)

### Return the time for a given time

```js
import "date"

date.time(t: 2020-02-11T12:21:03.29353494Z)// Returns 2020-02-11T12:21:03.293534940Z

```

### Return the time for a given relative duration

```js
import "date"

option now = () => 2022-01-01T00:00:00Z

date.time(t: -1h)// Returns 2021-12-31T23:00:00.000000000Z

```


text:
stomp-proxy
===========

A stomp proxy with frame re-writing and error handling

Stomp is a simple protocol for sending messages that works over both TCP or a websocket.

This project exposes a single simple class that allows for proxying and re-writing stomp frames,
this is useful for exposing a stomp endpoint publicly with custom authentication or custom functionality

# Usage
```JavaScript
var StompProxy = require("stomp-proxy")
// connect to STOMP server, such as rabbitmq speaking stomp
var upstreamConn = net.connect({port: 61613})

// attach a handler for each stomp command
function makeProxy(client, upstream) {
  var proxy = new StompProxy(client, upstream)
  // rewrite frames by passing them to the callback
  proxy.onSubscribe = function(session, frame, cb) {
    frame.headers.destination = "/topic/foo.bar.baz"
    cb(null, frame)
  }
  // send error frames by passing an error to the callback
  proxy.onSend = function(session, frame, cb) {
    cb(new Error("no sends allowed"))
  }
  // support custom auth and sessions (in-memory only)
  proxy.onConnect = function(session, frame, cb) {
    session.username = frame.headers.login
    if (frame.headers.passcode != "crispy_treats") {
      return cb(new Error("invalid passcode"))
    }
    frame.headers.login = "guest"
    frame.headers.passcode = "guest"
    cb(null, frame)
  }

  // rewrite frames from the server
  proxy.onMessage = function(session, frame, cb) {
    frame.headers["proxy-via"] = "stomp-proxy"
    frame.headers.user = session.username
    cb(null, frame)
  }

  // you need to call start after attaching handlers
  proxy.start()
  return proxy
}
// can be TCP
var server = net.createServer(function(conn) {
  var proxy = makeProxy(conn, upstreamConn)
})
// or a websocket
shoe(function(conn) {
  var proxy = makeProxy(conn, upstreamConn)
})
```

### Other random features
* Support arbitrary stomp commands to add your own functions
```JavaScript
// just attach a handler, client would send DELETE
proxy.onDelete = function(session, frame, cb) {
  myApi.delete(frame.headers.destination, function(err) {
    if (err) return cb(err)

    // don't send the frame back to the upstream as it wouldn't understand the method
    cb()
  })
}
```
* Passthrough mode when you don't want to rewrite in one direction
```JavaScript
// we don't wan to rewrite server frames, can be slightly more efficent
var proxy = new StompProxy(client, upstream, {serverPassthrough: true})
```
### More examples
* (backside-proxy)[https://github.com/backside/backside-proxy/blob/master/lib/proxy.js]

# License
MIT

# TODO
1. Add a way to sync/persist sessions between multiple hosts

# Contributing
Open a PR, add a test

repository: old-backside-byu-project/stomp-proxy

repo_name: old-backside-byu-project/stomp-proxy


text:
# Step by Step: Node Task with Typescript API

This step by step will show how to manually create, debug and test a cross platform task with no service, server or agent!

Tasks can be created using tfx as a convenience, but it's good to understand the parts of a task.
This tutorial walks through each manual step of creating a task.

Note: The tutorial was done on a mac.  Attempted to make generic for all platforms.  Create issues in the repo.

## Video

https://youtu.be/O_34c7p4GlM?t=1173

## Tools

[Typescript Compiler 2.2.0 or greater](https://www.npmjs.com/package/typescript)

[Node 4.4.7 (LTS) or greater](https://nodejs.org/en/)

[Npm 3.0 or greater recommended](https://www.npmjs.com/package/npm3) (comes with node >=5. creates flat dependencies in tasks)

This tutorial uses [VS Code](https://code.visualstudio.com) for great intellisense and debugging support

## Sample Files

Files used for this walk through are [located here in this gist](https://gist.github.com/bryanmacfarlane/154f14dd8cb11a71ef04b0c836e5be6e)

## Create Task Scaffolding

Create a directory and package.json.  Accept defaults of npm init for sample.
```bash
$ mkdir sampletask && cd sampletask
$ npm init
```

### Add vsts-task-lib

Add vsts-task-lib to your task.  Remember your task must carry the lib.  Ensure it's at least 0.9.5 which now carries typings.

The package.json should have dependency with ^.  Ex: ^0.9.5.  This means you are locked to 0.9.x and will pick up patches on npm install.

The npm module carries the .d.ts typecript definition files so compile and intellisense support will just work.

```
$ npm install vsts-task-lib --save
...
└─┬<EMAIL_ADDRESS>...
```

### Add Typings for Externals

Ensure typings are installed for external dependencies

```bash
$ npm install @types/node --save-dev
$ npm install @types/q --save-dev
```

Create a .gitignore and add node_modules to it.  Your build process should do `npm install` and `typings install`.  No need to checkin dependencies.

```bash
$ cat .gitignore
node_modules
```

### Create tsconfig.json Compiler Options

```bash
$ tsc --init
```

Change `tsconfig.json` file to ES6 to match the sample gist.  ES6 is for async await support.

## Task Implementation

Now that the scaffolding is out of the way, let's create the task!

Create a `task.json` file using `sample_task.json` as a starting point. The full task JSON schema is [here](https://aka.ms/vsts-tasks.schema.json).

Replace the `{{placeholders}}`.  The most important being a [unique guid](http://www.guidgen.com/).
Note: copy from web view since file needs property names in quotes (browser might strip in raw view)

Create a `index.ts` file using index.ts from the gist as a starting point.
Create a `taskmod.ts` file using taskmod.ts from the gist as a starting point.

Instellisense should just work in [VS Code](https://code.visualstudio.com)

The code is straight forward.  As a reference:

```javascript
import tl = require('vsts-task-lib/task');
import trm = require('vsts-task-lib/toolrunner');
import mod = require('./taskmod');

async function run() {
    try {
        console.log(process.env["INPUT_SAMPLESTRING"]);
        let tool: trm.ToolRunner;
        if (process.platform == 'win32') {
            let cmdPath = tl.which('cmd');
            tool = tl.tool(cmdPath).arg('/c').arg('echo ' + tl.getInput('samplestring', true));
        }
        else {
            let echoPath = tl.which('echo');
            tool = tl.tool(echoPath).arg(tl.getInput('samplestring', true));
        }

        let rc1: number = await tool.exec();

        // call some module which does external work
        if (rc1 == 0) {
            mod.sayHello();
        }

        console.log('Task done! ' + rc1);
    }
    catch (err) {
        tl.setResult(tl.TaskResult.Failed, err.message);
    }
}

run();
```

taskmod.ts:

```javascript
export function sayHello() {
    console.log('Hello World!');
}
```

Key Points:

 - Async code is linear.  Note the two executions written one after each other in linear fashion.
 - Must be wrapped in async function.
 - Greatly simplifies error handling.
 - Never process.exit your task.  You can sometimes lose output and often the last bit of output is critical

If we did our job well, the code should be pretty self explanatory.
But, see the [API Reference](vsts-task-lib.md) for specifics.

## Compile

Just type `tsc` from the root of the task.  That should have compiled an index.js

## Run the Task

The task can be run by simply running `node index.js`.  Note that is exactly what our agent will do.

```bash
$ node index.js
##vso[task.debug]agent.workFolder=undefined
##vso[task.debug]loading inputs and endpoints
##vso[task.debug]loaded 0
##vso[task.debug]task result: Failed
##vso[task.issue type=error;]Input required: samplestring
##vso[task.complete result=Failed;]Input required: samplestring
```

The task failed!  That's exactly what would happen if the task run and inputs were not supplied.

The agent runs the task and reads key information from command output output over stdout and stderr.  This allows for consistency in node, powershell or even ad-hoc scripts.  The lib is mostly a thin wrapper generating those commands.

Let's supply one of the inputs and try again.

```bash
$ export INPUT_SAMPLESTRING="Hello World"
$ node index.js
##vso[task.debug]agent.workFolder=undefined
##vso[task.debug]loading inputs and endpoints
##vso[task.debug]loading INPUT_SAMPLESTRING
##vso[task.debug]loaded 1
##vso[task.debug]samplestring=Hello World
##vso[task.debug]echo arg: Hello World
##vso[task.debug]exec tool: echo
##vso[task.debug]Arguments:
##vso[task.debug]   Hello World
[command]echo Hello World
Hello World
##vso[task.debug]rc:0
##vso[task.debug]success:true
##vso[task.debug]samplebool=null
Task done! 0,-1
$
```

> TIP: be careful with chars like ! in env vars.  [Example here](http://superuser.com/questions/133780/in-bash-how-do-i-escape-an-exclamation-mark)

Now let's set the sample bool.  This should fail since if sample bool is true, it should need the other input.  See the code.

```bash
$ export INPUT_SAMPLEBOOL=true
$ node index.js
##vso[task.debug]agent.workFolder=undefined
##vso[task.debug]loading inputs and endpoints
##vso[task.debug]loading INPUT_SAMPLEBOOL
##vso[task.debug]loading INPUT_SAMPLESTRING
##vso[task.debug]loaded 2
##vso[task.debug]samplestring=Hello World
##vso[task.debug]echo arg: Hello World
##vso[task.debug]exec tool: echo
##vso[task.debug]Arguments:
##vso[task.debug]   Hello World
[command]echo Hello World
Hello World
##vso[task.debug]rc:0
##vso[task.debug]success:true
##vso[task.debug]samplebool=true
##vso[task.debug]task result: Failed
##vso[task.issue type=error;]Input required: samplepathinput
##vso[task.complete result=Failed;]Input required: samplepathinput
$
```

So, as you can see, this offers powerful testing automation options to test all arcs with no agent or server.

## Interactive Use from Command Line (Advanced)

Node offers an interactive console and since the task lib is in your node_modules folder, you can interactively poke around.

```bash
$ node
> var tl = require('vsts-task-lib/task');
##vso[task.debug]agent.workFolder=undefined
##vso[task.debug]loading inputs and endpoints
##vso[task.debug]loaded 0
undefined
> tl.which('echo');
##vso[task.debug]echo=/bin/echo
'/bin/echo'
> tl.tool('echo').arg('Hello World!').args
##vso[task.debug]echo arg: Hello World!
[ 'Hello World!' ]
> .exit

```
## Debugging

Coming soon

## Unit testing your task scripts

This requires vsts-task-lib 0.9.15 or greater.

### Goals:

- Unit test the script, not the external tools it's calling.
- Run subsecond (often < 200 ms)
- Validate all arcs of the script, including failure paths.
- Run your task script unaltered the exact way the agent does (envvar as contract, run node against your script)
- Assert all aspects of the execution (succeeded, issues, std/errout etc...) by intercepting command output

### Install test tools

We will use mocha as the test driver in this examples.  Others exist.
```bash
npm install mocha --save-dev -g
npm install @types/mocha --save-dev
```

### Create test suite

Creates tests folder and _suite.ts.  [Example here](https://gist.github.com/bryanmacfarlane/154f14dd8cb11a71ef04b0c836e5be6e#file-_suite-ts)

### Success test

The success test will validate that the task will succeed if the tool returns 0.  It also confirms that no error or warning issues are added to the build summary.  Finally it validates that the task module is called if the tool succeeds.

The success test [from _suite.ts](https://gist.github.com/bryanmacfarlane/154f14dd8cb11a71ef04b0c836e5be6e#file-_suite-ts) looks like:

```javascript
    it('should succeed with simple inputs', (done: MochaDone) => {
        this.timeout(1000);

        let tp = path.join(__dirname, 'success.js');
        let tr: ttm.MockTestRunner = new ttm.MockTestRunner(tp);

        tr.run();
        assert(tr.succeeded, 'should have succeeded');
        assert.equal(tr.invokedToolCount, 1);
        assert.equal(tr.warningIssues.length, 0, "should have no warnings");
        assert.equal(tr.errorIssues.length, 0, "should have no errors");
        assert(tr.stdout.indexOf('atool output here') >= 0, "tool stdout");
        assert(tr.stdout.indexOf('Hello Mock!') >= 0, "task module is called");

        done();
    });
```

key code from [success.ts test file](https://gist.github.com/bryanmacfarlane/154f14dd8cb11a71ef04b0c836e5be6e#file-success-ts)

```javascript

let taskPath = path.join(__dirname, '..', 'index.js');
let tmr: tmrm.TaskMockRunner = new tmrm.TaskMockRunner(taskPath);

tmr.setInput('samplestring', "Hello, from task!");
tmr.setInput('samplebool', 'true');

// provide answers for task mock
let a: ma.TaskLibAnswers = <ma.TaskLibAnswers>{
    "which": {
        "echo": "/mocked/tools/echo",
        "cmd": "/mocked/tools/cmd"
    },
    "exec": {
        "/mocked/tools/echo Hello, from task!": {
            "code": 0,
            "stdout": "atool output here",
            "stderr": "atool with this stderr output"
        },
        "/mocked/tools/cmd /c echo Hello, from task!": {
            "code": 0,
            "stdout": "atool output here",
            "stderr": "atool with this stderr output"
        }
    }
};
tmr.setAnswers(a);

// mock a specific module function called in task
tmr.registerMock('./taskmod', {
    sayHello: function() {
        console.log('Hello Mock!');
    }
});

tmr.run();

```

### Fail test

This test validates that the task will fail if the tool returns 1.  It also validates that an error issue will be added to the build summary.  Finally, it validated that the task module is not called if the tool fails.

The fail test [from _suite.ts](https://gist.github.com/bryanmacfarlane/154f14dd8cb11a71ef04b0c836e5be6e#file-_suite-ts) looks like:

```javascript
    it('it should fail if tool returns 1', (done: MochaDone) => {
        this.timeout(1000);

        let tp = path.join(__dirname, 'failrc.js');
        let tr: ttm.MockTestRunner = new ttm.MockTestRunner(tp);

        tr.run();
        assert(!tr.succeeded, 'should have failed');
        assert.equal(tr.invokedToolCount, 1);
        assert.equal(tr.warningIssues, 0, "should have no warnings");
        assert.equal(tr.errorIssues.length, 1, "should have 1 error issue");
        if (process.platform == 'win32') {
            assert.equal(tr.errorIssues[0], '/mocked/tools/cmd failed with return code: 1', 'error issue output');
        }
        else {
            assert.equal(tr.errorIssues[0], '/mocked/tools/echo failed with return code: 1', 'error issue output');
        }
        assert(tr.stdout.indexOf('atool output here') >= 0, "tool stdout");
        assert.equal(tr.stdout.indexOf('Hello Mock!'), -1, "task module should have never been called");

        done();
    });
```

The [failrc.ts test file is here](https://gist.github.com/bryanmacfarlane/154f14dd8cb11a71ef04b0c836e5be6e#file-failrc-ts)

### Fail on required inputs

We also typically write tests to confirm the task fails with proper actionable guidance if required inputs are not supplied.  Give it a try.

### Running the tests

```bash
mocha tests/_suite.js
```

```bash
$ mocha tests/_suite.js

  Sample task tests
    ✓ should succeed with simple inputs (127ms)
    ✓ it should fail if tool returns 1 (121ms)

  2 passing (257ms)
```

If you want to run tests with verbose task output (what you would see in the build console) set envvar TASK_TEST_TRACE=1

```bash
export TASK_TEST_TRACE=1
```

## Add Task to an Extension

### Create your publisher

All extensions are identified as being provided by a publisher. If you aren't already a member of an existing publisher, you'll create one. Sign in to the [Visual Studio Marketplace Publishing Portal](http://aka.ms/vsmarketplace-manage). If you're not prompted to create a publisher, scroll down to the bottom of the page and select *Publish Extensions* underneath **Related Sites**.

### Create an extension manifest file

You will need to create the extension manifest file in the directory above `sampletask`.

Create a file `vss-extension.json`:
```json
{
    "manifestVersion": 1,
    "id": "sample-task",
    "name": "Sample Build Tools",
    "version": "0.0.1",
    "publisher": "samplepublisher",
    "targets": [
        {
            "id": "Microsoft.VisualStudio.Services"
        }
    ],
    "description": "Sample tools for building. Includes one build task.",
    "categories": [
        "Build and release"
    ],
    "//uncomment 'icons' below to include a custom icon": "",
    "//icons": {
        "default": "images/extension-icon.png"
    },
    "files": [
        {
            "//Relative path of the task directory": "",
            "path": "sampletask"
        }
    ],
    "contributions": [
        {
            "//Identifier of the contribution. Must be unique within the extension. Does not need to match the name of the build task, but typically the build task name is included in the ID of the contribution.": "",
            "id": "sample-build-task",
            "type": "ms.vss-distributed-task.task",
            "targets": [
                "ms.vss-distributed-task.tasks"
            ],
            "properties": {
                "//Name of the task. This must match the folder name of the corresponding self-contained build task definition.": "",
                "name": "sampletask"
            }
        }
    ]
}
```

### Publish, Install, Publish

Publish the extension to the Marketplace and grant your account the ability to see it. Sharing the extension with your account allows you to install and test it.

You will need a personal access token scoped to `All accessible` accounts.

```
tfx extension publish --manifest-globs your-manifest.json --share-with youraccountname
```

Install the extension into your account.

```
tfx extension install --vsix .\samplepublisher.sample-task-0.0.1.vsix --accounts youraccountname
```

Future publishes will automatically update the extension installed into your account. Add `--rev-version` when publishing updates to the extension, and also rev the task version (in the `task.json`).

repository: Cristie/vsts-task-lib

gha_language: TypeScript

repo_name: Cristie/vsts-task-lib


text:
# EEG-neural-decoding
EEG neural decoding for lower limb during gait using machine learning and neural networks

Still haven't added the instructions on how to use the code just yet. Will be working on this soon.

The data can be accessed via Scientific Data: https://www.nature.com/articles/sdata201874

repository: shonaka/EEG-neural-decoding

repo_name: shonaka/EEG-neural-decoding


text:
# Selfdriving car
A neuroevolutionary self driving (software) car.

## Components
###### Car
  - Direction
  - Speed (constant?)
  - Sensors
  - Connected to an ANN
  - Every step that it makes while driving flows trough the ANN

###### Road
  - Show cars
  - Detect collisions
  - Indicate distance traveled by each car

###### Artificial Neural Network
  - Nodes and weights.
  - Variable weights
  - Input: data from car sensors
  - Output: value between -1 (60 degrees to the left) and 1 (60 degrees to the right)

###### Evolver
  - Gets the ANN, tests and reproduces them:
    - Makes 50 cars with random weights
    - Applies the ANN to the cars and measure their distance
    - Selects the 25 best cars
    - Makes 25 new cars with weights inherited from the top 25
    - Applies a mutation
    - Replaces bad 25 with good 25
    - Repeat

repository: DiamondOperators/selfdriving-car

repo_name: DiamondOperators/selfdriving-car


text:
# GoogleNewsApi
A REST api which will give all news from any location of different categories and also related news

### Installation
```
npm i googlenewsapi

```
### Sample

```
News {
  title: 'Watch SpaceX get very close to catching its rocket’s nose cone during a test',
  description: 'SpaceX is getting excruciatingly close to catching a part of its rocket that falls out of the sky after each launch. A new video shows a recent drop test of the ...',
  thumbnail: 'https://lh3.googleusercontent.com/NKOw3Vvids-nNCfx1vmll-wR88AvTqnRK3vakt4XhxhtYzgWlurhPEOWY-lq413YTava0bMKdwIKrxQvRjQ=pf-w200-h200',
  publisher: 'The Verge',
  publishedAt: 'seconds:<PHONE_NUMBER>\n',
  category: 'Science',
  url: 'https://news.google.com/articles/CAIiEGxY4QfBKcLIDNpvJ1mDS2AqFggEKg4IACoGCAow3O8nMMqOBjD38Ak?hl=en-US&gl=US&ceid=US%3Aen',
  related:
   [ News {
       title: 'Watch SpaceX rocket\'s nose cone miss the boat, again',
       description: 'A ship named Mr. Steven keeps trying to catch the thing.',
       thumbnail: '',
       publisher: 'CNET',
       publishedAt:<PHONE_NUMBER>,
       category: 'Science',
       url: 'https://news.google.com/articles/CAIiECurGhHT3ziGtrQLkWIu6OIqEwgEKgwIACoFCAow4GowoAgwkRo?hl=en-US&gl=US&ceid=US%3Aen',
       related: []
       }
   ]
}

```
### Usage
```
var googleNewsApi=require('googlenewsapi');
// Initialize with storeFunction,country code.
googleNewsApi.initialize(storeFunction);
googleNewsApi.addCountryCode(code);
setInterval(googleNewsApi.refresh,1000*60*60);

//Search
googleNewsApi.search('Modi','IN').then((results)=>{

})
```

### Built With

Following is list of  libraries used in app.
1) async
2) cheerio
5) request

## Authors

* **Bhavneet Singh**  - [singhbhavneet](https://github.com/singhbhavneet)

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

## Acknowledgments

Its because of my mummy's motivation,god's blessing and family'support that I am able to complete this project.

repository: YasserDbeis/Refugee-Tracker

gha_language: Python

repo_name: YasserDbeis/Refugee-Tracker


text:
[![Build Status](https://travis-ci.org/hugopl/textui.svg?branch=master)](https://travis-ci.org/hugopl/textui)

# TextUI

A simple [Crystal](https://https://crystal-lang.org/) UI framework for terminal interfaces backed by [termbox](https://github.com/nsf/termbox).

**It still in a pre-alpha state**, API is not stable at all and this shard exists mainly because I'm using this lib in more than one project. In fact the repository is a fork from [QueryIt](https://github.com/hugopl/queryit), the original project were it was born.

So if you are brave enough to add this shard as a dependency into your project I suggest to use the `commit: ...` attribute into the textui dependency entry on your shard.yml.

## Installation

1. Add the dependency to your `shard.yml`:

   ```yaml
   dependencies:
     textui:
       github: hugopl/textui
   ```

2. Run `shards install`

## Usage

Example soon...

## Contributing

1. Fork it (<https://github.com/hugopl/textui/fork>)
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request

Feature suggestions are welcome, write them in github issues.

## Contributors

- [Hugo Parente Lima](https://github.com/hugopl) - creator and maintainer


text:
Thanks for your interest!

In order to clarify the intellectual property license granted for contributions to EA open source projects you must sign the [Contributor License Agreement (CLA)](https://go.ea.com/cla).

repository: Kingminghuang/orbit

repo_name: Kingminghuang/orbit


text:
---
layout: post
title: Making BlueMail Work For You | Adding a Gmail Account
date: 2016-09-14 09:05:45.000000000 +03:00
type: post
published: true
status: publish
categories:
- Accounts
tags: []
meta: /bluemail-adding-a-gmail-account/
---

In this episode of *Making BlueMail Work For You*, we learn how to add a Gmail account.

To add a Gmail account, tap *Unified* \| *More* \| *Add an Account* \| *Enter your Google Account info* \| *Allow \| ***Add your Name and Description***

repository: atesterlabs/bmtips

repo_name: atesterlabs/bmtips


text:
# Geode Enterprise Data Integration (GEDI) security extension

This project contains a Security Manager implementation for Apache Geode
version 1.2  (or GemFire version 9) and higher.

[Core Project](security-api) - core security module implementation

[Integration Test](security-int-tests) - runtime security verification tests


text:
# Adam Mansell's Anagrams & Antigrams Project

### Ruby Friday Project for Epicodus, 9/25/2020

## **By Adam Mansell**

## Description
In this project, we test whether two words are anagrams or antigrams.

## Setup
Clone this repository.

Open the project files in your text editor.

To view a file in the browser:

Right click one file at a time & copy it's path.

Open your prefered internet browser and paste the file path to view the file.
(If using mac, after pasting link in search bar, check for "file" dropdown option before hitting return).

## Technologies Used
For this project I used Ruby.

## Legal
_Copyright (c) 2020 Adam Mansell_

_This software is licensed under the MIT license._

repository: AdamMansell/ana-anti

repo_name: AdamMansell/ana-anti


text:
# What is Two Dimentional Array?

The two-dimensional array can be defined as an array of arrays. The 2D array is organized as matrices which can be represented as the collection of rows and columns. However, 2D arrays are created to implement a relational database lookalike data structure. It provides ease of holding the bulk of data at once which can be passed to any number of functions wherever required.

### Declaration of two dimensional Array in C:

The syntax to declare the 2D array is given below.

> arr_type arr_name [ rows ] [ cols ] ;

Where arr_name is the name of the array being declared, each element of which is of type arr_type. The expressions rows and cols enclosed in square brackets are constant integral expressions (or integral constants) that specify the number of rows and columns in the array. When a compiler comes across the declaration of an array, it reserves memory for the specified number of array elements. For a two-dimensional array, the memory required is given as rows* cols* sizeof(arr_type).

Note that we can declare matrices of built-in as well as user-defined types; we can also declare multiple matrices in a single declaration statement, separated by commas. Scalar variables, vectors and multidimensional arrays can also be declared along with matrices.

## Description of the algorithm

In the 1D array, we don't need to specify the size of the array if the declaration and initialization are being done simultaneously. However, this will not work with 2D arrays.
Just as with one-dimensional arrays, you cannot change
the size of a two-dimensional array once it has been
defined.
But you can initialize a 2-D array like this:

```python
int counts[COUNTRIES][MEDALS] =
 {
 { 0, 3, 0 },
 { 0, 0, 1 },
 { 0, 0, 1 },
 { 1, 0, 0 },
 { 0, 0, 1 },
 { 3, 1, 1 },
 { 0, 1, 0 }
 { 1, 0, 1 }
 };
```

## Time complexity

The time complexity of two dimentional array is O(n^2)

repository: myNameArnav/dsa-visualizer

repo_name: myNameArnav/dsa-visualizer


text:
# Bandit

Visit the [website][1].

Server details:
```
url:   bandit.labs.overthewire.org
port:  2220
user:  bandit<level>
```

For convenience use
```bash
./connect.sh <level>
```

### Storing solutions

Once you have a password add it to a `passwords` file in the form `<level> <password`. When you run the `./connect.sh`
script it will print the password for the level if it exists so you can easily copy and paste it.
For example
```
0 bandit0
1<PHONE_NUMBER>abcdef
2 fedcba0987654321
```

In the case where you get the private key for a level store it in a file named `bandit<level>_key` e.g. if you have found
the key for `bandit24` store it in the file `bandit24_key`. This key will be used automatically when running the
`./connect.sh` script for that level (e.g. `./connect.sh 24`).

The starting password for level `0` is `bandit0`.

### Levels

Instructions for each level can be found at
```
https://overthewire.org/wargames/bandit/bandit<level>.html
```

##### 0

`cat readme`

##### 1

`cat ./-`

##### 2

`cat` the filename

##### 3

`cat` the filename

##### 4

`grep . -- inhere/*`

##### 5

`find . -type f -size 1033c | xargs cat`

##### 6

`find / -type f -size 33c -group bandit6 -user bandit7 2>/dev/null | xargs cat`

##### 7

`grep millionth data.txt`

##### 8

`sort data.txt | uniq -u`

##### 9

`strings data.txt | grep "="`

##### 10

`base64 -d data.txt`

##### 11

`tr "[a-zA-Z]" "[n-za-mN-ZA-M]" < data.txt`

##### 12

```bash
tmp=/tmp/boooooooooom
mkdir $tmp
cd $tmp

cp ~/data.txt .
xxd -r data.txt > data2.bin.gz
gzip -d data2.bin.gz
bzip2 -d data2.bin
mv data2.bin.out data4.bin.gz
gzip -d data4.bin.gz
tar -xvf data4.bin
tar -xvf data5.bin
bzip2 -d data6.bin
tar -xvf data6.bin.out
mc data8.bin data9.bin.gz
gzip -d data9.bin.gz
cat data9.bin
```

##### 13

Copy the key to `bandit14_key` locally - `connect.sh` will pick this up automatically :)

##### 14

`cat /etc/bandit_pass/bandit14 | nc localhost 30000`

##### 15

`(cat /etc/bandit_pass/bandit15; sleep 1) | openssl s_client -connect :30001`

##### 16

Copy the output from below to `bandit17_key`.

```bash
ports=$(nc -zv localhost 31000-32000 2>&1 | awk '{ print $3 }')
key_found=0

for i in $ports
do
    (cat /etc/bandit_pass/bandit16; sleep 1) \
    | openssl s_client -connect :$i 2>/dev/null
done \
| while read line
    do
        if grep -q 'BEGIN RSA PRIVATE KEY' <<< $line
        then
            key_found=1
        fi
        if [ "${key_found}" -eq "1" ]
        then
            echo $line
        fi
        if grep -q 'END RSA PRIVATE KEY' <<< $line
        then
            key_found=0
        fi
    done
```

##### 17

`diff passwords.new passwords.old`

##### 18

```bash
# connect and set the command to sh e.g.
./connect 18 sh
# or
ssh ..... sh

cat readme
```

##### 19

`./bandit20-do cat /etc/bandit_pass/bandit20`

##### 20

```bash
cat /etc/bandit_pass/bandit20 | nc -lp 33333 &
./suconnect 33333
```

##### 21

```bash
# get the script name being run
cat /etc/cron.d/cronjob_bandit22
# find the temp file being created in the script and cat it
cat $(grep -o "/tmp/.*" /usr/bin/cronjob_bandit22.sh | uniq)
```

##### 22

```bash
# get the script name being run
cat /etc/cron.d/cronjob_bandit23
# regenerate the name and cat it
cat /tmp/$(echo I am user bandit23 | md5sum | cut -d ' ' -f 1
```

##### 23

```bash
cronfile=/var/spool/bandit24/andrewflbarnes24
outdir=/tmp/andrewflbarnes24
outfile=${outdir}/password

# add a shell script which the cronjob /etc/cron.d/cronjob_bandit24 will execute
cat << EOF > ${cronfile}
# make the output directory
mkdir -p ${outdir}
# copy the password file for bandit 24
cp /etc/bandit_pass/bandit24 ${outfile}
# make sure the file is readable
chmod 777 ${outfile}
EOF

# make sure the cron job file is executable
chmod a+x ${cronfile}

# wait for the output file to be created
while ! ls ${outfile} 2>/dev/null
do
    sleep 1
    echo "$(date -R) Checking for existence of ${outfile}"
done

# get the password!
cat ${outfile}

```

##### 24

```bash
for i in {0..9999}
do
    printf "%s %04d\n" $pass $i
done \
| nc localhost 30002 \
| grep -Ev "(Wrong|I am the pincode checker)"
```

##### 25

Copy SSH key

##### 26

Check the shell in use while logged in as `bandit25` e.g. `grep bandit26 /etc/passwd`.

Collapse your terminal real smallllllll so that `more` shows up, then you can
```
v
set shell=/bin/bash
:shell
./bandit27-do cat /etc/bandit_pass/bandit27
```

##### 27

```bash
mkdir -p /tmp/andrewflbarnes27
cd /tmp/andrewflbarnes27
git clone ssh://bandit27-git@localhost/home/bandit27-git/repo
# enter yes
# enter bandit27 password
cd repo
cat README
```

##### 28

```bash
mkdir -p /tmp/andrewflbarnes28
cd /tmp/andrewflbarnes28
git clone ssh://bandit28-git@localhost/home/bandit28-git/repo
# enter yes
# enter bandit28 password
cd repo

git log -p | grep "^+.*password:" | grep -vE "(xxxxxx|TBD)"
```

##### 29

```bash
mkdir -p /tmp/andrewflbarnes29
cd /tmp/andrewflbarnes29
git clone ssh://bandit29-git@localhost/home/bandit29-git/repo
# enter yes
# enter bandit29 password
cd repo

git log --all -p | grep "^+.*password:" | grep -v "no passwords"
```

##### 30

```bash
mkdir -p /tmp/andrewflbarnes30
cd /tmp/andrewflbarnes30
git clone ssh://bandit30-git@localhost/home/bandit30-git/repo
# enter yes
# enter bandit30 password
cd repo

git tag -l
# ohh look, a secret tag
git show secret
```

##### 31

```bash
mkdir -p /tmp/andrewflbarnes31
cd /tmp/andrewflbarnes31
git clone ssh://bandit31-git@localhost/home/bandit31-git/repo
# enter yes
# enter bandit31 password
cd repo

echo "May I come in?" > key.txt
> .gitignore
git commit -m "Get me the password :)"
git push
```

##### 32

```
# re-execute the shell - but why does this work?
$0

cat /etc/bandit_pass/bandit33
```

##### 33

Done!

[1]: <https://overthewire.org/wargames/bandit/> "Bandit wargames landing page"

repository: andrewflbarnes/overthewire

repo_name: andrewflbarnes/overthewire


text:
# Overview

A easy to use tmux configuration with basic mouse and system clipboard. This
configuration aims to simplify the navigation between sessions, windows, and
panes, creation of workspaces, and copy-pasting with OS integration. It tries to
do this all minimally.

# Table of Contents

- Requirements
- Installation
- Key Bindings
- Recommendations

# Requirements

tmux version 2.9a or above

You will need to have the following items installed on your system before making
use of this configuration.

- A program to interact with the system clipboard:

    - Linux: install `xsel`
    ```plaintext
    apt-get install xsel --no-install-recommends
    ```

    - macOS: `pbcopy`

# Installation

## macOS

To enable support for the meta key and shift modifer in `Terminal.app`, you will
need to do the following.

1. Enable Meta key and scroll support
    1. Open `Terminal.app`.
    2. Select `Terminal > Preferences` from the menu bar.
    3. Select the `Profiles` tab from the preferences window.
    4. In the pane on the left, select the profile marked with the text
    "Default" under the profile name.
    5. Select the `Keyboard` tab in the from the
    6. Enable the following options from the bottom of the <>:
        - `Use Option as Meta key`
        - `Scroll alternate screen`
2. Add correct keycodes for Shift modifiers.
    1. Select the `+`.

# Key Bindings

## Prefix Keys

`C-a`: enter command mode.

`C-s`: enter copy mode.

## Session Management

`<prefix> d`: detatch from session.

`<prefix> s`: list all sessions. Can also select a session to connect to by
              navigating with the arrow keys and selecting with enter.

`<prefix> (`: previous session
`<prefix> )`: next session

## Window/Pane Management

# Recommendations

## Use `Caps Lock` as `Control`

### macOS

repository: alam/tmux-configs

repo_name: alam/tmux-configs


text:
# Real-Time-Object-Detection-using-Tensorflow.js

### Made by sunny dhama

## Real Time Demonstration:
* [Click Here for Real Time Demo](https://sunny-od.netlify.com/)

  * Note: Loading the model can take several seconds.
  * It is best to load the model once and save a reference to it.
  * Use Google Chrome Browser on PC, Tablets, Phones. (Highly Recommended)

## Object Detection:
Object detection is the process of finding instances of real-world objects such as faces, bicycles, and buildings in images or videos. Object detection algorithms typically use extracted features and learning algorithms to recognize instances of an object category.

## Dependencies:

* [Tensorflow Models: Coco SSD](https://www.npmjs.com/package/@tensorflow-models/coco-ssd) - Object detection model that aims to localize and identify multiple objects in a single image.
* [Tensorflow.js](https://www.npmjs.com/package/@tensorflow/tfjs) - TensorFlow.js is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models.
* [React](https://www.npmjs.com/package/react) - React is a JavaScript library for creating user interfaces.
* [React DOM](https://www.npmjs.com/package/react-dom) - This package serves as the entry point to the DOM and server renderers for React. It is intended to be paired with the generic React package, which is shipped as react to npm.
* [React Scripts](https://www.npmjs.com/package/react-scripts) - This package includes scripts and configuration used by Create React App.

## Detecting Objects
To make object detection predictions, all we need to do is import the TensorFlow model, coco-ssd, which can be installed with a package manager like NPM or simply imported in a <script> tag. We can then load the model, and make a prediction.

 ```
import * as cocoSsd from "@tensorflow-models/coco-ssd";
const image = document.getElementById("image")
cocoSsd.load()
  .then(model => model.detect(image))
  .then(predictions => console.log(predictions))
```

The image we pass into the detection function is just a reference to the html <img> tag:
```
<img id="image" src="image_url">

```

After we get our prediction, we need a way to display it to the screen. It should look something like this:

```
[{
  bbox: [x, y, width, height],
  class: "cat",
  score: 0.8380282521247864
}]

```

We then use the <canvas> element:

```
<canvas id="canvas">
```

The canvas element allows us to use the strokeRect function, which maps perfectly with our prediction results:

```
const x = prediction.bbox[0];
const y = prediction.bbox[1];
const width = prediction.bbox[2];
const height = prediction.bbox[3];

const canvas = document.getElementById("canvas");
const ctx = canvas.getContext("2d");

ctx.strokeRect(x, y, width, height);
```

## Deployment
The Project is deployed in Codesandbox

## Streaming from the Webcam
To run real-time detection on a webcam stream is almost as easy as changing from an <img> tag, to a <video> tag …with the simple exception of this giant blob of code to start up the webcam:

 ```
const video = document.getElementById("video")

navigator.mediaDevices
  .getUserMedia({
    audio: false,
    video: {
      facingMode: "user",
      width: 600,
      height: 500
    }
  })
  .then(stream => {
    video.srcObject = stream
    video.onloadedmetadata = () => {
      video.play()
    }
  })

```

We can then just pass our video element to our model for detection. However, this time we are going to call requestAnimationFrame which will call our detection function over and over in an infinite loop as fast as it can, skipping frames when it can’t keep up.

```
function detectFrame() {
  model.detect(video).then(predictions => {
    renderOurPredictions(predictions)
    requestAnimationFrame(detectFrame)
  })
}
```


text:
## Cloud DevOps
Google Cloud DevOps Scripts and References

## Continuous Deployment on Google Cloud

![screenshot1](images/gcp_devops1.png?raw=true "GCP Continuous Deployment")

![screenshot2](images/gcp_devops2.png?raw=true "Continuous Delivery Pipeline")

## References:

 * [Google Cloud Build](https://cloud.google.com/cloud-build/)
 * [Google Cloud Container Registry](https://cloud.google.com/container-registry/)
 * [Google Cloud Source Repos](https://cloud.google.com/source-repositories/)
 * [Google Cloud Kubernetes Engine](https://cloud.google.com/kubernetes-engine/)
 * [Google Cloud App Engine](https://cloud.google.com/appengine/)
 * [Continuous Delivery with Spinnaker Kubernetes Engine](https://cloud.google.com/solutions/continuous-delivery-spinnaker-kubernetes-engine)


text:
# ArchComponentPlayground

A demo project used for investigating the implementation of [Android Architecture Components](https://developer.android.google.cn/topic/libraries/architecture/index.html).

Currently, for ViewModel, you can find my article (Chinese version) [here](https://www.jianshu.com/p/7505f79b3ae3).

repository: rayworks/ArchComponentPlayground

repo_name: rayworks/ArchComponentPlayground


text:
# RPi and Mbed are the best friends

### Mbed firmware

The code is available at : https://developer.mbed.org/users/vfdev/code/futurakart_mbed/

Compiled binary files are available [here](../mbed)

### Upload your firmware to `mbed`

This a particular case when you connect `mbed` cart to `rpi` and access
to `rpi` using ssh or sftp.

Use SSH access to
1) Mount `mbed` accessible point to `rpi`. Search it as `/dev/sd*`.
Make a mount point folder, e.g. `mkdir ~/mbed_repository`
```
sudo mount /dev/sda ~/mbed_repository -o gid=ubuntu,uid=ubuntu
```

To make a permanent mount see below.

2) Connect to `rpi` from the desktop using `sftp`:
- open nautilus
- "Connect to server" and write `sftp://username@<IP>`
- Copy firmware file to `~/mbed_repository`

### Permanent mount `mbed` to `rpi`

Assuming that you are already mounted the `mbed` and created mount point
at `~/mbed_repository`

Find out the UUID:
```
sudo blkid
...
/dev/sda: SEC_TYPE="msdos" LABEL="NUCLEO" UUID="2702-1974" TYPE="vfat"
```

Add to the `/etc/fstab` the following line:
```
UUID="2702-1974" /home/ubuntu/mbed_repository vfat nofail,gid=ubuntu,uid=ubuntu,x-systemd.device-timeout=10 0 0
```
See [this](http://www.pclosmag.com/html/Issues/200709/page07.html) and [this](http://unix.stackexchange.com/questions/314271/automount-disk-connected-through-usb-when-the-pc-is-turned-on) for more info

repository: RoseTeam/Futurakart

repo_name: RoseTeam/Futurakart


text:
# UnrealDI
[![MIT license](http://img.shields.io/badge/license-MIT-brightgreen.svg)](http://opensource.org/licenses/MIT)

Dependency Injection container for Unreal Engine. Container will take care of objects creation and their lifetime.

This implementation works on top of UObjects and UInterfaces.

## Getting started
First, you need to setup your container and register some types into it:
```cpp
// create builder object
FContainerBuilder Builder;

// register type UMyService as implementation of IMyService interface
Builder.RegisterType<UMyService>().As<IMyService>();

// create container itself
UObjectContainer* Container = Builder.Build();
```

After that you can ask it to resolve some instances:
```cpp
TScriptInterface<IMyService> MyService = Container->Resolve<IMyService>();
```

## Supported versions
UnrealDI was tested to work with Unreal versions 4.25 and 4.26. It should probably work with 4.27 as well, but no testing was done yet.

## Repository structure
This repository contains two plugins:
* UnrealDI — contains DI functionality to be used in game.
* UnrealDITests — contains Tests for main plugin and not intended for shipping.

## Installation
Copy UnrealDI folder into Plugins inside your project.

## Acknowledgment
UnrealDI is heavily inspired by [Hypodermic](https://github.com/ybainier/Hypodermic), but designed with Unreal Engine specifics in mind.

repository: denfrost/UnrealDI

repo_name: denfrost/UnrealDI


text:
---
layout: post
title: The European Union is what the United States was meant to be
date: 2016-06-27 17:32
description: American conservatives and libertarians should celebrate the EU and deplore Brexit.
categories: ["politics"]

show_meta: true
comments: true
mathjax: true
gistembed: true
published: true
noindex: false
nofollow: false
---

If there's one single issue which is the defining aspect of American conservatism,
one core principle which sits in conservatives' very bones, which they breathe in
and breathe out, it might be *limited government*, which you can recast without
too much trouble into *the federal government has too much damn power (and got it through underhanded means)*.

Yet for some reason, American conservatives and libertarians tend to look with approval
on the Brexiteers. A bold bid for independence, striking back at the chilly gray
halls of power in Brussels! The isles standing alone for freedom, once again!
If only we had some baddies to fight!

American conservatives shouldn't feel this way. The European Union is what the
American Founders had in mind.

Remember, we first took a stab at a union of states under the Articles of Confederation.
This was such a failure that the Founders were sent to modify the Articles, and
instead just started a Constitutional Convention to hammer out a brand-new governemnt.

This new federal government eliminated interstate tariffs and barriers, demanded
that each state give full faith and credit to each other's judgments, ensured that
no state could discriminate against the citizens of another state. Suddenly, the country
open up *within*.

However, over the past couple of centuries the federal government has also demanded
the power to regulate (for instance) how much wheat a single farmer can grow on his
land, to feed his own family. Arguably (because man do we argue about this), that's
too much power.

The EU has also dealt with all the problems that we had under the Articles of Confederation,
but all the states remain sovereign, a true federation.

Maybe you can think of a sort of Goldilocks process of governmental design.
First the American Founders tried the Articles of Confederation. This was too
little centralization, not enough unification.

Then American Founders tried the Constitution, which allows the federal
government, over time and after a war, to arrogate to itself every more powers.
Too much government!

Finally, over half a century, the European Union was formed, and this is just
about enough.

The Europeahn Union has so far managed to avoid excessive centralization for two
main reasons.

First, they built themselves bottom-up, piece by piece, over a long time.
I can hear the British Euroskeptics snorting now: "What do you mean it was built
bottom up?

Be that as it may, every power the European Council has gained has beeen granted
it by the member states. The United States binds itself in this way too, through
international treaties.

By accepting one treaty at a time, which are then part of the laws of the United
States, we've also built up international law piece by piece. It's a slow,
modular process.

Contrast this, again, with the relatively one-shot deal of the American Constitution.
Ratified by the states, and boom, all the core powers of the federal government
are right there. And if you're not sure if the federal government has a power, who
decides? Oh yeah, the federal government.

Which brings us to the second reason the EU has unified without over-centralization.

Any new government, or competency as they say in EU law, can be vetoed by the member
states. Who decides if the government has a new power? The states do.

Way back when, Alexander Hamilton argued that even though the Constitution didn't
explicitly give the government the power to establish a national bank, the federal
government had that power, because otherwise they couldn't do their job. The Constitution
*implicitly* granted the government that power.

Look, I'm not accusing anyone of bad faith. I'm just noting that if you ask the
government if they have a legal power, which would help them do their job better,
usually they're going to say *yes*. In the EU, they tend to ask the member states.

Freedom does not equal isolationism. No state should have to go all Idaho-militia,
live-in-the-woods-alone, to avoid surrendering local government to a central and
coordinating government.

The EU, so far, has managed to unify without over-regulating. American
conservatives and libertarians should celebrate the EU. The Europeans finally
figured out what our Founding Fathers were talking about - and they may have
done it even better than we did.


text:
---
title: Participants
---

## Workshop participants

### Organizers
The organizing committee consists of:

 - [Jacqueline O'Connor](http://www.mne.psu.edu/rfdl/), Penn State University
 - [Richard West](http://www.northeastern.edu/comocheng/index.php), Northeastern University
 - [Nicole Labbe](http://www.colorado.edu/mechanical/nicole-labbe), University of Colorado Boulder
 - [Kyle Niemeyer](http://kyleniemeyer.com), Oregon State University

### Participants

More information about the participants will be added here closer to the workshop. The total number of participants, including the organizers, will be limited to 45.


text:
Screen name validator
=============================
This plugin sample provides a customization of screen name validation.When deploying the plugin
liferay with use this validator to validate the screen name both client and server validation.

To modify this example you can modify the `com.liferay.blade.samples.screenname.validator.internal.CustomScreenNameValidator`.
The plugin check if a screen name contains reserved words that are configured  in

    Control Panel --> Configuration --> System Settings --> Foundation --> ScreenName Validator
    default value is  "admin|user"

It's possible to customize the configuration adding more properties in `com.liferay.blade.samples.screenname.validator.CustomScreenNameConfiguration`

Testing the validator
---------------------
   - Deploy the plugin
   - Go to `Control Panel --> Users --> Users and Organizations`
   - Create a new user
   - adding the screen name that contains the word `admin` or `user`

For more information on customizing the Validation sample to fit your
needs, see the Javadoc listed in this sample's
`CustomScreenNameValidator` class.

repository: bruinen/liferay-blade-samples

repo_name: bruinen/liferay-blade-samples


text:
# flutter music

一个自己拿来练手的flutter项目，仿照网易云音乐客户端编写，

## Getting Started

- 下载源代码

```$xslt
git clone https://github.com/dczxw/flutter_music.git
```

- 安装依赖
```$xslt
flutter packages get
```

## Demo截图

![alt](http://img.siuper.cn/ebd031b88e3f9d7b.png) ![alt](http://img.siuper.cn/0788fb7372e96f53.png)

## License
```
Copyright 2019 Philipp Jahoda

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
```


text:
# visage-verifie
Facial recognition software to detect if an ID matches a live video of a user
using [face-api.js](https://github.com/justadudewhohacks/face-api.js)

To use locally:
1. Clone the repository
2. `$ cd visage-verifie/examples/examples-browser/`
3. `$ npm install`
4. `$ npm start`
5. Go to localhost:3000 in the browser and have fun!

Note: on Macs, if having issues with webcam permissions in Chrome open localhost:3000 in Safari instead

repository: hypatiah/visage-verifie

branch_name: refs/heads/development

repo_name: hypatiah/visage-verifie