text:
# UNITY BENCHMARK SERVICE README
UNITY is a workflow benchmarking platform used for developing and comparing pipelines.

## SETUP

This application is built and deployed using [Docker](https://www.docker.com), specifically native [Docker for Mac OSX](https://docs.docker.com/docker-for-mac/).
Please refer to their online documentation for instructions on installing and creating a default VM for managing Docker images.

## BUILDING THE DOCKER IMAGE

Once all source files are checked out and Docker has been installed and your VM configured, open a terminal window and execute the following steps:

1. Navigate to the project directory
2. Build the Unity Benchmark image: docker build -t unity_benchmark_docker:[version_number] -f Dockerfile .

This will start the automated process of building the Docker image for running the service.  The image is built off of
the [Passenger-docker baseimage](https://github.com/phusion/passenger-docker) and comes with Ruby, Nginx, and Passenger
by default, with additional packages added to the [Broad Institute KDUX Rails baseimage](https://hub.docker.com/r/broadinstitute/kdux-rails-baseimage/)
which pulls from the original baseimage.

If this is your first time building the image, it may take several minutes to download and install everything.

## BEFORE RUNNING THE CONTAINER IN DEVELOPMENT

Since this project utilizes native Docker for Mac OSX, any resources on the host machine cannot be reached by the running
container (specifically, any database resources). Therefore, we will need to deploy a database container using Docker
as well.  This project uses [postgres](https://hub.docker.com/_/postgres/) as the primary datastore.

First, create a directory somewhere on your computer in which to store the raw database content (it doesn't matter where
as long as it has rw permissions, but preferably it would be inside your home directory).

To deploy the database container:

1. Pull the image: docker pull postgres
2. Navigate to the project directory
3. Run the helper script to start the DB container: bin/boot_postgres -d (path to data store directory) -u (postgres username) -p (postgres password)

Note: Once the container has been run once, you can stop & restart it using: docker stop postgres or
docker restart postgres

When deploying the service in production, the application will connect to a remote DB host that must be provisioned and configured separately.

## DEPLOYING A PRIVATE INSTANCE

If you are deploying a private instance of the Unity Benchmark Service, there are a few extra steps that need to be taken before the
portal is configured and ready to use:

* Create a GCP project: Even if you are deploying locally or in a private cluster, the portal requires a Google Cloud Plaform project in order to handle OAuth callbacks and service account credentials.  To create your project:
  * Visit https://console.developers.google.com
  * Click 'Select a project' in the top lefthand corner and click the + button
  * Name your new project and save
* OAuth Credentials: Once your project is created, you will need to create an OAuth Client ID in order to allow users to log in with their Google accounts.  To do so:
  * Log into your new GCP project
  * Click the navigation menu in the top left and select 'APIs & Services' > 'Credentials'
  * Click 'Create Credentials' > 'OAuth Client ID'
  * Select 'Web Application', and provide a name
  * For 'Authorized Javascript Origins', enter https://(your hostname)/
  * For 'Authorized redirect URIs', enter https://(your hostname)//omniauth/google_oauth2/callback
  * Save the client id
* Whitelisting your OAuth Audience
	* Once you have exported your OAuth credentials, you will need to have your client id whitelisted to allow it to make
	  authenticated requests into the FireCloud API as per [OpenID Connect 1.0](http://openid.net/specs/openid-connect-core-1_0.html#IDTokenValidation)
	* Send an email to<EMAIL_ADDRESS>with your OAuth2 client ID so it can be added to the whitelist
* GCP Service Account keys: Regardless of where the portal is deployed, it requires three Google Cloud Platform Service Accounts in order to make authenticated calls into FireCloud, Google Cloud Storage, and Cloud SQL (if deploying in Kubernetes).  Therefore, you must create these service account keys:
.  See https://developers.google.com/identity/protocols/OAuth2ServiceAccount for more information about service accounts.  To export the credentials:
  * Log into your new GCP project
  * Click the navigation menu in the top left and select 'IAM & Admin	' > 'Service Accounts'
  * On entry 'Compute Engine default service account', click the 'Options' menu (far right) and select 'Create key'
  * Select 'JSON' and export and save the key locally
  * Next, create two new service accounts with the following roles and export the keys:
    * Storage Admin
    * Cloud SQL Client (this key is only needed for Kubernetes-based deployments)
* Enable GCP APIs: The following Google Cloud Platform APIs must be enabled:
  * Google Compute Engine API
  * Google Cloud APIs
  * Google Cloud Billing API
  * Google Cloud Storage JSON API
  * Google+ API
  * Kubernetes Engine API (if deploying via Kubernetes)
* Registering your Service Account as a FireCloud user: Once you have configured and booted your instance of the portal, you will need to register your service account as a FireCloud user in order to interact with the FireCloud API.  To do so:
  1. Attach to the running instance of Unity (preferably a local instance in development mode, but can be a deployed instance)
docker exec -it unity_benchmark bash

  2. Enter the Rails console
bin/rails c [RAILS_ENV of running container]

  3. Instantiate the FireCloudClient
client = FireCloudClient.new

  4. Open and parse the sample FireCloud profile template
profile = JSON.parse(File.open(Rails.root.join('lib', 'assets', 'firecloud_profile_template.json')).read)

  5. Enter your desired values for every entry in the profile.
  6. Once the profile is completed, send your registration to the server
client.set_profile(profile)

## RUNNING THE CONTAINER

Once the image has successfully built and the database container is running, use the following command to start the container:

bin/boot_docker -u (sendgrid username) -P (sendgrid password) -E (encryption key) -k (service account key path) -K (gcs admin service account key path) -o (oauth client id) -S (oauth client secret) -l

This sets up several environment variables in your shell and then runs the following command:

docker run --rm -it --name $CONTAINER_NAME --link $DATABASE_HOST:$DATABASE_HOST -p 80:80 -p 443:443 -h localhost -v $PROJECT_DIR:/home/app/webapp:rw -e PASSENGER_APP_ENV=$PASSENGER_APP_ENV -e DATABASE_HOST=$DATABASE_HOST -e DATABASE_USER=$DATABASE_USER -e ENCRYPTION_KEY=$ENCRYPTION_KEY -e PROD_DATABASE_PASSWORD=$DATABASE_PASSWORD -e SERVICE_ACCOUNT_KEY=$SERVICE_ACCOUNT_KEY -e SENDGRID_USERNAME=$SENDGRID_USERNAME -e SENDGRID_PASSWORD=$SENDGRID_PASSWORD -e SECRET_KEY_BASE=$SECRET_KEY_BASE -e OAUTH_CLIENT_ID=$OAUTH_CLIENT_ID -e OAUTH_CLIENT_SECRET=$OAUTH_CLIENT_SECRET -e GOOGLE_CLOUD_KEYFILE_JSON="$GOOGLE_CLOUD_KEYFILE_JSON" -e GOOGLE_PRIVATE_KEY="$GOOGLE_PRIVATE_KEY" -e GOOGLE_CLIENT_EMAIL="$GOOGLE_CLIENT_EMAIL" -e GOOGLE_CLIENT_ID="$GOOGLE_CLIENT_ID" -e GOOGLE_CLOUD_PROJECT="$GOOGLE_CLOUD_PROJECT" unity_benchmark_docker:$DOCKER_IMAGE_VERSION

The container will then start running, and will execute its local startup scripts that will configure the application automatically.
You can then access your instance of Unity at https://localhost

You can also run the bin/boot_docker script in help mode by passing -H to print the help text
which will show you how to pass specific values to the above env variables.  Note: running the shortcut script with
an environment of 'production' will cause the container to spawn headlessly by passing the -d flag, rather
than --rm -it.

### DOCKER RUN COMMAND ENVIRONMENT VARIABLES
There are several variables that need to be passed to the Docker container in order to run properly:
1. *CONTAINER_NAME* (passed with --name): This names your container to whatever you want.  This is useful when linking containers.
1. *PROJECT_DIR* (passed with -v): This mounts your local working directory inside the Docker container.  Makes doing local development via hot deployment possible.
1. *PASSENGER_APP_ENV* (passed with -e): The Rails environment you wish to load.  Can be either development, test, or production (default is development).
1. *ENCRYPTION_KEY* (passed with -e): Salt value for encrypting sensitive information, like OAuth refresh tokens.  This must be at least 32 bytes in length, and cannot exceed 64 bytes.
1. *DATABASE_HOST* (passed with -e): Name of the container running postgres.  Even though our two containers are linked, this needs to be set to allow Rails to communicate with the database.
1. *DATABASE_USER* (passed with -e): Name of the container running postgres.  Even though our two containers are linked, this needs to be set to allow Rails to communicate with the database.
1. *DATABASE_PASSWORD* (passed with -e): Name of the container running postgres.  Even though our two containers are linked, this needs to be set to allow Rails to communicate with the database.
1. *SENDGRID_USERNAME* (passed with -e): The username associated with a Sendgrid account (for sending emails).
1. *SENDGRID_PASSWORD* (passed with -e): The password associated with a Sendgrid account (for sending emails).
1. *SECRET_KEY_BASE* (passed with -e): Sets the Rails SECRET_KEY_BASE environment variable, used mostly by Devise in authentication for cookies.
1. *SERVICE_ACCOUNT_KEY* (passed with -e): Sets the SERVICE_ACCOUNT_KEY environment variable, used for making authenticated API calls to FireCloud & GCP.
1. *GCS_ADMIN_GOOGLE_CLOUD_KEYFILE_JSON* (passed with -e): Sets the GCS_ADMIN_GOOGLE_CLOUD_KEYFILE_JSON environment variable, used for accessing user-controlled GCS assets.
1. *OAUTH_CLIENT_ID* (passed with -e): Sets the OAUTH_CLIENT_ID environment variable, used for Google OAuth2 integration.
1. *OAUTH_CLIENT_SECRET* (passed with -e): Sets the OAUTH_CLIENT_SECRET environment variable, used for Google OAuth2 integration.
1. *GOOGLE_[VARIOUS]* (passed with -e): If no SERVICE_ACCOUNT_KEY is present, the application will default to using the standard OAuth2 enviroment variables for authentication.  See [here](https://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-storage/v0.23.2/guides/authentication) for more information.
1. *DOCKER_IMAGE_VERSION*: sets the version of the unity_benchmark_docker image to use (defaults to _latest_)

### RUN COMMAND IN DETAIL
The run command explained in its entirety:
* *--rm:* This tells Docker to automatically clean up the container after exiting.
* *-it:* Leaves an interactive shell running in the foreground where the output of Nginx can be seen.
* --name CONTAINER_NAME: This names your container to whatever you want.  This is useful when linking other Docker containers to the portal container, or when connecting to a running container to check logs or environment variables.  The default is unity_benchmark.
* -p 80:80 -p 443:443 -p 587:587: Maps ports 80 (HTTP), 443 (HTTPS), and 587 (smtp) on the host machine to the corresponding ports inside the Docker container.
* --link $DATABASE_HOST:$DATABASE_HOST: Connects our webapp container to the postgres container, creating a virtual hostname inside the unity_benchmark container called postgres.
* -v [PROJECT_DIR]/:/home/app/webapp: This mounts your local working directory inside the running Docker container in the correct location for the portal to run.  This accomplishes two things:
  - Enables hot deployment for local development
  - Persists all project data past destruction of Docker container (since we're running with --rm), but not system-level log or tmp files.
* -e PASSENGER_APP_ENV= [RAILS_ENV]: The Rails environment.  Will default to development, so if you're doing a production deployment, set this accordingly.
* -e ENCRYPTION_KEY= [ENCRYPTION_KEY]: Salt value for encrypting sensitive information, like OAuth refresh tokens.  This value must be consistent, and between 32-64 bytes, or previously stored credentials will not decrypt correctly.
* -e DATABASE_HOST= [DATABASE_HOST]: Name of the container running postgres.  Even though our two containers are linked, this needs to be set to allow Rails to communicate with the database.
* -e DATABASE_USER= [DATABASE_USER] -e DATABASE_PASSWORD= [DATABASE_PASSWORD]: Credentials for authenticating into postgres.
* -e SENDGRID_USERNAME= [SENDGRID_USERNAME] -e SENDGRID_PASSWORD= [SENDGRID_PASSWORD]: The credentials for Sendgrid to send emails.  Alternatively, you could decide to not use Sendgrid and configure the application to use a different SMTP server (would be done inside your environment's config file).
* -e SECRET_KEY_BASE= [SECRET_KEY_BASE]: Setting the SECRET_KEY_BASE variable is necessary for creating secure cookies for authentication.  This variable automatically resets every time we restart the container.
* -e SERVICE_ACCOUNT_KEY= [SERVICE_ACCOUNT_KEY]: Setting the SERVICE_ACCOUNT_KEY variable is necessary for making authenticated API calls to FireCloud and GCP.  This should be a file path relative to the app root that points to the JSON service account key file you exported from GCP.
* -e GCS_ADMIN_SERVICE_ACCOUNT_KEY= [GCS_ADMIN_SERVICE_ACCOUNT_KEY]: Setting the GCS_ADMIN_SERVICE_ACCOUNT_KEY variable is necessary for accessing resources in GCP in user-controlled workspaces.  This service account is added with WRITER permissions to all Unity-associated workspaces.
* -e OAUTH_CLIENT_ID= [OAUTH_CLIENT_ID] -e OAUTH_CLIENT_SECRET= [OAUTH_CLIENT_SECRET]: Setting the OAUTH_CLIENT_ID and OAUTH_CLIENT_SECRET variables are necessary for allowing Google user authentication.  For instructions on creating OAuth 2.0 Client IDs, refer to the [Google OAuth 2.0 documentation](https://support.google.com/cloud/answer/6158849).
* *unity_benchmark_docker*: This is the name of the image we created earlier.  If you chose a different name, please use that here.
* *$DOCKER_IMAGE_VERSION*: Version number of the above Docker image.

## TESTS

### UNIT & INTEGRATION
To run all available rake tests (unit & integration), simply boot the container in test mode:

bin/boot_docker -e test -E (encryption key) -u (sendgrid username) -P (sendgrid password) -k (service account key path) -K (gcs admin service account key path) -o (oauth client id) -S (oauth client secret) -l

## PRODUCTION DEPLOYMENT

### KUBERNETES

Unity is designed to be deployed via [Kubernetes](https://kubernetes.io), although this is not the only way that Unity can be deployed (see below for details).
To deploy Unity in Kubernetes, you will need the following prerequisites:
* A Kubernetes cluster already [created](https://cloud.google.com/kubernetes-engine/docs/quickstart#create_cluster) in your GCP project
* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed
* Set your local kubernetes config to [point at your remote cluster](https://cloud.google.com/sdk/gcloud/reference/container/clusters/get-credentials):
  * gcloud container clusters get-credentials NAME -z ZONE

Unity is also pre-configured to leverage Cloud SQL when deployed to a Kubernetes cluster.  To configure this:
1. Log into your GCP Project and select 'SQL' from the left-hand hamburger menu
1. Select 'Create an instance'
1. Select 'PostgreSQL' and click 'Next'
1. Give the instance a name, a default user password, select a region, and click 'Create'
1. Once your instance has created, click on its name to load the instance details
1. Copy the instance connection name, and paste it into the line beginning with "-instances=" under the cloudsql-proxy container
details in config/unity-benchmark-deployment.yaml

You will then need to create a service account with the Cloud SQL Client role to be used to connect to the instance.

Before creating your deployment (but after your cluster is created), you will need to create the necessary secrets in Kubernetes to load
into your deployment.  These secrets must be created with the following key/value pairs:

1. cloudsql-db-credentials
   1. username: database username
   1. password: database password
1. oauth-client-credentials
   1. client_id: OAuth client ID
   1. client_secret: OAuth client secret
1. encryption-key
   1. encryption-key: 32-byte encryption key string
1. unity-service-account
   1. unity-benchmark-service-account.json: JSON contents of Unity project service account credentials (must be project owner/editor)
1. unity-gcs-admin-service-account
   1. unity-benchmark-gcs-admin.json: JSON contents of Unity GCS Admin service account credentials (must be have Google Cloud Storage Admin role)
1. cloudsql-instance-credentials
   1. credentials.json: JSON contents of CloudSQL service account credentials (must have Cloud SQL Client role)
1. google-site-verification
   1. verification-code: google-site-verification meta header value (for verifying site ownership in Google search console, required for OAuth verification)
1. ssl-certificate
   1. localhost.crt: a valid SSL certificate for your domain (filename of localhost does not affect certificate)
1. ssl-keyfile
   1. localhost.key: keyfile for your SSL certificate (filename of localhost does not affect certificate)
1. prod-secret-key-base
   1. secret-key-base: value for SECRET_KEY_BASE, which is used to encrypt secure cookies.  This can be set/updated by using bin/set_prod_secret_key_base
1. sendgrid-credentials
	 1. username: Username for your Sendgrid account
	 1. password: Password for your Sendgrid account

See [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) for more information on how to import and save secrets.

Once your secrets are loaded and kubectl is pointing at your remote cluster:
1. Navigate to the project directory
1. Create the deployment: kubectl apply -f config/unity-benchmark-deployment.yaml
1. Create the service: kubectl apply -f config/unity-benchmark-service.yaml
1. Once your service is running, get the EXTERNAL-IP address: kubectl get service unity-benchmark-service
1. You can change the external IP from ephemeral to static inside your VPC Network > [External IP Addresses](https://console.cloud.google.com/networking/addresses/list) console in GCP

Unity will now be available publicly on the above IP address.

TODO: Update deployment to load CloudSQL instance connection details from environment variables

#### PATCHING DEPLOYMENTS

If an update does not require incrementing a Docker image version number, you can force a restart by calling bin/patch_deployment.
This will 'patch' the current deployment by updating an annotation on the deployment (specifically, the date) and force
a full restart, including a docker pull.

NOTE: You must push an update to your docker image first via docker push.

### OTHER DEPLOYMENTS

Unity can also be deployed on any infrastructure that will support [Docker](https://www.docker.com).  This could be on
Google Cloud Platform, Amazon Web Services, or any other provider/operating system where Docker can be installed.  For instructions
on how to deploy, provision a virtual machine, install Docker, and then follow the instructions for 'RUNNING THE CONTAINER',
with the difference of adding -e production to bin/boot_docker.

The Unity database can be deployed either as a linked docker container (as in development, see 'BEFORE RUNNING THE CONTAINER IN DEVELOPMENT'
for more info, then linked by passing -l to bin/boot_docker) or a as a separate host, via -m [DATABASE HOST].

### FIRECLOUD INTEGRATION

The Unity Benchmark Service utilizes [FireCloud](https://software.broadinstitute.org/firecloud/) for storing data and launching workflow
submissions, which in turn store raw files in GCP buckets.  This is all managed through a GCP service account which in turn owns all workspaces
and manages them on behalf of users.  This is why your service account must be registered as a FireCloud user before the service
can function correctly.

### OAUTH VERIFICATION

Once you obtain a URL and a valid SSL certificate, you may go ahead and have your application verified for OAuth to remove
unverified application warnings.  Please refer to [this form](https://https://support.google.com/code/contact/oauth_app_verification)
for more information on the OAuth verification process.  You must also generate a separate OAuth client to use in production
that does not contain localhost URLs, as this will cause your request to be rejected.

The following scopes must be provided as part of the verification process (with the following justifications):

* https://www.googleapis.com/auth/userinfo.profile (to allow users to authenticate using their Google account)
* https://www.googleapis.com/auth/userinfo.email (to allow users to authenticate using their Google account)
* https://www.googleapis.com/auth/cloud-billing.readonly (to allow Unity to see available billing accounts to create projects on behalf of users)
* https://www.googleapis.com/auth/cloud-platform.read-only (to allow Unity to stream user-owned GCS objects back to the client)


text:
<- [[Windows - Event Logs]]
# Event Viewer
An #MMC (Microsoft Management Console) snap-in, Event View can be launched by simply right clicking the #Windows Icon in the taskbar and selecting `Event Viewer`

Also can be launched from CLI with `eventvwr.msc`

Like Registry Editor, you have to manually drill down to get to anything.

**Layout**
1. Left pane - hierarchical tree listing of event log providers
2. Middle Pane - general overview/summary or events specific to a selected provider
3. Right Pane - actions

## Middle Pane
A brief explanation for each column for events specific to provider:
-   The first column is **Level**, which is the event type. Recall from earlier there are 5 different event types. This first entry is labeled as **Information**.
-   Next is **Date and Time**, which is when the event was logged.
-   The third column **Source** is the name of the software that logs the event. From the above image, the source is PowerShell.
-   Events are identified by IDs (**Event ID**), which is the fourth column. Note that Event IDs are not unique. Meaning that Event ID 4103 in the above image is related to Executing Pipeline but will have an entirely different meaning in another event log.
-   Lastly is **Task Category**, which is an Event Category. This entry will help you organize events so Event Viewer can filter them. The event source defines this column.

More information is displayed in the split pane as the bottom of the middle pane:
This section has 2 tabs: **General** and **Details**.
-   General is the default view, and the rendered data is displayed.
-   The Details view has 2 options: Friendly view and #XML view.

## Right Pane - Actions
Most useful
- Create Custom View
- Filter Current Log

Basically identical but the latter, as the name implied *only* affects the current log while the former *Create Custom View* affects the global view.

license: MIT

repository: semi-Colin/CyberSecVault

path: /Event Viewer.md

branch_name: refs/heads/main

extension: md

filename: Event Viewer.md

license_type: permissive

repo_name: semi-Colin/CyberSecVault


text:
motivation

dynamically load file(image) and more from react spa to express using rest api

Usage

npm install in server and client directories

cd server directory , open terminal -> npm run dev

details

multer is used to handle the file and other input

file is saved into upload directory inside the  server root

client has two inputs : one is type text and the other is type file. this is done with out using form ,but formData is used in the client

client must use axios to get the image which was loaded dynamically

Points of interest

one can store the images on the client public directory. you will not need axios to load them but every time it is done create-react-app will cause browser refresh. this will not happen in production

the path for the stored images can be stored in database for further usage

repository: synthlace/react-express-load-image

branch_name: refs/heads/master

repo_name: synthlace/react-express-load-image


text:
# Storage APIs

# Introduction
This repo will provide following APIs to access or store data from a PostgresDB (9.6+) or MongoDB (3.6+)
* [Object Store APIs](http://storage-docs.pharmaledger.eu/#object-store-apis)
* [File Store APIs](http://storage-docs.pharmaledger.eu/#file-store-apis)
* [Schema Definition APIs](http://storage-docs.pharmaledger.eu/#schema-definition-apis)

N.B. If you use MongoDB as Database, to use [File Store APIs](http://storage-docs.pharmaledger.eu/#file-store-apis) you can choose following options as file storage:

* MongoDB Grid Storage (used by default and requires no setup); or
* Amazon S3 Storage; or
* Google Cloud Storage; or
* Local File Storage

N.B. If you use PostgresDB as Database, to use [File Store APIs](http://storage-docs.pharmaledger.eu/#file-store-apis) you can choose following options as file storage:

* Local File Storage; or
* Amazon S3 Storage; or
* Google Cloud Storage;

So you can setup the application in following ways:

| Option        | Database Store | File Store           | Local Setup                        | Cloud Setup
| ------------- | -------------  | -------------        | -------------                      | -------------
| #1            | MongoDB        | MongoDB Grid Storage | [Docker Compose](#option-1)        | [Kubernetes](k8s/README.md#option-1)
| #2            | MongoDB        | Amazon S3 Storage    | [Docker Compose](#option-2)        | [Kubernetes](k8s/README.md#option-2)
| #3            | MongoDB        | Google Cloud Storage | [Docker Compose](#option-3)        | [Kubernetes](k8s/README.md#option-3)
| #4            | MongoDB        | Local File Storage   | [Docker Compose](#option-4)        | [Kubernetes](k8s/README.md#option-4)
| #5            | PostgresDB     | Amazon S3 Storage    | [Docker Compose](#option-5)        | [Kubernetes](k8s/README.md#option-5)
| #6            | PostgresDB     | Google Cloud Storage | [Docker Compose](#option-6)        | [Kubernetes](k8s/README.md#option-6)
| #7            | PostgresDB     | Local File Storage   | [Docker Compose](#option-7)        | [Kubernetes](k8s/README.md#option-7)

Storage API application environment variables can be found [here](api/.env) and their description is given below.

| Variable Name       | Required / Optional | Description
| ------------------  | ------------------- | -------------
| APP_ID              | REQUIRED            | A unique identifier for your app. You can use any arbitrary string or UUID string.
| MASTER_KEY          | REQUIRED            | The master key that overrides all permissions. Keep this secret. You can use any arbitrary string or UUID string.
| RESTAPI_KEY         | REQUIRED            | Key for REST calls. You can use any arbitrary string or UUID string.
| SERVER_URL          | REQUIRED            | URL to Storage API server with http:// or https://.
| PUBLIC_SERVER_URL   | REQUIRED            | Public URL to Storage API server with http:// or https://.
| DATABASE_URI        | REQUIRED            | The connection string for your database. Supported databases are mongodb or postgres. Be sure to URL encode your password if your password has special characters.
| REDIS_URI           | REQUIRED            | The connection string to a redis cache database (0). This is to speed up schema verifications, user, roles and sessions lookup.
| JOB_REDIS_URI       | REQUIRED            | The connection string to a redis cache database (1). This is to manage backgroud jobs.
| DASHBOARD_USER      | REQUIRED            | Username to access the Dashboard. Keep this secret
| DASHBOARD_PASS      | REQUIRED            | Password to access the Dashboard. Keep this secret
| FILES_SUB_DIRECTORY | REQUIRED for [Option 4](#option-4) and [Option 7](#option-7) | Location to centralised file storage. e.g Docker volume.
| S3_ACCESS_KEY       | REQUIRED for [Option 2](#option-2) and [Option 5](#option-5) | The AWS access key for a user that has the required permissions.
| S3_SECRET_KEY       | REQUIRED for [Option 2](#option-2) and [Option 5](#option-5) | The AWS secret key for the user.
| S3_BUCKET           | REQUIRED for [Option 2](#option-2) and [Option 5](#option-5) | The name of your S3 bucket. Needs to be globally unique in all of S3.
| S3_REGION           | REQUIRED for [Option 2](#option-2) and [Option 5](#option-5) | The AWS region to connect to.
| S3_BUCKET_PREFIX    | OPTIONAL for [Option 2](#option-2) and [Option 5](#option-5) | Create all the files with the specified prefix added to the filename.
| S3_DIRECT_ACCESS    | OPTIONAL for [Option 2](#option-2) and [Option 5](#option-5) | Whether reads are going directly to S3 or proxied through Storage API. If set to true, files will be made publicly accessible, and reads will not be proxied.
| GCP_PROJECT_ID      | REQUIRED for [Option 3](#option-3) and [Option 6](#option-6) | The project ID from the Google Developer’s Console.
| GCP_KEYFILE_PATH    | REQUIRED for [Option 3](#option-3) and [Option 6](#option-6) | Full path to the a .json, .pem, or .p12 key downloaded from the Google Developers Console. e.g Docker volume path.
| GCS_BUCKET          | REQUIRED for [Option 3](#option-3) and [Option 6](#option-6) | The name of your GCS bucket.
| GCS_BUCKET_PREFIX   | OPTIONAL for [Option 3](#option-3) and [Option 6](#option-6) | Create all the files with the specified prefix added to the filename.
| GCS_DIRECT_ACCESS   | OPTIONAL for [Option 3](#option-3) and [Option 6](#option-6) | Whether reads are going directly to S3 or proxied through Storage API.

# Running the application with Docker Compose

## Option 1
Storage API with MongoDB and MongoDB Grid Storage

```shell
DATABASE_URI="mongodb://developer:secret@mongodb:27017/storage-db?authSource=admin" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.mongodb.yml -f docker-compose.api.yml up --build
```

## Option 2
Storage API with MongoDB and Amazon S3 Storage. Make sure to use correct value for following ENV variables:

* <tt>S3_ACCESS_KEY</tt>
* <tt>S3_SECRET_KEY</tt>
* <tt>S3_BUCKET</tt>
* <tt>S3_REGION</tt>

```shell
DATABASE_URI="mongodb://developer:secret@mongodb:27017/storage-db?authSource=admin" S3_ACCESS_KEY="<S3_ACCESS_KEY>" S3_SECRET_KEY="<S3_SECRET_KEY>" S3_BUCKET="<S3_BUCKET>" S3_REGION="<S3_REGION>" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.mongodb.yml -f docker-compose.api.yml up --build
```

## Option 3
Storage API with MongoDB and Google Cloud Storage. Make sure to use correct value for following ENV variables:

* <tt>GCP_PROJECT_ID</tt>
* <tt>GCP_KEYFILE_PATH</tt>
* <tt>GCS_BUCKET</tt>

```shell
DATABASE_URI="mongodb://developer:secret@mongodb:27017/storage-db?authSource=admin" GCP_PROJECT_ID="<GCP_PROJECT_ID>" GCP_KEYFILE_PATH="<GCP_KEYFILE_PATH>" GCS_BUCKET="<GCS_BUCKET>" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.mongodb.yml -f docker-compose.api.yml up --build
```

## Option 4
Storage API with MongoDB and Local File Storage

```shell
DATABASE_URI="mongodb://developer:secret@mongodb:27017/storage-db?authSource=admin" FILES_SUB_DIRECTORY="./files" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.mongodb.yml -f docker-compose.api.yml up --build
```

## Option 5
Storage API with PostgresDB and Amazon S3 Storage. Make sure to use correct value for following ENV variables:

* <tt>S3_ACCESS_KEY</tt>
* <tt>S3_SECRET_KEY</tt>
* <tt>S3_BUCKET</tt>
* <tt>S3_REGION</tt>

```shell
DATABASE_URI="postgres://developer:secret@postgres:5432/storage-db" S3_ACCESS_KEY="<S3_ACCESS_KEY>" S3_SECRET_KEY="<S3_SECRET_KEY>" S3_BUCKET="<S3_BUCKET>" S3_REGION="<S3_REGION>" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.postgres.yml -f docker-compose.api.yml up --build
```

## Option 6
Storage API with PostgresDB and Google Cloud Storage. Make sure to use correct value for following ENV variables:

* <tt>GCP_PROJECT_ID</tt>
* <tt>GCP_KEYFILE_PATH</tt>
* <tt>GCS_BUCKET</tt>

```shell
DATABASE_URI="postgres://developer:secret@postgres:5432/storage-db" GCP_PROJECT_ID="<GCP_PROJECT_ID>" GCP_KEYFILE_PATH="<GCP_KEYFILE_PATH>" GCS_BUCKET="<GCS_BUCKET>" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.postgres.yml -f docker-compose.api.yml up --build
```

## Option 7
Storage API with PostgresDB and Local File Storage

```shell
DATABASE_URI="postgres://developer:secret@postgres:5432/storage-db" FILES_SUB_DIRECTORY="./files" docker-compose -p storage_api -f docker-compose.redis.yml -f docker-compose.postgres.yml -f docker-compose.api.yml up --build
```

After running the application following url will be accessible:

* RESTful API endpoint: http://localhost:1337/v1/storage
* Dashboard: http://localhost:1337/dashboard
    * username: [Value of DASHBOARD_USER env variable](api/.env#L9)
    * password: [Value of DASHBOARD_PASS env variable](api/.env#L10)

# Documentations and Tools
Storage API uses [Parse Server](https://docs.parseplatform.org/parse-server/guide/) following features.
* Objects
* Queries
* Files
* Schema

Storage API has implemented Data Expiration feature which is not availbale in Parse Server.

You can interact with Storage API with [RESTful APIs](http://storage-docs.pharmaledger.eu) or [Official Parse JavaScript SDK](https://docs.parseplatform.org/js/guide/)

We created a [Postman Collection](https://www.getpostman.com/collections/51c483fe9b294a6183a8) to test the RESTful APIs. Import it and change the environment variables to connect to your setup environment.

We recommend to follow [Storage API Documentation](http://storage-docs.pharmaledger.eu). However, You can also check [Parse REST Guide](https://docs.parseplatform.org/rest/guide/) but we are not using all the features. So check only following sections.

* [Objects](https://docs.parseplatform.org/rest/guide/#objects)
* [Queries](https://docs.parseplatform.org/rest/guide/#queries)
* [Files](https://docs.parseplatform.org/rest/guide/#files)
* [Schema](https://docs.parseplatform.org/rest/guide/#schema)


text:
# CandidateCertification

This entity represents a certification that a Candidate may have.

| **CandidateCertification field** | **Type** | **Description** | **Not null** | **Read-only** |
| --- | --- | --- | --- | --- |
| id | Integer | Unique identifier for this entity. | X | X |
| boardCertification | String (100) | Set by certificationBoardCertificationList private label attribute | | |
| candidate | To-one association | Candidate linked to CandidateCertification. | X | |
| certification | To-one association | Certification linked to CandidateCertification. | X | |
| comments | String<PHONE_NUMBER>) | Free-text comments on this record. | | |
| compact | Integer | yes/no (1/0) Whether compact or not. | | |
| copyOnFile | Integer | yes/no (1/0) Whether on file. | | |
| dateCertified | Timestamp | Date candidate obtained certification. | | |
| dateExpiration | Timestamp | Date certification expires. | | |
| dateLastModified | Timestamp | Date the CandidateCertification was last updated. | X | X |
| fileAttachments | Many-to-many association | CandidateFileAttachments associated to entity. Association created by adding the ID(s) of the fileAttachment(s) to this field in an association PUT call of the form: {corpToken}/entity/CandidateCertification/{entity-id}/fileAttachments/{entity-id},*}. | | |
| certificationFileAttachments | Many-to-many association | CertificationFileAttachments associated to entity. | | |
| issuedBy | String (100) | Entity that issued certification to candidate. | | |
| licenseNumber | String (100) | License number of the certification. | | |
| licenseType | String (30) | License Type of the certification. | | |
| location | String (100) | Location where certification is valid. | | |
| modifyingUser | To-one association | Creator of CandidateCertification. | | |
| name | String (100) | Name of CandidateCertification. | X | |
| results | String (255) | Free text of certification results. | | |
| status | String (30) | Status of CandidateCertification. | | |

repository: amoslanka/rest-api-docs

repo_name: amoslanka/rest-api-docs


text:
# HUST material cloud database

This repo holds the demo for HUST material cloud database, based on Django3, Vue3 and tailwind css.

## Prepare the environment

To run the website, you should download the project source code, and prepare the environment for both frontend and backend:

- Download the repo source code and unzip it, you should see *frontend* and *backend* folder under the project root folder.
- For frontend: First run `cd frontend`; then run `npm install`.
- For backend: Back to the project root folder, first run `cd backend`; then run `python install -r requirements.txt`.

Now you should be set with the environment containing all dependencies.

## Run the website.

As you should be prepared by now, running the website is an easy task:

- Back to the root folder: First run `cd backend`; then run `python manage.py runserver`.
- Back to the root folder: First run `cd frontend`; then run `npm run dev`.

You can check the frontend logging console to see the address of this website. By default, it's at *http://<IP_ADDRESS>:3000*

## Dealing with authentication

If you don't have an account to login, you can create your own superuser using the backend terminal or contact the author for default account.


text:
# PSDocs

A PowerShell module with commands to generate markdown from objects using PowerShell syntax.

![ci-badge]

## Disclaimer

This project is open source and **not a supported product**.

If you are experiencing problems, have a feature request, or a question, please check for an [issue] on GitHub.
If you do not see your problem captured, please file a new issue, and follow the provided template.

## Getting the modules

You can download and install the PSDocs module from the PowerShell Gallery.

Module     | Description | Downloads / instructions
------     | ----------- | ------------------------
PSDocs     | Generate markdown from PowerShell | [latest][psg-psdocs] / [instructions][install]

For integration modules see [related projects](#related-projects).

## Getting started

The following example shows basic PSDocs usage.
For specific use cases see [scenarios](#scenarios).

### Define a document

A document provides instructions on how PSDocs should render an object into documentation.
To define a document, create the `Document` script block saved to a file with the `.Doc.ps1` extension.

For example:

```powershell
# File: Sample.Doc.ps1

# Define a document called Sample
Document Sample {
    # Define content here
}
```

Within the document body provide one or more instructions.

For example:

```powershell
# File: Sample.Doc.ps1

# Define a document called Sample
Document Sample {

    # Add an introduction section
    Section Introduction {
        # Add a comment
        "This is a sample file list from $TargetObject"

        # Generate a table
        Get-ChildItem -Path $TargetObject | Table -Property Name,PSIsContainer
    }
}
```

### Execute a document

To execute the document use `Invoke-PSDocument`.

For example:

```powershell
Invoke-PSDocument -InputObject 'C:\';
```

An example of the output generated is available [here](docs/examples/Get-child-item-output.md).

### Scenarios

- [Azure Resource Manager template example](docs/scenarios/arm-template/arm-template.md)
- [Integration with DocFX](docs/scenarios/docfx/integration-with-docfx.md)

## Language reference

PSDocs extends PowerShell with domain specific language (DSL) keywords and cmdlets.

### Keywords

The following language keywords are used by the `PSDocs` module:

- [Document](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#document) - Defines a named documentation block
- [Section](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#section) - Creates a named section
- [Title](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#title) - Sets the document title
- [Code](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#code) - Inserts a block of code
- [BlockQuote](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#blockquote) - Inserts a block quote
- [Note](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#note) - Inserts a note using DocFx formatted markdown (DFM)
- [Warning](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#warning) - Inserts a warning using DocFx formatted markdown (DFM)
- [Metadata](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#metadata) - Inserts a yaml header
- [Table](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#table) - Inserts a table from pipeline objects
- [Include](docs/keywords/PSDocs/en-US/about_PSDocs_Keywords.md#include) - Insert content from an external file

### Commands

The following commands exist in the `PSDocs` module:

- [Invoke-PSDocument](docs/commands/PSDocs/en-US/Invoke-PSDocument.md)
- [Get-PSDocument](docs/commands/PSDocs/en-US/Get-PSDocument.md)
- [Get-PSDocumentHeader](docs/commands/PSDocs/en-US/Get-PSDocumentHeader.md)
- [New-PSDocumentOption](docs/commands/PSDocs/en-US/New-PSDocumentOption.md)

The following commands exist in the `PSDocs.Dsc` module:

- [Get-DscMofDocument](docs/commands/PSDocs.Dsc/en-US/Get-DscMofDocument.md)
- [Invoke-DscNodeDocument](docs/commands/PSDocs.Dsc/en-US/Invoke-DscNodeDocument.md)

### Concepts

The following conceptual topics exist in the `PSDocs` module:

- [Configuration](docs/concepts/PSDocs/en-US/about_PSDocs_Configuration.md)
- [Conventions](docs/concepts/PSDocs/en-US/about_PSDocs_Conventions.md)
- [Options](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md)
  - [Configuration](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#configuration)
  - [Execution.LanguageMode](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#executionlanguagemode)
  - [Markdown.ColumnPadding](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#markdowncolumnpadding)
  - [Markdown.Encoding](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#markdownencoding)
  - [Markdown.SkipEmptySections](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#markdownskipemptysections)
  - [Markdown.UseEdgePipes](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#markdownuseedgepipes)
  - [Markdown.WrapSeparator](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#markdownwrapseparator)
  - [Output.Culture](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#outputculture)
  - [Output.Path](docs/concepts/PSDocs/en-US/about_PSDocs_Options.md#outputpath)
- [Variables](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md)
  - [$Culture](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#culture)
  - [$Document](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#document)
  - [$InstanceName](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#instancename)
  - [$LocalizedData](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#localizeddata)
  - [$PSDocs](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#psdocs)
  - [$TargetObject](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#targetobject)
  - [$Section](docs/concepts/PSDocs/en-US/about_PSDocs_Variables.md#section)

## Related projects

The following projects use or integrate with PSDocs.

Name          | Description
----          | -----------
[PSDocs.Dsc]  | Extension for PSDocs to generate markdown from Desired State Configuration.

## Changes and versioning

Modules in this repository will use the [semantic versioning](http://semver.org/) model to declare breaking changes from v1.0.0.
Prior to v1.0.0, breaking changes may be introduced in minor (0.x.0) version increments.
For a list of module changes please see the [change log](CHANGELOG.md).

> Pre-release module versions are created on major commits and can be installed from the PowerShell Gallery.
> Pre-release versions should be considered experimental.
> Modules and change log details for pre-releases will be removed as standard releases are made available.

## Maintainers

- [Bernie White](https://github.com/BernieWhite)

## License

This project is [licensed under the MIT License](LICENSE).

[install]: docs/install-instructions.md
[issue]: https://github.com/BernieWhite/PSDocs/issues
[ci-badge]: https://bewhite.visualstudio.com/PSDocs/_apis/build/status/PSDocs-CI?branchName=main
[psg-psdocs]: https://www.powershellgallery.com/packages/PSDocs
[psg-psdocs-version-badge]: https://img.shields.io/powershellgallery/v/PSDocs.svg
[psg-psdocs-installs-badge]: https://img.shields.io/powershellgallery/dt/PSDocs.svg
[PSDocs.Dsc]: https://www.powershellgallery.com/packages/PSDocs.Dsc

repository: joeygomes-avanade/PSDocs

repo_name: joeygomes-avanade/PSDocs


text:
# Toolbar Button Provider Sample

Copyright © Bentley Systems, Incorporated. All rights reserved.

This sample demonstrates how to use the [UiItemsProvider](https://www.imodeljs.org/reference/ui-abstract/uiitemsprovider/) API.  This API allows an AppUI interface to be altered when a package or extension is loaded.

## Purpose

The purpose of this sample is to demonstrate the following:

- Defining a UiItemsProvider.
- Using provideToolbarButtonItems() to specify a toolbar button to be added to a tool widget's toolbar.
- Using ToolbarItemUtilities.createActionButton() to create an button with an icon that launches a tool.
- Using UiItemsManager to register a UiItemsProvider.

## Description

An extension or package that would like to modify an app's UI can do so by defining and registering a UiItemsProvider to provide UI items that will be added to
the different areas in an AppUi app. For clarity, this sample is specific to adding a button to a toolbar in one of the tool widgets. This gives a package or extension
a seamless integration for its tools into the app's UI.

gha_language: TypeScript

gha_license_id: MIT


text:
# ~~tok5-tag-pos~~ tok-tag

*Tokenize and Tag Parts of Speech.*

View the git history from [@coarchive/tok5-tag-pos](https://github.com/coarchive/tok5-tag-pos).

This would have been so much easier if `wink-tokenizer` defined Token as

```ts
interface Token {
   token_type: TokenType,
   value: string,
};

// instead of

interface Token {
   tag: Tag,
   value: string,
};
```

I could even patch the source but I'm too lazy to go that far.
Whenever you read `.tag`, think `.token_type`.

repository: coarchive/brown-bear

repo_name: coarchive/brown-bear


text:
# TeamworkScrapeClient

Unofficial Teamwork.com scraping client to supplement the official API. Provides the ability to copy projects.

**WARNING:** This library uses undocumented APIs intended for use by the Teamwork.com web interface to communicate with the server. If they change the API, this library may break.

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'teamwork_scrape_client'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install teamwork_scrape_client

## Usage

```ruby
client = TeamworkScrapeClient::Client.new(
  email<EMAIL_ADDRESS>
  password: 'secret_password',
  base_url: 'https://yourdomain.teamwork.com'
)

client.copy_project(
    old_project_name: 'Template Project',
    new_project_name: 'Test Project',
    new_company_name: 'Test Company'
)
```

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ericmason/teamwork_scrape_client.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).


text:
# Change log

## 2021-10-17

### Import changes from https://github.com/bnosac/doc2vec/

- In paragraph2vec: close opened files directly after training instead of waiting for R garbage collection to kick in
- Start fixing valgrind issues
- more valgrind changes

## 2021-10-18

### Change

- replace custom hash with std::unordered_map, to fix issue with more than 30M documents
- print debug info to stderr instead of stdout
- rename Vocab.{cpp,h} to Vocabulary.{cpp,h}
- refactor by adding const where suitable
- remove document and word size limits and refactor to modernize code
- check return value from output file creation
- return 1 on argument error in train.cpp


text:
`Bootstrap` is a high-level framework for starting deep learning projects.
It aims at accelerating research projects and prototyping by providing a powerful workflow focused on your dataset and model only.

And it is:

- Scalable
- Modular
- Shareable
- Extendable
- Uncomplicated
- Built for reproducibility
- Easy to log and plot anything

It's not a wrapper over pytorch, it's a powerful extension.

## Quick tour

To display parsed options from the yaml file:
```
python -m bootstrap.run
       -o mnist/options/sgd.yaml
       -h
```

To run an experiment (training + evaluation):
```
python -m bootstrap.run
       -o mnist/options/sgd.yaml
```

Running an experiment will create 4 files:

- [options.yaml](https://github.com/Cadene/bootstrap.pytorch/blob/master/logs/mnist/sgd/options.yaml) contains the options used for the experiment,
- [logs.txt](https://github.com/Cadene/bootstrap.pytorch/blob/master/logs/mnist/sgd/logs.txt) contains all the information given to the logger.
- [logs.json](https://github.com/Cadene/bootstrap.pytorch/blob/master/logs/mnist/sgd/logs.json) contains the following data: train_epoch.loss, train_batch.loss, eval_epoch.accuracy_top1, etc.
- view.html contains training and evaluation curves with javascript utilities (plotly).

To save the next experiment in a specific directory:
```
python -m bootstrap.run
       -o mnist/options/sgd.yaml
       --exp.dir logs/mnist/custom
```

To reload an experiment:
```
python -m bootstrap.run
       -o logs/mnist/cuda/options.yaml
       --exp.resume last
```

## Documentation

The package reference is available on the [documentation website](http://remicadene.com/bootstrap).

It also contains some notes:

- [Installation](http://remicadene.com/bootstrap/#installation)
- [Concepts](http://remicadene.com/bootstrap/concepts.html)
- [Quickstart](http://remicadene.com/bootstrap/quickstart.html)
- [Directories](http://remicadene.com/bootstrap/directories.html)
- [Examples](http://remicadene.com/bootstrap/examples.html)

## Poster


text:
---
title: Intro to R
layout: workshop
---

# Introduction to R - with Dan and Erin

--------

**Wifi SSID**: `workshop`

---------
Please have these both installed (come early if you need help):

- **R**: [Download from here](https://cran.r-project.org/)
- **rstudio**: [Download from here](https://www.rstudio.com/products/RStudio/#Desktop)

Also, please install here awesome R packages (again, if you don't know how please come early):

+ 'rms' ()
+ 'ggplot2'
+ 'car'

--------
Please download the following files

1. [Rtutorial_data1.csv](/lrn2compute/workshops/data/Rtutorial_data1.csv)
2. [Rtutorial_data2.csv](/lrn2compute/workshops/data/Rtutorial_data2.csv)

---------
***1. RStudio environment***
---

Welcome to R studio! You should see 4 spaces:

1. Source
  + Your script! everything you do to your data should be typed here!! It saves what you are
  doing in a file ending in '.R'
2. Console
  + The place where the good stuff happens. When you run a command, you see the output here.
3. Enviroment/History
  + Enviroment: a list of everything in your "Workspace"
  + History: a list of every command you have run
4. Files/Plots/Packages/Help
  + Files: a file browser (like windows Explorer)
  + Plots: your plots will show up here
  + Packages: a list of all the packages you have installed with checkboxes showing if it is loaded
  + Help: your help documentation with be shown here.

----------
***2. Loading libraries / installing packages / basic scripting practice (comments/why?)***
---

for this tutorial, we are going to use the packages ggplot, rms and car.

+ Look at the Packages Tab
+ Click on the check box to the left of "ggplot2"
+ Notice that a bunch of lines start running in "Console" window to show you that the package is loading
+ Go to the History Tab (top right corner)
+ Notice that you should see lines starting in library("rms"..) and library("ggplot2"...) have just run
+ Click on these lines to highlight them, then click on "To Source" at the top of the panel
+ This moves the lines we just ran into our script so we can rerun this step tomorrow!

{% highlight r %}
library(rms)
library(ggplot2)
library(car)
{% endhighlight %}

--------
***3. Read in data***
---

The two datasets provided are as follows:

**Rtutorial_data1.csv:**
+ 5 variables: age, diagnosis (dx), ethnicity, sex, and subject identifier (subject_ID)

**Rtutorial_data2.csv:**
+ 4 variables: three cognitive scores (cog1, cog2, cog3), genotype, and subject identifier (subID)

In order to view and manipulate this data in R, we need to *import* the data into our R workspace (the same as you would open a file in excel to edit it).

*Rstudio trick:*

+ Click on the "Environment" Tab, then click on "Import Dataset" --> From text File
+ Navigate the browser window to the location of data2.csv and click Open
+ This opens a text reader window: You see the raw text on the top and what R will read in (the data frame) at the bottom
+ In my view, it looks like the R is not going to read in the first line as a header..to change this
+ switch the "Heading" option on the right to "yes"
+ Click "Import"
+ Now, if you look at the "Environment" tab you should see that "data1" has been loaded into R, It has 350 rows (or observations) and 5 variables
+ So that you do not have to type this again tomorrow - go to History, click on the line "data1 <- read.csv(...)" and then click on "To Source"
+ Repeat this whole process for data2.csv

{% highlight r %}
data1 <- read.csv("~/Downloads/Rtutorial_data1.csv")
data2 <- read.csv("~/Downloads/Rtutorial_data2.csv")
{% endhighlight %}

+ What you actually did was use the read.csv function... to find out more about this option you can type "?read.csv" in the Console
+ This is the basic syntax of R functions: some.function("stuff inside to do the function on")
+ The "help document" for the read.csv function is shown in the "Help" tab

Now we have two **"data frames"** loaded into our workspace. They are called data1 and data2.

-------
***4. Basic data summaries and visualization ( head, tail, describe() )***
---

+ Now that we have the data loaded, how do we just look at it? The simplest way is with the "View" function within rstudio.
+ In Enviroment tab. Click on the little spreadsheet to teh far right of the data1.csv row... this shows you your data in what looks like a spreadsheet - but you cannot edit it!

To look at the top six rows of your data:

{% highlight r %}
head(data1)
{% endhighlight %}

To look at the bottom six rows:

{% highlight r %}
tail(data2)
{% endhighlight %}

Using the function names() tells us what all the variables in our dataframe are called.
the ls() function does the same thing, except it returns the variables in alphabetical order

{% highlight r %}
names(data1)
ls(data1)
{% endhighlight %}

That was all nice, but we want to find out more about this data we can use "summary"

{% highlight r %}
summary(data1)
summary(data2)
{% endhighlight %}

Another very useful function is describe()  - (from the rms package)

{% highlight r %}
describe(data1)
describe(data2)
{% endhighlight %}

---------
***5. Data cleaning***
---

+ Now that you have looked at your data - you might have noticed that there are a couple probems
The RA that you have been working with have coded missing values in three different ways ("9999", "missing",and "NA")
+ We first need to set these all to NA - which R recognizes as missing value:

The following will take all values in data1 that are equal to "", "missing", or "9999", and code them as missing in a way that R understands:

{% highlight r %}
data1[data1==""] <- NA
data1[data1=="missing"] <- NA
data1[data1=="9999"] <- NA
{% endhighlight %}

Because R is "smart", it categorizes data types automatically when data are loaded. Before working with new data, especailly if it is real (i.e. messy), it is important to tell R what kind of data
you are dealing with. This will be especially important when we discuss our statistical analyses... after all, R is statistical software.

The following will correctly format our variables for analyses:

+ age is a numeric variable
+ ethicity is a discrete factor
+ sex is a discrete factor
+ diagnosis is a discrete factor

{% highlight r %}
data1$age <- as.numeric(as.character(data1$age))
data1$ethnicity <- factor(data1$ethnicity,levels=c("Cauc","AA","As","In","Other"))
data1$sex <- factor(data1$sex, levels=c(0,1), labels=c("Male","Female"))
data1$dx <- factor(data1$dx, levels=c(0,1), labels=c("Control","Case"))
{% endhighlight %}

By indicating the levels of our factors, we have erased from R the memory that we once had values of
"", "9999", and "missing" (which up until now R had no reason to assume were not observations).

Let us now apply the same cleanup steps to our second data frame:

Remove missing:

{% highlight r %}
data2[data2==""] <- NA
data2[data2=="missing"] <- NA
data2[data2=="9999"] <- NA
{% endhighlight %}

Correctly format variables for analyses:

{% highlight r %}
data2$genotype <- factor(data2$genotype, levels=c(0,1,2), labels=c("AA","AG","GG"))
data2$cog1 <- as.numeric(as.character(data2$cog1))
data2$cog2 <- as.numeric(as.character(data2$cog2))
data2$cog3 <- as.numeric(as.character(data2$cog3))
{% endhighlight %}

---------
***6. Merging data frames***
---

In order to analyze the effect of sex on diagnosis, or perform any other comparison across our data frames, we should merge them. If you remember only this and nothing else today, it will still have been worth your time.

Conceptually, merging two data frames assumes that the rows in one correspond to rows in the other, even if they are not in the same order. In order to match up the correct rows between data frames we need to make sure that one column in each spreadsheet can act as a "key" (i.e. each row has a unique value in this key that is the same in both spreadsheets). In our case, we have one subject identifier column in each of our spreadsheets.

First we need to make sure that the values in these columns are the same:

{% highlight r %}
data2$subID <- gsub(data2$subID,pattern="subject",replacement="SUB_")
{% endhighlight %}

We can then merge the two datasets by specifying their names (in order x,y) and then specifying which columns are to be used as the key to merging the two data frames (by.x and by.y):

{% highlight r %}
alldata <- merge(data1,data2,by.x="subject_ID",by.y="subID")
{% endhighlight %}

Skipping ahead a little - now we can look at histograms of our numeric variables, just to see what we are dealing with:

{% highlight r %}
hist(data2$cog1)
hist(data2$cog2)
hist(data2$cog3)
hist(data1$age)
{% endhighlight %}

Now that our data are loaded, cleaned, and merged, it is time to do some basic statistics!

---------
***STUDY QUESTION 1: What is the relationship between sex and diagnosis?***
---

**For this question, our null hypothesis is that there is no difference in the number of males and females between our case and control diagnosis groups**

The ftable() function will give us a 2 x 2 contingency table of the frequency of observations in each category. the formula syntax "y ~ x" is common in R!

{% highlight r %}
ftable(data=alldata,dx~sex)
{% endhighlight %}

We now want to save that table as an *object* called "dxXsex_table":

{% highlight r %}
dxXsex_table <- ftable(data=alldata,dx~sex)
{% endhighlight %}

Now, in order to test our null hypothesis using a chi-squared test, we simply apply the chisq.test() function to that table:

{% highlight r %}
chisq.test(dxXsex_table)
{% endhighlight %}

Similarly, we can use the nonparametric Fisher test to get a more exact test statistic:

{% highlight r %}
fisher.test(dxXsex_table)
{% endhighlight %}

*A bit more advanced!*
This will accoplish the same thing as ftable(), except that here we are *indexing* our alldata dataframe with the R syntax [<row>,<column>]. the blank value for <row> tells R that we want all rows.
The c("dx","sex") value for <columns> means we want to use the columns named "dx" and "sex". the table() function knows to arrange these as a 2 x 2 contingency table.

{% highlight r %}
table(alldata[ ,c("dx","sex")])
{% endhighlight %}

---------
***STUDY QUESTION 2: What is the relationship between genotype and diagnosis?***
---

**for this question, our null hypothesis is that there is no difference in the number of males and females between our case and control diagnosis groups**

{% highlight r %}
ftable(data=alldata,dx~genotype)
dxXgene_table <- ftable(data=alldata,dx~genotype)
chisq.test(dxXgene_table)
fisher.test(dxXgene_table)
{% endhighlight %}

---------
***P.S.****
Here is an R script with all of the steps we went over today!!
[Download Intro R script](/lrn2compute/workshops/data/R_tutorial_intro.R)

branch_name: refs/heads/gh-pages

gha_language: CSS


text:
# SwiftWasmPlaygrounds
https://elmetal.github.io/SwiftWasmPlaygrounds/

## Get Started
### Installation
`brew install swiftwasm/tap/carton`

### Usage
`carton dev`

## License
This software includes the work that is distributed in the Apache License 2.0

repository: elmetal/SwiftWasmPlaygrounds

repo_name: elmetal/SwiftWasmPlaygrounds


text:
# Introduce

enum is a tool for generating enum struct

# Install

```
go install github.com/Natata/enum
```

# How to use

## Write enum file

File format

```
type:
    [int|string]
name:
    [package name]
list:
    [enum element] = [value]
```

filename doesn't matter

## Generate enum.go

```
enum -fp=exam.enum
```

## Example

### int enum
```
type:
    int
name:
    direction
list:
    West = 0
    east = 1
    North = 2
    South = 3
```

result:
```
package direction

// Alias hide the real type of the enum
// and users can use it to define the var for accepting enum
type Alias = int

type list struct {
    West Alias
    East Alias
    North Alias
    South Alias
}

// Enum for public use
var Enum = &list{
    West: 0,
    East: 1,
    North: 2,
    South: 3,
}
```

### int enum use iota keyword
```
type:
    int
name:
    direction
list:
    West = 9
    east = 8
    North = iota
    South
```

result:
```
package direction

// Alias hide the real type of the enum
// and users can use it to define the var for accepting enum
type Alias = int

type list struct {
    West Alias
    East Alias
    North Alias
    South Alias
}

// Enum for public use
var Enum = &list{
    West: 9,
    East: 8,
    North: 0,
    South: 1,
}
```

### string enum
```
type:
    string
name:
    direction
list:
    West = "W"
    East = "E"
    North = "N"
    South = "S"
```

result:
```
package direction

// Alias hide the real type of the enum
// and users can use it to define the var for accepting enum
type Alias = string

type list struct {
    West Alias
    East Alias
    North Alias
    South Alias
}

// Enum for public use
var Enum = &list{
    West: "W",
    East: "E",
    North: "N",
    South: "S",
}
```

repository: rahamannaik/enum

repo_name: rahamannaik/enum


text:
# Gitsearch

[![Build Status](https://travis-ci.org/Tom-Alexander/gitsearch.svg?branch=master)](https://travis-ci.org/Tom-Alexander/gitsearch)

This is a tool for indexing and searching git repositories. Provides github
and bitbucket web hooks for automatic indexing. Allows searching within branches/tags across repositories.

## Installing

Make sure your have [elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html) installed and running, then install gitsearch with:

```
npm install gitsearch -g
```
## Usage
### Indexing
#### `gitsearch index remote`
Indexes a git repository from a remote machine.

#### `gitsearch index local`
Indexes a git repository from your local machine.

### Serve
#### `gitsearch serve`
Starts the client application for searching.

### Watch
#### `gitsearch watch`
Starts the client application for watching github and bitbucket webhooks.

### Arguments

##### `--url <string>`
The path to the repository, for example `https://github.com/Tom-Alexander/gitsearch.git`. This argument is *required* for local repositories and is not set by default.

##### `--path <string>`
The path to the repository, for example `~/gitsearch`. This argument is *required* for local repositories and is not set by default.

##### `--name <string>`
The name of the repository, for example `Tom-Alexander/gitsearch`. This argument is *required* and is not set by default.

##### `--type <GITHUB|BITBUCKET>`
The type of repository, for example `GITHUB`. This argument is *optional* and is not set by default.

##### `--host <string>`
The elasticsearch host, for example `localhost:9200`. This argument is *required* but `localhost:9200` will used by default.

##### `--index <string>`
The elasticsearch index, for example `repositories`. This argument is *required* but `repositories` will used by default.

##### `--port <integer>`
The port for the server to listen to, for example `5900`. This argument is *optional*  for `serve` and `watch`.

##### `--user <integer>`
The username for the basic auth credentials. This argument is *optional*  for `watch`.

##### `--pass <integer>`
The password for the basic auth credentials. This argument is *optional*  for `watch`.

## Ignoring files
The indexer will look for glob path patterns in a file named `.gitsearchignore` in the `master` branch for your repository. Paths that match these patterns will be ignored, along with the patterns in the  `.gitsearchignore` file in this library.

## API
#### `indexFromURL(url, type, name, index)`
#### `indexFromPath(path, type, name, index)`
#### `indexFromRepository(source, type, name, index)`

## License
The MIT License (MIT)

Copyright (c) Tom Alexander

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

repository: WooodHead/gitsearch

repo_name: WooodHead/gitsearch


text:
# Memory Streams JS
_Memory Streams JS_ is a light-weight implementation of the `Stream.Readable` and `Stream.Writable` abstract classes from node.js. You can use the classes provided to store the result of reading and writing streams in memory. This can be useful when you need pipe your test output for later inspection or to stream files from the web into memory without have to use temporary files on disk.

## Installation
Install with:

```bash
npm install memory-streams --save
```

## Usage
Sample usage, using the `ReadableStream` class and piping:

```js
var streams = require('memory-streams');

// Initialize with the string
var reader = new streams.ReadableStream('Hello World\n');

// Send all output to stdout
reader.pipe(process.stdout); // outputs: "Hello World\n"

// Add more data to the stream
reader.append('Hello Universe\n'); // outputs "Hello Universe\n";
```

Using the `ReadableStream` class and reading manually:

```js
var streams = require('memory-streams');

// Initialize with the string
var reader = new streams.ReadableStream('Hello World\n');

// Add more data to the stream
reader.append('Hello Universe\n'); // outputs "Hello Universe\n";

// Read the data out
console.log(reader.read().toString()); // outputs: "Hello World\nHello Universe\n"
```

Using the `WritableStream` class and piping the contents of a file:

```js
var streams = require('memory-streams')
  , fs      = require('fs');

// Pipe
var reader = fs.createReadStream('index.js');
var writer = new streams.WritableStream();
reader.pipe(writer);
reader.on('readable', function() {

  // Output the content as a string
  console.log(writer.toString());

  // Output the content as a Buffer
  console.log(writer.toBuffer());
});
```

You can also call the `write` method directly to store data to the stream:

```js
var streams = require('memory-streams');

// Write method
var writer = new streams.WritableStream();
writer.write('Hello World\n');

// Output the content as a string
console.log(writer.toString()); // Outputs: "Hello World\n"
```

For more examples you can look at the tests for the module.

## License
MIT

Copyright (c) 2017 Paul Jackson

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

repository: MohammedAlaboud/HackferenceProject

repo_name: MohammedAlaboud/HackferenceProject


text:
> **Overfitting**: the classifier works well on training set but really bad on evaluation set
>
> **Consistency**: classifier able to flawlessly predict the class of all training instance
>
> **Generalisation**: how well does the classifier generalise from the specifics of the training example to predict the target function

### Factors to consider

- Overfitting
  - Plot learning curves based on split sizes
  - If hold-out is used
    - Trade-off between more training instances and more evaluation instances (more reliable estimate of effectiveness)
- Generalisation
  - A good model should fit the training data well, and generalise well to unseen data
  - The expectation is that training and test data are randomly selected from the same population, but neither are the entire population.
- Possible evidences of overfitting
  - Decision boundary distorted by noice
    - ![image-20190616125346785](assets/image-20190616125346785.png)
  - Could be due to small numbers of examples
  - Could be due to non-randomness in training sample (sample bias)

### Model Bias and Variance

- Model bias
  - Classifier makes systematically wrong predictions
- Sampling bias
  - Training or evaluation dataset isn't representative of the population
- Model variance
  - Different training sets to produce different model/predictions with the same learner
- High bias and high variance are often bad, but low bias and low variance are no guarantee of good
  - Low bias
    - Weighted random classifier
  - Low variance (zero variance)
    - 0-R

### Evaluation Bias and Variance

- Evaluation bias
  - Evaluation strategy to over or under estimate the **effectiveness** of our classifier
- Evaluation variance
  - Our estimate of the **effectiveness** of a model changes a lot as we alter the instances in the evaluation set.
- Difference between evaluation bias and model bias
  - Whether we need to compare with external labeled data.
- Control evaluation basis and variance
  - Holdout partition size
    - More training data: less model variance, more evaluation variance
  - Repeated random subsampling and K-fold Cross-Validation
  - Stratification
    - less model bias
  - Leave-one-out Cross-Validation
    - No possibility of sampling bias, lowest bias/variance in general

repository: siriusctrl/UniPublic

repo_name: siriusctrl/UniPublic


text:
# Facebook Chat Bot Demo

This demo shows the manual integration with [Facebook Messenger](https://developers.facebook.com/) and
[API.ai](https://console.api.ai) using webhook.

This bot allows you to create, join, or leave a group on Facebook Messenger.

## Demo1: Create user
![alt text](./images/demo/demo1.gif "Demo1")

## Demo2: Create group
![alt text](./images/demo/demo2.gif "Demo2")

## Demo3: Show groups
![alt text](./images/demo/demo3.png "Demo3")

# Usage
To use it, you'll have to open a server. (It's closed now. I used [ngrok](https://ngrok.com) to open one)

Also, change the url in both facebook messenger and api.ai. (as shown below)

## In [facebook_developer](https://developers.facebook.com):
![alt text](./images/changes/fb_webhook_changes.gif "fb_webhook_changes")

## In api.ai:
![alt text](./images/changes/apiai_chages.png "apiai_chages")

gha_language: JavaScript


text:
# eslint-etc

[![GitHub License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/cartant/eslint-etc/blob/master/LICENSE)
[![NPM version](https://img.shields.io/npm/v/eslint-etc.svg)](https://www.npmjs.com/package/eslint-etc)
[![Downloads](http://img.shields.io/npm/dm/eslint-etc.svg)](https://npmjs.org/package/eslint-etc)
[![Build status](https://img.shields.io/circleci/build/github/cartant/eslint-etc?token=43cceb8f45da20a867f4cee8b7068fdf5ceb1269)](https://app.circleci.com/pipelines/github/cartant)
[![dependency status](https://img.shields.io/david/cartant/eslint-etc.svg)](https://david-dm.org/cartant/eslint-etc)
[![devDependency Status](https://img.shields.io/david/dev/cartant/eslint-etc.svg)](https://david-dm.org/cartant/eslint-etc#info=devDependencies)
[![peerDependency Status](https://img.shields.io/david/peer/cartant/eslint-etc.svg)](https://david-dm.org/cartant/eslint-etc#info=peerDependencies)

More utils for use with `eslint`.

I use these utils to implement and test my own ESLint rules. That's their primary purpose, so the documentation is ... light.

## fromFixture

`fromFixture` allows TSLint-like fixtures to be used to test ESlint rules. Using fixtures means that you don't have to specify lines and columns. Instead, you underline the failure locations within the fixture, like this:

<!-- prettier-ignore -->
```ts
{
  invalid: [
    fromFixture(stripIndent`
      const name = "alice";
            ~~~~            [foo { "identifier": "name" }]
                   ~~~~~~~  [bar]
      const role = 'cto';
            ~~~~            [foo { "identifier": "role" }]
    `),
    fromFixture(stripIndent`
      const name = "alice";
            ~~~~            [foo { "identifier": "name" }]
      const role = 'cto';
            ~~~~            [foo { "identifier": "role" }]
    `, {
      options: [{ bar: false }]
    }),
  ]
}
```

which is equivalent to the following:

<!-- prettier-ignore -->
```ts
{
  invalid: [{
    code: `const name = "alice";
const role = 'cto';`,
    errors: [{
      column: 7,
      endColumn: 11,
      line: 1,
      endLine: 1,
      messageId: "foo",
      data: {
        identifier: "name",
      },
    }, {
      column: 14,
      endColumn: 21,
      line: 1,
      endLine: 1,
      messageId: "bar",
      data: {},
    }, {
      column: 7,
      endColumn: 11,
      line: 2,
      endLine: 2,
      messageId: "foo",
      data: {
        identifier: "role",
      },
    }]
  }, {
    code: `const name = "alice";
const role = 'cto';`,
    errors: [{
      column: 7,
      endColumn: 11,
      line: 1,
      endLine: 1,
      messageId: "foo",
      data: {
        identifier: "name",
      },
    }, {
      column: 7,
      endColumn: 11,
      line: 2,
      endLine: 2,
      messageId: "foo",
      data: {
        identifier: "role",
      },
    }],
    options: [{
      bar: false
    }]
  }]
}
```

Specifying `data` in the fixture is optional. If it's omitted, `data` defaults to `{}`.

The second, optional, argument passed to `fromFixture` can be used to pass additional test case properties - `options` and `output`, etc.

### Suggestions

A `suggestions` array can be passed to `fromFixture` via its second, optional parameter and, when passed, a `suggest` annotation can be used within the fixture, like this:

<!-- prettier-ignore -->
```ts
fromFixture(stripIndent`
  const a = "alice";
            ~~~~~~~ [foo suggest]
`, {
  suggestions: [{
    messageId: "suggestionForFoo",
    output: "/* suggestion output goes here */"
  }]
}),
```

And it's possible to include multiple `suggest` annotations and suggestions in a single fixture:

<!-- prettier-ignore -->
```ts
fromFixture(stripIndent`
  const a = "alice";
            ~~~~~~~ [foo suggest 0]
  const b = "bob";
            ~~~~~ [bar suggest 1]
`, {
  suggestions: [{
    messageId: "suggestionForFoo",
    output: "/* suggestion for foo output goes here */"
  }, {
    messageId: "suggestionForBar",
    output: "/* suggestion for bar output goes here */"
  }]
}),
```

The `suggest` annotations work with the `suggestions` array in the following manner:

-   If `suggest` is specified with no indices, all suggestions are associated with the annotated error.
-   If `suggest` is specified with indices, suggestions at those indices are associated with the annotated error.
-   If `suggest` is not specified, no suggestions are associated with the annotated error.
-   And if `suggestions` are specified without a `suggest` annotation being used, `fromFixture` will throw an error.

And if the rule has both a fixer and suggestions, you can specify an `output` - for the fixer - in conjunction with a `suggestions` array:

<!-- prettier-ignore -->
```ts
fromFixture(stripIndent`
  const a = "alice";
            ~~~~~~~ [foo suggest]
`, {
  output: "/* fixer output goes here */",
  suggestions: [{
    messageId: "suggestionForFoo",
    output: "/* suggestion output goes here */"
  }]
}),
```

repository: cartant/eslint-etc

repo_name: cartant/eslint-etc


text:
This repo is a copy of golang std crypto on tag go1.13.6, hash 14b79df428fd.

## modified

- elliptic/elliptic.go
- elliptic/elliptic_test.go
- elliptic/p224.go
- elliptic/p256.go
- elliptic/p256_asm.go
- elliptic/p256_asm_amd64.s
- x509/x509.go
- x509/root_cgo_darwin.go
- tls/common.go
- tls/key_schedule.go
- copy go/internal to internal

## added
- elliptic/secp256k1_test.go
- elliptic/secp256k1.go
- elliptic/sm2p256v1.go

## Reference

- [crypto/elliptic: generalize elliptic.Curve to allow A != -3](https://github.com/golang/go/pull/26873/files)

repository: Ning-Qing/crypto

repo_name: Ning-Qing/crypto


text:
# DiscoverSDK
[![Build Status](https://travis-ci.com/locally-io/ios-discover-sdk.svg?branch=master)](https://travis-ci.com/locally-io/ios-discover-sdk) ![GitHub](https://img.shields.io/github/license/locally-io/ios-discover-sdk.svg) ![GitHub last commit](https://img.shields.io/github/last-commit/locally-io/ios-discover-sdk.svg)

## Requirements
----
* iOS 10 +
* Xcode 10 +
* Swift 4.2 +
* Cocoapods

## Installation
----
## CocoaPods
Cocoapods is a dependency management platform to install, update and delete the libraries used on the project.

You can  install Cocoapods with the following terminal command

```ruby
$ sudo gem install cocoapods
```

To initialize Cocoapods on your project, navigate through the terminal to your project directory and run this command:
```ruby
$ cocoapods init
```

This will create a `.podfile` on the root of your project. The `.podfile` is the configuration file that Cocoapods use to declare the project dependencies.

Add the Discovery SDK as a dependency to your project like this.

```ruby
source 'https://github.com/CocoaPods/Specs.git'
platform :ios, '10.0'
use_frameworks!

target 'Your Target Name' do
pod 'DiscoverySDK'
end
```

On the root of your project where the `.podfile` was created run the following command to install the Discovery SDK as a dependency.

```ruby
$ pod install
```

##  Permissions

Discover SDK will need location and bluetooth permissions. Add the following keys to  your App plist.

- NSBluetoothPeripheralUsageDescription
- NSLocationAlwaysAndWhenInUseUsageDescription
- NSLocationWhenInUseUsageDescription

# Usage
----
## Connecting the Discover SDK

To start using the Discover SDK simply call the **connect** operation.

```swift
DiscoverSDK.shared.connect()
```

The discover SDK will request location permission from your app and after authorized  it will start recording immediately. The default configuration records data every 20 seconds and streaming it every minute.

This is all you have to have the DiscoverSDK installed and running.

# Fine grain configuration using Delegates

Discover SDK can add a more fine grain control over each step of the permissions or recording process in your App.

## Location Permissions Delegate

To receive updates about location permissions just add a location delegate and implement the required operations.

```swift
DiscoverSDK.shared.locationDelegate = self
DiscoverSDK.shared.connect()
```

Implement the following operations in your delegate

```swift
extension MyClass: DiscoverSDKLocationDelegate {

func didAuthorizedLocationMonitoring() {
// Your code here
}

func didNotAuthorizedLocationMonitoring() {
// Your code here
}
}
```

repository: locally-io/ios-discover-sdk

repo_name: locally-io/ios-discover-sdk


text:
+++
title = "Utilidades"
date = 2018-09-20T23:16:14-03:00
draft = false

# Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`.
# authors = ["Joshua Foer"]

# Tags and categories
# For example, use `tags = []` for no tags, or the form `tags = ["A Tag", "Another Tag"]` for one or more tags.
tags = []
categories = []

# Featured image
# Place your image in the `static/img/` folder and reference its filename below, e.g. `image = "example.jpg"`.
# Use `caption` to display an image caption.
#   Markdown linking is allowed, e.g. `caption = "[Image credit](http://example.org)"`.
# Set `preview` to `false` to disable the thumbnail in listings.
[header]
image = ""
caption = ""
preview = true

+++
- [On Mind, Language and Machines](https://0x0f0f0f.github.io/posts/2020/02/on-mind-language-and-machines/)

- [Utopian for Beginners](https://www.newyorker.com/magazine/2012/12/24/utopian-for-beginners)

- [Design Patterns and the Tower of Babel](https://effectivesoftwaredesign.com/2011/07/11/design-patterns-and-the-tower-of-babel/)

- [Terence McKenna - Searching For Patterns](https://www.youtube.com/watch?v=QJBOMwsGsYs)

repository: mvrozanti/academic-kickstart

repo_name: mvrozanti/academic-kickstart


text:
# Trackit-BackEnd

> This project is the back-end of an mobile-app to track the progress of some activities like working on challenges, looking for job, working on your network or if you are just taking a break you can get track of it.

![nativeapp](https://user-images.githubusercontent.com/11781597/76324032-176a6600-62ab-11ea-8e51-aa084159089f.png)

## Built With

- Ruby version 2.6.4,
- Rails version 6.0.0.

## Live Demo
> Install expo on your device. You can download the app from the [Play Store](https://play.google.com/store/apps/details?id=host.exp.exponent&hl=es_SV) then go to [Expo page](https://expo.io/@juakata/Trackit) and scan the QR code.

## The Front-End was built with React Native.

[Link to the Front-End Repo](https://github.com/Juakata/Trackit)

## Dependencies

-rack-cors
-pg
-rspec-rail
-factory_bot_rails
-rubocop

## To Run the project locally
- Clone the repo project https://github.com/Juakata/Trackit.git,
- yarn install --check-files
- bundle install
- rails server
- You can go to routes to see the options I have available.

## Authors

👤 **Andoni Uzquiano**

- Github: [@Juakata](https://github.com/Juakata)
- Twitter: [@HUzquiano](https://twitter.com/HUzquiano)
- Linkedin: [@Andoni](https://www.linkedin.com/in/andoni-uzquiano-31304818a/)

## 🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check the [issues page](https://github.com/Juakata/Trackit-BackEnd/issues).

## Show your support

Give a ⭐ if you like this project!

## 📝 License

This project is [MIT](https://opensource.org/licenses/MIT) licensed.

repository: Juakata/Trackit-BackEnd

branch_name: refs/heads/development

gha_language: Ruby

repo_name: Juakata/Trackit-BackEnd


text:
[![PyPI](https://img.shields.io/pypi/pyversions/saseg_runner.svg)](#)
[![PyPI](https://img.shields.io/pypi/status/saseg_runner.svg)](#)
[![PyPI](https://img.shields.io/pypi/v/saseg_runner)](https://pypi.org/project/pcm/)
[![PyPI](https://img.shields.io/pypi/l/saseg_runner.svg)](#)

# Overview
saseg_runner get you run egp file from python or command line.

# Requirment
* windows
* python 3.6 later
* SAS Enterprise guide

# Install
```bash
pip install saseg_runner
```

# Usage

## as python library

```python
from saseg_runner import run_egp, SASEGRuntimeError
run_egp(egp_path='test_success.egp', eg_version='7.1', profile_name='Your Profile')

# opening SAS Enterprise Guide 7.1
# -> application instance created
# activating profile:[SAS Asia]
# -> profile:[SAS Asia] activated
# opening test_success.egp
# -> egp file opened
# running test_success.egp
# -> run finished
# -> saved to test_success_20191031-1102.egp
# getting logs from test_success_20191031-1102.egp
# -> log created
# successfully finished exectuing test_success.egp
# elapsed_time:40[sec]
```

In case some tasks in the egp file fails, 'run_egp' can raise Exception of SASEGRuntimeError!
```python
from saseg_runner import run_egp, SASEGRuntimeError
run_egp(egp_path='test_fail.egp', eg_version='7.1', profile_name='Your Profile')

# opening SAS Enterprise Guide 7.1
# -> application instance created
# activating profile:[SAS Asia]
# -> profile:[SAS Asia] activated
# opening test_fail.egp
# -> egp file opened
# running test_fail.egp
# -> run finished
# -> saved to test_fail_20191031-1056.egp
# getting logs from test_fail_20191031-1056.egp
# -> log created
# [task 2] failed in test_fail.egp
# ERROR: CLI error occured: [DataDirect][ODBC lib] Specified driver could not be loaded
# Traceback (most recent call last):
#   File "saseg_runner/runner.py", line 123, in <module>
#     run_egp(SCRIPTDIR_PATH.parent.parent / 'test/test_fail.egp')
#   File "saseg_runner/runner.py", line 103, in run_egp
#     raise SASEGRuntimeError
# __main__.SASEGRuntimeError
```

### Parameters
* egp_path : Union[str, Path]
    * SAS Enterprise Guide file path.
* profile_name : str
    * profile name to use
* overwrite: bool
    * controls whether to save the egp file after exection. if False, timestamp is added to filename. The default is False.
* remove_log: bool
    * whether to remove log files or not. the default is True.
* verbose: bool
    * default is False

## from command line
```bash
run_egp <your egp file path>
```

gha_language: Python


text:
* _2.2.0_
  * Support for adding and removing child data. Adding `AddChildData` and `RemoveChildData` actions and `addChildData` and `removeChildData` action creators.

* _2.1.0_
  * entitiesProjector takes an optional array of id strings ([@juanpmarin](https://github.com/juanpmarin)) Closes #18

* _2.0.0_
  * Serialization support for actions. _Details:_  The normalization of entities is now perfomed in the action constructor. Previously it was handled by the reducer. As ([@PachowStudios](https://github.com/PachowStudios)) pointed out in Issue #16, ngrx-normalizr actions were not serializable. This could raise issues with other redux/ngrx libraries. The normalizr `schema.Entity` is not part of the action payload anymore, hence the interfaces for describing the payload have changed and the action constructor does no longer take the payload itself as an argument. As long as you did not type any action parameters in your code or dispatched actions directly with a simle pojo by using the exported action type names, you should have no problem updating, since the arity/keys of the constructor API did not change - see Breaking Changes. Closes #16

  * **Breaking Changes**
    * Action constructor parameters changed from  `NormalizeDataPayload` to `NormalizeActionConfig` and from `NormalizeRemovePayload` to `NormalizeRemoveActionConfig`
    * Action `payload` property types changed from `NormalizeDataPayload` to `NormalizeActionPayload` and from `NormalizeRemovePayload` to `NormalizeRemoveActionPayload`
    * (might break) Internal used interface `SchemaPayload` replaced by `NormalizeActionSchemaConfig`

* _1.0.4_
    * exporting `SchemaSelectors` interface ([@JSantha](https://github.com/JSantha))

* _1.0.3_
    * fixed typos ([@hoisel](https://github.com/hoisel))
    * exporting types for reuse in other libraries

* _1.0.2_
    * improved documentation
    * improved code coverage (100%)
    * `SetData` action for setting entity data instead of updating and adding data

* _1.0.1_
    * `actionCreators` for creating schema bound actions
    * improved code coverage

* _1.0.0_
    * first production version

* _0.0.1_
    * first development version


text:
# hubot-remote-ark
## hubot-remote-ark is a script/extension for [GitHub's Hubot](https://hubot.github.com/) for controlling your Linux based ARK server.

Currently **very** basic and assumed you already have a start script for the server.

Driven out of wanting to give more players ability to control the ARK server without being given access to the host.

## Commands
- ark start - starts the ark server
- ark stop - stops the ark server
- ark status - responds with if the server is already up or not

# Setup

Add `hubot-remote-ark` to the `external-scripts.json` for your Hubot.

## Environment Variables

- SSHHOST - domain or IP of host
- SSHUSER - user to connect with (likely `steam` if following ARK setup)
- SSHPASS - password for user
- SSHPORT - port to connect on (default: 22)
- ARKSTARTSCRIPT - name of ARK script. currently assumes located in user home directory
- ARKPROCESSNAME - name of ARK process, left as variable for easy updating in case server process name is changed by devs (default: ShooterGameServ)

## Example Start Scripts
This hubot script currently requires you have to have startup and update shell scripts on your server. Example startup scripts are provided in the example_scripts directory.

- ark_server.sh - ARK script to run located in user home directory and/or parent directory of `steamcmd`
- ark_update.sh - login to steam and check updates for ARK (place in `steamcmd`)
- server_start.sh - example command to start server (place in `steamcmd/ark/ShooterGame/Binaries/Linux/`)

repository: iamtjg/hubot-remote-ark

repo_name: iamtjg/hubot-remote-ark


text:
# BLE-periferal-sample

This is sample code to use BLE at ESP32 that is behaved periferal

ESP32 adverties macaddress itself.

## How to use

Build, upload, and see monitor.

```shell-session
$ platformio run
$ platformio run --target upload
$ platformio device monitor
```

## Recommend

You can use PlatformIO IDE for VSCode when you build, upload, see monitor.
http://docs.platformio.org/en/latest/ide/vscode.html

## Require

- PlatfomrIO IDE

## Ref

If you need BLE scan sample, You can see below.

https://github.com/szkkentaro/BLE-scan-sample

repository: szkkentaro/BLE-periferal-sample

repo_name: szkkentaro/BLE-periferal-sample


text:
## Introduction

A Sudoku game written in Java.

## Build

Package:

```shell
mvn package
```

Run:

```shell
java -jar target/sudoku-*-jar-with-dependencies.jar
```

## Screenshot

![A snapshot](./sudoku-screenshot.png)

repository: cloudecho/sudoku

repo_name: cloudecho/sudoku


text:
# tags.py

This python script will create an sqlite database of local music and allow very simple queries.

It is used mainly on Linux, but should work on other OSes.

## Installation

This script requires [mutagen](https://github.com/quodlibet/mutagen) to pull the mp3 tags out of your files.

Mutagen is needed to read local mp3 tag information that is inserted into the database.

## Usage examples

```
$ ./tags.py

./tags.py , version  0.8.1
--------------------
database path:  /media/archerja/Extreme SSD/bin/id3.db3
   music path:  /media/archerja/Extreme SSD/Music
--------------------

Usage:  ./tags.py  [ help | summary | search | update | m3u | db-build ] {argument} {query}

```

```
$ ./tags.py summary
 Question: Which summary are you looking for?
   [ all | artist | genre | group | year | bitrate ]

```

```
$ ./tags.py search
 Question: Which search are you looking for?
   [ artist | album | title | year | genre | discog | group | bitrate | below320 ]
$ ./tags.py search artist
 Question: Which artist are you looking for?
$ ./tags.py search artist "Heart"

```

## Help

```
$ ./tags.py --help

./tags.py , version  0.8.1
--------------------
database path:  /media/archerja/Extreme SSD/bin/id3.db3
   music path:  /media/archerja/Extreme SSD/Music
--------------------

Usage:  ./tags.py  [ summary | search | update | m3u | db-build ] {argument} {query}

                    db-build    [your music root path]
                                (only use first time, will delete all records)

                    update      [music path to update]
                                (updates the database)

       Database Searches:
                    search      artist      "string" (using "like")
                    search      album       "string" (using "like")
                    search      title       "string" (using "like")
                    search      year        "string" (using "like")
                    search      genre       "string" (using "like")
                    search      discog      "string" (using "like", for artist)
                    search      group       "string" (using "like", for groups)
                    search      below320    "group"  (using "like")
                    search      bitrate     "string" (128,256,320,etc.)

       Database Summaries:
                    summary     all         (total albums, artists, records)
                    summary     artist      (total albums per artist)
                    summary     genre       (total records per genre)
                    summary     group       (total records per group)
                    summary     year        (total records per year)
                    summary     bitrate     (total records per bitrate range)

       Music list to export to other applications:
                    m3u         "artist" "album" "title" "year"

           Examples:
                    ./tags.py m3u "james taylor" "greatest hits" "" "1976" | mpv --playlist=-

                    mpv --playlist <(./tags.py m3u "james taylor" "greatest hits" "" "1976")

       Notes:
                    current "groups"    (artist,christmas,classical
                                          ,compilation,lounge,soundtrack)

```

## Release History

* 0.8.1
    * Show error if db and music paths differ
    * Added year search
* 0.8.0
    * Rewrite and cleanup
    * Added m3u search
* 0.7.7
    * Added list search
    * More work on help section
* 0.7.6
    * Added discog search
    * Worked on help section
* 0.7.5
    * Added bitrate searches
* 0.7.0
    * Added several summary queries
    * Added genre search
* 0.3.0
    * The first proper release
    * Added simple searches
* 0.0.1
    * Work in progress

## Author

Joseph Archer (C) 2017

## License

The code is covered by the MIT.


text:
# Essex lorry death victims 'all Vietnamese' after 39 bodies found

Published at: **2019-11-01T22:41:10+00:00**

Author: **James Rodger, https://www.facebook.com/jamesrodgerjournalist**

Original: [birminghammail](https://www.birminghammail.co.uk/news/midlands-news/essex-lorry-death-victims-all-17188885)

The 39 migrants found dead in the back of a lorry are believed to be Vietnamese nationals, Essex police has said.
Assistant Chief Constable Tim Smith said on Friday evening: "At this time, we believe the victims are Vietnamese nationals, and we are in contact with the Vietnamese Government.
"We are in direct contact with a number of families in Vietnam and the UK, and we believe we have identified families for some of the victims whose journey ended in tragedy on our shores."
"The confirmatory evidence needed to formally present cases to HM Senior Coroner for her consideration has not yet been obtained.
"This evidence is being gathered across a number of jurisdictions worldwide. As a result, we cannot at this time announce the identity of any of the victims."
Mr Smith said specially trained people, supported by Vietnamese interpreters, were manning its dedicated hotline between 9am to midnight, seven days a week, for people seeking to be in touch about the incident.
In a statement, the Vietnamese Embassy in London sent its "heartfelt condolences" to the families of the victims.

It said it was "deeply saddened" that Essex Police's initial findings were that the victims may be Vietnamese.
"Specific identities of the victims still need to be identified and confirmed by the relevant authorities of Vietnam and UK," the embassy said.
Its statement added: "The Embassy of Vietnam in London will closely co-ordinate with the relevant authorities of Vietnam and UK to support the families of the Vietnamese victims, if any, to bring their loved ones home."
"We strongly condemn the inhuman activities of the traffickers of illegal immigrants that have caused such a tragedy for the victims and their families.

"Vietnam will work closely with the UK to bring the criminals to justice and find effective measures to prevent a similar tragedy from happening again in the future."

license: Unlicense

repository: dsadlollioygjmzvcxv/tdc-exported

repo_name: dsadlollioygjmzvcxv/tdc-exported


text:
# Sidecar Containers

Containers that exist to assist your microservices. In Kubernetes, sidecars run within the same namespace, i.e. the same pod, and have the same lifecycle as the service it supports.

Istio is an example of a sidecar container. By routing all traffic to and from a service through the istio sidecar, it is able to control ingress and egress routing, circuit breaker support, provide observability, and more.

The goal of a sidecar container is such that support for these and other features of cloud native computing are contained outside of your source code, i.e. you don’t need to import libraries and write code to support, for example, a circuit breaker.