text:
# Python API Deployment
This repo contains guided examples of deploying Python applications via multiple ways.

## 1. Flask
Flask is a micro-framework for deploying Python apps in a straight-forward manner.

You can install flask by pip:

```
pip install Flask
```

### Flask Examples
Under [1_Flask](1_Flask), 2 simple flask apps are provided.

After cloning the files or copying the codes by youself, use command line (bash) to navigate to the directories containing the files.

### Example 1 - GET method
Take a look at the code of [app.py](1_Flask/app.py)
1. On bash:
    1. export FLASK_APP=app.py
    2. flask run
2. Go to your browser: http://localhost:5000/
    1. You should see ```{"dateOfBirth": null, "numbers": null, "occupation": null}```
3. Change the url to: http://localhost:5000/?dateOfBirth=2000-01-01&numbers=1234567&occupation=datacleaning
    1. You should see ```{"dateOfBirth": "2000-01-01", "numbers": "1234567", "occupation": "datacleaning"}```

**set* instead of *export* on windows

#### Request & Response
All interactions between and client (browser) the server (flask app) are request & response.
When you make a query (the keys & values in url) and send the request to your server, you flask app processes the query and return the response.

In the example, we make a query from the address bar, which is a GET request.

You can get the user input by ```request.args.get('key')```.

### Example 2 - POST method
[app2.py](1_Flask/app2.py) differs from app.py

* It gives the route name as `"/predict"`
* It is a POST method.

You cannot make a post call in a url from browser address bar.

We use a third-party tool called [postman](https://www.getpostman.com/). It is the easiest way to make requests to a server.
It supports all request methods and authorisation headers.

Download and install the postman tool:
1. On bash:
    1. `export FLASK_RUN_PORT=5001`
    2. `export FLASK_APP=app2.py`
    3. `flask run`
2. Browser: http://localhost:5001/predict
    1. You should see a *Method Not Allowed* error.
3. Postman: http://localhost:5001/predict -> click SEND button
    1. You should see a *Method Not Allowed* error. The same as your browser.
    2. Choose from the scroll down button and change GET -> POST.
    3. In *Body*, type in the keys ('name','age') and any values you want. Hit SEND.
    4. You should see your input at bash.

### Example 3 - Textbox

[app3.py](1_Flask/app3.py) is an example with textbox input.

You can see the [demo hosted on Repl](https://uselessflaskapp--jsnc.repl.co/). (Wait for a while for the packages to
    be loaded if you are the first visitor of the day.)

The app3.py looks for the designated html file under /templates directory, renders that html, accepts user input in textbox and return the results after processing calculation.

As you may have realised, this is how normal web users can submit a POST request to the server via submitting through a textbox.

You can always host the app3.py on your localhost with the same flask commands as before.

### Flask Documentation
Check out the [flask documentation](http://flask.pocoo.org/).
___
# Deploy for Production
While flask is easy to use, it is intended for development use but not for direct production. You can always develop and test your application on your local machine. But when it comes to development, it is a standard to deploy on vitural private servers, as they are scalable, cheap and convenient.

## 2. Digital Ocean

[Digital Ocean](https://m.do.co/c/e7833ee793bd) is one of the commonly used VPS. A droplet means a server. So you can create multiple servers all maintained under one account.

After registration, create the simplest ubuntu server (1GB-memory/25GB-disk).

### Connect to Your Remote Machine
1. Open a new command line, type `$ ssh root@ipv4` where ipv4 is the ip address found in your droplet page.
2. You can find your initial password in your email box. You are forced to change it at your first login to server. Change it to something memorable (E.g *The best apples are from south california*).
3. Reconnect ssh and you are done! Type `python` and you can use the default python on your remote machine.

### Example 4 - Caesar Cipher
[app4.py](1_Flask/app4.py) is a caesar cipher application delopyed on Digital Ocean.

You can see the demo on [http://<IP_ADDRESS>:8080/](http://<IP_ADDRESS>:8080/).

It allows user inputs of a name and a message to encrypt. The app will return the encrypted message. At the backend, all user inputs and the timestamp will be recored in the `record.csv` file.

### Working with Linux Server
System administration is rarely taught even for computer science students.

Here I will guide you through step-by-step how to set up your linux server and deploy the python app as I do.

Every time you open a new machine you would update it to the latest one:
1. `$ sudo apt-get update && sudo apt-get upgrade`

Install Python3
1. `$ sudo apt install python3`
2. `$ sudo install python3-pip`

#### Gunicorn
Python apps require wsgi, a compatible server to run properly.
1. Use Ngnix to set up wsgi with flast (too complicated).
2. Use Gunicorn, another python module, to deploy your flask app (much easier).

#### Pip3 install modules
1. `$ pip3 install gunicorn flask numpy scipy`
2. Check the installed packages using `pip3 list`.

#### Git clone something
1. `$ git clone https://github.com/jsrpy/Python_Deployment`

#### Deploy the app with Gunicorn
1. Nevigate to the app's directory... `$ cd Python_Deployment/1_Flask`.
2. Create an empty csv file `$ touch record.csv`.
3. Deploy the App: `$ gunicorn --bind <IP_ADDRESS>:8080 app4`.
4. After getting an error, do the following:

```
$ nano app4.py
```
Change line 5 & 6 into:
```
application = Flask(__name__)
app = application
```
Press `Ctrl-O` to save, then press `enter` & `Ctrl-X` to exit. Now run the gunicorn code again.

The `gunicorn` command basically sends your app4 onto the ipv4 address at port 8080.

By this point, you should see your app running on `ipv4:8080`. Simply go to your browser and visit that page!

Submit an entry. Then, check the record by simply printing out the standard output `$cat record.csv` (You should either open a new command line or stop the gunicorn process by pressing `Ctrl+C`).


text:
# Macaw demo

For convenience, we provide the minimally documented file [`macaw/demo.py`](macaw/demo.py) with self-contained code
for starting a local demo to interact with Macaw models in
a browser (or through a REST API). It uses the [streamlit](https://streamlit.io/) framework, but the code
can easily be modified for other use cases.

To configure the demo, simply modify the variables near the top of the file, e.g.,

```
# CONFIGURATION
MODEL_NAME_OR_PATH = "allenai/macaw-large"  # Name or path to model
CUDA_DEVICES = []   # List of available CUDA devices if any, e.g., [0, 2]
```

This defines which model to use (downloaded to local cache from the Hugging Face model hub, or point to
a local directory where model is stored) as well as any
GPU devices available (recommended for good performance for the models larger than Macaw-large).

Also make sure the dependencies are installed (e.g., run `pip install -r requirements.txt`, ideally in a
dedicated virtual environment), and the demo is started
by running `streamlit run macaw/demo.py`. It might take a few minutes for the model to load and be ready, and
then the demo can be accessed in a browser by going to http://localhost:8501/.

The input format in the demo expects each slot to be given on separate lines, prefixed by the slot capital
letter, with output slots given without value. E.g., a request for angle QM->AE is given as:

```
Q: Which characteristic of a cheetah is more likely to be learned rather than inherited?
M: (A) speed (B) a spotted coat (C) hunting strategies (D) claws that do not retract
A
E
```

which, for the Macaw-large model, should give the output:

```
Answer: hunting strategies
Explanation: Skills are learned characteristics. Hunting is a kind of skill.
```

The demo also supports a special mode for scoring different answer options, this only works when there is a single
output slot, and makes the most sense when the answer options are also present as multiple-choice options. This
is indicated by the special "X" slot, an example would be:

```
Q: What material is magnetic?
M: (A) iron (B) copper (C) paper
X: (A) iron (B) copper (C) paper
A
```
which (again for Macaw-large) should produce the output:

```
Output rank 1:
Answer: iron

Explicit outputs ranked:
Answer: iron (0.9998)
Answer: copper (0.0002)
Answer: paper (0.0000)
```
which has both the original generated answer, as well as the 3 explicit answers along with the products of the
associated output token probabilities.

The demo also has a second page accessed by selecting "Generator settings" in the side bar, where the
generator parameters (like beam size and number of results) can be adjusted.

## Memory requirements

Running the main Macaw (11B) model requires around 100GB of CPU RAM to load properly (to temporarily store
two copies of the 44GB worth of weights), and 48GB of GPU RAM to run on
a GPU. It's possible to squeeze onto a single 48GB GPU, but normally you would need to split across multiple GPUs
by specifying a list of more than one device in the `CUDA_DEVICES` variable.

(It's technically possible to load the 11B model with ~50GB of CPU RAM and 48GB of GPU RAM by carefully loading
an empty model, moving it to the GPU, load the weights, move weights to GPU, delete from CPU)

## REST API

The demo code also includes a REST api available at port 8502 (by default), it takes an input string in same format, along with
optional generator options. A couple of examples using curl (with newlines inserted for readability), using the
(inferior) Macaw-large model:

```
$ curl --data-urlencode $'input=Q: What material is magnetic?\nM: (A) iron (B) copper (C) paper\nA' -G http://localhost:8502/api
{"generator_options":{"do_sample":false,"length_penalty":1.0,"max_length":128,"min_length":1,"num_beams":1,"num_return_sequences":1,
      "repetition_penalty":1.0,"temperature":1.0,"top_k":50,"top_p":1.0},
  "input_raw":"$answer$ ; $question$ = What material is magnetic? ; $mcoptions$ = (A) iron (B) copper (C) paper",
  "input_slots":{"MCOptions":"(A) iron (B) copper (C) paper","Question":"What material is magnetic?"},
  "output_raw_list":["$answer$ = iron"],
  "output_slots_list":[{"answer":"iron"}],"requested_angle":"QM->A"}
```

Include "X" options for explicit output scoring:
```
$ curl --data-urlencode $'input=Q: What material is magnetic?\nM: (A) iron (B) copper (C) paper\nX: (A) iron (B) copper (C) paper\nA' -G http://localhost:8502/api
{"explicit_output_angle":"Answer",
 "explicit_outputs":[
    {"input_raw":"$answer$ ; $question$ = What material is magnetic? ; $mcoptions$ = (A) iron (B) copper (C) paper",
     "loss":3.015684887941461e-05,"output_prob":0.9997889405240201,
     "output_raw":"$answer$ = iron","output_text":"iron",
     "output_token_probs":[1.0,1.0,1.0,1.0,1.0,0.99979168176651,0.9999972581863403],
     "output_tokens":["\u2581$","ans","wer","$","\u2581=","\u2581iron","</s>"],"score":0.9999698436058337},
    {"input_raw":"$answer$ ; $question$ = What material is magnetic? ; $mcoptions$ = (A) iron (B) copper (C) paper",
     "loss":1.2147718667984009,"output_prob":0.0002027770735510033,
     "output_raw":"$answer$ = copper","output_text":"copper",
     "output_token_probs":[1.0,1.0,1.0,1.0,1.0,0.00020277731528040022,0.9999988079071045],
     "output_tokens":["\u2581$","ans","wer","$","\u2581=","\u2581copper","</s>"],"score":0.29677771142124026},
   {"input_raw":"$answer$ ; $question$ = What material is magnetic? ; $mcoptions$ = (A) iron (B) copper (C) paper",
    "loss":1.836950659751892,"output_prob":2.6034949917854637e-06,
    "output_raw":"$answer$ = paper","output_text":"paper",
    "output_token_probs":[1.0,1.0,1.0,1.0,1.0,2.603500888653798e-06,0.9999977350234985],
    "output_tokens":["\u2581$","ans","wer","$","\u2581=","\u2581paper","</s>"],"score":0.15930245360755435}],
  "generator_options":{"do_sample":false,"length_penalty":1.0,"max_length":128,"min_length":1,"num_beams":1,"num_return_sequences":1,
      "repetition_penalty":1.0,"temperature":1.0,"top_k":50,"top_p":1.0},
   "input_raw":"$answer$ ; $question$ = What material is magnetic? ; $mcoptions$ = (A) iron (B) copper (C) paper",
   "input_slots":{"MCOptions":"(A) iron (B) copper (C) paper","Question":"What material is magnetic?"},
   "output_raw_list":["$answer$ = iron"],
   "output_slots_list":[{"answer":"iron"}],"requested_angle":"QM->A"}
```

Change some of the generator options (now `output_slots_list` has two elements, the second one isn't very good though):
```
$curl --data-urlencode $'input=Q: What material is magnetic?\nM: (A) iron (B) copper (C) paper\nA' --data 'num_beams=4' --data 'num_return_sequences=2' -G http://localhost:8502/api
{"generator_options":{"do_sample":false,"length_penalty":1.0,"max_length":128,"min_length":1,"num_beams":4,"num_return_sequences":2,
    "repetition_penalty":1.0,"temperature":1.0,"top_k":50,"top_p":1.0},
  "input_raw":"$answer$ ; $question$ = What material is magnetic? ; $mcoptions$ = (A) iron (B) copper (C) paper",
  "input_slots":{"MCOptions":"(A) iron (B) copper (C) paper","Question":"What material is magnetic?"},
  "output_raw_list":["$answer$ = iron","$answer$ = iron (B) copper"],
  "output_slots_list":[{"answer":"iron"},{"answer":"iron (B) copper"}],"requested_angle":"QM->A"}
```

branch_name: refs/heads/main

extension: md

license_type: permissive


text:
----
Title: Getting started
Order: 100
TileLink: true
TileLinkOrder: 10
----
### SPMeta2 essentials

SPMeta2 is a hassle-free fluent API for code-based SharePoint artefact provisioning.
It offers a consistent provisioning API via SSOM/CSOM for SharePoin 2010, 2013 and O365.

The library provides an abstraction level over SharePoint API, so it is highly desired to understand a few concepts on which SPMeta2 library is built - definitions, models and provision services.

SPMeta2 introduces a domain of c# POCO objects, then it maps every single POCO object on SharePoint artifacts.

 "SPMeta2" -> SharePoint;    } '>

Have a look on the same idea expressed in a c# code.
Don not focus on understanding every single bit of the code, it will be much clear to you as we refine ideas later in the article.

### Zoom in, 1/3 scale - definitions

Let's zoom in and have a closer look.
SPMeta2 provides c# POCP objects, we call them **definitions** for every SharePoint artifact.
So you describe what you want to provision in definitions, and then SPMeta2 takes care about everything else.

 "SPMeta2";
   "Field definition" -> "SPMeta2";
   "Content type definition" -> "SPMeta2";
   "List definition" -> "SPMeta2";
   "List view definition" -> "SPMeta2";
   "Web part page definition" -> "SPMeta2";
   "Web part definition" -> "SPMeta2";
   "... other definitions ..." -> "SPMeta2";  "SPMeta2" -> SharePoint;  }' >

And the same idea in the code. Now field definition and content type definition make sense, don't they?
Don't focus on understanding every single bit of the code, it will be much clear to you as we refine ideas later in the article.

### Zoom in, 2/3 scale - models

All SharePoint artifacts have relationships between each other.
For intance, here a few to mentions:

* fields belong to site, web or list
* content types belong to site or web
* lists belong to web
* list views belong to list
* etc..

Having definitions of not enought, we need to describe relationshop between them, so that SPMeta2 would know how to provision them correctly.
Here is a refined view on how SPMeta2 works for site and web level provision:

 SharePoint;
   "Site model" -> "SPMeta2";
   "Web model" -> "SPMeta2";
   "Field definition"  -> "Site model";
   "Content type definition" -> "Site model";
   "User Custom Action" -> "Site model";
   "List definition" -> "Web model";
   "List view definition" -> "Web model";
   "Web partpage" -> "Web model"; }' >

And the same idea in the code. NewSiteModel(), AddField() and AddContentType() help to build a relationships between artifacts
Don't focus on understanding every single bit of the code, it will be much clear to you as we refine ideas later in the article.

### Zoom in, 3/3 scale - provision services

We can have SharePoint 2010, SharePoint 2013 or SharePoint Online along with either SSOM or CSOM API.
SPMeta2 supports both SSOM and CSOM API abstracting them via 'provisioning services' implementations.

Here is an idea for SSOM or CSOM based provision:

 "Choose your provision strategy";
   "Web model" -> "Choose your provision strategy";
   "Field definition" -> "Site model";
   "Content type definition" -> "Site model";
   "User Custom Action" -> "Site model";
   "List definition" -> "Web model";
   "List view definition" -> "Web model";
   "Web partpage" -> "Web model";
    "Choose your provision strategy" -> "CSOM provision service";
    "Choose your provision strategy" -> "SSOM provision service";
    "CSOM provision service" -> "SharePoint 2013";
    "CSOM provision service" -> "SharePoint Online";
    "SSOM provision service" -> "SharePoint 2010";
    "SSOM provision service" -> "SharePoint 2013; }' >

Finally, the code should be absolutely clear to you. We create definitions, we build a logical model and then we use a provision service to push the model to the SharePoint site. Easy enough?

license: MS-PL

branch_name: refs/heads/master

gha_language: C#


text:
# sfdx-einstein-nba

RC_NBA_Recommendations lightning component allows you to active/deactivate recommendations that displayed in a Next Best Action strategy.

See the 'NBA Samples' app. This component works also in Communities.

## Development

To work on this project in a scratch org:

1. [Set up CumulusCI](https://cumulusci.readthedocs.io/en/latest/tutorial.html)
2. Run `cci flow run dev_org --org dev` to deploy this project.
3. Run `cci org browser dev` to open the org in your browser.

license: MIT

repository: congmingwudi/sfdx-einstein-nba

repo_name: congmingwudi/sfdx-einstein-nba


text:
# Shoe Stores
### Created By: Ben Ronda
***
## Project Description:
This site was designed to demonstrate proficiency in many-to-many database relationships, where *many* stores have *many* brands, and *many* brands belong to *many* stores.

It was built using the following:
* PHP
* Twig & Silex
* MySQL
* HTML and CSS
* Tests ran using PHPUnit

***
## MySQL Commands:
1. Use `/Applications/MAMP/Library/bin/mysql --host=localhost -uroot -proot` to access the MySQL Shell.
2. `CREATE DATABASE shoes;`
3. `USE shoes;`
4. `CREATE TABLE stores (id serial PRIMARY KEY, name VARCHAR (100));`
5. `CREATE TABLE brands (id serial PRIMARY KEY, name VARCHAR (100));`
6. `CREATE TABLE brands_stores (id serial PRIMARY KEY, store_id INT, brand_id INT);`

***
## Installation:
To install Shoe Stores clone the following repository by:
1. Open up your terminal and type the following line:
`git clone https://github.com/ben-ronda/shoe_stores.git`
2. Use this line to change directories to the project folder: `cd shoe_stores`
3. In order to load the pages correctly make sure you run: `composer install`
3. Use this line to change into the web directory: `cd web`
4. Once you are in your web directory, run this line to start a local webserver `php -S localhost:8000`
5. Finally open your web browser and type `localhost:8000` into the URL bar of your browser to load the page.
***
## License
Copyright (c) 2016 Ben Ronda

This software is licensed under the MIT license.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


text:
# Serilog.Sinks.Fluentd.Http.Json [![verify][]](https://github.com/Georgy-Losenkov/serilog-sinks-fluentd-http-json) [![Nuget][]](https://www.nuget.org/packages/Serilog.Sinks.Fluentd.Http.Json)

[verify]: https://github.com/Georgy-Losenkov/serilog-sinks-fluentd-http-json/workflows/verify/badge.svg

[nuget]: https://img.shields.io/nuget/v/Serilog.Sinks.Fluentd.Http.Json

## Table of contents

* [What is this sink](#what-is-this-sink)
* [Features](#features)
* [Quick start](#quick-start)
* [Parameters](#parameters)
  * [bodySizeLimit](#bodysizelimit)
  * [flushPeriod](#flushperiod)
  * [httpTimeout](#httptimeout)
  * [maxQueueSize](#maxqueuesize)
  * [jsonFormatter](#jsonformatter)
  * [url](#url)

## What is this sink

The Serilog Fluentd Http Json sink project is a sink (basically a writer) for the Serilog logging framework. Structured log events are written to sinks and each sink is responsible for writing it to its own backend, database, store etc. This sink delivers the data to the http endpoint of the Fluentd daemon in json format. By default it uses Serilog.Formatting.ElasticSearch json formatter, making it suitable for forwarding to ElasticSearch.

## Features

* Simple configuration to get log events published to Fluentd daemon. Only endpoint url is needed.
* Log events are accumulated in the memory and periodically are sent in batches  LoPeriodic batching

## Quick start

Install the Serilog.Sinks.Fluentd.Http.Json package from NuGet:

```powershell
Install-Package Serilog.Sinks.Fluentd.Http.Json
```

To configure the sink in C# code, call WriteTo.Fluentd() during logger configuration:

```csharp
var log = new LoggerConfiguration()
    .WriteTo.Fluentd()
    .CreateLogger();
```

To configure the sink through configuration file using [Serilog.Settings.Configuration][], just add { "Name": "Fluentd" } into "WriteTo" section:

[Serilog.Settings.Configuration]: https://github.com/serilog/serilog-settings-configuration#serilogsettingsconfiguration--

```json
"WriteTo": [
  ...
  { "Name": "Fluentd" }
]
```

## Parameters
```csharp
public static LoggerConfiguration Fluentd(
    this LoggerSinkConfiguration sinkConfiguration,
    Int32 bodySizeLimit = 32 * 1024 * 1024,
    TimeSpan? flushPeriod = null,
    TimeSpan? httpTimeout = null,
    Int32? maxQueueSize = 10000,
    ITextFormatter jsonFormatter = null,
    String url = "http://localhost:8888/logging.log")
```

### bodySizeLimit
The size limit of the json sent to fluentd. Default value is 32Mb (33554432 bytes). This corresponds to default value of parameter [body_size_limit][] of fluentd http input plugin.

[body_size_limit]: https://docs.fluentd.org/input/http#body_size_limit

### flushPeriod
Interval between communication sessions with fluentd. Default value is null value. That corresponds to 4 sec.
When sink is being disposed this value is used as timeout for sending remaining log events to fluentd.

### httpTimeout
Time to wait for submitting to complete. Default value is null value. That corresponds to 3 sec.

### maxQueueSize
Maximum size of the queue accumulating log events. Default value is 10000. Null value means no limits.

### jsonFormatter
The JSON formatter. By default [Serilog.Formatting.ElasticSearch][] formatter is used.

[Serilog.Formatting.ElasticSearch]: https://github.com/serilog/serilog-sinks-elasticsearch#elasticsearch-formatters

### url
The URL of the fluentd http input endpoint appended by the tag name.
Because of limitaions of the fluentd http input plugin tag must contain period, e.g logging.log.


text:
# Googly Eyes Demo using Google Cloud Vision API

This is a very basic [Cloud Vision API](https://cloud.google.com/vision/) example. Originally prepared for my talk at Google DevFest Karachi 2016.

## Prerequisites
1. [Sign up for Google Cloud Platform Account](http://cloud.google.com)
2. [Enable Cloud Vision API](https://cloud.google.com/vision/docs/getting-started)
3. [Create a Browser Key](https://console.cloud.google.com/apis/credentials) for making calls to Cloud Vision API

## Setup
1. Rename the file *key.js.example* to *key.js*.
2. Update the line `window.apiKey = 'YOUR API KEY HERE';` in *key.js* with your API key.
3. Start a web server in this folder and navigate to index.html.

From here, the demo allows you to upload images, process them using the [Cloud Vision API annotation types](https://cloud.google.com/vision/reference/rest/v1/images/annotate#Type) and add googly eyes to them.


text:
# SRE-Onboarding
Broad collection of learning plan to getting up to speed as a Site Reliability Engineer.  This is highly opinionated due to my sample size of me.  I plan to cover the things I wish I knew and the things I want my new hires to know.

## External Sites
* Get a [Linux Academy](https://linuxAcademy.com) subscription.  Besides it's focus on learning linux it provides several guided training plans for mastering many of the technologies and concepts you'll need.  Curated and consistent quality videos that walk you through far enough to then learn on your own.

* Peruse and then re-read the [Google SRE books](https://landing.google.com/sre/books/).  There are some concepts that only really become crystalline once you've had a few near misses.

* [Brendan Gregg's](http://www.brendangregg.com/overview.html) is a dense bit of knowledge to read but you will live or die by mastering what's in it.  Yes his focus is on linux debugging but his approach highlights a methodical and evidence based approach that you will do well to emulate

## Linux

### OS
1.  You should be proffecient enough to pass the [LPIC-1 objectives](https://www.lpi.org/our-certifications/lpic-1-overview) at a minimum.  Preferrably the [LPIC-2 objectives](https://www.lpi.org/our-certifications/lpic-2-overview).

### Debugging
I can write an explanation for all of these or you can just go read the excellent overview by [Julie Evans's Debugging Zine](http://jvns.ca/debugging-zine.pdf)
1.  strace - [Tutorial](https://www.howtoforge.com/linux-strace-command/)
1.  perf - [Brendan Gregg's](http://www.brendangregg.com/perf.html) amazing set of examples.
1.  netcat - [Tutorial](https://www.binarytides.com/netcat-tutorial-for-beginners/)
1.  ngrep
1.  tcpdump
1.  lsof
1.  Print [this](http://www.brendangregg.com/Perf/linux_observability_tools.png) image out and memorize it.

### Statistics

### Networking

### Automation

### Databases

repository: iamshreeram/SRE-Onboarding

repo_name: iamshreeram/SRE-Onboarding


text:
# Identity Management Using SPIRE for Kubeedge

## Background

Security is a paramount requirement for edge computing architecture as security breaches can make a complete organization to come to a halt (IIot) , data breach can lead to privacy issues and also control of the complete edge computing infrastructure.

To enable better security, following needs to be satisfied for kubeedge framework
* Only verified and authorized edge nodes should be able to join cluster and connect to cloud.
* Applications connecting to edge computing framework should be verified against the assigned identities.
* Applications should be authorized to send and receive data from edge computing framework.
* Certificate rotation should be possible at each service-level.
* Audit at service-level for all communciation between services.

## SPIFFE and SPIRE

The Secure Production Identity Framework For Everyone (SPIFFE) Project defines a framework and set of standards for identifying and securing communications between services.
SPIFFE enables enterprises to transform their security posture from just protecting the edge to consistently securing all inter-service communications deep within their applications.
SPIFFE recommends industry standards like TLS and JWT for forming a identity document for every service. The service-level identity AuthN and AuthZ removes the dependency of complex network-level ACL strategies.

More information about SPIFFE can be found at [*https://github.com/spiffe/spiffe*](https://github.com/spiffe/spiffe).

SPIRE (SPIFFE Runtime Environment) is a reference implementation of SPIFFE specification. SPIRE manages identities for node and workloads.
 It provides API for controlling attestation policies and identity issuance and rotation strategies.

More information about SPIRE can be found at [*https://github.com/spiffe/spire*](https://github.com/spiffe/spire).

## Benefits

* Node attestation: Only verifiable edge nodes can join the edge clusters. Every node is issued an identity on verification.
 In case of failed node attestations, no identity documents can be issued for services running on the node.

* Workload attestation: Only verifiable workload can run on edge nodes. In case of failed workload attestations, there are no identities issues for the workloads.
All communications are blocked from unverified workloads.

* Certificate rotation: Short-lived certificates are generated and rotation policies can be configured for every service communication.
There is no need for custom agents and reliance on specific orchestrators for certificate rotation configuration and management.

* Automated non-root CA certificate heirarchical deployments: Edge spire servers can be configured to not share any root CA chain for downstream nodes and workloads.

## SPIRE-based Identity Management Reference Architecture

Goal of this example is to achieve workload identity management for
cloud and edge processes . A reference architecture is shown below. A
list of things to-do is listed below in version information section.
For further integration , an extension add-on will be developed
integrated with kubeege for the same.

![image](./doc/images/demo-arch.png)

## Functional sequence diagrams

### Cloud component sequence
1\. Start cloud spire server.

2\. Generate token for node with spiffeID association for cloud node. (e.g: spiffe id = spiffe://example.org/upstream-cloud-node).

3\. Start spire agent with generated token. Node attestation is performed.

4\. Create entry with cloud spire server for cloud component (cloudapp in this example). The entry is associated with node spiffeID and selectors used for attestation of cloud component.

5\. Start spiffe-helper with helper.conf configuration. Spiffe-helper fetches certificates and starts ghostunnel. Spiffe-helper also updates the certificates and signals (SIGUSR1) to ghostunnel to reload certificates. Workload attestation is performed.

6\. Start cloud component. Cloud component uses localhost ip and port to transfer data. Ghostunnel fetches data from this port and securely sends the data to peer ghostunnel.

7\. SpiffeIDs are validated by ghostunnel to control authorization by using tls peer verification method. Authorized spiffeID are configured as arguments to ghostunnel in helper.conf.

![image](./doc/images/cloudsequence.png)

### Edge component sequence

1\. Generate token for edge with spiffeID association for edge node. (e.g:spiffe id = spiffe://example.org/upstream-edge-node).

2\. Start spire agent (cloud spire agent) on edge node with configuration to connect to cloud spire server and associated token. Node attestation is performed.

3\. Create entry with cloud spire server for edge component. The entry is associated with node spiffeID and selectors used for attestation of edge component.

4\. Start edge spire server. Edge spire server starts with upstreamCA configured to cloud spire server.

5\. Edge spire server fetches certificate from cloud spire agent (running on edge node) which invokes workload attestation and receives workload certificates in response.

6\. Edge spire server connects to cloud spire server using workload certificates to fetch intermediate CA using spire node api.

7\. Generate token for edge with spiffeID association for edge node. (e.g:spiffe id = spiffe://example.org/downstream-edge-node).

8\. Start edge spire agent on edge node with configuration to connect to edge spire server and associated token. Node attestation is performed.

9\. Start spiffe-helper. Spiffe-helper fetches workload certificate for edge component (edge-hub) from cloud spire agent (running on edge node) and starts ghostunnel. Spiffe-helper also updates the certificates and signals (SIGUSR1) to ghostunnel to reload certificates. Workload attestation is performed.

10\. Start edge-hub (execute edge_core binary). Edge-hub uses localhost ip and port to transfer data. Ghostunnel fetches data from this port and securely sends the data to peer ghostunnel.

11\. SpiffeIDs are validated by ghostunnel to control authorization by using tls peer verification method. Authorized spiffeID are configured as arguments to ghostunnel in helper.conf.

![image](./doc/images/edgehubsequence.png)

### Edge application sequence

1\. Create entry with edge spire server for edge applications (eventbus and lightmapper in this example). The entry is associated with edge node spiffeID (token parameter used to register edge spire agent) and selectors used for attestation of edge application.

2\. Start spiffe-helper with helper.conf configuration (refer event-bus-helper.conf/user-app-helper.conf files). Spiffe-helper fetches certificates and starts ghostunnel. Spiffe-helper also updates the certificates and signals (SIGUSR1) to ghostunnel to reload certificates. Workload attestation is performed.

3\. Start edge applications. Edge application uses localhost ip and port to transfer data. Ghostunnel fetches data from this port and securely sends the data to peer ghostunnel.

4\. SpiffeIDs are validated by ghostunnel to control authorization by using tls peer verification method. Authorized spiffeID are configured as arguments to ghostunnel in helper.conf.

![image](./doc/images/edgeappsequence.png)

## Source Folders
* release : Directory has configurations and scripts to be used for deployment of identity management infrastructure for Kubeedge.

* app-agent-conf  : Configurations for spire agent interfacing with edge (event-bus) and user applications . Spire agent communicates to edge spire server.

* conf : Configurations for spire server (cloud and edge) and spire agent (agent communicating to cloud spire server).

* certs : Spiffe helper communicates with spire agents and download certificates in this folder . Ghostunnel is loaded with these certificates . For edge to user interface application , spiffe helper configuration and ghostunnel certs are present in event-bus and user-app folders respectively.

* app-binaries : All binaries for edge , cloud and test applications.

* log : Logs for different processes run as part of the example.

## Scripts
* deploy-cloud.sh : Deploys cloud part of identity management infrastructure. Used to

                    1) Start cloud spire server.
                    2) Registers cloud node and cloud agent.
                    3) Registers and starts cloud hub (test cloud hub command simulator).

* deploy-edge.sh  : Deploys edge part of identity management infrastructure.

                    1) Registers edge node and edge agent communication with cloud spire server.
                    2) Registers and starts edge spire server.
                    3) Registers and starts edge agent for edge to user app communication interface.

* start-spiffe-helper.sh  : Based on the configuration (helper.conf) , starts spiffe-helper communication for certificate download and rotation. Ghostunnel is run using spiffe-helper.
* commands.sh  : Abstracts spire cli commands.

## Configurations Used
### Cloud node configuration

*Upstream CA – Cloud spire server configuration:*
<SPIRE\_PATH>/conf/server/server.conf

*Upstream CA Agent – Cloud node agent configuration :*
<SPIRE\_PATH>/conf/agent/agent.conf

Following fields might require modification based on the deployment environment

    bind_address = "<IP_ADDRESS>" <Cloud node vm IP>
    server_address = "<IP_ADDRESS>" <Cloud Spire Server IP>
    server_port = "8081" <Cloud Spire Server port>
    socket_path =/tmp/agent.sock"

### Edge node configuration

*Edge agent configuration to **connect to cloud spire server** :*
<SPIRE\_PATH>/conf/agent/agent.conf

Following fields might require modification based on the deployment environment

    bind_address = "<IP_ADDRESS>"
    bind_port = "8088"
    data_dir = "./.data"
    server_address = "<IP_ADDRESS>" // <Cloud spire server IP>
    server_port = "8081" // <Cloud spire server port>
    socket_path ="/tmp/upstream_agent.sock"

*Edge spire server configuration :*
<SPIRE\_PATH>/conf/server/server.conf

    UpstreamCA "spire" {
    plugin_data {
      server_address = "<IP_ADDRESS>" // <Cloud spire server IP>
      server_port = 8081 // <Cloud spire server port>
      workload_api_socket = "/tmp/upstream\_agent.sock"
    }

*Edge application agent configuration to connect to edge spire server :
<SPIRE\_PATH>/conf/app-agent-conf/agent/agent.conf*

    bind_address = "<IP_ADDRESS>"
    bind_port = "9088"
    data_dir = "./.app-data"
    server_address = "<IP_ADDRESS>" // <Edge spire server IP>
    server_port = "8081" // <Edge spire server port>
    socket_path ="/tmp/app-agent.sock"

### Spiffe helper configuration (IMPORTANT)

Spiffe helper is used to execute ghostunnel for creating communciation
channel over TLS for cloud , edge and user workloads . Please refer to
the specific configurations in the folders section.

All the helper configurations need to be updated with IP addresses and
ports as per the deployment in

1\. <SPIRE\_PATH>/helper.conf

2\. <SPIRE\_PATH>/event-bus/event-bus-helper.conf

3\. <SPIRE\_PATH>/user-app/user-app-helper.conf

In the list of To-Dos , there is an item to generate these
configurations automatically based on environment configuration to avoid
errors.

Following is a sample configuration for kubeedge event-bus interface which invokes ghostunnel in server mode:

    agentAddress = "/tmp/app-agent.sock"
    cmdArgs = "**server** --listen <IP_ADDRESS>:18884 --target <IP_ADDRESS>:2884 --cacert /opt/spire/event-bus/certs/svid\_bundle.pem --keystore /opt/spire/event-bus/certs/new\_bundle.p12 **--allow-uri-san spiffe://example.org/downstream-app-event-bus**"
    cmd = "/opt/spire/ghostunnel"
    certDir = "/opt/spire/event-bus/certs"
    renewSignal = "SIGUSR1"
    svidFileName = "svid.pem"
    svidKeyFileName = "svid\_key.pem"
    svidBundleFileName = "svid\_bundle.pem"

Following is a sample configuration for user-app interface which invokes ghostunnel in client mode:

    agentAddress = "/tmp/app-agent.sock"
    cmdArgs = "client --listen <IP_ADDRESS>:1884 --target <IP_ADDRESS>:18884 --cacert /opt/spire/user-app/certs/svid_bundle.pem --keystore /opt/spire/user-app/certs/new_bundle.p12 --verify-spiffe-id spiffe://example.org/downstream-app-event-bus"
    cmd = "/opt/spire/ghostunnel"
    certDir = "/opt/spire/user-app/certs"
    renewSignal = "SIGUSR1"
    svidFileName = "svid.pem"
    svidKeyFileName = "svid_key.pem"
    svidBundleFileName = "svid_bundle.pem"

# Deployment of example setup
## Prerequisites
* Bash version 4.4.19
* Go compiler version 1.11.4
* Go dep and glide package managers
* sshpass
* Kubernetes installation based on the kubeedge installation guide [*https://github.com/kubeedge/kubeedge#install-kubernetes*](https://github.com/kubeedge/kubeedge#install-kubernetes)

## How to configure
### Script Configurations

Environment variable configurations : <SPIRE\_PATH>/edge.env

    export CLOUD_VM_USER=vm1
    export CLOUD_VM_PASS=<cloud vm password used for ssh>
    export CLOUD_VM_IP=<IP_ADDRESS>
    export SPIRE_PATH=/opt/spire
    export NODE_ID=fakenodeid
    export PROJECT_ID=dummyprojectid
    export MQTT_EXT_PORT=2883
    export MQTT_INT_PORT=2884
    export EDGE_HUB_IP=<IP_ADDRESS>
    export EDGE_HUB_PORT=20000
    export EDGE_VM_IP=<IP_ADDRESS>

Environment variable configurations : <SPIRE\_PATH>/cloud.env

    export CLOUD_HUB_IP=<IP_ADDRESS>
    export CLOUD_HUB_PORT=20000
    export KUBECONFIG="/home/vm1/.kube/config"
    export CLOUD_VM_IP=<IP_ADDRESS>

## How to setup and use

1\. Copy the release files to cloud node and edge node at path /opt/spire/.

2\. Update the IP , port and spire path in the above listed configurations.

3\. Create an edge node using kubectl using the steps provided at [*https://github.com/kubeedge/kubeedge#run-as-a-binary-1*](https://github.com/kubeedge/kubeedge#run-as-a-binary-1).

4\. In cloud node , execute deploy-cloud.sh.

5\. In edge node , execute deploy-edge.sh. Edged reports ready status to edgecontroller. To verify node status, in cloud vm, execute

    `kubectl get nodes`

## How to test device application

1\. Following hack needs to be used as v0.2 does not support device deployment using kubectl.
Kill edgecontroller process in cloudvm. Execute cloud-app application located at $SPIRE_PATH/app-binaries/cloud/ folder.

2\. Register an example device with cloud using following command. Please
note, in the current version, cloud test application opens 30000 port
for metadata creation (create pod or device) and 20000 port for
communication with kubeedge edgehub. In cloud node, execute

    `curl -XGET http://<IP_ADDRESS>:30000/device -H 'content-type:application/json'<EMAIL_ADDRESS>
3\. Run the light\_mapper application from app\_binaries in edge node. Light mapper
application is a binary built from
[*https://github.com/kubeedge/examples/tree/master/led-raspberrypi*](https://github.com/kubeedge/examples/tree/master/led-raspberrypi).
Usage of the application can be referred in the same page.

For spire server cli usage , please refer to
[*https://github.com/spiffe/spire*](https://github.com/spiffe/spire).

## Present support

1\. Initial infrastructure deployment.

2\. 1-1 cloud and edge node. Multiple edge nodes are not supported in
scripts.

3\. Communication from user application and edge application using
certificates issued by cloud spire server and edge spire server.

4\. Supports kubeedge v0.2.

## ToDo

1\. Optimization for redundancies in configuration and scripts.

2\. Test and support upstream\_bundle=false to prune rootCA at edge spire server.

3\. Trusted cloud spire server for communication between edge spire agents, edge spire server, cloud spire agent and cloud spire server.

4\. Secure communication between event bus and internal mqtt server.

repository: arcanique/examples

repo_name: arcanique/examples


text:
# crnk.io - Crank up the development of RESTful applications!

[![Build Status](https://travis-ci.org/crnk-project/crnk-framework.svg?branch=master)](https://travis-ci.org/crnk-project/crnk-framework)
[![Gitter](https://img.shields.io/gitter/room/crkn-io/lobby.svg)](https://gitter.im/crnk-io/Lobby)
[![License](https://img.shields.io/badge/License-Apache%202.0-yellowgreen.svg)](https://github.com/crnk-project/crnk-framework/blob/master/LICENSE.txt)
[![Coverage Status](https://coveralls.io/repos/github/crnk-project/crnk-framework/badge.svg?branch=master)](https://coveralls.io/github/crnk-project/crnk-framework?branch=master)

## What is Crnk?

Crnk is an implementation of the [JSON API](https://http://jsonapi.org/) specification and recommendations in Java to
facilitate building RESTful applications. It provides many conventions and building blocks that application can benefit from.
This includes features such as  sorting, filtering, pagination, requesting complex object graphs, sparse
field sets, attaching links to data or atomically execute multiple operations. Further integration
with frameworks and libraries such as Spring, CDI, JPA, Bean Validation, Dropwizard, Servlet API, Zipkin and
and more ensure that JSON API plays well together with the Java ecosystem. Have a look at
[www.crnk.io](http://www.crnk.io) and the [documentation](http://www.crnk.io/releases/stable/documentation/) for more detailed
information.

## Repository

Crnk Maven artifacts are available from jcenter/bintray: https://bintray.com/crnk-project.

@crnk/angular-ngrx is available from https://www.npmjs.com/package/@crnk/angular-ngrx.

## Requirements

Crnk requires Java 1.8 or later and an SLF4J setup for logging.

## Licensing

Crnk is licensed under the Apache License, Version 2.0.
You can grab a copy of the license at http://www.apache.org/licenses/LICENSE-2.0.

## Building from Source

Crnk make use of Gradle for its build. To build the complete project run

    gradlew clean build

Note as part of the build a local Node installation is downloaded to build the frontend parts (crnk-ui) of the project.

## Links

* [Homepage](http://www.crnk.io)
* [Documentation](http://www.crnk.io/releases/stable/documentation/)
* [Source code](https://github.com/crnk-project/crnk-framework/)
* [Issue tracker](https://github.com/crnk-project/crnk-framework/issues)
* [Forum](https://gitter.im/crnk-io/Lobby)
* [Build](https://travis-ci.org/crnk-project/crnk-framework/)

repository: yashprit/crnk-framework

repo_name: yashprit/crnk-framework


text:
[![NPM version](https://img.shields.io/npm/v/@gmod/bgzf-filehandle.svg?style=flat-square)](https://npmjs.org/package/@gmod/bgzf-filehandle)
[![Build Status](https://img.shields.io/travis/GMOD/bgzf-filehandle/master.svg?style=flat-square)](https://travis-ci.org/GMOD/bgzf-filehandle) [![Greenkeeper badge](https://badges.greenkeeper.io/GMOD/bgzf-filehandle.svg)](https://greenkeeper.io/)
[![codecov](https://codecov.io/gh/GMOD/bgzf-filehandle/branch/master/graph/badge.svg)](https://codecov.io/gh/GMOD/bgzf-filehandle)

Transparently read [indexed block-gzipped (BGZF)](http://www.htslib.org/doc/bgzip.html) files, such as those created by bgzip, using coordinates from the uncompressed file.

Also provides a `unzip` utility function that properly decompresses BGZF chunks in both node and the browser. Uses `pako` when running in the browser, native `zlib` when running in node. The `unzipChunk` function is another

## Install

    $ npm install --save @gmod/bgzf-filehandle

## Usage

```js
const { BgzfFilehandle, unzip, unzipChunk } = require('@gmod/bgzf-filehandle')

const f = new BgzfFilehandle({path: 'path/to/my_file.gz'})
// assumes a .gzi index exists at path/to/my_file.gz.gzi. can also
// pass `gziPath` to set it explicitly. Can also pass filehandles
// for the files: `filehandle` and `gziFilehandle`

// supports a subset of the NodeJS v10 filehandle API. currently
// just read() and stat()
const myBuf = Buffer.alloc(300)
await f.read(myBuf, 0, 300, 23234)
// now use the data in the buffer

const { size } = f.stat() // stat gives the size as if the file were uncompressed

// unzip takes a buffer and returns a promise for a new buffer
const chunkDataBuffer = readDirectlyFromFile(someFile, 123, 456)
const unzippedBuffer = await unzip(chunkDataBuffer)

// unzipChunk takes a buffer and returns a decompressed buffer plus the offsets
// of the block boundaries in the bgzip file in compressed (cpositions) and
// decompressed (dpositions) coordinates
// you can ignore dpositions/cpositions if your code doesn't care about stable feature IDs
const {buffer, dpositions, cpositions} = await unzipChunk(chunkDataBuffer)

// similar to the above unzipChunk but takes extra chunk argument and trims
// off (0,chunk.minv.dataPosition) and (chunk.maxv.dataPosition)
// used especially for generating stable feature IDs across chunk boundaries
// normal unzip or unzipChunk can be used if this is not important
const {buffer, dpositions, cpositions} = await unzipChunkSlice(chunkDataBuffer, chunk)
```

## Academic Use

This package was written with funding from the [NHGRI](http://genome.gov) as part of the [JBrowse](http://jbrowse.org) project. If you use it in an academic project that you publish, please cite the most recent JBrowse paper, which will be linked from [jbrowse.org](http://jbrowse.org).

## License

MIT © [Robert Buels](https://github.com/rbuels)

repository: manzt/bgzf-filehandle

gha_license_id: MIT

repo_name: manzt/bgzf-filehandle


text:
# Learning Spark

Authors: Holden Karau, Andy Konwinski, Patrick Wendell, and Matei Zaharia

## Contents

* [Chapter 1: Introduction to Data Analysis with Spark](chapter-1-intro-to-data-analysis-with-spark.md)
* [Chapter 2: Downloading and Getting Started](chapter-2-downloading-and-getting-started.md)
* [Chapter 3: Programming with RDDs](chapter-3-programming-with-rdds.md) (in progress)

gha_language: Jupyter Notebook


text:
|Branch      | OSX / Linux / Windows     |
|------------|---------------------------|
|master      | [![Build Status](https://travis-ci.org/JCash/jctest.svg?branch=master)](https://travis-ci.org/JCash/jctest?branch=master) |
|dev         | [![Build Status](https://travis-ci.org/JCash/jctest.svg?branch=dev)](https://travis-ci.org/JCash/jctest?branch=dev) |

# Documentation

https://jcash.github.io/jctest/ (or [here](./docs/index.md))

# jctest

A C++ header only test framework

See [documentation](https://jcash.github.io/jctest) for a full list of features

# Disclaimer

This software is supplied "AS IS" without any warranties and support

# License

[The MIT license](http://choosealicense.com/licenses/mit/)

# Contributing

* Make pull requests towards the `dev` branch.
* All tests must pass (on Travis-CI)
* No added warnings to the code (according to Travis-CI)
* New features must have unit tests

I don't want to put performance (compile time/code size/runtime) explicitly in the list, since
one might sometimes need to fix a bug, which might have those effects.
But as a general rule, take care to keep the performance high in this library.

# Code of conduct

[Contributor covenant](./CODE_OF_CONDUCT.md)

repository: mtvee/jctest

repo_name: mtvee/jctest


text:
# es6-functional-programming
Repository to test some basic concepts of functional programming using es6 high order functions.

## How to start
Open `/test/exercisesTest.js` file and start solving all the tests in that file.

## Rules
Make sure not to change any assertion (expect clause), just implement the function marked with `/*TODO*/` comment.

Some exercises needs to be solved using a specific function (i.e compose), and all of them must be coded using a functional approach (don't use for/forEach statemenents).

If you want to view one way of solving this exercise, just take a look at `solved` branch.

Good luck!

license: Unlicense


text:
# vue-like
a vue-like web framework.

# design

## virtual node
```javascript
{
    type: null, // 1 =  host element；2 = 自定义组件
    tag: null, // 标签，对于host element，是 HTML 标签；对于component，是首字母大写自定义标签
    props: null, // 外部输入
    children: null, // 对于host组件 子元素
    slots: null, // 对于自定义组件 的子元素
    ref: null, // host 元素 或者 component 实例 或者 虚拟 组件（if、text、loop等）
    // Fiber
    _return: null,
    child: null,
    sibling: null,
}
```

## how it works

```
We image that every data you can bind to template is from the component's field $state.

Component can receive data from parent by props, and props also can be bound to template.

The data declared as state is reactive by RXJS.

Every update of state will lead to the updates of DOM.

```

### template

We use fast-xml-parser as the parser to parse the HTML user declares to json.
Then we use the json to generate virtual dom by calling component's render method.

### component

A component is a class with it's own state and methods to process state over the lifecycle.


text:
# SettingX.Repositories.Tests
## About
This solution contains unit/integration tests helpful for developing DB layer implementations. Since the DB layer is deliberately separated from the API/Service layer, it might be a bit troublesome to test each and every small change in DB implementation to make the contracts and basic functionality were not broken. This solution is intended to take care of that - it allows for easier fixes and development of existing and new DB implementations. It runs simple CRUD type operations with dummy data, therefore **do not run them on production environment** as data integrity is not guaranteed!

Currently contains following projects:
1. `SettingX.Repositories.Tests.Rest` - for testing REST HTTP implementation.
## Prerequisites
The solution runs on `dotnet core 3.1` and uses `XUnit` as the testing framework.
## Usage
1. Clone the repository
```sh
    git clone https://github.com/SettingX/SettingX.Repositories.Tests.git
   ```
 2. Go to the project directory
 ```sh
    cd SettingX.Repositories.Tests.Rest
   ```
2. Configure environment variables: `SERVICE_URL` indicating the to-be-tested DB implementation endpoint and `DEDICATED_DB` indicating whether or not the instance of the DB implementation was set up specifically for these unit tests and the database itself is, therefore, empty. This will enable additional tests which will be possible only on a clean, unused environment. If not set explicitly, `DEDICATED_DB` defaults to `false`
```sh
    SERVICE_URL=http://localhost:5000
    DEDICATED_DB=true
   ```
3. Run
```sh
    dotnet test
   ```

## License
Distributed under the MIT License. See `LICENSE` for more information.


text:
---
id: 4411
title: Dakota Fanning
date: 2021-04-06T16:49:22+00:00
author: Laima
layout: post
guid: https://ukdataservers.com/dakota-fanning/
permalink: /04/06/dakota-fanning/
---

* some text
{: toc}

## Who is Dakota Fanning

As a child actress, she appeared in films such as Man on Fire, War of the Worlds, and Charlotte’s Web. She had her breakthrough role in I Am Sam at age seven and, in 2002, became the youngest person in history to be nominated for a Screen Actors Guild Award.

## Prior to Popularity

She began her career in a national commercial for Tide when she was five years old.

## Random data

She lent her voice to the 2009 animated film Coraline.

## Family & Everyday Life of Dakota Fanning

Her parents, Heather Joy and Steven Fanning, raised her as a Southern Baptist. Her younger sister, Elle Fanning, also became an actress. In 2013, she began dating model Jamie Strachan. She would later date Henry Frye in 2018.

## People Related With Dakota Fanning

She starred in the 2010 film The Runaways with Kristin Stewart.


text:
Title: Hypothes.is Info
Date: 2013-10-15 13:40
Category: assignment3
Slug: assign3
Tags: i290m-ocpp
Author: Jake Hartnell
Summary: Briefly update us on your project participation. Include links.

In this brief post, link to email archives, issue trackers, wiki pages, or other appropriate infrastructure where you have been participating in these projects.

Main website: http://hypothes.is/
Email archives: http://list.hypothes.is/archive/dev/
Issue tracker: https://github.com/hypothesis/h/issues?state=open
Wiki-pages: https://github.com/hypothesis/h/wiki

gha_language: TeX


text:
# LESS Effort: Standings Scraper

Transposes the standings table available on collegehockeynews.com and prints the data by column for easy pasting into the RPI TV Titling System. Any conference may be selected for display.

## Deployment

### Prerequisites

  - [Python 3](https://www.python.org/)
  - [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/)

repository: jzblee/less-chn

gha_language: Python

repo_name: jzblee/less-chn


text:
# Deploying Stakater Stacks

## Overview
This guide provides guidelines on the deployment of Stakater Stacks. Currently, there are two ways to install Stakater Stacks on kubernetes:

- Jenkins CI/CD pipeline.
- Gitlab CI/CD pipeline.

## Jenkins CI/CD Pipeline

To deploy Stakater Stacks using Jenkins CI/CD, create pipeline for stack with following values:

### Configuration
- Create a new folder `Stacks` that will hold pipeline of each stack by using the `New Item` feature.
- Inside the `Stacks` folder add a `Pipeline` for each stack by using the `New Item` feature.
- In `General` settings UI enter stack repository URL in `GitHub Project`'s `Project url` field.
- Select the checkbox `This project is parameterized` and add the following parameters

  | Name           | Default Value | Type   | Description                                               |
  |----------------|--------|--------|-----------------------------------------------------------|
  | REPO_URL       | URL of pipeline repository | String | link to stack repo                                        |
  | BRANCH         | master | String | name of branch to deploy                                  |
  | CREDENTIALS_ID | Name of the user | String | Id of Jenkins credentials to use to clone above git repo  |
  | PROVIDER       | `azure` or `aws` | Choice | Allow either aws or azure                                 |
  | MAKE_TARGET    | Stack's Makefile targets | Choice | Allow targets available in Stack's Makefile               |
  | NAMESPACE      | Stack's Namespace | String | Specify the name of namespace that should be forwarded to MakeFile |
  | KUBE_CONFIG    | None | Multi line String | Kube Config of Kubernetes stack in which it should deploy the stack |
  | INSTALL_PVC  | None | String | Flag to install PVC or not. Its value can be either `true` or `false`. |

- There are two ways to add the pipeline script:

    1- Add the script given below in Pipeline Script of pipeline configuration.

    2- Enable the `Pipeline script from SCM` feature, configure it and add a `Jenkinsfile` in each stack repository with the following content.

  ```groovy
    #!/usr/bin/env groovy
<EMAIL_ADDRESS>_

    String repoURL = ""
    try {
        repoURL = REPO_URL
    } catch (Throwable ignored) {
        repoURL = ""
    }

    if (repoURL == "") {
        error("REPO_URL must be specified")
    }

    String branch = ""
    try {
        branch = BRANCH
    } catch (Throwable ignored) {
        branch = ""
    }

    if (branch == "") {
        error("BRANCH must be specified")
    }

    String credentialsID = ""
    try {
        credentialsID = CREDENTIALS_ID
    } catch (Throwable ignored) {
        credentialsID = ""
    }

    if (credentialsID == "") {
        error("CREDENTIALS_ID must be specified")
    }

    String provider = ""
    try {
        provider = PROVIDER
    } catch (Throwable ignored) {
        provider = ""
    }

    if (provider == "") {
        error("PROVIDER must be specified")
    }

    String makeTarget = ""
    try {
        makeTarget = MAKE_TARGET
    } catch (Throwable ignored) {
        makeTarget = ""
    }

    if (makeTarget == "") {
        error("MAKE_TARGET must be specified")
    }

    String namespace = ""
    try {
        namespace = NAMESPACE
    } catch (Throwable ignored) {
        namespace = ""
    }

    if (namespace == "") {
        error("NAMESPACE must be specified")
    }

    Boolean installPVC = false
    try {
        installPVC = INSTALL_PVC
    } catch (Throwable ignored) {
        installPVC = false
    }

    String kubeConfig = ""
    try {
        kubeConfig = KUBE_CONFIG
    } catch (Throwable ignored) {
        kubeConfig = ""
    }

    if (kubeConfig == "") {
        error("KUBE_CONFIG must be specified otherwise default cluster is used")
    }

    def utils = new io.fabric8.Utils()
    def flow = new io.stakater.StakaterCommands()

    timeout(time: 20, unit: 'MINUTES') {
        timestamps {
            stakaterNode() {
                container(name: 'tools') {
                    stage('checkout') {
                        git branch: branch , url: repoURL, credentialsId: credentialsID
                    }

                    stage('configure') {
                        writeFile file: '/home/jenkins/.kube/config', text: kubeConfig
                    }

                    stage('apply') {
                        executeMakeTargets {
                            target = makeTarget
                            NAMESPACE = namespace
                            PROVIDER_NAME = provider
                            INSTALL_PVC = installPVC
                        }
                    }
                }
            }
        }
    }
  ```

## Gitlab CI/CD Pipeline

This section provides guidelines on deploying Stakater Stacks on kubernetes cluster using Gitlab CI/CD pipeline.

### Configuration

* Github Personal Access Token will be used to pull stacks repository.

* Create a repository on Gitlab and configure its [pipeline](/content/processes/bootstrapping/gitlab-pipeline-configuration.html).

* Gitlab CI/CD pipeline requires some environment variables, their description is given below:

    | Environment Variables | Type | Description | Source |
    |---|---|---|---|
    | KUBE_CONFIG | String (base64 encoded) | Kubernetes configuration file. | Gitlab CI/CD Enviroment Variable |
    | GITHUB_TOKEN  | String (encoded) | Github Personal Access token. It can be generated by following this [guideline](https://github.com/stakater/til/blob/master/gitlab/gitlab-integration-with-github.md). | Gitlab CI/CD Environment Variable |
    | STACK  | String | Name of the stack that will be deployed. Possible values are `global`, `release`, `logging`, `monitoring`, `tracing` |  Gitlab CI/CD Environment Variable |
    | NAMESPACE  | String | Deployment namespace. | Config file |
    | PROVIDER  | String | Cloud provider's name. Currently, we support AWS and Azure. Its value can be provided as `aws` for AWS and `azure` for Azure. | Config file |
    | INSTALL_PVC  | String | Flag to install PVC or not. Its value can be either `true` or `false`. | Config file |
    | REPO_URL  | String | URL of the stack repository that need to be deployed. From each repository URL only the part after this section `https://` is required. Like from `https://github.com/stakater/til.git` URL we require only the `github.com/stakater/til.git` part | Config file |
    | BRANCH  | String | Branch name that will be deployed. | Config file |
    | TARGET  | String | Makefile target's name that will be executed. Makefile exists in Tracing stack Github repository. | Config file |

* `Gitlab CI/CD environment variables` can be configured from the dashboard `Gitlab Project Setting -> CI/CD -> Variables`.

* `Config variables` can be configured by creating a `global.sh` file in the root folder and export the stack specific variables from there. It will reduce the chances of errors in configuration because now few CI/CD variables needs to be changed for each deployment. Example of config file is given below:
    ```bash
    export BRANCH="master"
    export INSTALL_PVC="true"
    export NAMESPACE="<stack-namespace>"
    export PROVIDER="aws"
    export REPO_URL="github.com/stakater/<stack-name>.git"
    export TARGET="install-dry-run"
    ```

* Once environment variables are configured create a `.gitlab-ci.yml` file in project root directory and paste the configuration given below in it:

    ```yaml
    image:
        name: stakater/gitlab:0.0.3

    before_script:

        - if [ $STACK == "global" ]; then \
        -     echo "Gloabl Stack"; \
        -     source ./global.sh; \

        - elif [ $STACK == "release" ]; then \
        -     echo "Release Stack"; \
        -     source ./release.sh; \

        - elif [ $STACK == "logging" ]; then \
        -     echo "Logging Stack"; \
        -     source ./logging.sh; \

        - elif [ $STACK == "monitoring" ]; then \
        -     echo "Monitoring Stack"; \
        -     source ./monitoring.sh; \

        - elif [ $STACK == "tracing" ]; then \
        -     echo "Tracing Stack"; \
        -     source ./tracing.sh; \

        - else \
        -     echo "Invalid stack name provided"
        -     exit 1
        - fi

        - echo "configuration kubernetes access in pipeline"
        - mkdir ~/.kube/
        - echo $KUBE_CONFIG | base64 -d > config
        - mv config ~/.kube/

        - echo "Extracting repository name from the URL"
        - BASE_NAME=$(basename $REPO_URL)
        - REPO_NAME=${BASE_NAME%.*}

    stages:
        - deploy

    deploy:
        stage: deploy
        script:

        - echo "Cloning repository and redirecting the output to black hole because we don't want GITHUB_TOKEN to be visible on pipeline logs"
        - git clone https://$GITHUB_TOKEN@$REPO_URL > /dev/null

        - echo "Moving inside cloned repository directory"
        - cd $REPO_NAME

        - echo "Checkout to the deployment branch"
        - git checkout $BRANCH

        - echo "Deploying stack on kubernetes cluster"
        - if [ -z "$NAMESPACE" ]; then \
        -       make $TARGET PROVIDER_NAME=$PROVIDER INSTALL_PVC=$INSTALL_PVC; \
        - else \
        -       make $TARGET NAMESPACE=$NAMESPACE PROVIDER_NAME=$PROVIDER INSTALL_PVC=$INSTALL_PVC; \
        - fi
    ```

* `Config variables` file is just a way to easy the deployment process, it is also possible to configure all `configuration variables` through Gitlab CI/CD environment variable. If you configure all the configuration variables through Gitlab CI/CD env variable feature then remove the code snipped(that checks which stack to deploy) from the `before_script` section of `.gitlab-ci.yml` file and also remove the `STACK` CI/CD environment variable.

* Once configuration is done, run the Gitlab pipeline for deployment by pushing a commit or manually trigger the pipeline.

Run the pipelines in the following order:

- [Global](https://github.com/stakater/StakaterKubeHelmGlobal)
- [Release](https://github.com/stakater/StakaterKubeHelmRelease)
- [Logging](https://github.com/stakater/StakaterKubeHelmLogging)
- [Monitoring](https://github.com/stakater/StakaterKubeHelmMonitoring)
- [Tracing](https://github.com/stakater/StakaterKubeHelmTracing)

repository: stakater/Playbook

gha_language: Dockerfile

repo_name: stakater/Playbook


text:
---
layout: default
---

# 1993-08-18 - Re: encrypted anonymous traffic

## Header Data

From: "Perry E. Metzger"<EMAIL_ADDRESS>To: Marc Horowitz<EMAIL_ADDRESS>Message Hash: ecf5fdcf801c522dff7b4ad618240e6a077aeb8d896a88f6f22cb4d072dfad44

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1993-08-18 16:25:42 UTC

Raw Date: Wed, 18 Aug 93 09:25:42 PDT

## Raw message

```
{% raw  %}From: "Perry E. Metzger"<EMAIL_ADDRESS>Date: Wed, 18 Aug 93 09:25:42 PDT
To: Marc Horowitz<EMAIL_ADDRESS>Subject: Re: encrypted anonymous traffic
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Marc Horowitz says:
> I have to disagree with Lance here.
>
> >> No one can send any `bomb' through mere text, and to compare harassing
> >> mail (which is definitely not to be condoned) to it is to expose your
> >> naive and self-serving view of the matter.
>
> I know people who would probably rather receive an explosive in the
> mail than receive email from certain individuals, or about certain
> subjects.

To begin with, the ban on encrypted messages makes no sense because
people who do not have the key to read the messages can obviously not
receive them, and even people who do have the key must make an active
effort to read the messages. I will ignore that for the moment,
however, and address this pervasive notion that words can cause more
harm than letter bombs.

I'm sorry, but its completely irrational to prefer to be killed by an
explosive over getting email from someone you hate. This insane notion
that words are somehow worse than physical blows has to stop.

It leads to insane conclusions, among others, the conclusion that we
must all be restricted in our speech at all times lest we offend
other's feelings. This is the same argument that fundamentalist
christians who would like to ban certain books from our libraries
would use -- that harm can be caused by people accidently reading the
books. This is no speculative notion -- the argument was once actually
used regularly in our country.

The real world contains lots of harmful things.  They are too fragile for this world and
likely should be locked up for their own good until psychiatrists can
manage to heal them, as the preference of death to being offended is
suicidal and the incapacity to deal with the real world will obviously
cripple them. The rest of the world should not be constrained to
handle the needs of these obviously very mentally unbalanced
individuals.

Perry Metzger

{% endraw %}
```

## Thread

+ Return to [August 1993](/archive/1993/08)

+ Return to<EMAIL_ADDRESS>(Allan Bailey)](/authors/allan_at_elvis_tamu_edu_allan_bailey_)"
+ Return to<EMAIL_ADDRESS>(Carl Ellison)](/authors/cme_at_ellisun_sw_stratus_com_carl_ellison_)"
+ Return to "["L. Detweiler" <ld231782@longs.lance.colostate.edu>](/authors/l_detweiler_ld231782_at_longs_lance_colostate_edu_)"
+ Return to "[Marc Horowitz <marc@Athena.MIT.EDU>](/authors/marc_horowitz_marc_at_athena_mit_edu_)"
+ Return to "[Mike Rose <mrose@stsci.edu>](/authors/mike_rose_mrose_at_stsci_edu_)"
+ Return to "["Perry E. Metzger" <pmetzger@lehman.com>](/authors/perry_e_metzger_pmetzger_at_lehman_com_)"
+ Return to "[T. William Wells <bill@twwells.com>](/authors/t_william_wells_bill_at_twwells_com_)"

+ 1993-08-18 (Tue, 17 Aug 93 17:20:28 PDT) - [encrypted anonymous traffic](/archive/1993/08/1f3c9fec5f23379bd07bb4449ed6a4a3fca35f8a5a505e5a0eb30eda17a18043) - _"L. Detweiler"<EMAIL_ADDRESS>  + 1993-08-18 (Tue, 17 Aug 93 19:05:29 PDT) - [Re: encrypted anonymous traffic](/archive/1993/08/c63fc5278634804c4f7d6f8a7ecf4672697af9d99a0321ba21065310b1643508) - _T. William Wells<EMAIL_ADDRESS>  + 1993-08-18 (Tue, 17 Aug 93 19:15:28 PDT) - [Re: encrypted anonymous traffic](/archive/1993/08/6ec5bd582dd64a0a1946abc97e96be8022d53523ed0cc7534fa24b26f1caf9ad) - _Marc Horowitz<EMAIL_ADDRESS>    + 1993-08-18 (Wed, 18 Aug 93 09:25:42 PDT) - Re: encrypted anonymous traffic - _"Perry E. Metzger"<EMAIL_ADDRESS>      + 1993-08-18 (Wed, 18 Aug 93 10:10:41 PDT) - [Re: encrypted anonymous traffic](/archive/1993/08/5e7aa4a98f8b9e1aa874e920b0c541e2057294a40e714c304ae76e2454797601) - _Mike Rose<EMAIL_ADDRESS>        + 1993-08-18 (Wed, 18 Aug 93 13:05:46 PDT) - [Re: encrypted anonymous traffic](/archive/1993/08/6e7f69e2b4481199307802700dbb96e4e5f711757b70d89ebdfd61d67f33cc46) -<EMAIL_ADDRESS>(Carl Ellison)_
    + 1993-08-18 (Wed, 18 Aug 93 09:50:39 PDT) - [Re: encrypted anonymous traffic](/archive/1993/08/4501e576b3d67bd135dcc809de9cf8ff48ca6f7ed9dfccfbdb502bfebe3df2a5) -<EMAIL_ADDRESS>(Allan Bailey)_


text:
---
layout: default
---

# 1997-01-10 - Re: IBM Key Recovery Propaganda on United Airlines

## Header Data

From<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)

To<EMAIL_ADDRESS>Message Hash: da7b622d528727c625fa0044af9bb2e175ddbefdb79107d6cdb3ab6f9581c068

Message ID<EMAIL_ADDRESS>Reply To: \<Pine.SUN.3.94.970109181157.707C-100000@polaris\>

UTC Datetime: 1997-01-10 12:52:13 UTC

Raw Date: Fri, 10 Jan 1997 04:52:13 -0800 (PST)

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)
Date: Fri, 10 Jan 1997 04:52:13 -0800 (PST)
To<EMAIL_ADDRESS>Subject: Re: IBM Key Recovery Propaganda on United Airlines
In-Reply-To: <Pine.SUN.3.94.970109181157.707C-100000@polaris>
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Black Unicorn<EMAIL_ADDRESS>writes:

> On Thu, 9 Jan 1997, Dr.Dimitri Vulis KOTM wrote:
>
> > "William H. Geiger III"<EMAIL_ADDRESS>writes:
> >
> > > Well I guess I will not need to worry about them since I will never run a
> > > reguardless how may laws the jackboots in DC pass.
> >
> > That's from the same guy who endorses censorship on this mailing list...
>
> Wait a second, isn't that what you were just calling for in my case?
>
> To wit: "I suggest zero tolerance of Black Unicorn unless and until
> someone reveals his real name."

Not quite.

          No censorship
          is better than
          consistent censorship
          is which is better than
          arbitrary censorship.

*IF* this list is to be moderated, then Unicorn and Plucky Green have
to banned from it - unless their real names are revealed.

> For the guy who has been screaming his head off about censorship to the
> point of nausea you were sure quick to turn your coat in my case.  That
> more than anything else you have ever said puts the cap on your
> credibility as far as I'm concerned.

OK, let's guess one letter at a time. Does your last name start with 'M'?

---

Dr.Dimitri Vulis KOTM
Brighton Beach Boardwalk BBS, Forest Hills, N.Y.: +1-718-261-2013, 14.4Kbps

{% endraw %}
```

## Thread

+ Return to [January 1997](/archive/1997/01)

+ Return to "[Black Unicorn <unicorn@schloss.li>](/authors/black_unicorn_unicorn_at_schloss_li_)"
+ Return to "[Dale Thorn <dthorn@gte.net>](/authors/dale_thorn_dthorn_at_gte_net_)"
+ Return to<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)](/authors/dlv_at_bwalk_dm_com_dr_dimitri_vulis_kotm_)"
+ Return to "["Eric Verheul" <everheul@mail.rijnhaave.nl>](/authors/eric_verheul_everheul_at_mail_rijnhaave_nl_)"
+ Return to "[Toto <toto@sk.sympatico.ca>](/authors/toto_toto_at_sk_sympatico_ca_)"
+ Return to "["William H. Geiger III" <whgiii@amaranth.com>](/authors/william_h_geiger_iii_whgiii_at_amaranth_com_)"

+ 1997-01-09 (Wed, 8 Jan 1997 23:42:32 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/c674d8600b57cbbed5266fbf55b88d2233f92015d4cc20bc0c5ff135ca68148d) - _"Eric Verheul"<EMAIL_ADDRESS>  + 1997-01-09 (Thu, 9 Jan 1997 02:08:57 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/c540faa0e7fea23fc6a698943415a94d65158fa1b161a0b940ef3ea09a0a256d) - _"William H. Geiger III"<EMAIL_ADDRESS>    + 1997-01-09 (Thu, 9 Jan 1997 10:33:14 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/fa4e2e9db5c54de6790966d486eb3dcd6a1f38924a091389827a79a66c4cf070) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
      + 1997-01-09 (Thu, 9 Jan 1997 15:17:48 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/77b61cebfdeac2e25e53dde54a9cf6fb4d0e225c42ae30b61cd501eaacde0aed) - _Black Unicorn<EMAIL_ADDRESS>        + 1997-01-10 (Thu, 9 Jan 1997 21:48:33 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/20dbac660d43fcd60d0cccb32a2497758e05936bd44ad2bcbe72d679b8b5c429) - _Toto<EMAIL_ADDRESS>        + 1997-01-10 (Fri, 10 Jan 1997 04:52:13 -0800 (PST)) - Re: IBM Key Recovery Propaganda on United Airlines -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
          + 1997-01-10 (Fri, 10 Jan 1997 12:47:04 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/378264d204641a4b63bfac9b68faeb136fbd99f0f5b94ed79ab5a050452fa499) - _Toto<EMAIL_ADDRESS>  + 1997-01-09 (Thu, 9 Jan 1997 07:38:29 -0800 (PST)) - [Re: IBM Key Recovery Propaganda on United Airlines](/archive/1997/01/3c674a1432599fefb01ebb80e13546af73e458eed77fe8ccdaf962e7b79f4f9d) - _Dale Thorn<EMAIL_ADDRESS>


text:
# Blue Cloud Mirror - IBM Cloud Technology Showcase

This project contains a game where players need to show five specific emotions and do five specific poses in two levels. The fastest player wins. The game uses various key cloud technologies to demonstrate the value of a diverse, interconnected system, with both public and private cloud environments.

[Play the Game.](https://blue-cloud-mirror.mybluemix.net/)

Here is an example of a successful game:

![alt text](images/game-results-900.jpeg "game results")

**Showcase of IBM Cloud Technologies**

The game utilizes various key IBM Cloud technologies to demonstrate the value of the IBM Cloud. There are three main parts:

* Core game: Implemented as serverless web application since it is primarily used during conferences only
* Users service: Implemented via IBM Cloud Private to avoid storing personal data in a public cloud
* Scores service: Implemented via Cloud Foundry Enterprise Edition to highlight easy changes via the ‘cf push experience’ (for example theme adoptions for different conferences)

**Note:** In order to allow developers to easily set up the game with an IBM Cloud Lite account, this project contains documentation how to use Cloud Foundry Publication Applications rather than Cloud Foundry Enterprise Environment and how to use Minikube rather than IBM Cloud Private.

The following diagram shows the key components that are used in this project. More diagrams, including a diagram showing IBM Cloud Private and Cloud Foundry Enterprise Environment, can be found in the [images](https://github.com/IBM/blue-cloud-mirror/tree/master/images) folder.

![alt text](images/diagram-overview-lite-900.jpeg "architecture diagram")

This is a list of used technologies:
* [IBM Cloud](https://cloud.ibm.com/)
* [IBM Cloud Private](https://www.ibm.com/cloud/private)
* [IBM Cloud Functions](https://cloud.ibm.com/openwhisk)
* [IBM Cloud Foundry Enterprise Edition](https://cloud.ibm.com/cfadmin/create)
* [IBM Cloud Foundry Public Applications](https://cloud.ibm.com/catalog?search=runtime)
* [IBM Cloud Object Storage](https://cloud.ibm.com/catalog/services/cloud-object-storage)
* [IBM API Connect](https://cloud.ibm.com/catalog/services/api-connect)
* [IBM Secure Gateway](https://cloud.ibm.com/catalog/services/secure-gateway)
* [IBM Watson Studio](https://dataplatform.cloud.ibm.com/)
* [IBM App ID](https://cloud.ibm.com/catalog/services/app-id)
* [IBM Cloudant](https://cloud.ibm.com/catalog/services/cloudant)
* [Apache CouchDB](https://couchdb.apache.org/)
* [TensorFlow.js](https://js.tensorflow.org)
* [Vue.js](https://vuejs.org/)
* [Minikube](https://github.com/kubernetes/minikube)

## Authors

* [Niklas Heidloff](https://twitter.com/nheidloff)
* [Thomas Südbröcker](https://twitter.com/tsuedbroecker)
* [Harald Uebele](https://twitter.com/harald_u)

## Documentation and Articles

* Code Pattern: [Mirror game app showcases 15 cloud technologies and components](https://developer.ibm.com/patterns/cloud-showcase-blue-mirror/)
* [Introducing Blue Cloud Mirror, a Fun IBM Cloud Showcase](http://heidloff.net/article/introducing-blue-cloud-mirror)
* [Blue Cloud Mirror Architecture Diagrams](http://heidloff.net/article/blue-cloud-mirror-architecture-diagrams)
* [Three Minutes Video](https://www.youtube.com/watch?v=jkDWoHjMAig)
* [Screenshots and Diagrams](https://github.com/IBM/blue-cloud-mirror/tree/master/images)
* [Three awesome TensorFlow.js Models for Visual Recognition](http://heidloff.net/article/tensorflowjs-visual-recognition)
* [Serverless Vue.js Applications on the IBM Cloud](http://heidloff.net/article/serverless-vuejs-ibm-cloud)
* [Blue Cloud Mirror — A fun IBM Cloud showcase](https://haralduebele.blog/2019/01/31/blue-cloud-mirror-a-fun-ibm-cloud-showcase/)
* [Blue Cloud Mirror – Of Kubes and Couches](https://haralduebele.blog/2019/02/01/blue-cloud-mirror-of-kubes-and-couches/ )
* [Awesome Developer Experience with Vue.js](http://heidloff.net/article/awesome-developer-experience-vuejs)
* [Videos: How to setup the Highscores part in IBM Cloud?](https://www.youtube.com/playlist?list=PLUte4WEyMEjWjJSL_MG692rL_hOna7MbD)
* [Highscores overview blog post](https://suedbroecker.net/2019/02/01/bluecloudmirror-game-highscores/)

## Setup

In the easiest case the game can be set up on a local machine in less than five minutes. In this case only the game web application is run without being able to register users and having highscores. To set up the game locally, invoke the following commands:

```
$ git clone https://github.com/IBM/blue-cloud-mirror.git
$ cd blue-cloud-mirror/game
$ yarn install
$ yarn run serve
```

or

[![Deploy to IBM Cloud](https://cloud.ibm.com/devops/setup/deploy/button.png)](https://cloud.ibm.com/devops/setup/deploy?repository=https://github.com/thomassuedbroecker/blue-cloud-mirror.git&branch=master)

The full setup includes IBM Cloud Functions, Cloud Foundry and an on-premises environment. Please refer to the instructions of the three main components:

* [Game](game/README.md): Web application hosted on the IBM Cloud
* [Scores service](scores/README.md): API and web application hosted on the IBM Cloud
* [Users service](users/README.md): API hosted on-premises

## Prerequisites

For all three components, you need the following prerequisites.

Create an IBM Cloud lite account (free, no credit card required):

* [IBM Cloud account](https://ibm.biz/nheidloff)

Make sure you have the following tools installed:

* [git](https://git-scm.com/downloads)
* [ibmcloud](https://console.bluemix.net/docs/cli/index.html)
* [node](https://nodejs.org/en/download/)
* [curl](https://curl.haxx.se/download.html)
* [vue](https://cli.vuejs.org/)
* [yarn](https://yarnpkg.com/lang/en/docs/install/#mac-stable)
* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)

## Local Environment Setup

Invoke the following commands:

```
$ git clone git@github.ibm.com:niklas-heidloff/visual-recognition-demo.git
$ cd visual-recognition-demo/
$ ibmcloud login
$ ibmcloud target --cf
$ ibmcloud iam api-key-create emotion-recognition-application \
  -d "emotion-recognition-application" \
  --file emotion-recognition-application.json
$ cat emotion-recognition-application.json
$ cp template.local.env local.env
```

In [local.env](local.env) define 'IBMCLOUD_API_KEY', 'IBMCLOUD_ORG', 'IBMCLOUD_SPACE' and 'BLUEMIX_REGION' to match the apikey in [emotion-recognition-application.json](emotion-recognition-application.json) and the org, space and region name that you're using (see the outputs in your terminal when following the steps above).

repository: thomassuedbroecker/blue-cloud-mirror

repo_name: thomassuedbroecker/blue-cloud-mirror


text:
---
layout: post
title: Building BAIL app's backend - Part 1
category: [engineering]
comments: true
---

I saw a tweet the other day and it really piqued my interest. "The app is called “You’re Cancelled.” When you’ve made plans that you wish you could cancel, you go into the app and press a little button. If the other person presses theirs too, congratulations! Confetti exploded and your plans are cancelled."

The app is called “You’re Cancelled.” When you’ve made plans that you wish you could cancel, you go into the app and press a little button. If the other person presses theirs too, congratulations! Confetti exploded and your plans are cancelled.
— mattie kahn (@mattiekahn) February 2, 2020

Needless to say, I was immediately sold. I was drinking beer when I saw it so I told my friend about it and asked him to hold my beer. Honestly, the idea might promote something anti-social but I just thought it would be a good exercise to flex my android app and side project skills. I also think that it's somewhat of an appropriate app idea especially for people of Seattle. Have you heard of the Seattle freeze?

Anyways, I fully intend to finish this and release it on iOS and Google Play store if I can find a solid designer who can finish the UI for me. I will get the rest done. So if you like to build cool things and have some experience with Android/iOS design, please inquire within<EMAIL_ADDRESS>
I already have the perfect cover picture idea for the app. I just need to email the artist to see if he would be okay with me using it - or see what he would want in return. Check him out: https://www.willmcphail.com/about I'm a big fan of his New Yorker cartoons.

Without further due, let's get right into it. First things first. Pick a playlist. Honestly, if I'm coding something exciting but also kinda edgy, I'll go with the Social network soundtrack. Enjoy: https://open.spotify.com/album/1ijkFiMeHopKkHyvQCWxUa?si=3SGegSNkTom9JHvelWcyQg. If I'm more in the melodic/dark techno mood, then I'll go with Mind Against or Tale of Us. Pick your poison basically.

Next is obviously alcohol. But I will not take my beer back from my friend. Instead, let's get funky. Pour some kahlua with club soda. And lots of ice.

Alright, ladies and gentleman, we are ready to code... is what a junior engineer would say. First, we need to design the interface, use cases and the data model. The rest, the implementation, we can just hand-off to a junior engineer. Just kidding.

Let's start with our requirements aka the use cases. There might be a slight difference between these terms but it's ok.

### Requirements

```
1. Authenticate the user of the application
2. Secretly enter the plans that you wish you could cancel
3. View all the (confirmed and otherwise) plans that you entered
4. Be nofitied when the cancellation desire happens to be mutual!
     -- Bonus: Celebrate the cancellation of your plans with confetti.
```

That's it. We got to keep it simple because, you know, I have a day job. Also starting simple and getting feedback is perfectly fine. Worse is better. Reduce the scope. Feature down. Underdo your competition. *Winks at 37signals, you guys know it.

Alrighty, so now that we know our requirements, let's make them crystal clear from the technical standpoint. This is where the engineering team usually chimes in. We take the "more abstract" requirements and turn them into tangible technical capabilities.

Security is a huge thing and being able to authenticate the users is essential for almost all applications. We can't take many shortcuts here. However, I will go with the simplest approach here. Normally, I dislike that we still use SIM cards and phone numbers but they do simplify things. I hate social-media app based auth models even more so I'll go with phone numbers.

Obviously, the right thing would be to interact with the UICC SIM card, maybe retrieve IMEI etc, authenticate via some JavaCard applet (https://nelenkov.blogspot.com/2013/09/using-sim-card-as-secure-element.html) but ain't nobody got time for that. Funny, I actually have a solid amount of expertise in JavaCard development but that's for a later post perhaps. We will do the second most feasible approach. One-time OPT codes sent via SMS. Here's the API we are going to build.

```
Given a phone number, generate a one-time use time-based OTP, send it via SMS and ask the user to enter the OTP on the screen
```

or in API form
``` java
/*
Generates an OTP, sends it over SMS, stores it with a TTL for a phone number auth attempt
*/
void requestOtp(PhoneNumber number);
```

Next, we need to be able to retrieve the auth token OTP and complete authentication. After this point, the user should be able to use the app until they switch to a different device. Obviously, we can limit the client auth to a certain period (3 months?) and re-ask them to do the challenge.

``` java
/*
Given an OPT and a phone number, authenticates the user or throw AccessDenied (if auth fails or time has expired upon which the client should retry the flow)
*/
ClientId authenticate(PhoneNumber number, OTP authToken) throws AccessDeniedException;
```

Assuming we can authenticate users, we can now focus on the core use case. Here, it is extremely critical to agree on the time of the dates. I like to go with the granularity level of daily plans, nothing more or less granular. I mean, who makes a plan with the same person more than once on any given day? Our app will allow you to secretly enter your plans on a daily basis and with however many people.

This means, our date consists of 2 questions: 1) With whom (contacts prompt) 2) When? (calendar prompt). Simple. I guess our model does extend to dates with more than just two people. But for simplicity I'll keep it at two for now. I'll try to pay attention to that to avoid making a one-way door decision that would restrict our ability to support dates with multiple people. In such a case, I presume we would need to wait for all parties' cancellation request before the plans can be considered cancelled.

```java
/*
Given a date and a contact, stores the plans that the user wishes they could cancel
*/
void schedule(ClientId authenticatedAndAuthorizedUser, Date when, PhoneNumber withWhom);
```

It's important to note that all these APIs will have to carry the authenticated user id so that the backend can check whether or not this user is authorized to set a date on behalf of the owner of the client id. Usually, those client ids or auth tokens should be short-lived and periodically refreshed. Ignoring that for the time being.

We are almost there. We need one more API. Whenever I open the app, I'd like to see my unconfirmed plans. Remember, we are sending notifications when both parties enter their plans that they wish to cancel but otherwise the plans are still happening and you will hopelessly look at your dashboards. For that, we need to retrieve the unconfirmed plans from the backend starting from today into the future

```java
/*
Given a user retrieves all the future plans in paginated form
*/
List<Plan> getPlans(ClientId authenticatedAndAuthorizedUser, NextToken nextContinuationToken);
```

where ```Plan``` is
```java
public class Plan {
  User withWhom;
  Date date;
  Status status; // (enum: UNCONFIRMED, CANCELLED etc.)
}
```

This API will query all the future unconfirmed and confirmed(?) plans that the user has with others. The continuation token is set if the result size is bigger than X. Let's not worry about pagination just yet but yeah it's obviously gonna be there in this API.

We can build another version of this for archived cancelled and not cancelled plans that the user has for, you know, bragging points. What else do we need?

We need to be able to send notifications only when both parties cancel on each other. I love the confetti idea.  Animations would be dope. We could technically provide an option to revive the date? I'm just thinking out loud.

```java
/*
Given a notification token retrieved via SMS and the authorized user, retrieves the cancelled plan details to view.
*/
Plan getCancelledPlan(ClientId authenticatedAndAuthorizedUser, NotificationToken token);
```

We all love push notifications but you know that's more work. Here we will take advantage of the SMS integration and will force the app to make a backend call to retrieve just the cancelled event. Similarly, we need another API to retrieve all the notifications so that if they miss the SMS or something, they can go back to the app and hit on the notification page and go from there.

One thing that's critical is to NOT send the notification immediately AFTER the second person puts in the plan details. That would make it obvious to both parties who entered the details first. Which is why we should be careful building that core logic. We could introduce some jizzer between the time plans are mutually confirmed and the time notifications are sent out. The exact jizzer should depend on the time that's left until the date. Nothing difficult, meh, implementation detail even but the idea is essential to success. More importantly, we need to be able to convey this feature to our users so that they don't assume the plans are always wished to be cancelled by the other party. Because that's how dating apps work unlike this app.

A couple of geeky ideas I have on this is to show users "secretly" that somebody is wishing to bail on them by sending them a notification without revealing who and when. That way we can lure them into buying the "premium" version of the app that can reveal the identity of the person that's bailing. Maybe. Haven't thought deeply about this so let's get back to our non-existing free version.

Basically, we are done with the APIs. I think I covered all the requirements. Now, assuming these are my right access patterns, I can go ahead and design my data schema/model/state management. Let's talk about state.

### State Management

We need to be able to persist the auth token/client id somewhere on the device that can only be modified by the app. I was thinking of using SharedPreferences in Android world. Once the authentication is complete and the backend sends the client id back to the app, this is where the app will store it. It will be durable across app crashes/device reboots. The app will keep this state until the authentication needs to be repeated due to access expiry. This is pretty much the only state that the app needs to keep track of. Obviously, it can do some caching to minimize redundant calls to the backend.

On the cloud side, we need to keep track of all the entered unconfirmed plans so we can match them. We need durable storage. To make the app fast enough, we will need to define our keys and indices well. We should go with a database - a NoSQL should do it. Good thing, we have a solid idea of our access patterns. Otherwise, I'd recommend going with a SQL approach and just storing the entities per table. A relational model would keep the flexibility in the queries you are going to construct as you develop the idea.

I want to have the fewest number of tables that we can get away it. We should also strive to make the runtime of our queries linear. I came up with the following:

```
Table: Plan
Primary (hash) key: Number1_Number2_Date (Number1_Number_2_..._NumberN_Date to support dates with multiple people)
Sort (range) key: Number that entered the plan details first
```

This way both our write & read (match) operations will be O(1). Here's how it works:

1. Let's say a User 2 with Number 2 enters the plan they wish to cancel with User 1 with Number 1 on Date X.
2. Backend sorts Number 1 and Number 2 numerically and concats them with the date. Let's say it turned out to be Number1_Number2_DateX.
3. Backend first queries the DB with that being the hash key, PK=Number1_Number2_DateX. It finds out there are no records.
4. Backend writes the record into the DB with PK=Number1_Number2_DateX and SK=Number2
5. A day after User 1 with Number 1 also realizes they wish to cancel so they submit the request on BAIL APP.
6. Backend sorts Number 1 and Number 2 numerically and concats them with the date. It becomes Number1_Number2_DateX.
7. Backend queries the DB with that being the hash key, PK=Number1_Number2_DateX. It finds a record in O(1) constant time because that was our hash key. This is good news- confetti time.
8. Backend sets up the notication event and time for further processing.

Unfortunately, we can't query this table for retrieving all the plans given a number so we would have to store the details in a parallel table with PK=Number, and sort keys being withWhom and date and status. Actually mirroring the plan in-memory data structure. That should work and retrieve the plans (sorted by today onwards) in linear time given a phone number (user)

That wraps up the data model. I shall now briefly cover the dependencies that we will utilize and proceed to the implementation in the next blog post.

### Infrastructure and Dependencies

Since I'm an AWS geek I won't look elsewhere. We need a persistent NoSQL data storage and we have no better option than DynamoDB. Simple, consistent API. Easy enough.

We will use SNS with SMS integration. SMS is not inherently "reliable" unless we use Transaction feature supported by SNS. I think OPT may fall into this category. I will have to double check. We will also use SNS for notification texts and they will have to be promotional. Cost should be minimal at first but with more users SMS would cost the company money so we would have to look into actual push notification integration.

CW Events can be used in a fairly straightforward way to integrate "cancel notification" scheduling which would then trigger SNS-SMS push notifications. Once the storage returns a record to step #7, the backend should schedule a CW event.

We should be able to build the entire backend in Lambda. No need for any persistence or customization in the compute layer. Nothing complicated. Short, plain simple functions with no timeout concerns. The cold start would be annoying but now they have reserved dedicated lambdas that we can take advantage of. At first, Lambda would be great since it will scale with usage. If the app became extremely popular, we may have to switch to dedicated servers. Very hypothetical.

Finally, we need to write some APIs. Yeah, yeah, we may need an API Gw but what was the first rule? Simplicity. So we will ignore the GW at first. Instead, we will setup an unauthenticated Cognito pool (since we are doing authentication out of band) and that pool shall give the android app's aws sdk permissions to call our Lambdas. To keep things warm, we can write all the APIs in one lambda or 1 per API. Not so sure yet, implementation detail so I'll cover this in the next post.

Getting excited to build this. Hope you enjoyed this guide around system design and keeping things simple as you build your next idea.

url: https://raw.githubusercontent.com/tameravci/tameravci.github.io/1a96919cf5f2533f3aa3e3f9bbf435d2588585fa/_posts/2020-2-24-Developing BAIL's backend - Part 1.md


text:
## Frappe Client

Python Library for interacting with Frappe / ERPNext API

### Usage

### Create a Customer, and then fetch it back from the HTTP API
```python
from frappeclient import FrappeClient

client = FrappeClient("example.com"<EMAIL_ADDRESS>"password")

# Prepare a customer dict that we will use to create a new
# customer in ERPNext
doc = {"doctype": "Customer",
       "customer_name": "Example Inc.",
       "customer_type": "Company",
       "website": "example.net"}

# create new record in ERPNext
client.insert(doc)

# Query the erpnext HTTP API for the name of customer whos
# website is example.net
customer_name = client.get_value("Customer",
                                 "name",
                                 {"website": "example.net"})
# Fetch customer
customer = client.get_doc("Customer", customer_name['name'])
```

#### Fetch a list of Items from the HTTP API
```python
from frappeclient import FrappeClient

client = FrappeClient("example.com"<EMAIL_ADDRESS>"password")
notes = [{"doctype": "Note", "title": "Sing", "public": True},
         {"doctype": "Note", "title": "a", "public": True},
         {"doctype": "Note", "title": "Song", "public": True},
         {"doctype": "Note", "title": "of", "public": True},
         {"doctype": "Note", "title": "sixpence", "public": True}
         ]

for note in notes:
    print(client.insert(note))

# Query for Note using only filters, and fields arguments. Returns a list
# of four dicts with the respective titles Sing, Song and sixpence.
notes_starting_with_S = client.get_doc(
    'Note',
    filters=[["Note", "title", "LIKE", "S%"]],
    fields=["title", "public"])
```

### Example

See example.py for more info

### License

MIT

repository: DigiThinkIT/frappe-client

repo_name: DigiThinkIT/frappe-client