text:
---
# Display name
name: Zhuowan Li

# Username (this should match the folder name)
authors:
- admin

# Is this the primary user of the site?
superuser: true

# Role/position
role: 李卓婉

# Organizations/Affiliations
organizations:
- name: PhD in Computer Science
- name: Johns Hopkins University
  url: ""

# Short bio (displayed in user profile at end of posts)
bio:

interests:
#- Artificial Intelligence
#- Computational Linguistics
#- Information Retrieval

education:
  courses:
  - course: PhD in Computer Science
    institution: Johns Hopkins University
    year: 2018 - Now
  - course: BE in Electronic Engineering
    institution: Tsinghua University
    year: 2014 - 2018

# Social/Academic Networking
# For available icons, see: https://sourcethemes.com/academic/docs/page-builder/#icons
#   For an email link, use "fas" icon pack, "envelope" icon, and a link in the
#   form<EMAIL_ADDRESS>or "#contact" for contact widget.
social:
- icon: envelope
  icon_pack: fas
  link<EMAIL_ADDRESS># For a direct email link, use<EMAIL_ADDRESS>- icon: github
  icon_pack: fab
  link: https://github.com/Lizw14
- icon: google-scholar
  icon_pack: ai
  link: https://scholar.google.com/citations?user=Fft1WvwAAAAJ&hl=en
- icon: twitter
  icon_pack: fab
  link: https://twitter.com/zhuowanli

# Link to a PDF of your resume/CV from the About widget.
# To enable, copy your resume/CV to `static/files/cv.pdf` and uncomment the lines below.
# - icon: cv
#   icon_pack: ai
#   link: files/cv.pdf

# Enter email to display Gravatar (if Gravatar enabled in Config)
email<EMAIL_ADDRESS>
# Organizational groups that you belong to (for People widget)
#   Set this to `[]` or comment out if you are not using People widget.
user_groups:
#- Researchers
#- Visitors
---

I am a 5th-year PhD student at Johns Hopkins University, supervised by [Prof. Alan Yuille](https://cs.jhu.edu/~ayuille/) and [Prof. Benjamin Van Durme](https://www.cs.jhu.edu/~vandurme/index.html). I am a member of [CCVL lab](https://ccvl.jhu.edu/team/). Before this, I received my B.E. degree at Tsinghua Univeristy in 2018. I have also spent wonderful times at Facebook, Adobe and Sensetime.

My research interest focuses on **computer vision and natural language processing**. My works relates to both large-scale pretraining and compositional models. I am also interested in model diagnosis including robustness, generaliation, compositionality, etc. I believe that the joint learning of vision and language offers mutual benefits!

I am <span style="color:red"> **on the job market** </span> for research positions in 2023, including full-time/internship positions in industry and post-docs positions. Don't hesitate to reach out to me (zli110 at jhu dot edu) if there is a potential fit:)

In part time, I am a big fan of outdoor sports including rock climbing (recovering from a finger injury 😭), snowboarding, skiing, hiking, mountaineering, etc. Recently I also spent time in learning piano.


text:
> Lean Runup model

Runup-lean contains a [Runup](https://github.com/everbuild/runup) model with a fairly complete set of web (app) development tools.
It's rather opinionated as it contains those tools I currently prefer to work with.
Use what you like, tweak what you don't (and share).

I try to find tools that do the job well and do it efficiently.
Currently these are:

For JS:

* [Rollup](https://rollupjs.org/) for bundling ES modules
* [Babel](https://babeljs.io/) for transpiling them to JS in prd mode
* [Uglify](http://lisperator.net/uglifyjs/) for minifying them in prd mode
* [ESLint](http://eslint.org/) for linting
* [Thintest](https://github.com/everbuild/thintest) for testing

For CSS:

* [Node-sass](https://github.com/sass/node-sass) for compiling from [Sass](http://sass-lang.com/)
* [Autoprefixer](https://github.com/postcss/autoprefixer) (through [PostCSS](http://postcss.org/)) to add vendor prefixes
* [CSSO](https://github.com/css/csso) to minify it in prd mode

# Install

```
$ npm install --saveDev runup runup-lean
```

# Usage

If you haven't yet, [setup Runup](https://github.com/everbuild/runup#usage).

Then let your model extend runup-lean:

```ecmascript 6
// file: project.js

module.exports = {

    extend: 'runup-lean',

    // you can override tool options:
    sass: {
        // usual Node-sass options...
        outputStyle: 'compact'
    }
    // there are similar hooks for all tools, see the flow files

    // ... other settings ...
}
```

Then start Runup in dev or prd mode as usual.
For more info on Runup, see [the wiki](https://github.com/everbuild/runup/wiki).

license: MIT

repository: everbuild/runup-lean

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: everbuild/runup-lean


text:
### Description

vtkArrowSource object appends a cylinder to a cone to form an arrow.

The shaft base is always at (0,0,0). The arrow tip is always at (1,0,0). If "Invert" is true, then the ends are flipped i.e. tip is at (0,0,0) while base is at (1, 0, 0).

The resolution of the cone and shaft can be set and default to 6.

The radius of the cone and shaft can be set and default to 0.03 and 0.1. The length of the tip can also be set, and defaults to 0.35.

repository: Kitware/vtk-examples

repo_name: Kitware/vtk-examples


text:
# BlurHash Algorithm

## Summary

BlurHash applies a simple [DCT transform](https://en.wikipedia.org/wiki/Discrete_cosine_transform) to the image data,
keeping only the first few components, and then encodes these components using a base 83 encoding, with a JSON,
HTML and shell-safe character set. The DC component, which represents the average colour of the image, is stored exactly
as an sRGB value, for easy use without impleneting the full algorithm. The AC components are encoded lossily.

## Reference implementation

[Simplified Swift decoder implemenation.](Swift/BlurHashDecode.swift)

[Simplified Swift encoder implemenation.](Swift/BlurHashEncode.swift)

## Structure

Here follows an example of a BlurHash string, with the different parts labelled:

    Example: LlMF%n00%#MwS|WCWEM{R*bbWBbH
    Legend:  12333344....................

1. **Number of components, 1 digit.**

   For a BlurHash with `nx` components along the X axis and `ny` components along the Y axis, this is equal to `(nx - 1) + (ny - 1) * 9`.

2. **Maximum AC component value, 1 digit.**

   All AC components are scaled by this value. It represents a floating-point value of `(max + 1) / 166`.

3. **Average colour. 4 digits.**

   The average colour of the image in sRGB space, encoded as a 24-bit RGB value, with R in the most signficant position. This value can
   be used directly if you only want the average colour rather than the full DCT-encoded image.

4. **AC components, 2 digits each, `nx * ny - 1` components in total.**

   The AC components of the DCT transform, ordred by increasing X first, then Y. They  are encoded as three values for `R`, `G` and `B`,
   each between 0 and 18. They are combined together as `R * 19^2 + G * 19 + B`, for a total range of 0 to 6859.

   Each value represents a floating-point value between -1 and 1. 0-8 represent negative values, 9 represents zero, and 10-18
   represent positive values. Positive values are encoded as `((X - 9) / 9) ^ 2`, while negative
   values are encoded as `-((9 - X) / 9 ) ^ 2`. `^` represents exponentiation. This value is then multiplied by the maximum AC
   component value, field 2 above.

## Base 83

A custom base 83 encoding is used. Values are encoded individually, using 1 to 4 digits, and concatenated together. Multiple-digit
values are encoded in big-endian order, with the most signficant digit first.

The character used set is `0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~`.

## Discrete Cosine Transform

To decode a single pixel of output, you loop over the DCT components and calculate a weighted sum of cosine functions. In
pseudocode, for a normalised pixel position `x`, `y`, with each coordinate ranging from 0 to 1, and components `Cij` ,
you calculate the following for each of R, G and B:

    foreach j in 0 ... ny - 1
        foreach i in 0 ... nx - 1
            value = value + Cij * cos(x * i * pi) * cos(y * j * pi)

The `C00` component is the DC component, while the others are the AC components. The DC component must first be converted
from sRGB to linear RGB space. AC components are already linear.

Once the R, G and B values have been calculated, the must be converted from linear to your output colourspace, usually sRGB.

repository: ankane/blurhash

gha_license_id: MIT

repo_name: ankane/blurhash


text:
# pyAutomationTesting ![](https://img.shields.io/badge/python-3.7-brightgreen)
python3.7 automation test framework(unittest + BeautifulReport+ pytest + pytest-html + selenium)

## 1. Setup Project
```
cd path/to/pyAutomationTesting
python -m venv venv
venv\Scripts\activate.bat
(venv) path/to/pyAutomationTesting>pip install -r requirements.txt
```

## 2. Run testcase
```
cd path/to/pyAutomationTesting
venv\Scripts\activate.bat
(venv) path/to/pyAutomationTesting>python main.py
```

## 3. Add your own testcases
- just add your unittest testcase to cases_unittest dir
- just add your pytest testcase to cases_pytest dir

## 4. Log and Report
- log and report was write in path/to/pyAutomationTesting/result/ if you not set
- unittest report located at: path/to/pyAutomationTesting/result/unittest_report.html
- pytest report located at: path/to/pyAutomationTesting/result/pytest_report.html

## 5. Selenium WebDriver Download for Windows 64 Bit
- [IE WebDriver](http://selenium-release.storage.googleapis.com/index.html): [IEDriverServer.exe](http://selenium-release.storage.googleapis.com/index.html?path=3.9/IEDriverServer_x64_3.9.0.zip)
- [Firefox WebDriver](https://github.com/mozilla/geckodriver/releases): [geckodriver.exe](https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-win64.zip)
- [Chrome WebDriver](http://npm.taobao.org/mirrors/chromedriver): [chromedriver.exe](http://npm.taobao.org/mirrors/chromedriver/77.0.3865.40/chromedriver_win32.zip)
> Extract WebDriver excutable file to C:\WebDriver and then add C:\WebDriver into %PATH%

## 6. Thanks
- [BeautifulReport](https://github.com/TesterlifeRaymond/BeautifulReport)

gha_language: Python


text:
[![Build status](https://github.com/bento-dbaas/host-provider/actions/workflows/main.yml/badge.svg?branch=master)](https://github.com/bento-dbaas/host-provider/actions) [![codecov](https://codecov.io/gh/bento-dbaas/host-provider/branch/master/graph/badge.svg?token=GQN1ZN9FJQ)](https://codecov.io/gh/bento-dbaas/host-provider)

## Host Provider

### How to run

#### Native:
 - Copy the base env file to dev env file:
```shell
$cp .export-host-provider-local-base.sh .export-host-provider-local-dev.sh
```
 - Replace variables in `.export-host-provider-local-dev.sh` file.
 - install requirements:
  ```shell
 $pip install -r requirements.txt
 ```
 - load environment variables:
  ```shell
$source .export-host-provider-local-dev.sh
  ```

 - run project: `$make run`

#### Docker Compose:
`todo`

### Configure DBaaS:
Go to your `DBaaS local instance > DBaaaS_Credentials > Credentials` and point the `Host Provider` Cretentials to your host provider instance (<IP_ADDRESS>:5002)

### Setup MySQL Database

#### Initialize Script
```shell
$make db_initialize
```

#### Schema Migration
```shell
$make db_migrate
```

repository: bento-dbaas/host-provider

repo_name: bento-dbaas/host-provider


text:
# PuttyLauncher

PuttyLauncher is a plugin for [WinCommandPalette](https://github.com/DarkIrata/WinCommandPalette).

This plugin will load all your saved PuTTY sessions and create a command for them.
Also it creates a command to load all your private keys into pageant.

You can also create your own putty commands for SSH, Telnet, Rlogin, Raw and Serial.

# Installation

Download the latest release and extract the .zip file. Copy the *PuttyLauncher* directory into your *WinCommandPalette* directory
and finally restart WinCommandPalette.

# Reload sessions

To reload newly saved sessions you have to restart WinCommandPalette, as there is no API yet to add/remove commands.


text:
---
title: "Scaling with Apache SkyWalking"
date: 2022-01-24
author: Han Liu, Sheng Wu
description: "Introduce how to load balance the traffic between SkyWalking backend and agent through SkyWalking Satellite"

tags:
- Performance
---

## Background

In the Apache SkyWalking ecosystem, the OAP obtains metrics, traces, logs, and event data through SkyWalking Agent, Envoy, or other data sources. Under the gRPC protocol, it transmits data by communicating with a single server node. Only when the connection is broken, the reconnecting policy would be used based on DNS round-robin mode. When new services are added at runtime or the OAP load is kept high due to increased traffic of observed services, the OAP cluster needs to scale out for increased traffic. The load of the new OAP node would be less due to all existing agents having connected to previous nodes. Even without scaling, the load of OAP nodes would be unbalanced, because the agent would keep the connection due to random policy at the booting stage. In these cases, it would become a challenge to keep up the health status of all nodes, and be able to scale out when needed.

In this article, we mainly discuss how to solve this challenge in SkyWalking.

## How to Load Balance

SkyWalking mainly uses the gRPC protocol for data transmission, so this article mainly introduces load balancing in the gRPC protocol.

### Proxy Or Client-side

Based on the [gRPC official Load Balancing blog](https://grpc.io/blog/grpc-load-balancing/), there are two approaches to load balancing:

1. **Client-side**: The client perceives multiple back-end services and uses a load-balancing algorithm to select a back-end service for each RPC.
2. **Proxy**: The client sends the message to the proxy server, and the proxy server load balances the message to the back-end service.

From the perspective of observability system architecture:

|     | Pros | Cons |
| --- | ---- | ---- |
|Client-side|
High performance because of the elimination of extra hop
 |
Complex client (cluster awareness, load balancing, health check, etc.)

Ensure each data source to be connected provides complex client capabilities
 |
|Proxy|
Simple Client
|
Higher latency
|

We choose Proxy mode for the following reasons:

1. Observable data is not very time-sensitive, a little latency caused by transmission is acceptable.
2. As an observability platform, we cannot/should not ask clients to change. They make their own tech decisions and may have their own commercial considerations.

### Transmission Policy

In the proxy mode, we should determine the transmission path between downstream and upstream.

Different data protocols require different processing policies. There are two transmission policies:

1. **Synchronous**: Suitable for protocols that require data exchange in the client, such as SkyWalking Dynamic Configuration Service. This type of protocol provides real-time results.
2. **Asynchronous batch**: Used when the client doesn’t care about the upstream processing results, but only the transmitted data (e.g., trace report, log report, etc.)

The synchronization policy requires that the proxy send the message to the upstream server when receiving the client message, and synchronously return the response data to the downstream client. Usually, only a few protocols need to use the synchronization policy.

As shown below, after the client sends the request to the Proxy, the proxy would send the message to the server synchronously. When the proxy receives the result, it returns to the client.

![](./transmission-policy-synchronous.png)

The asynchronous batch policy means that the data is sent to the upstream server in batches asynchronously. This policy is more common because most protocols in SkyWalking are primarily based on data reporting. We think using the queue as a buffer could have a good effect. The asynchronous batch policy is executed according to the following steps:

1. The proxy receives the data and wraps it as an Event object.
2. An event is added into the queue.
3. When the cycle time is reached or when the queue elements reach the fixed number, the elements in the queue will parallel consume and send to the OAP.

The advantage of using queues is:

1. Separate data receiving and sending to reduce the mutual influence.
2. The interval quantization mechanism can be used to combine events, which helps to speed up sending events to the OAP.
3. Using multi-threaded consumption queue events can make fuller use of network IO.

As shown below, after the proxy receives the message, the proxy would wrap the message as an event and push it to the queue. The message sender would take batch events from the queue and send them to the upstream OAP.

![](./transmission-policy-asynchronous.png)

### Routing

Routing algorithms are used to route messages to a single upstream server node.

The Round-Robin algorithm selects nodes in order from the list of upstream service nodes. The advantage of this algorithm is that the number of times each node is selected is average. When the size of the data is close to the same, each upstream node can handle the same quantity of data content.

![](./routing-round-robin.png)

With the Weight Round-Robin, each upstream server node has a corresponding routing weight ratio. The difference from Round-Robin is that each upstream node has more chances to be routed according to its weight. This algorithm is more suitable to use when the upstream server node machine configuration is not the same.

![](./routing-weight-round-robin.png)

The Fixed algorithm is a hybrid algorithm. It can ensure that the same data is routed to the same upstream server node, and when the upstream server scales out, it still maintains routing to the same node; unless the upstream node does not exist, it will reroute. This algorithm is mainly used in the SkyWalking Meter protocol because this protocol needs to ensure that the metrics of the same service instance are sent to the same OAP node.
The Routing steps are as follows:

1. Generate a unique identification string based on the data content, as short as possible. The amount of data is controllable.
2. Get the upstream node of identity from LRU Cache, and use it if it exists.
3. According to the identification, generate the corresponding hash value, and find the upstream server node from the upstream list.
4. Save the mapping relationship between the upstream server node and identification to LRU Cache.

The advantage of this algorithm is to bind the data with the upstream server node as much as possible, so the upstream server can better process continuous data. The disadvantage is that it takes up a certain amount of memory space to save the corresponding relationship.

As shown below, the image is divided into two parts:

1. The left side represents that the same data content always is routed to the same server node.
2. The right side represents the data routing algorithm. Get the number from the data, and use the remainder algorithm to obtain the position.

![](./routing-fixed.png)

We choose to use a combination of Round-Robin and Fixed algorithm for routing:

1. The Fixed routing algorithm is suitable for specific protocols, mainly used when passing metrics data to the SkyWalking Meter protocol
2. The Round-Robin algorithm is used by default. When the SkyWalking OAP cluster is deployed, the configuration of the nodes needs to be as much the same as possible, so there would be no need to use the Weight Round-Robin algorithm.

## How to balance the load balancer itself?

Proxy still needs to deal with the load balancing problem from client to itself, especially when deploying a Proxy cluster in a production environment.

There are three ways to solve this problem:

1. **Connection management**: Use the `max_connection` config on the client-side to specify the maximum connection duration of each connection. For more information, please read the [proposal](https://github.com/grpc/proposal/blob/master/A9-server-side-conn-mgt.md).
2. **Cluster awareness**: The proxy has cluster awareness, and actively disconnects the connection when the load is unbalanced to allow the client to re-pick up the proxy.
3. **Resource limit+HPA**: Restrict the connection resource situation of each proxy, and no longer accept new connections when the resource limit is reached. And use the HPA mechanism of Kubernetes to dynamically scale out the number of the proxy.

|     | Connection management | Cluster awareness | Resource Limit+HPA |
| --- | ---- | ---- | ---- |
|Pros|
Simple to use
 |
Ensure that the number of connections in each proxy is relatively
 |
Simple to use
 |
|Cons|
Each client needs to ensure that data is not lost

The client is required to accept GOWAY responses
|
May cause a sudden increase in traffic on some nodes

Each client needs to ensure that data is not lost
|
Traffic will not be particularly balanced in each instance
|

We choose Limit+HPA for these reasons:

1. Easy to config and use the proxy and easy to understand based on basic data metrics.
2. No data loss due to broken connection. There is no need for the client to implement any other protocols to prevent data loss, especially when the client is a commercial product.
3. The connection of each node in the proxy cluster does not need to be particularly balanced, as long as the proxy node itself is high-performance.

## SkyWalking-Satellite

We have implemented this Proxy in the [SkyWalking-Satellite](https://github.com/apache/skywalking-satellite) project. It’s used between Client and SkyWalking OAP, effectively solving the load balancing problem.

After the system is deployed, the Satellite would accept the traffic from the Client, and the Satellite will perceive all the nodes of the OAP through Kubernetes Label Selector or manual configuration, and load balance the traffic to the upstream OAP node.

As shown below, a single client still maintains a connection with a single Satellite, Satellite would establish the connection with each OAP, and load balance message to the OAP node.

![](./skywalking-satellites.png)

When scaling Satellite, we need to deploy the [SWCK](https://github.com/apache/skywalking-swck) adapter and configure the HPA in Kubernetes. SWCK is a platform for the SkyWalking users, provisions, upgrades, maintains SkyWalking relevant components, and makes them work natively on Kubernetes.

After deployment is finished, the following steps would be performed:

1. **Read metrics from OAP**: HPA requests the SWCK metrics adapter to dynamically read the metrics in the OAP.
2. **Scaling the Satellite**: Kubernetes HPA senses that the metrics values are in line with expectations, so the Satellite would be scaling automatically.

As shown below, use the dotted line to divide the two parts. HPA uses SWCK Adapter to read the metrics in the OAP. When the threshold is met, HPA would scale the Satellite deployment.

![](swck-hpa.png)

## Example

In this section, we will demonstrate two cases:

1. SkyWalking Scaling: After SkyWalking OAP scaling, the traffic would auto load balancing through Satellite.
2. Satellite Scaling: Satellite’s own traffic load balancing.

NOTE: All commands could be accessed through [GitHub](https://github.com/mrproliu/sw-satellite-demo-scripts/tree/1180c23e8f3bb36778307f9ae15395274ca039b3).

### SkyWalking Scaling

We will use the [bookinfo application](https://istio.io/latest/docs/examples/bookinfo/) to demonstrate how to integrate Apache SkyWalking 8.9.1 with Apache SkyWalking-Satellite 0.5.0, and observe the service mesh through the Envoy ALS protocol.

Before starting, please make sure that you already have a Kubernetes environment.

#### Install Istio

Istio provides a very convenient way to configure the Envoy proxy and enable the access log service. The following step:

1. Install the istioctl locally to help manage the Istio mesh.
2. Install Istio into the Kubernetes environment with a demo configuration profile, and enable the Envoy ALS. Transmit the ALS message to the satellite. The satellite we will deploy later.
3. Add the label into the default namespace so Istio could automatically inject Envoy sidecar proxies when you deploy your application later.

```shell
# install istioctl
export ISTIO_VERSION=1.12.0
curl -L https://istio.io/downloadIstio | sh -
sudo mv $PWD/istio-$ISTIO_VERSION/bin/istioctl /usr/local/bin/

# install istio
istioctl install -y --set profile=demo \
	--set meshConfig.enableEnvoyAccessLogService=true \
	--set meshConfig.defaultConfig.envoyAccessLogService.address=skywalking-system-satellite.skywalking-system:11800

# enbale envoy proxy in default namespace
kubectl label namespace default istio-injection=enabled
```

#### Install SWCK

SWCK provides convenience for users to deploy and upgrade SkyWalking related components based on Kubernetes. The automatic scale function of Satellite also mainly relies on SWCK. For more information, you could refer to the [official documentation](https://github.com/apache/skywalking-swck/blob/master/docs/operator.md#guides-of-operator-deployment).

```shell
# Install cert-manager
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.yaml

# Deploy SWCK
mkdir -p skywalking-swck && cd skywalking-swck
wget https://dlcdn.apache.org/skywalking/swck/0.6.1/skywalking-swck-0.6.1-bin.tgz
tar -zxvf skywalking-swck-0.6.1-bin.tgz
cd config
kubectl apply -f operator-bundle.yaml
```

#### Deploy Apache SkyWalking And Apache SkyWalking-Satellite

We have provided a simple script to deploy the skywalking OAP, UI, and Satellite.

```shell
# Create the skywalking components namespace
kubectl create namespace skywalking-system
kubectl label namespace skywalking-system swck-injection=enabled
# Deploy components
kubectl apply -f https://raw.githubusercontent.com/mrproliu/sw-satellite-demo-scripts/5821a909b647f7c8f99c70378e197630836f45f7/resources/sw-components.yaml
```

#### Deploy Bookinfo Application

```shell
export ISTIO_VERSION=1.12.0
kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/platform/kube/bookinfo.yaml
kubectl wait --for=condition=Ready pods --all --timeout=1200s
kubectl port-forward service/productpage 9080
```

Next, please open your browser and visit `http://localhost:9080`. You should be able to see the Bookinfo application. Refresh the webpage several times to generate enough access logs.

Then, you can see the topology and metrics of the Bookinfo application on SkyWalking WebUI. At this time, you can see that the Satellite is working!

#### Deploy Monitor

We need to install OpenTelemetry Collector to collect metrics in OAPs and analyze them.

```shell
# Add OTEL collector
kubectl apply -f https://raw.githubusercontent.com/mrproliu/sw-satellite-demo-scripts/5821a909b647f7c8f99c70378e197630836f45f7/resources/otel-collector-oap.yaml

kubectl port-forward -n skywalking-system  service/skywalking-system-ui 8080:80
```

Next, please open your browser and visit `http://localhost:8080/` and create a new item on the dashboard. The SkyWalking Web UI pictured below shows how the data content is applied.

![](./mesh-count-conf.png)

#### Scaling OAP

Scaling the number of OAPs by deployment.

```shell
kubectl scale --replicas=3 -n skywalking-system deployment/skywalking-system-oap
```

#### Done!

After a period of time, you will see that the number of OAPs becomes 3, and the ALS traffic is balanced to each OAP.

![](./mesh-count-list.png)

### Satellite Scaling

After we have completed the SkyWalking Scaling, we would carry out the Satellite Scaling demo.

#### Deploy SWCK HPA

SWCK provides an adapter to implement the Kubernetes external metrics to adapt the HPA through reading the metrics in SkyWalking OAP. We expose the metrics service in Satellite to OAP and configure HPA Resource to auto-scaling the Satellite.

Install the SWCK adapter into the Kubernetes environment:
```shell
kubectl apply -f skywalking-swck/config/adapter-bundle.yaml
```

Create the HPA resource, and limit each Satellite to handle a maximum of 10 connections:
```shell
kubectl apply -f https://raw.githubusercontent.com/mrproliu/sw-satellite-demo-scripts/5821a909b647f7c8f99c70378e197630836f45f7/resources/satellite-hpa.yaml
```

Then, you could see we have 9 connections in one satellite. One envoy proxy may establish multiple connections to the satellite.
```shell
$ kubectl get HorizontalPodAutoscaler -n skywalking-system
NAME       REFERENCE                                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
hpa-demo   Deployment/skywalking-system-satellite   9/10      1         3         1          5m18s
```

#### Scaling Application

The scaling application could establish more connections to the satellite, to verify whether the HPA is in effect.
```shell
kubectl scale --replicas=3 deployment/productpage-v1 deployment/details-v1
```

#### Done!

By default, Satellite will deploy a single instance and a single instance will only accept 11 connections. HPA resources limit one Satellite to handle 10 connections and use a stabilization window to make Satellite stable scaling up. In this case, we deploy the Bookinfo application in 10+ instances after scaling, which means that 10+ connections will be established to the Satellite.

So after HPA resources are running, the Satellite would be automatically scaled up to 2 instances. You can learn about the calculation algorithm of replicas through the [official documentation](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#algorithm-details). Run the following command to view the running status:

```shell
$ kubectl get HorizontalPodAutoscaler -n skywalking-system --watch
NAME       REFERENCE                                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
hpa-demo   Deployment/skywalking-system-satellite   11/10     1         3         1          3m31s
hpa-demo   Deployment/skywalking-system-satellite   11/10     1         3         1          4m20s
hpa-demo   Deployment/skywalking-system-satellite   11/10     1         3         2          4m38s
hpa-demo   Deployment/skywalking-system-satellite   11/10     1         3         2          5m8s
hpa-demo   Deployment/skywalking-system-satellite   6/10      1         3         2          5m23s
```

By observing the “number of connections” metric, we would be able to see that when the number of connections of each gRPC exceeds 10 connections, then the satellite automatically scales through the HPA rule. As a result, the connection number is down to normal status (in this example, less than 10)

```shell
swctl metrics linear --name satellite_service_grpc_connect_count --service-name satellite::satellite-service
```
![](satellite-connection-metrics.png)

repository: apache/skywalking-website

gha_language: CSS

repo_name: apache/skywalking-website


text:
[NEX Protocols](NEX-Protocols.md) > Friends (0x14)
---

Even though this protocol is implemented on Nintendo game servers, it is never actually used. Instead, Nintendo has implemented its own friend services:
* [Friends Protocol (3DS)](Friends-Protocol-(3DS).md)
* [Friends Protocol (Wii U)](Friends-Protocol-(Wii-U))

| Method ID | Method Name |
| --- | --- |
| 1 | [AddFriend](#1-addfriend) |
| 2 | [AddFriendByName](#2-addfriendbyname) |
| 3 | [AddFriendWithDetails](#3-addfriendwithdetails) |
| 4 | [AddFriendByNameWithDetails](#4-addfriendbynamewithdetails) |
| 5 | [AcceptFriendship](#5-acceptfriendship) |
| 6 | [DeclineFriendship](#6-declinefriendship) |
| 7 | [BlackList](#7-blacklist) |
| 8 | [BlackListByName](#8-blacklistbyname) |
| 9 | [ClearRelationship](#9-clearrelationship) |
| 10 | [UpdateDetails](#10-updatedetails) |
| 11 | [GetList](#11-getlist) |
| 12 | [GetDetailedList](#12-getdetailedlist) |
| 13 | [GetRelationships](#13-getrelationships) |

# (1) AddFriend
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |
| Uint32 | uiDetails |
| [String] | strMessage |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (2) AddFriendByName
## Request
| Type | Name |
| --- | --- |
| [String] | strPlayerName |
| Uint32 | uiDetails |
| [String] | strMessage |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (3) AddFriendWithDetails
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |
| Uint32 | uiDetails |
| [String] | strMessage |

## Response
| Type | Name |
| --- | --- |
| [RelationshipData](#relationshipdata) | relationshipData |

# (4) AddFriendByNameWithDetails
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |
| Uint32 | uiDetails |
| [String] | strMessage |

## Response
| Type | Name |
| --- | --- |
| [RelationshipData](#relationshipdata) | relationshipData |

# (5) AcceptFriendship
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (6) DeclineFriendship
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (7) BlackList
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |
| Uint32 | uiDetails |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (8) BlackListByName
## Request
| Type | Name |
| --- | --- |
| [String] | strPlayerName |
| Uint32 | uiDetails |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (9) ClearRelationship
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (10) UpdateDetails
## Request
| Type | Name |
| --- | --- |
| Uint32 | uiPlayer |
| Uint32 | uiDetails |

## Response
| Type | Name |
| --- | --- |
| Bool | %retval% |

# (11) GetList
## Request
| Type | Name |
| --- | --- |
| Uint8 | byRelationship |
| Bool | bReversed |

## Response
| Type | Name |
| --- | --- |
| [List]<Uint32> | lstFriendsList |

# (12) GetDetailedList
## Request
| Type | Name |
| --- | --- |
| Uint8 | byRelationship |
| Bool | bReversed |

## Response
| Type | Name |
| --- | --- |
| [List]<[FriendData](#frienddata)> | lstFriendsList |

# (13) GetRelationships
## Request
| Type | Name |
| --- | --- |
| [ResultRange] | resultRange |

## Response
| Type | Name |
| --- | --- |
| Uint32 | uiTotalCount |
| [List]<[RelationshipData](#relationshipdata)> | lstRelationshipsList |

# Types
## FriendData
| Type | Name |
| --- | --- |
| Uint32 | m_pid |
| [String] | m_strName |
| Uint8 | m_byRelationship |
| Uint32 | m_uiDetails |
| [String] | m_strStatus |

## RelationshipData
| Type | Name |
| --- | --- |
| Uint32 | m_pid |
| [String] | m_strName |
| Uint8 | m_byRelationship |
| Uint32 | m_uiDetails |
| Uint8 | m_byStatus |

[List]: NEX-Common-Types#list
[String]: NEX-Common-Types#string
[ResultRange]: NEX-Common-Types#resultrange-structure

repository: amicuchu/NintendoClientsWiki

repo_name: amicuchu/NintendoClientsWiki


text:
This is a very simple demo of using Cookie Authentication is ASP.NET Core

It uses kind of a cheap hack for you to login, I was too lazy to make a login form for this.

To authentication, do:

http://localhost:5000/home/login?password=dumb

Where you replace localhost with your machine name or whatever is appropriate for where you are running.

It shows how to create policies and setup the Cookie Authentication Middleware in the Startup.cs file.

In the HomeController you can see the code for the authentication and sign in.

repository: tylerlrhodes/CookieAuth

repo_name: tylerlrhodes/CookieAuth


text:
How to Define Custom Options for Entity Properties
==================================================

This article explains how to define custom options for any entity property and
how to use those options in the `list`, `search` or `show` views.

Default Property Options
------------------------

Properties included in the `fields` option of any entity can define several
options (`property`, `label`, `template`, `type`, `help` and `css_class`):

```yaml
# app/config.yml
easy_admin:
    entities:
        User:
            class: AppBundle\Entity\User
            list:
                fields:
                    - { property: 'email', label: 'Contact' }
                    - { property: 'biography', help: 'Markdown allowed' }
                    # ...
```

Custom Property Options
-----------------------

Adding custom options is as simple as defining their names and values in each
property configuration. Imagine that you want to translate the contents of a
property called `name` in the `list` view. To do so, define a custom option
called `trans` which indicates if the property should be translated and another
option called `domain` which defines the name of the translation domain to use:

```yaml
# app/config.yml
Product:
    class: AppBundle\Entity\Product
    label: 'Products'
    list:
        fields:
            - id
            - { property: 'name', trans: true, domain: 'messages' }
            # ...
```

Using Custom Property Options in Templates
------------------------------------------

Property templates receive a parameter called `field_options` which is an array
that contains all the options defined in the configuration file for that
property. If you add custom options, they will also be available in that
`field_options` parameter. This allows you to add custom logic to templates very
easily.

Considering that the `name` property is of type `string`, override the built-in
`field_string.html.twig` templateto add support for the `trans` and `domain`
options:

```twig
{# app/Resources/views/easy_admin/field_string.html.twig #}

{% if field_options.trans|default(false) %}
    {# translate fields defined as "translatable" #}
    {{ value|trans({}, field_options.domain|default('messages')) }}
{% else %}
    {# if not translatable, simply include the default template #}
    {{<EMAIL_ADDRESS>}}
{% endif %}
```

If the custom logic is too complex, it may be better to render the property with
its own custom template to not mess the default templates too much. In the
following example, the backend wants to display a collection of tags with the
colors configured for the property.

Since this business logic is too specific, it's better to not reuse the
corresponding default template. The solution is to define a custom template just
for this property and make use of the `label_colors` custom option:

```yaml
# app/config.yml
easy_admin:
    entities:
        Product:
            class: AppBundle\Entity\Product
            list:
                fields:
                    # ...
                    - { property: 'tags', template: 'tag_collection.html.twig',
                        label_colors: ['primary', 'success', 'info'] }
```

The custom `tag_collection.html.twig` template would look as follows:

```twig
{# app/Resources/views/easy_admin/tag_collection.html.twig #}

{% set colors = field_options.label_colors|default(['primary']) %}

{% for tag in value %}
    <span class="label label-{{ cycle(colors, loop.index) }}">{{ tag }}</span>
{% endfor %}
```

And this property would be rendered in the `list` view as follows:

![Default listing interface](../images/easyadmin-design-customization-custom-data-types.png)

repository: Danzabar/EasyAdminBundle

repo_name: Danzabar/EasyAdminBundle


text:
# fluids

This library is a tiny glue layer for observable events.

- Create a tree of observable values
- Let parent nodes send arbitrary events to children (for maximum flexibility)
- Stay small yet provide helpers for easier integration

## API

- `hasFluidValue(target: any): boolean`

  Returns `true` if the given value is a fluid object.

- `getFluidValue(target: any): any`

  Returns the current value of the fluid object (if possible),
  otherwise the argument is passed through as-is.

- `getFluidConfig(target: any): FluidConfig`

  Returns the `FluidConfig` object that allows for observing the argument

- `setFluidConfig(target: object, config: FluidConfig): void`

  Defines the hidden property that holds the `FluidConfig` object.
  Newer calls override older calls.

- `addFluidObserver(target: object, observer: FluidObserver): () => void`

  Attach an observer to a fluid object, and get an unsubscribe function back.
  Returns `undefined` if the first argument is not a fluid object.

## Types

- `class FluidValue<T, EventType> implements FluidConfig<T, EventType>`

  The `FluidValue` class is convenient for automatic TypeScript compatibility
  with other libraries using `fluids`. The constructor sets each instance as
  its own `FluidConfig` (eg: `setFluidConfig(this, this)`), so you need to
  implement the `onParentChange(event)` method in your subclass. Remember,
  you're not required to use `FluidValue` at all. The `setFluidConfig`
  function is enough if you don't want TypeScript interop OOTB.

- `interface FluidConfig<T, EventType>`

  The `FluidConfig` interface has three methods: `get`, `addChild`, and
  `removeChild`. These methods act as a compatibility layer between
  libraries and their custom event-based solutions.

- `interface FluidObserver<EventType>`

  The `FluidObserver` interface has one method, `onParentChange(event)`,
  which is called by observed `FluidConfig` objects whenever a new event
  occurs (like a "change" event).

- `interface FluidEvent<T>`

  The basic shape that every `FluidConfig` event must adhere to.

- `interface ChangeEvent<T>`

  The basic shape that every "change" event must adhere to.

## `FluidValue` example

Extending the `FluidValue` class guarantees free TypeScript compatibility.

Your `FluidValue` subclass must provide the following methods:
```ts
// Get the current value
get(): T

// Add an observer
addChild(child: FluidObserver<Event>): void

// Remove an observer
removeChild(child: FluidObserver<Event>): void
```

Here's a basic example:

```ts
import { FluidValue, FluidObserver } from 'fluids'

export class MyObservable<T> extends FluidValue<T> {
  protected _value: T
  protected _observers = new Set<FluidObserver>()
  constructor(value: T) {
    super()
    this._value = value
  }
  get() {
    return this._value
  }
  set(value: T) {
    this._value = value
  }
  addChild(observer: FluidObserver) {
    this._observers.push(observer)
  }
  removeChild(observer: FluidObserver) {
    this._observers.push(observer)
  }
}
```

## `FluidConfig` example

This example adds observability to a ref object, like what `React.useRef` returns.

Any object can conform to the `FluidConfig` interface **without needing to change its public API.**

```ts
import { setFluidConfig, FluidObserver, FluidEvent } from 'fluids'

/** Create a ref object that can be observed */
function createRef(current) {
  const ref = {}

  // Observer tracking
  const children = new Set<FluidObserver>()
  const emit = (event: FluidEvent) =>
    children.forEach(child => child.onParentChange(event))

  // Change tracking
  const get = () => current
  Object.defineProperty(ref, 'current', {
    enumerable: true,
    get,
    set: newValue => {
      if (current !== newValue) {
        current = newValue
        emit({
          type: 'change',
          parent: ref,
          value: newValue,
        })
      }
    }
  })

  // Observer API
  setFluidConfig(ref, {
    get,
    addChild: child => children.add(child),
    removeChild: child => children.delete(child),
  })

  return ref
}
```

## `FluidObserver` example

This example shows how to observe a fluid object.

```ts
import { addFluidObserver } from 'fluids'

const ref = createRef(0)
const stop = addFluidObserver(ref, {
  onParentChange(event) {
    if (event.type === 'change') {
      console.log(event.value, event.parent)
    }
  }
})

ref.current++
stop()
ref.current++
```

branch_name: refs/heads/main


text:
# CSI metrics exposed

Here is the list of csi volume metrics exposed by k8s:-

```
kubelet_volume_stats_available_bytes
kubelet_volume_stats_capacity_bytes
kubelet_volume_stats_inodes
kubelet_volume_stats_inodes_free
kubelet_volume_stats_inodes_used
kubelet_volume_stats_used_bytes
```

repository: kmova/monitoring

branch_name: refs/heads/develop

repo_name: kmova/monitoring


text:
---
layout: post
title: Openstax Textbooks Reader on GitHub using Kramdown
tags: books
---

Over the weekend I converted all of our popular textbooks from our new HTML format to [kramdown](http://kramdown.gettalong.org) (a Markdown variant that allows classes) and [tossed it on GitHub](http://philschatz.com/books/).

Since [gh-pages](http://pages.github.com) automatically converts kramdown files to HTML using [jekyll](http://jekyllrb.com), I also created a [tiny book reader](https://github.com/philschatz/book-viewer) and linked to it in the book.
Click the screenshot to [*read all the books!*](http://philschatz.com/books/)

[![image](https://cloud.githubusercontent.com/assets/253202/3496567/d862235a-05e3-11e4-876e-80bdcf516938.png)](http://philschatz.com/books/)

# GitHub.com Bonus Features

By using a Markdown variant instead of HTML, there are several things GitHub provides for free.

- repo pages will render instead of just showing the markup
- links between pages *just work*
- diffs and Pull Requests render and add changebars/colors to make the changes clearer

## Clickable Table of Contents

[![image](https://cloud.githubusercontent.com/assets/253202/3496418/69b29f3a-05e2-11e4-8b72-de53ad718207.png)](https://github.com/philschatz/anatomy-book/blob/gh-pages/SUMMARY.md)

**Note:** `{: ...}` is kramdown's way of adding classes

[The markup](https://github.com/philschatz/anatomy-book/blob/gh-pages/SUMMARY.md):

    ---
    layout: page
    ---

    1.  {: .preface} [Preface](contents/m46844.md)
    2.  {: .part} Unit 1: Levels of Organization
        1.  {: .chapter} [An Introduction to the...](contents/m45981.md)
            1. [Overview of Anatomy and Physiology](contents/m45983.md)
            2. [Structural Organization of the...](contents/m45985.md)
            3. [Functions of Human Life](contents/m45986.md)

        2.  {: .chapter} [The Chemical Level of...](contents/m45996.md)
            1. [Elements and Atoms: The Building...](contents/m45998.md)
            2. [Chemical Bonds](contents/m46000.md)
            3. [Chemical Reactions](contents/m46004.md)

## Rendering in normal GitHub.com

The markdown files render on GitHub.com repo pages ([see example page on GitHub](https://github.com/philschatz/anatomy-book/blob/gh-pages/contents/m46377.md))

[![image](https://cloud.githubusercontent.com/assets/253202/3496428/7f327c7c-05e2-11e4-9c90-6a4dde496886.png)](https://github.com/philschatz/anatomy-book/blob/gh-pages/contents/m46377.md)

and the [raw kramdown](https://raw.githubusercontent.com/philschatz/anatomy-book/gh-pages/contents/m46377.md):

    ---
    title: "Anatomy of Selected Synovial Joints"
    layout: page
    ---

    <div data-type="abstract" markdown="1">
    By the end of this section, you will be able to:

    * Describe the bones that articulate together to form selected...
    * Discuss the movements available at each joint
    * Describe the structures that support and prevent excess movements

    </div>

    # Articulations of the Vertebral Column

    In addition to being held together by the intervertebral discs,
    vertebrae also articulate with each other at synovial joints formed
    between the superior and inferior articular processes called
    **zygapophysial joints**{: data-type="term" :} (facet joints) (see
    [link text](m46383.md#fig-ch09_01_02)). These are
    plane joints that provide for only limited motions between the
    vertebrae. The orientation of the articular processes at these joint
    varies in different regions of the vertebral column and serves to
    determine the types of motions available in each vertebral region.
    The cervical and lumbar regions have the greatest ranges of motions.

    ![Figure alt text](../resource/fig1.jpg "Figure caption"){: data-title="Figure Title" .half-page :}

### Rendered Changes

GitHub will also render the changes of kramdown files instead of showing a diff ([see example of deleting a chapter](https://github.com/philschatz/anatomy-book/commit/bd695b8c50bbdfccc4d892e521b7f8b48d1b55ba?short_path=b4f3573#diff-b4f3573b2f24d5af026c33acf52ff716))

[![image](https://cloud.githubusercontent.com/assets/253202/3496449/b512862a-05e2-11e4-8264-19ed006cfa77.png)](https://github.com/philschatz/anatomy-book/commit/bd695b8c50bbdfccc4d892e521b7f8b48d1b55ba?short_path=b4f3573#diff-b4f3573b2f24d5af026c33acf52ff716)

# Features

- unobtrusive design
- big next/prev buttons
- reading progress bar
- keyboard left/right buttons work
- Table of Contents shows pages you have read
- spinner shows when page is loading
- just 300 lines of code
- works on any book hosted on *cnx.org* too! [Examples of all the Openstax College books](http://mountainbunker.org/~schatz/ed/simple-webview/)
- search button reuses the Table of Contents
- does not require building the files for `gh-pages` (unlike [GitBook](http://gitbook.io))

# Book Reader Links

- [reader code](https://github.com/philschatz/book-viewer) (it's in `./dist` because I'm lazy)
- [reader for Openstax books hosted in GitHub](http://philschatz.com/books/)
- [Openstax College books using the *same* reader code](http://mountainbunker.org/~schatz/ed/simple-webview/) (loads archive.cnx.org)

# Notes

- the reader works even if files are stored in places other than `/contents` (thanks to URI.js)
- kramdown does not support `<figure>` (unless you HTML escape it) so I added attributes to the `<img>` tag (for the title) and JS converts them to a `<figure>`
- internal links do not work on the OSC cnx.org books because I ran out of time and used `<base href="archive.cnx.org">`
- Jekyll requires all Markdown that is converted to HTML *must* have a YAML header
- I removed autogenerated ids on paragraphs and lists so the kramdown is cleaner (`id="fs-id*"`)


text:
# Formio

# Configuration Notes

## href Links to other pages

The formio form is loaded in a iframe for various capability and control purposes.

When `href` is required in a form, you typically want to redirect the parent window and not the iframe.

To do this, ensure that your `href` has the attribute `target="_top"`; this will ensure that the parent window is redirected whene the user selects the `href` link.

![formio config for href link of parent window](resources/href-link.png)

repository: DigitalState/Documentation

repo_name: DigitalState/Documentation


text:
OnlineRandomForest
==================

Online Random Forest (for regression)

This is a simple prototype for an Online Random Forest machine learning algorithm.
The Driver file has an interactive and batch testing mode.

umm...

    cabal install hmatrix easyplot rvar random-fu

Hi ConCon,

Anyhow, the code is pretty fucked up right now because I keep changing my mind about things.
Unfortunately, right now my approach has TONS of free parameters (meaning difficult for an end user, which is not the case for RF.  I hope to fix this).

Heres an extremely breif tutorial.

There is a Driver.hs file which allows you to try it out on a fake example with a bunch of different parameter configurations and then graph the error rate as it goes.

Basically, baked_online_predictor takes
1) a datafile
2) a list of (random forest,string) pairs to test on (the string is used for graphing)
3) the dimension of the problem (yes, this is known both from the data file and the forest, but whatever)
4) the bin size for reporting error rates, which you should just set to like 100

The datafile is a file where each line is one observation, which is a bunch of floats separated by spaces.
The first float is the thing being predicted, and the rest are the predictors.
baked_online_predictor will see each line and try to predict on it before observing the datapoint.
One way of generating these things is with mixtureModel.
If you want one to get started with, type "mixtureStdBasis 10", which gives a ten dimensional problem, and saves it in the file "mix_10".

The function some_tree_predictors is an example of using it, although there are too many forests used so the graph you get is a big convoluted.

The alpha_grow is the minimum number of guys you need to make a new branch, the alpha_prune is the minimum number of guys you need on a branch to prune it.  beta_grow is the improvement you need to justify a new branch and beta_prune is the improvement you need to justify pruning it.
For offline trees you don't need these params since you can look at all the data at once, but online trees not only require this, but are somehow a bit sensitive to it.

Anyway, if you want to play with it this might be enough to get you started.  If you want more details, just email me.


text:
The aim os this project is to save all the knowledge needed for installing and setup a linux working environment in form os automatic installation procedures.

The goals are:

- DRY.
- Save time in maintenance operations.
- Save the effort and hours invested in solving issues.
- Use as a knowledge database.

The project uses Fabric to specify and automate configurations.

Each folder contains specific files that are needed outside the official package (as licenses or code patches) when is needed.

The project is in a very early state, but it reach my initials goals:

- Working environment: Sublime 3, WingIDE, python tools, etc.
- Editing and publishing tools: pelican, markdown, etc.
- Propietary drivers: NVIDIA cards, etc.
- Web browsers: Opera, Vivaldi and Chrome.

Contributions and sugestions are very welcome!

repository: asteriogonzalez/housekeeping

repo_name: asteriogonzalez/housekeeping


text:
---
layout: post
title: Central limit theorem
tags: statistics
---

The central limit theorem states that, given a parent distribution of mean $\mu$ and variance $\sigma^2$, if you independently draw $k$ samples from this parent distribution, average them to get one value, then repeat this process over and over, the resulting distribution formed by these values will approach a normal distribution of mean $\mu$ and variance $\sigma^2/k$ as $k \to \infty$.

Because I have always had poor intuition for this theorem, below I'll discuss (with very simple examples) how this convergence to normality occurs for different parent distributions, along with some illustrations.

## Uniform distribution
Consider a uniform distribution defined between 0 and 1 (note that an unbounded uniform distribution is a trivial example of a distribution with an infinite variance for which no convergence would occur).

The most curious feature is the kinked line that occurs when $k=2$. The functional form of this kinked line can be found theoretically. In particular, we need to obtain the total probability that two values $x_1$ and $x_2$ sum to a particular value $u$ (for now, this is defined between $0$ and $2$ -- we will divide by $2$ later to turn this sum into an average). This can be written as follows:

$$P(u = x + y) = \int_{-\infty}^{\infty} P(x) P(u - x) \ dx$$

(note this is a convolution -- for a fantastic, more thorough explanation of the above equation, see [Understanding Convolutions](http://colah.github.io/posts/2014-07-Understanding-Convolutions/) by Christopher Olah). $P(x) = 1$, so the integrals should be quite simple. The only trick here is to solve for the piecewise behavior... $P(u - x)$ is only $1$ where $0 \leq u - x \leq 1$, or in other words, where $ -1 +u \leq x \leq u $, and $P(x)$ is only $1$ when $0 \leq x \leq 1$. Therefore, between $0 \leq u \leq 1$, we have

$$P(u) = \int_{0}^{u} dx = u$$

And for $u>1$, we have

$$P(u) = \int_{-1+u}^{1} dx = 2 - u$$

We can define $G(u)$, the probability density function for the average of two values $x_1$ and $x_2$, by moving the threshold to $u=1/2$ and doubling the density, as follows:

$$G(u) =
\begin{cases}
4 u & 0 \leq u \leq 0.5 \\
4 - 4u & 0.5 \leq u \leq 1 \\
0 & \text{otherwise}
\end{cases}
$$

... consistent with our figure!

## Beta distribution
The beta distribution is a normalized version of:

$$P(x) \sim x^{\alpha-1} (1-x)^{\beta -1}$$

Because it's only defined between 0 and 1, it certainly satisfies the condition that the second moment is finite.

## Log-normal distribution
Curiously, this theorem also works for the log-normal distribution:

$$P(x) \sim \frac{1}{x} e^{- (\ln x - \mu)^2 / 2 \sigma^2}$$

It takes a bit longer for the convergence to occur, so let's try log-spacing the number of samples to average.

Because there are so many distributions that are log-normal, the central limit applied here has some strong practical ramifications. For example, if price distributions tend to be log-normal, then a revenue distribution for which multiple orders are placed could perhaps be more accurately described as one of these in-between shapes. Or, at a coarser scale, this could mean that the amount of money spent by a person each year is normally distributed.

Or, considering [prices tend to be between power law or log normal](https://link.springer.com/content/pdf/10.1007%2Fs00191-009-0142-z.pdf), if log normality is observed, could it somehow be a step on the way to normalcy, from the summing of random independent variables that are power-law distributed? Let's move onto power-law distributions to find out!

## Pareto distributions
The "Pareto distribution" is a power law distribution $P(x) \sim 1/x^a$. To avoid the singularity at $x=0$, this is often shifted over, or only defined starting at $x=1$ (or a different value, depending on the scaling).

`numpy.random.pareto` is used here. The formula appears to be:

$$P(x) \sim \frac{1}{(1+x)^{\alpha+1}}$$

For $\alpha = 2$, for which the variance is finite, the normal distribution appears.

The first few steps indeed look somewhat log-normal!

However, for $\alpha =1$, for which the variance is infinite, no normal distribution appears to be in sight.

While this empirical test does seem to indicate that no convergence to a normal distribution occurs, it does still [converge](https://en.wikipedia.org/wiki/Central_limit_theorem) to an [alpha-stable distribution](https://en.wikipedia.org/wiki/Stable_distribution).

## Final notes
Note that [the density of the sum of two independent real-valued random variables equals the convolution of the density functions of the original variables](https://en.wikipedia.org/w/index.php?title=Illustration_of_the_central_limit_theorem&gettingStartedReturn=true). We could therefore recreate these graphs analytically.

A couple remaining open questions:

* **How does the peak of the pdf change with $N$?**
 This peak increases in height as the distribution narrows as mentioned above, but for the log-normal case it decreases before it increases.
* **Why does the power law appear to converge slower than the other distributions?**
 Explicitly, the question amounts to: how does changing the functional form of a convolving function affect the convergence of a series of such convolutions to the convergent distribution? A simple way to approach this would be to check the Kolmogorov-Smirnov distance between the the converging distribution and the normal distribution (note: the [Berry-Esseen theorem](https://en.wikipedia.org/wiki/Berry%E2%80%93Esseen_theorem) appears to do just that!), but perhaps there exists a more elegant way to do this, while also answering the first question above...


text:
# Rancher Developer Starter Template

The aim of this project is to create a template development solution that will run on Docker for Mac and Docker for Windows that
creates an instance of Rancher server locally and then adds itself as a host. It also creates an environment script to export the
variables needed to connect to Rancher from the command line.

*Note Docker Toolkit is not currently supported*

#Usage

## Pre-requisites Mac

* Install [Docker for Mac](https://docs.docker.com/docker-for-mac/)
* brew install make

## Pre-requisites Windows 10

* Install [Docker for Windows](https://docs.docker.com/docker-for-windows/)
* Install [Redhat Cygwin](http://www.redhat.com/services/custom/cygwin/) add make sure to select "curl" and "make"

## Access from localhost

Running rancher on Docker for Mac/Windows will not expose ports on localhost. To get round this you can use the
proxy. This creates an haproxy instance that passes through the ports to the host so that you can use localhost.
The haproxy is generated automatically from a list of ports either configured in the make file or by passing it
explicitly via the ports variable.

You cannot run the proxy before deploying your system into Rancher. If you do the containers that expose ports
to the host will not work because the proxy blocks their use. Once the system is deployed then you should start
the proxy.

## Make file

To make things easy I created a Makefile wrapper for the script which can then be extended. The commands are:
* `make up <ports=>` stand up Rancher server, Rancher Agent, Create Dev API key and generate a rancher-env.sh script
* `make down` tear it all down  build-proxy
* `make build-proxy <ports=>` build the proxy, either change the ports in the makefile or pass it to the make command
* `make run-proxy <ports=>` run the proxy
* `make remove-proxy` stop and remove the proxy
* `make generate-rancher-env` re-generate a rancher-env.sh script

# Thanks
Chris Urwin - providing me with various snippets to perform Rancher tasks.

repository: muo-ltd/local-dev-rancher-template

repo_name: muo-ltd/local-dev-rancher-template


text:
## Guitar Tabs

Full Stack Webapp which allows the users to do the following
- Sign Up
- Login
- Search Songs
- Add Songs
- Edit Songs
- View Songs
- About Page

## Web Stack

Material UI

Vue.js

Express.js

Sequlize

repository: ajomadlabs/GuitarTabs

repo_name: ajomadlabs/GuitarTabs


text:
JProbe
======

*JProbe is a servlet for inspecting running thread inside a servlet container and detect those gone haywire.*

About
-----
JProbe is designed to be a minimalisitc tool to list and inspect all threads running on the JVM. Packaged in a deployable web archive, JProbe can be deployed next to a running servlet and inspect this servlet's threads. This allows for finding threads that are stuck with a minimal impact on the running web applications.

Build using Ant
---------------
JProbe can be build using [Apache Ant](http://ant.apache.org/). The build script is located in `build.xml` and can be configured in `build.properties`.

The minimum configuration required is setting the `servlet-libs.location` property to point to a folder containing the Java Servlet library (servlet-api.jar). This library is bundled with all Java Servlet implementions, e.g. [Apache Tomcat](http://tomcat.apache.org/).

A war (Web ARchive) file can be bundled by running: `ant all`

*The default password is 's3cr3t' and should be set to a secure password in web.xml before deployment in a production environment.*

License
-------
JProbe is licensed under the terms of the Apache License version 2.0, see the included LICENSE file.

Author
------
[Leonard van Driel](http://www.leonardvandriel.nl/)

repository: mohanaraosv/JProbe

repo_name: mohanaraosv/JProbe


text:
---
layout: post
title: '18: Ports by example'
audioURL: e018-ports-by-example
categories: episodes
excerpt_separator: <!--more-->
---

It's time to dive right in with several uses of Ports in Elm, what better way than by example! Ports are Elms way of interacting with the outside world, currently with JavaScript, but the sky is the limit.

<!--more-->

<!-- TOC -->

* [More Ports](#more-ports)
  * [Ports, the Actor Model.](#ports-the-actor-model)
  * [Ports, the example](#ports-the-example)
* [Picks](#picks)
* [Resources](#resources)
* [Follow](#follow)

<!-- /TOC -->

## More Ports

### Ports, the Actor Model.

* Ports is not server/http, but rather the Actor Model
* Talking with messages. Messages In, Messages Out.
* Erlang/Elixir, webworkers, Scala
* Let's just have 1 port. Singular. Let's **NOT** wrap the JS API.
* Murphy's "Outside Info" of a single pair of ports
* The Actor Model
  * The actor model is a conceptual model to deal with concurrent computation.
  * The actor is just a _thing_ that receives a message and then does something based on that message.
  * The actor can have private state, that cannot be changed directly by another actor.
  * Actors work in a system of actors. If you only have a single actor, then there's no one to send messages to or worse, no one to get messages to. In our case, Elm is an actor, and JS is an actor, sending messages back and forth to each other.
  * Messages sent to an actor or processed sequentially.
  * Needs 3 things of computation:
    1. getting something done
    2. store things
    3. be able to communicate
  * Actors can only do one of the following:
    1. Create more actors;
    2. Send messages to other actors; (Factorial, many messages at the same time, relates to concurrency)
    3. Designates what to do with the next message, to itself. (This relates to the state of the actor)
  * Mailboxes, concurrency, and queues
  * EO Wilson, "One ant, is no ant"
  * Actors will need an address, in order send messages to.
  * "the actor has to agree to chop off it's own head. "

### Ports, the example

* Before Diving into the implementation of the Ports Example, I wanted describe what it was like to get stuff from Elm and be able to simply `console.log` it out into my familiar dev tools.
* Incoming Cmd hand to the runtime in Elm.
* Outgoing Subscription, message tagger.
* Change the module declaration to `port module`
* We've got `port check : String -> Cmd msg`
  * function that takes a string and returns a Cmd msg. This is used by our update function,
  * so, with the onClick we take the string in the current model word, which has been captured with the onChange, and we and in the update function we call `check` and the model.word creating a Cmd for the Elm runtime.
  * as a port it will handle making that available to JS from `app.ports.check.subscribe` which takes a callback to invoke whenever Elm passes a message. This is the sort of gritty part of the actor model setup.
  * So we've wired up the outgoing message from Elm,
* Now we've got `port suggestions : (List String -> msg) -> Sub msg`
  * This is going to be our List of Strings from JS, or is JS speak, Array
  * so you can convert that list of strings to your Msg type immediately.
  * This makes it easy to deal with in your update function, but it is just for convenience.

The JS Side

```js
var app = Main.fullscreen();
/*
Methods we delcared as 'port' in Elm will be available on app.ports that
will get invoked/called when we send a cmd message out via Elm.
*/
app.ports.check.subscribe(function(word) {
  var suggestions = spellCheck(word); //--> invoke JS function, return array of strings
  app.ports.suggestions.send(suggestions); //--> sending to the Elm sub
});

function spellCheck(word) {
  console.log(word); // Words from Elm!!!!!
  return [];
}
```

The Elm Side

```Elm
port check : String -> Cmd msg

---- Subscription ---
-- port for listening for suggestions from JavaScript

port suggestions : (List String -> msg) -> Sub msg

subscriptions : Model -> Sub Msg
subscriptions model =
    suggestions Suggest

...

type Msg
    = Change String
    | Check
    | Suggest (List String)

...

Suggest newSuggestion ->
            ( Model model.word newSuggestion, Cmd.none )
```

## Picks

* [State of Elm](https://discourse.elm-lang.org/t/state-of-elm-2018-questions-draft-1/161/11)

## Resources

* [Elm Docs Docs](https://guide.elm-lang.org/interop/javascript.html)
* [Murphy Randle — The Importance of Ports](https://www.youtube.com/watch?v=P3pL85n9_5s)
* [Actor Model logic programming](https://en.wikipedia.org/wiki/Actor_model#Relationship_to_logic_programming)
* [The Actor Model](http://www.brianstorti.com/the-actor-model/)
* [Actors from the guy who invented them, Hewitt, Meijer and Szyperski](https://www.youtube.com/watch?time_continue=11&v=7erJ1DV_Tlo)
* [Elm Port Example ](https://github.com/jtomchak/elm-port-example)

## Follow

* JavaScript to Elm
  * Twitter: [@jstoelm](https://twitter.com/jstoelm)
  * Email: [hello<EMAIL_ADDRESS>* Jesse Tomchak
  * Twitter: [@jtomchak](https://twitter.com/jtomchak)

repository: jtomchak/jstoelm

repo_name: jtomchak/jstoelm


text:
* [General directory structure](#general-directory-structure)
* [HTML](#html)
  * [Directory structure](#directory-structure)
* [Stylesheets](#stylesheets)
  * [Directory structure](#directory-structure-1)
  * [Coding standards/style and linting](#coding-standardsstyle-and-linting)
* [JavaScript](#javascript)
  * [Directory structure](#directory-structure-2)
  * [Coding standards/style and linting](#coding-standardsstyle-and-linting-1)
* [Images](#images)
* [Linting](#linting)

# Project structure

## General directory structure
All source files are located in [src](./src). Continue reading to see how the workflow is working. Once the project is built, you can find the generated site in `dist`. In [_gulpfile](./_gulpfile) you will find parts of the [gulpfile.babel.js](./gulpfile.babel.js).

## HTML
The HTML is written in [Pug (formerly Jade)](https://pugjs.org) which is getting compiled to browser understandable HTML. It can be filled with data provied via JSON files in [_data](./src/_data). The compiling is done with a [Gulp](http://gulpjs.com) task. This task can also watch for changes so that the source files are getting recompiled when changed.

### Directory structure
The source files are directly placed under the [src](./src) directory like the [index.pug](./src/index.pug). You can also nest them in sub directories which will be reflected in the URL (**Note**: You can access the built HTML files without the suffix .html because of the [mod_rewrite](https://httpd.apache.org/docs/current/mod/mod_rewrite.html) in the [.htaccess](./src/.htaccess#L27)). Because of Pug you can seperate your code in seperate files and include them. For these files I created the sub directory [_templates](./src/_templates). Files in this directory like the [base.pug](./src/_templates/base.pug) will not get compiled to HTML, they should only be included in other files like the [index.pug](./src/index.pug#L1).

## Stylesheets
The styles are written in [SCSS](http://sass-lang.com) which is getting compiled to browser understandable CSS. The compiling is done with a [Gulp](http://gulpjs.com) task. This task can also watch for changes so that the Stylesheets are getting recompiled when changed. The SCSS will also getting concatenated. You will get a `styles.css` which should contain all SCSS which will be used on every page. This file will get included on every page (look at [base.pug](./src/_templates/base.pug#L53)). You can also create seperate SCSS files like the [front.bundle.scss](./src/assets/styles/front.bundle.scss) which contains SCSS only used on this page. You can individually include them on the corresponding pages (look at [front-page.pug](./src/_templates/pages/front.pug#L9)).

### Directory structure
You will find the source files under [src/assets/styles](./src/assets/styles). The directory is seperated in three main directories. In [_atoms](./src/assets/styles/_atoms) you will place all basic styling of the HTML tags like `<table>` or `<ul>`. In [_base](./src/assets/styles/_base) you will place all variables, mixins, global classes and so on. In [_modules](./src/assets/styles/_modules) you will place your modules of your site like the header or a footer. The files are getting compiled to `dist/assets/css`. Files and directories beginning with an underscore are only getting included in other files.

### Coding standards/style and linting
I included [stylelint](https://stylelint.io/) as linter for the styles source files. You will find the rules in [.stylelintrc](./.stylelintrc). You should configure your editor/IDE to lint your code on the fly.

## JavaScript
The JavaScript is written in [ES6/ECMAScript 2015](http://www.ecma-international.org/ecma-262/6.0/index.html) which is getting compiled to browser understandable [ES5](http://www.ecma-international.org/ecma-262/5.1). The compiling is done with a [Gulp](http://gulpjs.com) task. This task can also watch for changes so that the JavaScript is getting recompiled when changed. The JavaScript will also getting concatenated. You will get a `global.js` which should contain all JavaScript which will be used on every page. This file will get included on every page (look at [base.pug](./src/_templates/base.pug#L61)). You can also create seperate JavaScripts files like the [front.js](./src/assets/scripts/front.js) which contains JavScript only used on this page. You can individually include them on the corresponding pages (look at [front-page.pug](./src/_templates/pages/front.pug#L11)).

### Directory structure
You will find the source files under [src/assets/scripts](./src/assets/scripts).

### Coding standards/style and linting
I included [ESLint](http://eslint.org) as linter for the JavaScript source files. You will find the rules in [.eslintrc](./.eslintrc). You should configure your editor/IDE to lint your code on the fly.

## Images
_wip_

## Linting
The linters don't run when using `npm start` or `npm run default`. It will run when using `npm test`. We recommend that you use feature branches and a CI which runs this tasks when you push to it. This decision was made to improve the duration of the watch tasks. You should configure your editor/IDE to lint your code on the fly (there are plugins for every popular editor/IDE).

repository: simonknittel/static-site-generator

gha_language: JavaScript

repo_name: simonknittel/static-site-generator