text:
# TrendTerms
## Visualisation of terms in document abstracts

The TrendTerms package analyses the relevant terms in the abstract of an eprint and
its related eprints, including their time evolution. Word statistics and positions
from the Xapian search engine index are used to select the relevant terms and to
calculate the relations between the terms. The terms and their relations are
saved in graph XML files. The graph XML files are visualised using ProcessingJS
or can be further analysed.

The package consists of
- a script and plugin to generate the graph files
- the ProcessingJS program (including the Processing source files) to visualize the
  TrendTerms graph files
- some JavaScript helper code
- phrase files in English and German

For a demo, see e.g. http://www.zora.uzh.ch/97201/

## Requirements

- EPrints 3.3.x with installed Search::Xapian extension. For details see
https://wiki.eprints.org/w/API:EPrints/Plugin/Search/Xapian
- JQuery is required for scaling the visualisation canvas.

## General setup

The setup procedure consists of the following steps

- Installation of the required files
- Configuration of the look of the visualisation
- Initial generation of trendterms data
- Linking the trendterms_data directory
- Initial test
- Full generation of trendterms data
- Running updates

## Installation

Copy the content of the bin and cfg directories to the respective
{eprints_root}/archives/{yourarchive}/bin and {eprints_root}/archives/{yourarchive}/cfg
directories.

## Configuration

### Configure the position of the TrendTerms visualisation

archives/{archive}/cfg/cfg.d/z_trendterms.pl allows you to configure the position
of the TrendTerms box in the summary page of an eprint.

If you use a custom archives/{archive}/cfg/cfg.d/eprint_render.pl to render the
summary page, you can turn off the box in archives/{archive}/cfg/cfg.d/z_trendterms.pl by
setting

$c->{plugins}->{"Screen::EPrint::Box::TrendTerms"}->{params}->{disable} = 1;

You can then copy the code in the render() and make_trendtermsbox() in
archives/{archive}/cfg/plugins/EPrints/Plugin/Screen/EPrint/Box/TrendTerms.pm over to
eprint_render.pl and adapt it to your needs.

### Edit the look of your visualisation

You can configure the look of your visualisation (fonts, colors, position of the HUDs, ...) in
archives/{archive}/cfg/static/trendterms/configuration.xml:

```XML
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <callback>
    <base_url>/cgi/search/advanced?abstract=</base_url>
  </callback>
  <node>
    <font name="Verdana" size="12"/>
  </node>
  <edge>
    <line_weight>2.0</line_weight>
    <color>FF808080</color>
  </edge>
  <colors>
    <set id="1" name="general">
      <color id="1" name="red" value="80FF0000"/>
      <color id="2" name="orange" value="80FF6633"/>
      <color id="3" name="yellow" value="80FFCC33"/>
      <color id="4" name="green" value="8000FF00"/>
      <color id="5" name="blue" value="800000FF"/>
      <color id="6" name="pink" value="80FF33CC"/>
    </set>
  </colors>
  <version>
    <font name="Verdana" size="8" color="FFC0C0C0"/>
  </version>
  <timeline>
    <background>C0E0E0E0</background>
    <font name="Verdana" size="8" color="E0808080"/>
  </timeline>
  <depthlayers xshift="5" yshift="5" color="20282828"/>
  <trendanalysis start="2008" end="9999"/>
  <huds>
    <zoombutton size="30" margin="6" padding="5" align="right" valign="top" background="C0FFFFFF"
      color="E0808080"/>
    <trenddirhud size="30" margin="6" padding="5" align="left" valign="top" background="C0FFFFFF"
      color="E0808080" distcolor="E033CCFF" font="Verdana" high="↗" mid="→" low="↘"/>
    <trendsizehud size="30" margin="6" padding="5" align="left" valign="center"
      background="C0FFFFFF" color="E0808080" distcolor="E033CCFF" font="Verdana" high="H" mid="M"
      low="L"/>
  </huds>
</configuration>
```

Some explanations:

The `<base_url>` element contains the callback URL fragment for an advanced search in the
eprints' abstracts when a user clicks on a term.

All fonts sizes are in pt.

All color values are 4-Byte hexadecimal values in the order ARGB (alpha, red, green, blue channel).
A value of FF for the alpha channel means that the color is opaque, smaller values increase transparency.

The `<colors>` and `<set>` element group the colors used for the nodes. Currently, only one
set is read by the visualisation. You can define as many `<color>` elements as you wish. The
`name` attributes can be filled with arbitrary values, they are just here for your documentation.

`<depthlayers>` defines the pseudo-3D effect used to display the time variation of a hovered node.
The `xshift` and `yshift` attributes are in pixels. The `color` attribute is not used currently (it was in
a earlier version to indicate depth).

`<trendanalysis>` is used to define the year boundaries for the analysis of the timelines used
by the filter HUDs ("head-up-display"), e.g. trend direction and trend size HUD. If `end="9999"`
is specified, the upper boundary is determined by the TrendTerms visualisation.

`<huds>` stands for "head-up-displays". These are the buttons that are displayed when a
user mouse-hovers the canvas. There is a HUD for the zoom buttons, one for the filter by
trend direction, and one for the filter by trend size. For the latter two, the TrendTerms
visualisation applies analysis algorithms to the data.

The detailed format (including XML Schemas) is described in
[TrendTerms_io.pde](https://github.com/eprintsug/TrendTerms/blob/master/Processing/TrendTerms/TrendTerms_io.pde)

### Restart the web server

After you have edited the configuration files, restart the web server.

## Initial generation of TrendTerms data

To initialize and test your setup, create TrendTerms graph data for a single eprintid (note that
the eprint should have an abstract)

```
sudo -u apache {eprints_root}/archives/{repo}/bin/generate_trendterms {repo} 1 --verbose
```

The generate_trendterms script does the following:
- It creates the directory `{eprints_root}/archives/{archive}/html/trendterms_data`
- For eprint 1, it creates two files in this directory which together make a graph
  - terms_1.xml containing the terms (=nodes of the graph) and its timelines
  - edges_1.xml containing the edges of the graph

(as a side note: the format of the TrendTerms graph files is described in
https://github.com/eprintsug/TrendTerms/blob/master/Processing/TrendTerms/TrendTerms_io.pde
https://github.com/eprintsug/TrendTerms/tree/master/Processing/TrendTerms/xml_schemas )

### Linking the trendterms_data directory

The trendterms_data directory must be linked to all your language-specific HTML trendterms
directories so that the data can be accessed by the Processing code. Do the following:

```
cd {eprints_root}/archives/{repo}/html/{language}/trendterms/
ln -s {eprints_root}/archives/{repo}/html/trendterms_data data
```

Repeat these commands for every language, e.g. en, de, and so on.

### Initial test

In your browser, load the summary page of the eprint for which the TrendTerms data was
created above.

The TrendTerms should be displayed in a box at the end of the summary page.

If the graph is displayed, you are nearly set. If there are less than 100 terms,
the optimization of the term positions should start, otherwise, you will just see 4 or 5
concentric rings of bubbles.

Mouse-hover the graph to check whether the HUDs do appear.
Hover over a term to display the timeline.
Try the zoom buttons.
Click on a term to check whether the advanced search of the publications does work
(see Edit the look of your visualisation).

### The TrendTerms Visualisation

The TrendTerms visualisation uses a physical model to optimize the positions of the terms.
To each pair of terms, a repulsive force is assigned. To each pair of terms that are connected
with an edge (e.g. which have a close distance in the abstract), a spring force is a assigned.
The spring strength is inversely proportional to the distance between two terms. Terms
that have high connectivity are initially placed to the center, terms with low connectivity outside.
The positions are optimized until the total force is below a given threshold.
Optimisation is only carried out for graphs with less than 100 terms.

## Generating the TrendTerms data: The generate_trendterms script

Now a full run to generate the TrendTerms data must be carried out:

```
sudo -u apache {eprints_root}/archives/{repo}/bin/generate_trendterms {repo}
```

Depending on the number of eprints, this may take a long time. Assume
a computation time of about 1 hour per 1'000 eprints.
Be also prepared that your file system has reserved enough space for the graph files.
About 1.5 GB are required for 100'000 eprints.

`sudo -u apache {eprints_root}/archives/{repo}/bin/generate_trendterms --help` lists all options.

## Running updates

There are two options in for running updates with the generate_trendterms script:

`--update`: Generates TrendTerms graph files for a daily segment of eprints, so that
within one month all eprints are processed once, including the newly added eprints.
`--new`: Generates TrendTerms graph files only for eprints that were
added to the live archive yesterday.

For a small repository with a few 1000 eprints, we recommend to use the `--update` option.
This keeps the TrendTerms data for all eprints up-to-date.

For a large repository with several 10000 eprints, we recommend to use `--new` in a nightly
cronjob, which reduces processing time to about 10-15 minutes, and to carry out a
a complete run every 6-12 months.

`--new` has the following effect:
The terms and edges for the new eprints are correctly calculated. However, because IDF
has changed because documents have been added to the total document set
(for IDF, see theory below), the term selection in the graph for the older eprints is now
only approximative, since terms may have been dropped out because their WDF\*IDF value for a
given document may be now lower than the given boundary for selection. However, we assume,
that IDF only changes slowly, but recommend to carry out periodically complete runs to
correct for the IDF change.

## Theory behind the selection of terms

In information retrieval, the product WDF*IDF is a popular expression for the weight of a
term in relevance ranking.

WDF(i) is the within-document-frequency of a term i in a document. It remains constant.
IDF(i) = 1 / DF(i) is the inverse of the document frequency DF(i), which is the number of
documents that contain the term i in a document set. This value changes with addition of
new documents to the document set.

Theoretically, plotting WDF\*IDF against the frequency of a term in a complete document set
should yield a bell-like curve. Very frequent terms have a low WDF\*IDF because of their high
IDF value - these are the stop words. Terms which are highly specific have a low WDF\*IDF too,
because of their low WDF. Thus, choosing terms with a WDF\*IDF above a given boundary should
yield relevant terms that occur also several times in a document set.

The Xapian search engine stores both WDF and DF with the terms. These can be used to
calculate WDF*IDF.

The generate_trendterms script processes an eprint in 3 passes:

Pass 1: Selection of terms within the own abstract

The steps of pass 1 are:

*	Get abstract terms + WDF + positions. Filter stop words.
*	Get DF --> IDF
*	Filter terms: choose WDF*IDF within boundary conditions --> set of terms {F1}
*	For all combinations of terms in the set {F1}, sum up distance-dependent edge weights.
Distance is calculated from positions of the terms
*	Coloring of the term bubbles is done according to the edge weights

Pass 2: Selection of related terms in abstracts of related eprints

This pass is carried out analogously to pass 1. Abstracts of related eprints are found by
carrying out queries for every term in {F1} that has a DF > 1. This leads to n < |{F1}|
queries. The new terms are added to set {F2}.

Pass 3: Get timelines

The timelines are gathered for all terms of the combined set {F1 ∪ F2}

## Editing the Processing code (for developers)

The TrendTerms.pde file being used in the EPrints repo can be found in cfg/static/trendterms
and can be used as is.

If you need or want to modify the visualisation itself, the individual functional modules
of the Processing code are available in Processing/TrendTerms. From these, you can create
the combined TrendTerms.pde with the help of the Processing Development Environment
(aka "Processing").

Processing can be obtained from

https://www.processing.org/download/

After you have installed Processing, the JavaScript mode must be installed as
well. Start Processing, and create a new sketch. In the top right corner of the
Processing window, there is a dropdown menu called "Java". Choose "Add mode ...", and
select "JavaScript Mode" from the list, then choose "Install".

Copy the folder "TrendTerms" to your sketchbook location (see Processing Preferences, where
you can configure the sketchbook location).

Switch to JavaScript mode and edit the modules.

To create TrendTerms.pde, use menu File > Export. A directory web-export is created in the
TrendTerms folder that contains TrendTerms.pde and all other necessary files.
The visualisation can be tested by loading index.html in a Web browser.

repository: buildvoc/TrendTerms

repo_name: buildvoc/TrendTerms


text:
# psps
Small Linux tool as an alternative to ps

Usage:
  psps procname

where procname is the name of the processes we want listed. Example:

$ psps vas

Process 2193
Command: /var/vas/vasd
Parameter: ptcs.vas

Process 7120
Command: /var/vas/vascd
Parameter: core.vas

Process 12667
Command: /var/vas/vasd
Parameter: codecs.vas

Total processes found: 3

That's all it does! Enjoy!

license: MIT

repository: paulscottkildare/psps

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: paulscottkildare/psps


text:
---
id: version-2.0.0-alpha.11-react-native.reactnativeasyncstoragestoragedriver
title: ReactNativeAsyncStorageStorageDriver class
hide_title: true
original_id: react-native.reactnativeasyncstoragestoragedriver
---
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

## ReactNativeAsyncStorageStorageDriver class

<b>Signature:</b>

```typescript
export declare class ReactNativeAsyncStorageStorageDriver implements StorageDriver
```

## Methods

|  Method | Modifiers | Description |
|  --- | --- | --- |
|  [del(key)](./react-native.reactnativeasyncstoragestoragedriver.del.md) |  |  |
|  [get(key)](./react-native.reactnativeasyncstoragestoragedriver.get.md) |  |  |
|  [set(key, value)](./react-native.reactnativeasyncstoragestoragedriver.set.md) |  |  |

repository: SkygearIO/skygear-SDK-JS

gha_language: TypeScript

gha_license_id: NOASSERTION

repo_name: SkygearIO/skygear-SDK-JS


text:
# Basics for coding optimization problems
This document will give the reader an introduction to numerical optimization and the various trade-offs associated with the most common optimization algorithms. We will also cover how to code and evaluate an optimization problem in R. At the end, we will discuss what the cost function looks like for the most well-known statistical models and ML algorithms

## Basics of functional optimization

- All statistical estimators are either minima of some error (aka "cost") function or maxima of some likelihood function
    * Examples:
    * linear regression (OLS)
    * logistic regression
    * naive Bayes classification
    * Multi-Layer Perceptron (MLP) aka artifical neural networks (ANN)
    * support vector machine (SVM)
    * ...
    * many others
- Some estimators (e.g. OLS, Generalized Method of Moments) have **closed form** solutions for the extremum value
    * we covered this in our previous lecture
- Other estimators (e.g. MLE, some non-linear least squares) require **numerical methods** to find the extremum value
    * we also showed this in the previous lecture (the case of logistic regression)
- There are a number of different algorithms to numerically find extrema ("guess-and-check")
- You can use these in R, Python, Julia, Matlab, ...

## Graph of the optimization problem

This is a graph of a likelihood function. The parameters are on the `x` and `y` axes and the likelihood is on the `z` axis.

## How numerical optimization works
- User provides an **objective function** and a **starting value**
- The optimizer then calculates the magnitude and direction of the *gradient vector* at the starting value
- Given these values of the gradient vector, it decides
    * in which direction to move to get closer to the optimum
    * how far it should move to get closer to the optimum
- Given the magnitude and direction of the gradient vector at the new point, it repeats these steps until convergence is achieved

## How to determine convergence? (How do we know when we've "arrived"?)
There are generally three ways to determine if a numerical optimization algorithm has reached the optimum
1. The gradient vector (or Jacobian matrix) is numerically close to 0
    - Intuition: we may be at an extremum if the derivative is 0
2. The location of the previous guess is numerically close to the location of the updated guess
    - Intuition: The optimizer didn't move very far in updating the guess, so we're probably close to the answer
3. The value of the objective function at the previous guess is numerically close to the value of the objective function at the updated guess
    - Intuition: The objective function is flat in the area of the previous and updated guesses, which is kind of like having a gradient of zero

Note that "numerically close" generally refers to being within 10^-6

## Local or global minimum?
- Note that in the previous slide, no mention of second-order conditions was ever made
- Hence, it is unclear if the answer our optimizer gave is a local optimum, a saddle point, or a global optimum
- Luckily, most common statistical estimators have been proven to have **globally concave** objective functions
- If you are unsure whether or not your objective function is globally concave, you can try different sets of starting values and see how much the answers differ

*Note:* "globally concave" simply means that the function has only one "hump" or only one "dip" which means we will eventually get to the only extremum.

## Most commonly used optimization algorithms

- Gradient descent
- Stochastic gradient descent
- BFGS/L-BFGS
- Nelder-Mead (gradient-free simplex methods)
- Gauss-Newton

## When should I use which algorithm?

- It turns out that optimization can often be more of an "art" than a "science"
- Often times your choice of objective function will dictate which optimization algorithm you should choose
- We'll talk more about these considerations a bit later today

## How to use these in R?
R has several packages for calling these commonly used optimizers, as well as many others. In a future problem set, you will get experience with the packages `optimr` and `nloptr`.

Sadly, R is not the best language to be using if you need to optimize your own functions. I recommend Python or (especially) Julia!

## Gradient descent
What is gradient descent?

- Pretend you are on a mountain and there's a valley below that you're trying to get to
- There's a fog over the whole mountain, such that you can't see down the mountain
- You want to get down the mountain as quickly as possible
- You have a special compass that will tell you the steepness of the hill in every direction
- It's costly to look at the compass
- You need to decide how often to look at the compass (i.e. how far to walk before checking the compass and potentially changing direction)

The best way down the mountain is to follow the path of steepest descent while taking the minimum number of stops to look at the compass

## The math behind Gradient descent

- Start with some initial value (call it x0) and some step size (call it γ)
- The new guess of the solution is equal to

x_n = x0 - γ∇f(x0)

where ∇f(x0) is the gradient vector of our objective function f(·)

Repeat this process until convergence (i.e. until ∇f(x0) is arbitrarily close to 0, until x doesn't change across iterations, or until f(x) doesn't change across iterations

## Famous quote
"Victory will never be found by taking the line of least resistance." -- Winston Churchill

- But in actuality, with gradient descent, victory *is* found by taking the line of (steepest descent)!

## Gradient descent in R
Suppose we want to find the minimum of `f(x) = x^4 - 3*x^3 + 2`

```r
# set up a stepsize
alpha <- 0.003

# set up a number of iteration
iter <- 500

# define the gradient of f(x) = x^4 - 3*x^3 + 2
gradient <- function(x) return((4*x^3) - (9*x^2))

# randomly initialize a value to x
set.seed(100)
x <- floor(runif(1)*10)

# create a vector to contain all xs for all steps
x.All <- vector("numeric",iter)

# gradient descent method to find the minimum
for(i in 1:iter){
        x <- x - alpha*gradient(x)
        x.All[i] <- x
        print(x)
}

# print result and plot all xs for every iteration
print(paste("The minimum of f(x) is ", x, sep = ""))
```
Want to do it in Julia? see [here](https://github.com/tyleransom/DScourseS19/blob/master/ModelingOptimization/simpleGD.jl)

## How to write a function in R
How do you write a function in R? The function has four distinct elements:

1. Name
2. Arguments (aka inputs)
3. Statements (i.e. code)
4. Outputs

```r
myfunction <- function(arg1, arg2, ... ){
statements
return(object)
}
```

Note the braces!

## Calling a function
Once you have your function written, you call it as follows:

```r
output <- myfunction(arg1, arg2, ...)
```

where `output` can be any name you'd like, and `arg1` etc. are the specific arguments you're interested in

## Example of a simple function
```r
customMean <- function(x){
mean <- sum(x)/length(x)
return(mean)
}

# Now let's call the function

sample.average <- customMean(iris$Sepal.Width)
print(sample.average)

# Check that it's correct
mean(iris$Sepal.Width)
```

## Function scope

- It doesn't matter what you name your variables when you call the function
- i.e. when we called `customMean()` above, we sent it `iris$Sepal.Width` even though we called the input `x` in the function definition
- This is all to say that when a function is *evaluated*, its new workspace is whatever is sent to the function
- This is important to keep in mind when trying to optimize your own objective function

## Using `nloptr`
Now that we know how to write a function, let's optimize the same function we did before (using gradient descent) but instead now let's use one of the optimizers in the `nloptr` package.

To use the package we need to provide the following information:

- Objective function
- Gradient vector of the objective function
- Algorithm
- Initial value
- Tolerance parameter

## Code
```r
library(nloptr)
## Our objective function
eval_f <- function(x) {
return( x[1]^4 - 3*x[1]^3 + 2 )
}

## Gradient of our objective function
eval_grad_f <- function(x) {
return( 4*x[1]^3 - 9*x[1]^2 )
}

## initial values
x0 <- -5

## Algorithm parameters
opts <- list("algorithm"="NLOPT_LD_LBFGS","xtol_rel"=1.0e-8)

## Find the optimum!
res <- nloptr( x0=x0,eval_f=eval_f,eval_grad_f=eval_grad_f,opts=opts)
print(res)
```
Want to do it in Julia? see [here](https://github.com/tyleransom/DScourseS19/blob/master/ModelingOptimization/LBFGS.jl)

## Comparison with previous result
As you can see, we got the same result using this method as we did with our simple gradient descent.

## What if my function isn't differentiable?
By now you might be wondering how we can find the optimum of a function that is not differentiable. The answer is to use what's called a "simplex method."

A simplex is a multi-dimensional triangle. Rather than find the "steepest" path, the triangle moves around to find lower spots of the objective function. The triangle can do one of many operations:

- reflect
- expand
- contract
- shrink

The alogrithm uses the ordering of the points of the mutli-dimensional triangle to figure out which operation it should next use in order to find the minimum.

## Nelder-Mead code
```r
library(nloptr)
## Our objective function
objfun <- function(x) {
return( x[1]^4 - 3*x[1]^3 + 2 )
}

## initial values
xstart <- 5

## Algorithm parameters
options <- list("algorithm"="NLOPT_LN_NELDERMEAD","xtol_rel"=1.0e-8)

## Find the optimum!
res <- nloptr( x0=xstart,eval_f=objfun,opts=options)
print(res)
```

## Computing OLS and MLE estimates using the `nloptr` package
Now let's see how we can actually optimize a statistical model using the `nloptr` package in R.

OLS objective function:
```r
library(nloptr)
## Our objective function
objfun <- function(beta,y,X) {
return (sum((y-X%*%beta)^2))
# equivalently, if we want to use matrix algebra:
# return ( crossprod(y-X%*%beta) )
}

## Gradient of our objective function
gradient <- function(beta,y,X) {
return ( as.vector(-2*t(X)%*%(y-X%*%beta)) )
}

## read in the data
y <- iris$Sepal.Length
X <- model.matrix(~Sepal.Width+Petal.Length+Petal.Width+Species,iris)

## initial values
beta0 <- runif(dim(X)[2]) #start at uniform random numbers equal to number of coefficients

## Algorithm parameters
options <- list("algorithm"="NLOPT_LD_LBFGS","xtol_rel"=1.0e-6,"maxeval"=1e3)

## Optimize!
result <- nloptr( x0=beta0,eval_f=objfun,eval_grad_f=gradient,opts=options,y=y,X=X)
print(result)

## Check solution
print(summary(lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width+Species,data=iris)))
```

## MLE estimation using the `nloptr` package
Now let's estimate the same model, but this time using the MLE formulation of the linear regression model:

MLE objective function:
```r
library(nloptr)
## Our objective function
objfun  <- function(theta,y,X) {
# need to slice our parameter vector into beta and sigma components
beta    <- theta[1:(length(theta)-1)]
sig     <- theta[length(theta)]
# write objective function as *negative* log likelihood (since NLOPT minimizes)
loglike <- -sum( -.5*(log(2*pi*(sig^2)) + ((y-X%*%beta)/sig)^2) )
return (loglike)
}

## read in the data
y <- iris$Sepal.Length
X <- model.matrix(~Sepal.Width+Petal.Length+Petal.Width+Species,iris)

## initial values
theta0 <- runif(dim(X)[2]+1) #start at uniform random numbers equal to number of coefficients
theta0 <- append(as.vector(summary(lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width+Species,data=iris))$coefficients[,1]),runif(1))

## Algorithm parameters
options <- list("algorithm"="NLOPT_LN_NELDERMEAD","xtol_rel"=1.0e-6,"maxeval"=1e4)

## Optimize!
result <- nloptr( x0=theta0,eval_f=objfun,opts=options,y=y,X=X)
print(result)
betahat  <- result$solution[1:(length(result$solution)-1)]
sigmahat <- result$solution[length(result$solution)]

## Check solution
print(summary(lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width+Species,data=iris)))
```

## Stochastic gradient descent
**Stochastic gradient descent (SGD)** is the most popular way to optimize a neural network. (What does "stochastic" mean, anyway, and why is it called stochastic gradient descent?)

Recall regular gradient descent (often called *batch gradient descent*):

- Start with some initial value (call it x0) and some step size (call it γ)
- The new guess of the solution is equal to

x_n = x0 - γ∇f(x0)

where ∇f(x0) is the gradient vector of our objective function f(·)

Repeat this process until convergence (i.e. until ∇f(x0) is arbitrarily close to 0, until x doesn't change across iterations, or until f(x) doesn't change across iterations

## Stochastic gradient descent (continued)
With SGD, we instead update the gradient for *each observation*.

- each iteration now has N updates within it
- we re-shuffle the data between each iteration

## Let's see how it works, using OLS as our objective function.

Batch gradient descent first:
```r
# set up a stepsize
alpha <- 0.00003

# set up a number of iterations
maxiter <- 500000

## Our objective function
objfun <- function(beta,y,X) {
return ( sum((y-X%*%beta)^2) )
}

# define the gradient of our objective function
gradient <- function(beta,y,X) {
return ( as.vector(-2*t(X)%*%(y-X%*%beta)) )
}

## read in the data
y <- iris$Sepal.Length
X <- model.matrix(~Sepal.Width+Petal.Length+Petal.Width+Species,iris)

## initial values
beta <- runif(dim(X)[2]) #start at uniform random numbers equal to number of coefficients

# randomly initialize a value to beta
set.seed(100)

# create a vector to contain all beta's for all steps
beta.All <- matrix("numeric",length(beta),maxiter)

# gradient descent method to find the minimum
iter  <- 1
beta0 <- 0*beta
while (norm(as.matrix(beta0)-as.matrix(beta))>1e-8) {
    beta0 <- beta
    beta <- beta0 - alpha*gradient(beta0,y,X)
    beta.All[,iter] <- beta
    if (iter%%10000==0) {
        print(beta)
    }
    iter <- iter+1
}

# print result and plot all xs for every iteration
print(iter)
print(paste("The minimum of f(beta,y,X) is ", beta, sep = ""))

## Closed-form solution
print(summary(lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width+Species,data=iris)))
```

Now SGD:
```r
# set up a stepsize
alpha <- 0.0003

## Our objective function
objfun <- function(beta,y,X) {
return ( sum((y-X%*%beta)^2) )
}

# define the gradient of our objective function
gradient <- function(beta,y,X) {
return ( as.vector(-2*X%*%(y-t(X)%*%beta)) )
}

## read in the data
y <- iris$Sepal.Length
X <- model.matrix(~Sepal.Width+Petal.Length+Petal.Width+Species,iris)

## initial values
beta <- runif(dim(X)[2]) #start at uniform random numbers equal to number of coefficients

# randomly initialize a value to beta
set.seed(100)

# create a vector to contain all beta's for all steps
beta.All <- matrix("numeric",length(beta),iter)

# stochastic gradient descent method to find the minimum
iter  <- 1
beta0 <- 0*beta
while (norm(as.matrix(beta0)-as.matrix(beta))>1e-12) {
    # Randomly re-order the data
    random <- sample(nrow(X))
    X <- X[random,]
    y <- y[random]
    # Update parameters for each row of data
    for(i in 1:dim(X)[1]){
        beta0 <- beta
        beta <- beta0 - alpha*gradient(beta0,y[i],as.matrix(X[i,]))
        beta.All[,i] <- beta
    }
    alpha <- alpha/1.0005
    if (iter%%1000==0) {
        print(beta)
    }
    iter <- iter+1
}

# print result and plot all xs for every iteration
print(iter)
print(paste("The minimum of f(beta,y,X) is ", beta, sep = ""))

## Closed-form solution
print(summary(lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width+Species,data=iris)))
```

## Why is SGD useful?
Pros:

- much faster (if properly tuned)
- can be parallelized fairly easily
- can be used for streaming data
- randomness in data across observations can help it find global optima
- Gauss-Newton, L-BFGS can't be used for Big Data sets

Cons:

- convergence is more complicated
- slower in R (since loops are awful)
- must slowly decrease `alpha` (the learning rate) to achieve convergence

## Mini-batch gradient descent

- combines batch gradient descent and SGD
- loop over groups of observations (instead of each observation)
- can improve performance

# Cost functions of commonly used algorithms

## Linear regression (least squares)

```r
objfun <- sum((y-X%*%beta)^2)
objfun <- crossprod(y-X%*%beta)
```

or (if normal MLE):

```r
loglike <- -sum( -.5*(log(2*pi*(sig^2)) + ((Y-X%*%beta)/sig)^2) )
```

## Logistic regression (foundation of neural networks)

```r
loglike <- -sum( (y==1)*(log(h(X%*%beta))) + (y==0)*(log(1-h(X%*%beta))) )

where h(X%*%beta) = exp(X%*%beta)/(1+exp(X%*%beta))
```

Note: h is also known as the *sigmoid function* or the *logistic function*

## Classification and Regression Trees (CARTs)

The cost function for a CART is a little more complicated because it is a two-step process:

1. Minimize within-leaf deviation (i.e. want all observations within a leaf to be as similar as possible)
2. Maximize across-leaf deviation (i.e. want each leaf to be as different as possible from all other leaves)

Have to try various cut points (defining leaf boundaries) and see which configuration best satisfies the above criteria

## Naive Bayes (classification or regression)

This is called "naive" because the researcher makes the "naive" assumption that all of the X's are independent. This is a powerful assumption because it significantly reduces the computation cost.

To estimate the model, simply compute the mean and standard deviation of each X variable

Then plug into the pdf of the normal distribution and create predictions of y that satisfy Bayes' rule

## Support Vector Machine (SVM)

The cost function is similar to least squares, except it does not count errors that are sufficiently small:

```r
costfun <- sum(max(0,1-y*(w%*%X))) + lambda/(sum(w^2))
```

where `w` is the support vector which divides the sample space

## Penalizing complexity
As we will discuss in the coming lectures, prediction problems do better the more predictors there are. But we should also penalize complexity.

The three ways to penalize complexity are:

1. LASSO model (aka L1 regularization) : (absolute value of each additional parameter counts agains the objective function)
2. Ridge model (aka L2 regularization) : (squared value of each additional parameter counts agains the objective function)
3. Elastic net model: A weighted average of 1. and 2.

To penalize complexity, just add this term to the cost function (if minimizing) or subtract from objective function (if maximizing)

# Helpful resources

* [An overview of gradient descent optimization algorithms](http://ruder.io/optimizing-gradient-descent/)

gha_language: Jupyter Notebook

gha_license_id: MIT


text:
---
title: EdWord and EdWeb
date: 2016-06-21 17:46:46
tags:
- Symbol Library
- Text to Speech
- Switch Input Software
- Alternative and Augmentative Communication
- Text input Projects
- Accessing the Web
- Symbols
- Alternative Access
- Possible-404
---

{% alert info no-icon %}
EdWord is a talking word processor that allows you to combine text with symbols.  EdWeb is a talking web browser that can display web pages as a combination of text and symbols.
{% endalert %}

### Links:
- Help
- <a<EMAIL_ADDRESS>Address
- Project Home Page

### Download: http://www.oatsoft.org/Software/edword-and-edweb/downloads/<IP_ADDRESS>/Sense_EdWord-EdWeb_setup.exe (Warning: possible dead-link)

repository: willwade/OATSoft

repo_name: willwade/OATSoft


text:
# NGLessPy: NGLess in Python

**Note** As of Oct 2017, NGLess is considered beta software (we believe it
works, but there may still be a few rough edges), while NGLessPy is alpha
software (very experimental).

## Install

```bash
    pip install NGLessPy
```

(or from source, using the standard `python setup.py install`)

## Basic Tutorial

This tutorial expects a certain familiarity with general ngless concepts and
functions.

We start by importing the `NGLess` object:

```python
    from ngless import NGLess
```

We now build an `NGLess.NGLess` object, giving it the version of ngless we wish
(this is like the version declaration at the top of an NGLess file:

```python
    sc = NGLess.NGLess('0.8')
```

To simplify the rest of the script, we are going to use the short name `e` to
refer to the environment of the script we are generating. The environment is
what will hold the ngless variables we will use:

```python
    e = sc.env
```

We can import ngless modules using the `import_` function (using `name` and
`version`):

```python
    sc.import_('mocat', '0.0')
```

Now, we can use all NGLesss functionality. Functions get an underscore at the
end, like this:

```python
    e.sample = sc.load_mocat_sample_('testing')
```

`preprocess_` is special because it takes a block in ngless, which maps to it
taking a function in Python. We can use decorator syntax to do it all
compactly:

```python
    @sc.preprocess_(e.sample, using='r')
    def proc(bk):
        # bk is the block environment, where `r` is defined
        bk.r = sc.substrim_(bk.r, min_quality=25)
```

Now, we map against `hg19` and filter it. As you can see, ngless functions are
called with an extra underscore and variables are kept in the `e` object:

```python
    e.mapped = sc.map_(e.sample, reference='hg19')
    e.mapped = sc.select_(e.mapped, keep_if=['{mapped}'])

    sc.write_(e.mapped, ofile='ofile.sam')
```

Finally, we execute the resulting script:

```python
    sc.run(auto_install=True)
```

This will even install NGLess if it is not available in the PATH.

## Full script

```python
    from ngless import NGLess

    sc = NGLess.NGLess('0.8')
    e = sc.env

    sc.import_('mocat', '0.0')

    e.sample = sc.load_mocat_sample_('testing')
    @sc.preprocess_(e.sample, using='r')
    def proc(bk):
        # bk is the block environment, where `r` is defined
        bk.r = sc.substrim_(bk.r, min_quality=25)

    e.mapped = sc.map_(e.sample, reference='hg19')
    e.mapped = sc.select_(e.mapped, keep_if=['{mapped}'])

    sc.write_(e.mapped, ofile='ofile.sam')

    sc.run(auto_install=True)
```


text:
# General.numerical_replace

| Parameter Name | Default value | Types | Value range | Description |
| :---: | :---: | :---: | :---: | :---: |
| string |No default | string | N/A  | A string built from numbers |
| *args | No Default | N/A | N/A | The strings that will replace the numbers in the given string |

Returns the `string` after replacing the numbers appearing in it with the given strings in `*args` according to the order.

## Code Examples:
```
>>> string = "I went to the 0 to buy 1, 2, and 3"
>>> numerical_replace(string, "supermarket", "eggs", "milk", "flour")
'I went to the supermarket to buy eggs, milk, and flour'
```
</br> </br>
```
>>> string = "If 1 = 1 then 0 must equal 5"
>>> numerical_replace(string, "5")
'If 1 = 1 then 5 must equal 5'
```
</br></br>
```
>>> string = "If 1 = 1 then 0 must equal 3"
>>> numerical_replace(string, "1", "2", "3", "4")
'If 2 = 2 then 1 must equal 4'
```


text:
# Hashmap or Hash tables

### 1. Hashmap or Hash tables

Many applications require a dynamic set that supports only the dictionary operations INSERT, SEARCH, DELETE. Under reasonable assumptions, the average time to search for an element in a hash table is O\(1\). The worst case is O\(n\) when there are as many as n collision when we are doing searching.

A hash table typically uses an array of size proportional to the number of keys actually stored. Instead of using the key as an array index _directly_, the array index is computed from the key using a _hash function_. When more than one key maps to the same array index, this is called _“collision”_, which is usually handled by “Chaining” that we store a linked list in the collide index position.

**Hash function**

Hash function maps the universe U of keys into the slots of a hashtable T\[0..m-1\]. With hash function the element is stored in h\(k\).

`h: U -> {0,1,..., m-1},` where the size `m` of the hash table is much less than `|U|` . With hash function, the range of array indices and the size of the array is reduced.

**Collision**

When two keys hash to the same slot, this situation is called collision. The idea to solve collision is to avoid collisions.

One idea is to make `h` appear to be “random”, thus avoiding collisions or at least minimizing their number. Because `|U|>m` , there must be at least two keys that have the same hash value; avoiding collisions altogether is therefore impossible. Thus, we use a well-designed, “random”-looking hash function to minimize the number of collisions. However, we still need a method for resolving the collisions that do occur.

**Resolving Collision by Chaining**

In chaining, we place all the elements that hash to the same slot into the same linked list.![](https://miro.medium.com/max/60/0*4B9Ia13byYuGwKpc.png?q=20)![](https://miro.medium.com/max/1063/0*4B9Ia13byYuGwKpc.png)Fig 1: Hash table and chaining. Figure from [here](http://abutko.github.io/418project/).

**Common Used Hash Functions**

A good hash function satisfied the condtion of simple uniform hashing: each key is equally likely to has to any of the `m` slots.

Step 1: interpreting keys as nature numbers N = {0,1,2, …}. For string or character, we might translate pt as the pair of decimal integers `(112,116)` with their ASCII character; then we express it as a radix-128 integer, then the number we get is \(112\*128\)+116 = 14452.

Step 2: Define hashing functions

* The division method, `h(k) = k mod m`
* The multiplication method, `h(k)=`⌊m\(kA mod 1\)⌋. `kA mod 1` means the fractional part of `kA` , which can be noted as `{kA}` , and equals to `kA-`⌊ kA ⌋, `0<A<1` . e.g. for 45.2 the fractional part of it is .2.

[**Implementation with Python List**](http://interactivepython.org/runestone/static/pythonds/SortSearch/Hashing.html)

**Python built-in hashmap data structure**

* [dictionary](https://www.laurentluce.com/posts/python-dictionary-implementation/)
* [set](https://docs.python.org/2/library/sets.html)

The set classes are implemented using dictionaries. Accordingly, the requirements for set elements are the same as those for dictionary keys; namely, that the element defines both [`__eq__()`](https://docs.python.org/2/reference/datamodel.html#object.__eq__) and [`__hash__()`](https://docs.python.org/2/reference/datamodel.html#object.__hash__). As a result, sets cannot contain mutable elements such as lists or dictionaries. However, they can contain immutable collections such as tuples or instances of [`ImmutableSet`](https://docs.python.org/2/library/sets.html#sets.ImmutableSet). For convenience in implementing sets of sets, inner sets are automatically converted to immutable form, for example, `Set([Set(['dog'])])` is transformed to `Set([ImmutableSet(['dog'])])`.

* OrderedDict

Standard dictionaries are unordered, which means that any time you loop through a dictionary, you will go through every key, but you are not guaranteed to get them in any particular order.

The OrderedDict from the collections module is a special type of dictionary that keeps track of the order in which its keys were inserted. Iterating the keys of an orderedDict has predictable behavior. This can simplify testing and debugging by making all the code deterministic.

* defaultDict

Dictionaries are useful for bookkeeping and tracking statistics. One problem is that when we try to add an element, we have no idea if the key is present or not, which requires us to check such condition every time.

```text
dict = {}
key = "counter"
if key not in dict:
    dict[key]=0
dict[key] += 1
```

The defaultdict class from the collections module simplifies this process by pre-assigning a default value when a key does not present. For different value type it has different default value, for example, for int, it is 0 as the default value.

```text
from collections import defaultdict
dict = defaultdict(int)
dict['counter']+=1
```

**Time Complexity for Operations**

Search, Insert, Delete: O\(1\). Check [here ](https://www.ics.uci.edu/~pattis/ICS-33/lectures/complexitypython.txt)for more details.

### 2. Queue, Stack, and Heap

Stacks and queue are dynamic lists in which the element removed from the list by the DELETE operation is prespecified. The operation of PUSH and POP takes O\(1\) time complexity. These two structure are good for problems that we deal with element in either FIFO or LIFO manner.

**queue:** it is first in, first out, FIFO, which can be used to implement iterative BFS. The following code is the basic implementation with Python.

```text
q = []
q.insert[0,v]
q.pop()
```

**stack:** the element deleted is the most recently inserted, Last in first out, LIFO, which can be used to implement iterative DFS. The following code is the basic implementation with Python.

```text
stack = []
stack.append(v)
stack.pop()
```

[**Deque** ](https://docs.python.org/2/library/collections.html#collections.deque)\(operation from both side\): In Python, the _deque_ class from the _collections_ module is a double-ended queue. It provides constant time opearations for inserting ore removing items from its beginning or end. It can be used to implement both stack and queue structure we mentioned above and usually is more time efficient than the above implementation. Mainly because inserting or removing elements from the head of a list takes linear time.

```text
from collections import deque
#1. implement Queue
fifo=deque()
fifo.append(l)
fifo.popleft()
#2. implement Stack
lifo = deque()
lifo.append(l)
lifo.pop()
```

**Heap**

The \(binary\) heap data structure is an array that we can view as a nearly complete binary tree. The tree is completely filled on all levels except possibly the lowest, which is filled from left up to a point.![](https://miro.medium.com/freeze/max/60/0*rcNN6jqCrCsKNGRU.gif?q=20)![](https://miro.medium.com/max/726/0*rcNN6jqCrCsKNGRU.gif)\(a\) binary tree \(b\) an array

As we can see we can implement either the max-heap or the min-heap as an array. Because the tree is complete, the left child of a parent \(at position _p_\) is the node that is found in position 2_p_ in the list. Similarly, the right child of the parent is at position 2_p_+1 in the list. To find the parent of any node in the tree, we can simply use Python’s integer division. Given that a node is at position _n_ in the list, the parent is at position _n_/2.

There are two kinds of binary heaps: max-heaps and min-heaps. In both kinds, the values in the nodes satisfy a _heap property_. For max-heap, the property states as for every node `i` other than root.

```text
A[PARENT(i)]>= A[i];
```

Thus, the largest element in a max-heap is stored at the root.

Heap can be used into heapsort and a priority-queue data structure. Operations include:

* MAX-HEAPIFY, runs in O\(lgn\), is the key to maintaining the max-heap property
* BUILD-MAX-HEAP, runs in linear time, produces a maxheap from an unordered input arrary
* MAX-HEAP-INSERT, HEAP-EXTRACT-MAX, HEAP-INCREASE-KEY, and HEAP-MAXIMUM, runs in O\(lgn\) time, allow the heap data structure to implement a priority queue.

[**heapq**](https://docs.python.org/2/library/heapq.html)**:** heapq from collections is an implementation of heap, which can be used to maintain a priority queue. Operations include heappush, heappop, and nsmallest. heapq in python to maintain a priority queue with O\(logn\)

```text
heap = []
heappush(heap, 5)
heappush(heap, 3)
heappush(heap, 7)
heappush(heap, 4)
```

Items are removed by the highest priority or say the lowest number first. Also, accessing the 0 index of the heap will return the smallest item.

```text
print(heappop(a), heappop(a), heappop(a), heappop(a))
3 4 5 7
assert heap[0] == nsmallest(1, heap)[0] == 3
```

More materials can be found [here](https://www.geeksforgeeks.org/heap-queue-or-heapq-in-python/).

[**Monotonous Stack**](https://zhuanlan.zhihu.com/p/26465701): For monotonous increasing stack, which only allow the increasing element to be put in the stack, smaller one came will kick out the larger one in the previous position, untill we found one that is smaller than the current element. For the monotonous decreasing stack, the larger elements will force the stack to kick out the previous smaller element untill larger one found. In monotonous stack, we only operate at the end of the stack. To summarize, monotonous stack has two features:

* monotonic
* when adding new element, we will delete all the previous elements that break the first monotonic feature.
* For increasing stack: we can find the first element to the left that is larger than current element
* For decreasing stack: we can find the first element to the left that is smaller than current element.

### 3. Linked List

Like arrays, Linked List is a linear data structure. Unlike arrays, linked list elements are not stored at contiguous location; the elements are linked using pointers.![](https://miro.medium.com/max/60/0*GoKtsAI7vqlhLJ0w.png?q=20)![](https://miro.medium.com/max/1063/0*GoKtsAI7vqlhLJ0w.png)

**Why Linked List?**
Arrays can be used to store linear data of similar types, but arrays have following limitations.
**1\)** The size of the arrays is fixed: So we must know the upper limit on the number of elements in advance. Also, generally, the allocated memory is equal to the upper limit irrespective of the usage.
**2\)** Inserting a new element in an array of elements is expensive, because room has to be created for the new elements and to create room existing elements have to shifted.

**Advantages over arrays**
**1\)** Dynamic size
**2\)** Ease of insertion/deletion

**Drawbacks:**
**1\)** Random access is not allowed. We have to access elements sequentially starting from the first node. So we cannot do binary search with linked lists.
**2\)** Extra memory space for a pointer is required with each element of the list.

For Linked List, we can only iterate over elements, for python code example:

```text
#Definition for singly-linked list.
class ListNode(object):
     def __init__(self, x):
         self.val = x
         self.next = None#First, construct a head and point and record the head
pointer=head = ListNode(None)
#loop through all node
while pointer:
    print pointer.val
    pointer = pointer.next
```

Dummy Node in Linked List

* Remove Duplicates from Sorted List II
* Reverse Linked List II
* Partition List

Basic Linked List Skills

* Sort List
* Reorder List

Two Pointers in Linked List \(Fast-slow pointers\)

* Merge K Sorted Lists

### [4. Tree](https://medium.com/algorithms-and-leetcode/solving-tree-problems-on-leetcode-d0b7a9b4a7a4)

Binary Tree and Binary Search Tree, please see my following post.[Solving Tree Problems on LeetCodePart of this great node comes from blog:medium.com](https://medium.com/algorithms-and-leetcode/solving-tree-problems-on-leetcode-d0b7a9b4a7a4)

### Reference

\[1\] Cormen, Thomas H. _Introduction to algorithms_. MIT press, 2009.

\[2\] Géron, Aurélien. _Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems_. “ O’Reilly Media, Inc.”, 2017.

\[3\][https://www.geeksforgeeks.org/heap-queue-or-heapq-in-python/](https://www.geeksforgeeks.org/heap-queue-or-heapq-in-python/)


text:
---
layout: page
title: Principles of Electrical Engineering 1
permalink: /education/principles-of-ee-1/
keywords: 14:332:221, principles, electrical, engineering
description: A page full of notes and useful links for succeeding in principles of electrical engineering 1.
---

Here you will find various links and notes related to Principles of EE 1 at Rutgers University.

Notes come from the version of the class which was taught during the fall of 2015.

Links:

- [**Course Notes**](notes/)
- [Rutgers ECE Course Page](http://www.ece.rutgers.edu/332_221)
- [Wikipedia - Nodal Circuit Analysis](https://en.wikipedia.org/wiki/Nodal_analysis)
- [Wikipedia - Mesh Current Circuit Analysis](https://en.wikipedia.org/wiki/Mesh_analysis)

gha_language: CSS


text:
# Behaviour Clonning

# Introduction

Eight laps of track 1 (4 in each direction) were used as initial training data.

More data was generated using Data.py. It does the following things to generate data:

1. It takes as input, images from all three cameras.
2. For images from the left camera, a correction of 0.2 is added to the measurements
3. For images from the right camera, a correction of 0.2 is subtracted from the measurements
4. All images are horizontally flipped, and their measurements are also reversed. This is done to help the model generalise better.
5. The Data.py stores these images into a Numpy array.

 ![Histogram](ReadMePics/P1.png)

# Architecture

I have used the Nvidia's CNN architecture. For this the following operations were performed

1. Cropping the top and bottom of image
2. Resize to 66 by 200 pixels
3. Normalise between -.5 and .5
4. 5 layers of convolution
5. 3 fully connected layers
6. Dropout with keep\_prob = 0.75 in first fully connected layer

I also used LeNet, but, no amount of training and tweaking could get it to steer the car successfully about the last corner.

Initially, learning parameter is 0.0001. For fine tuning, the learning rate is set at 0.00001.

 ![NN model](ReadMePics/P2.png)

I was unable to generate good data on my hardware, hence I ended up using Udacity's dataset

repository: shantnavagarwal/BehaviourCloning

repo_name: shantnavagarwal/BehaviourCloning


text:
This is a Python 3 implementation of the Visual Information Fidelity (VIF) Image Quality Assessment (IQA) metric. This code uses the [pyrtools](https://pyrtools.readthedocs.io/en/latest/) library to compute the Steerable Pyramid decomposition, and [integral images](https://en.wikipedia.org/wiki/Summed-area_table) to accelerate filtering by averaging filters.

To compute VIF, follow this template.
```
# img_ref, img_dist are two images of the same size.
from vif_utils import vif
print(vif(img_ref, img_dist))
```

repository: abhinaukumar/vif

branch_name: refs/heads/main

repo_name: abhinaukumar/vif


text:
---
published: true
type: post
title: Network Analysis with Netlytic
---
## Netlytic Workshop at the University of Ottawa with Elizabeth Dubois

### Workshop Details:

This workshop introduced participants to the free online tool Netlytic. We reviewed how to set up data collection from various social media sites and collect a small Twitter dataset. Next we reviewed the fundamentals of online social network analysis using Netlytic.

### Basics of Social Network Analysis - Online and Offline

**What is Social Network Analysis?**

Social network analysis helps us figure out relationships between people through their relationships online which helps us link people's relationships online. People have different ways in which their interactions can be mapped - pointing to what can be learned from their attributes. We are interested in the flow of information and the relationship between different nodes. General idea is to mathematically map out social relationships to determine what the influence is. There are small communities within network graphs. When doing online social network analysis we look at the data left when individuals nagivate the web but we are not able to see what is going on in the real world.

**Netlytic**

Netlytic is a community-supported text and social networks analyzer that can automatically summarize and discover social networks from online conversations on social media sites. It is made by researchers for researchers, no programming/API skills required.

**Tips**

- Name your dataset your search term so you do not forget your search term
- Different social networks have different rules
- You can only collect 1000 tweets every fifteen minutes so that servers do not get overloaded with requests
- After the first time you can pull the tweets that occur after you started you will only get what occured within that 15 minutes.
- You can export data into a CSV and open it in Excel to access more data.
- You can also do a text analysis - create wordmaps of what conversations are occuring in the community you are looking at.
- The more ties the denser your charts will be.
- Different layouts are useful for different kinds of graphs - use the one easiest for your to interpret.
- Stars can show those engageing in bot behaviour or acting as promoters.
- You cannot go back in time with Twitter's data collection unless you pay a data broker.
- When you search a hashtag that is trending you often get a lot of people piggybacking on it.

**Definitions:**

- Node: An individual or user
- Edges (Also called ties):The link between individuals - mentions, retweets, likes, etc.
- Cascade: When one piece of information flows through a network.
- In Degree: How many people engage with it.
- Out Degree: How many people does it engage with.
- Total Degree: All engagement.
- Clusters: Where nodes are grouped together.
- Centrality:
- Betweeness Centality: Connection between different groups.
- Weak Ties: People you are connected to but do not know well. Weak ties are very important because they connect us with different communities.
- Homopholy: Most of the time we connect with people who are just like us.
- Hubs: The key players in a conversation.
- Trace Data: Data we leave as we navigate the web.
- API: A backdoor to the network which researchers can use to pull data.
- Streaming: Opens a door to Twitter and says push through the tweets as they happen.
- Search API: Check in every 15 minutes to allow you to push the data through.

[An example of Netlytic analysis by Elizabeth Dubois.](https://medium.com/@lizdubois/the-canadian-open-dialogue-community-codf16-77edf0c35c9d)


text:
# Compare Experiments

DVC makes it easy to iterate on your project using Git commits with tags or Git
branches. It provides a way to try different ideas, keep track of them, switch
back and forth. To find the best performing experiment or track the progress,
[project metrics](/doc/command-reference/metrics) are supported in DVC (as
described in one of the previous chapters).

Let's run evaluate for the latest `bigrams` experiment we created in previous
chapters. It mostly takes just running the `dvc repro`:

```dvc
$ git checkout master
$ dvc checkout
$ dvc repro evaluate.dvc
```

`git checkout master` and `dvc checkout` commands ensure that we have the latest
experiment code and data respectively. And `dvc repro`, as we discussed in the
[Reproduce](/doc/tutorials/get-started/reproduce) chapter, is a way to run all
the necessary commands to build the model and measure its performance.

```dvc
$ git commit -am "Evaluate bigrams model"
$ git tag -a "bigrams-experiment" -m "Bigrams experiment evaluation"
```

Now, we can use `-T` option of the `dvc metrics show` command to see the
difference between the `baseline` and `bigrams` experiments:

```dvc
$ dvc metrics show -T

baseline-experiment:
    auc.metric: 0.588426
bigrams-experiment:
    auc.metric: 0.602818
```

DVC provides built-in support to track and navigate `JSON`, `TSV` or `CSV`
metric files if you want to track additional information. See `dvc metrics` to
learn more.


text:
# ExpressionAble

### Background

**The context**: Gene-expression levels help in determining cellular function. To understand these roles, scientists have performed thousands of gene-expression studies using microarray assays and next-generation sequencing. Such data are typically stored in tabular files (with rows and columns).

**The problem**: Different functional-genomics repositories and preprocessing tools store gene-expression data in a cacophony of file formats. Analysis tools (such as Python, R, and Excel) do not support all these formats natively. It is time consuming and labor intensive for bioinformaticians to translate data between file formats and identify samples that match specific criteria.

**The solution**: We developed the ExpressionAble tool to reduce the time from data acquisition to analysis for researchers studying gene expression. This tool can help researchers more easily work with their own data as well as the immense gene-expression resources in public databases.

### expressionable Python Module

This is the official repository for the ExpressionAble Python module, which allows for:

* Transforming tabular data sets from one format to another.
* Querying large data sets to extract useful data.
* Selecting columns/features to include in the resulting data set.
* Merging data sets of various formats into a single file.
* Gzipping resulting data sets, as well as the ability to read gzipped files.

**Most people will prefer to use the [ExpressionAble command-line tool](https://github.com/srp33/ShapeShifter-CLI), which combines the features of ExpressionAble with the ease of the command-line.**

Basic use of the Python module is described below. You can see full documentation on the functions in this module [here](https://shapeshifter.readthedocs.io/en/latest/).

### Install

`pip install expressionable`

### Basic Use

After installing, import the ExpressionAble class with `from expressionable import ExpressionAble`. An ExpressionAble object
represents the file to be transformed. It is then transformed using the `export_filter_results` method. Here is a simple
example of file called `input_file.tsv` being transformed into an HDF5 file called `output_file.h5`, while filtering
the data on sex and age:

```python
from expressionable import ExpressionAble

my_expressionable = ExpressionAble("input_file.tsv")
my_expressionable.export_filter_results("output_file.h5", filters="Sex == 'M' and Age > 40")
```

Note that the type of file being read and exported to were not stated explicitly but inferred by ExpressionAble based on
the file extensions provided. If necessary, `input_file_type` and `output_file_type` can be named explicitly.

### Contributing

We welcome contributions that help expand ExpressionAble to be compatible with additional file formats. If you are
interested in contributing, please follow the instructions [here](https://github.com/srp33/ExpressionAble/wiki/Adding-Support-for-Additional-File-Types-in-ExpressionAble).

### Currently Supported Formats

##### Input Formats:

* CSV
* TSV (samples as rows, variables as columns)
* Transposed TSV (variables as rows, samples as columns)
* JSON
* Excel
* HDF5
* Parquet
* MsgPack
* Stata
* Pickle
* SQLite
* ARFF
* GCT
* GCTX
* PDF
* Gene Expression Omnibus
* Kallisto (RNA-Sequencing)
* Salmon (RNA-Sequencing)
* STAR (RNA-Sequencing)
* HT-Seq (RNA-Sequencing)
* CBio Portal (RNA expression format)

##### Output Formats:

* CSV
* TSV
* JSON
* Excel
* HDF5
* Parquet
* MsgPack
* Stata
* Pickle
* SQLite
* ARFF
* GCT
* RMarkdown notebook
* Jupyter notebook

gha_language: HTML


text:
# No Global jQuery

**TL;DR** Do not use global `$` or `jQuery`.

In general, we want application code to reference the version of jQuery that's been directly pinned
to the version of Ember used. This helps avoid version conflicts, and ensures that code inside modules
isn't reliant on global variables.


text:
---
layout: page
title: Senses
description: A sonic sculpture
img: assets/img/senses.jpg
importance: 2
category: soundart
---

The traditional five senses depicted in the context of the loudspeaker.

        {% include figure.html path="assets/img/senses.jpg" title="Title" class="img-fluid rounded z-depth-1" %}

    Senses, a sonic sculpture - Ted Apel - sandpaper, fruit, mirror, loudspeaker, perfume bottle top


text:
> :DarkLight
> > :InDark
> >
> >
>
> > :InLight
> >
> >

# Refs

You can reference DOM elements with variables:

```tsx | --no-wmbar
const div = <div>Hellow World!</div>;
setTimeout(() => div.textContent = 'GoodBye, Blue Sky!', 2000);

renderer.render(div).on(document.body);
```

But typically its easier to have all the DOM tree described in one place. For that purpose,
you can use `ref` objects to reference elements (and generally any value) that will be resolved
later:

```tsx | --no-wmbar
/*!*/import { ref } from 'render-jsx/common';

// ...

/*!*/const div = ref();
/*!*/setTimeout(() => div.$.textContent = 'GoodBye, Blue Sky!', 2000);

renderer.render(
/*!*/  <div _ref={div}>Hellow World!</div>
).on(document.body);
```

---

## Generic Use

Refs can also be used to reference any other not-yet-resolved value:

```ts | --no-wmbar
const answer = ref<number>();

// ...

answer.resolve(42);
```

---

## Safety

If you access a ref's `.$` property before the ref is resolved, it will throw an error:

```ts | --no-wmbar
const myRef = ref();
/*~*/myRef.$/*~*/;
```

If a ref is also resolved multiple times, it will throw an error:

```ts | --no-wmbar
const myRef = ref<string>();
myRef.resolve('hellow');
/*~*/myRef.resolve('goodbye!')/*~*/;
```

You can check whether a ref is resolved via its `.resolved` property:

```ts | --no-wmbar
const myRef = ref<string>();
myRef.resolved;  // --> false

myRef.resolve('hellow');
myRef.resolved;  // --> true
```

> :ToCPrevNext

repository: loreanvictor/render-jsx

repo_name: loreanvictor/render-jsx


text:
# Pholiota lignicola (Peck) Jacobsson, 1989 SPECIES

#### Status
SYNONYM

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Mycotaxon 36(1): 138 (1989)

#### Original name
Agaricus lignicola Peck, 1872

### Remarks
null

repository: mdoering/backbone

path: /life/Fungi/Basidiomycota/Agaricomycetes/Agaricales/Strophariaceae/Pholiota/Kuehneromyces lignicola/ Syn. Pholiota lignicola/README.md

repo_name: mdoering/backbone


text:
# Basics of OTP

Supervisor, GenServer, Agent, and Task

## What is a process?

In IEx, we can use
```
iex(1)> spawn(fn -> IO.puts("Hello OTP") end)
Hello OTP
#PID<0.409.0>
iex(2)> pid = spawn(fn -> IO.puts("Hello OTP") end)
Hello OTP
#PID<0.411.0>
iex(3)> Process.alive?(pid)
false
```
to create a new process (that dies immediately).

Supervisors are also processess. Their main purpose is to monitoring other processes. If a child process fails, then the supervisor restarts it with a fresh state.

OTP allow us to configure how supervisors will handle failures. For example,

- Whether or not to restart the process
- Restart only the process that failed or all processes linked to the supervisor
- Restarting just when the child process terminates abnormally

These configurations are called Strategies.

## GenServer

GenServer is an implementation for starting and handling all in and out messages in your process.

We only need to  `use GenServer` in our module and implement the 3 following functions:

`GenServer.start(__MODULE__, :ok, [])`

This triggers an `init` function in the module. This function sets the initial process state.

`handle_cast()`

This casts any message that we receive. The message will be pattern-matched with any `handle_cast()` function declared in our module. At the end of this function we return a two-element tuple `{:noreply, new_state}` to give a new state to our server.

`handle_call()`

This sends any message back the the process that pinged us. At the end, we reply with a three-element tuple {:reply, state_to_reply, new_state} where the second element is actually the message that we will send back, and the third one is the new state for our process.

repository: fdeage/til

repo_name: fdeage/til


text:
# Groovy Language Server with Sublime Text

Learn how to configure [Sublime Text](https://www.sublimetext.com/) to add code intelligence for Groovy using the [Groovy Language Server](https://github.com/prominic/groovy-language-server).

1. Install the [LSP package](https://github.com/tomv564/LSP) for Sublime Text.

2. Download the [Groovy Language Server](https://github.com/prominic/groovy-language-server) source code from Github, and build the project using the instructions in the [_README_ file](https://github.com/prominic/groovy-language-server/blob/master/README.md).

3. In Sublime Text, go to the **Preferences** menu → **Package Settings** → **LSP** → **Settings**. This will open the LSP package settings.

4. In _LSP.sublime-settings — User_, add the configuration for the Groovy Language Server:

   ```json
   {
     "clients": {
       "groovy-language-server": {
         "enabled": true,
         "command": [
           "java",
           "-jar",
           "/absolute/path/to/groovy-language-server-all.jar"
         ],
         "languageId": "groovy",
         "syntaxes": ["Packages/Groovy/Groovy.sublime-syntax"],
         "scopes": ["source.groovy"]
       }
     }
   }
   ```

   Be sure to change to _/absolute/path/to/groovy-language-server-all.jar_ to the real location of this _.jar_ file on your computer. It **must** be an absolute path.

repository: bgshacklett/groovy-language-server

gha_language: Java

repo_name: bgshacklett/groovy-language-server


text:
---
title: Getting Started with CSS
description: Get started with the CSS version of Foundation for Emails.
---

Foundation for Emails takes the pain out of developing HTML emails by giving you a set of powerful, tested components out of the box. This includes a fully-responsive grid, buttons, callouts, menus, and more.

In this Getting Started guide, we'll download Foundation for Emails, construct the basic grid, and then inline our email so it's ready to test.

---

## Installing

If you haven't yet, download the starter files for Foundation for Emails. They include the boilerplate HTML needed for an email, and all of the CSS for Foundation.

Download Foundation for Emails

Unzip the folder and open it in your text editor of choice.

---

## File Structure

Here's a breakdown of the files in the folder you got:

- `css/foundation-emails.css`: the Foundation for Emails CSS.
- `index.html`: a blank boilerplate to get started.
- `templates/`: a set of pre-made templates following common email layouts.

We'll be writing a layout from scratch, so open up `index.html`.

---

## Boilerplate

Inside `index.html`, you can see the boilerplate needed to make an HTML email work, with comments explaining what does what.

```html
<!-- Emails use the XHTML Strict doctype -->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="https://www.w3.org/1999/xhtml">
<head>
  <!-- The character set should be utf-8 -->
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta name="viewport" content="width=device-width"/>
  <!-- Link to the email's CSS, which will be inlined into the email -->
  <link rel="stylesheet" href="assets/css/foundation-emails.css">
  <style>
    <!-- Your CSS to inline should be added here -->
  </style>
</head>

<body>
  <!-- Wrapper for the body of the email -->
  <table class="body" data-made-with-foundation>
    <tr>
      <!-- The class, align, and <center> tag center the container -->
      <td class="float-center" align="center" valign="top">
        <center>
          <!-- The content of your email goes here. -->
        </center>
      </td>
    </tr>
  </table>
</body>
</html>
```

---

## Grid Basics

Foundation for Emails includes many common elements needed to make HTML emails: a grid, typography styles, buttons, callouts, and more.

The markup required to create HTML emails that work in all email clients is *complicated*, and involves writing many tables. However, all of Foundation's components are thoroughly tested in every major email client, including the problematic ones like Outlook. The rigid structure ensures your designs don't look off in any email client.

Let's start by building a basic grid. To do that, we need three pieces: a container, a row, and then some columns.

### Container

Most Foundation for Emails components are full tables, including the container. Inside the `<center>` tag of `index.html`, add this table code:

```html
<table class="container">
  <tr>
    <td></td>
  </tr>
</table>
```

The **container** will wrap the body of your entire email. It applies a maximum width to the body of the email. Every email needs this bit of boilerplate.

### Row

Next, let's build the grid itself, starting with the row. **Rows** group columns together into a unit. Inside of your container, add this table:

```html
<table class="row">
  <tr>
    <th></th>
  </tr>
</table>
```

### Columns

Columns divide your layout into horizontal sections that sit side-by-side. On small screens, these columns stack on top of each other to save space—unless you set them up to keep their layout on small screens.

Inside of your row (the innermost `<tr>`), add one column using this code:

```html
<table class="row">
  <tr>
    <th class="small-12 large-6 first columns">
      Column One
    </th>
    <th class="expander"></th>
  </tr>
</table>
```

In the above example, we used the classes `.small-12` and `.large-6` to define the size of the column on small vs. large email clients. Foundation uses a 12-column grid, so on mobile clients, the column stretches the full width of the page, and on desktop clients, the column is half the width of the row.

Since this first column is half-width, we need a second one to go with it. *After* the `<th>` for the first column, add the code for the second column:

```html
<table class="row">
  <tr>
    <th class="small-12 large-6 first columns">
      Column One
    </th>
    <th class="small-12 large-6 last columns">
      Column Two
    </th>
    <th class="expander"></th>
  </tr>
</table>
```

You may have noticed the `.first` and `.last` classes on the column. The first column in a row needs the `.first` class, and the last column in a row needs the `.last` class. This is explained more in the [First and Last Classes](grid.html#first-and-last-classes) section of the grid documentation.

That's a lot of HTML! Let's look at what it is at a high level:

```html
<!-- Container -->
  <!-- Row -->
    <!-- Column One -->
    <!-- Column Two -->
```

The CSS classes are always on the outermost table element, so that's an easy way to identify where a component starts. The container starts at the table with the `.container` class, the row starts at the table with the `.row` class, and so on.

---

## Inlining

Now that we have a basic email, the last thing we need to do before we can send it is *inline* it. This is the process of injecting all of the CSS for the email into the HTML, so that it works as a self-contained file.

Email clients like Gmail and Outlook strip out `<style>` tag from the `<head>` and Gmails strips it from the `<body>` of the email as well. It's best to have your CSS written inline within your markup. Hand writing all your CSS inline in a style tag would be a real pain and would take a long time.

Our [web inliner](https://get.foundation/emails/inliner.html) automates this process for you. To use it, paste in the contents of `index.html` into the HTML field, paste in the contents of `css/foundation-emails.css` into the CSS field, and then press "Inline!". Once it's done, you'll see a large soup of HTML that is your inlined email.

Your email's CSS will include media queries for responsive styling which the inliner tool will move into the `<body>` so they are preserved.

---

## Testing

Now that you have an inlined email, you'll need to test it in real email clients to verify that it looks correct. All of Foundation's built-in components have already been tested in every major email client, so we've done a lot of the work for you. However, you'll want to test your email before you send it out to the masses.

The most popular tool for testing emails is [Litmus](https://litmus.com/). All you have to do is paste in the HTML of an email, and you get a live preview in any email client you want.

It's up to you to choose what email clients are important to test in, but you can [see our compatibility list](compatibility.html) for recommendations.

---

## Next Steps

You've successfully installed Foundation for Emails, and written, inlined, and tested your first email! To keep going with the framework, you can check out the documentation for the other framework components, including [buttons](button.html), [callouts](callout.html), [menus](menu.html).

If you're interested in going in-depth on the framework with the Foundation team, [check out our master class on Foundation for Emails](https://zurb.com/university/responsive-emails-foundation), an on-demand video series that explores every aspect of email design workflow.

repository: hmartos/foundation-emails

branch_name: refs/heads/develop

repo_name: hmartos/foundation-emails


text:
# WebGL Fluid Simulation

This repo is a simple modified version for the Wallpaper Engine.

Find this workshop item [here](https://steamcommunity.com/sharedfiles/filedetails/?id=1846923623)!

[Preview here](https://paveldogreat.github.io/WebGL-Fluid-Simulation/)

<img src="/screenshot.jpg?raw=true" width="880">

## References

http://developer.download.nvidia.com/books/HTML/gpugems/gpugems_ch38.html

https://github.com/mharrys/fluids-2d

https://github.com/haxiomic/GPU-Fluid-Experiments

## License

The code is available under the [MIT license](https://github.com/PavelDoGreat/WebGL-Fluid-Simulation/blob/master/LICENSE)

repository: Jasonnor/WebGL-Fluid-Simulation

repo_name: Jasonnor/WebGL-Fluid-Simulation


text:
---
title: Welcome to the CFPB Design System
layout: variation
collection_name: special-pages
is_homepage: true
description: >-

The Design System is an open-source resource for teams at the Consumer Financial Protection Bureau (CFPB). It's intended to help teams produce consistent, effective, and accessible products for the American public.

  ![Homepage image](/design-system/images/uploads/cfpb_ds_home.png)

  ## Getting started

  Our [general principles](https://cfpb.github.io/design-system/general-principles) form the strategic underpinnings for the CFPB’s design and development standards. Our [accessibility principles](https://cfpb.github.io/design-system/accessibility) lay the foundation for ensuring that our web content is available for all users.

  ## Browse the Design System

  ### Foundation

  This section includes our visual identity standards, such as color, grid, and typography. It forms the foundation for the CFPB’s website and external-facing materials.

Browse foundation

  ### Components

  Components contain the key building blocks of the design system that, when combined, can be used to create a website. Examples of components include buttons, text inputs, tables, and alerts.

Browse components

  ### Patterns

  Patterns are combinations of components used in conjunction to achieve a goal. Interaction patterns are best practice design solutions to common user tasks. Layout patterns are used by designers to organize content into clear, accessible web pages.

Browse patterns

  ### Pages

  Find out more about the common page types that we use within our content management system, which are documented for easy reference.

Browse pages

  ### Development

  The development section includes utilities, such as variables, helper classes, and mixins, and layout options, such as blocks.

Browse development

  ### Guidelines

  This section includes guidelines for creating specific product experiences not covered in other sections.

Browse guidelines

  ## Get started as a developer

  ### Setup and contribution

The Design System provides a set of modular HTML, CSS, and JavaScript patterns that can be integrated into your project. Integrate the Design System into your project.

  Learn how to contribute to the Design System code base and update the documentation.  Get started.

  ### Using source code

  The Design System also includes the source code for the common components that power the design of [consumerfinance.gov](https://www.consumerfinance.gov).

  View source code on GitHub

  ## Contribute to the Design System

  We strongly encourage you to participate in the growth and maintenance of the Design System. To make contribution easier, the Design System is built on a tool called Netlify CMS, which allows for editing of pages in a web browser, without needing to use git or other command-line tools.

View instructions for contributing to the Design System

About the CFPB Design System

  Our design system has been created to work with a wide range of devices and browsers. Following a modern, mobile first responsive approach, sites built with our Design System easily adapt to a wide range of screen sizes, all while carefully following accessibility best practices.

  As a work of the United States government, all code is open source and in the public domain. We encourage you to use this framework in your own projects and to contribute back.

License

  All content has been released as open source under the CC0 1.0 Universal Public Domain Dedication, and we’d love for other agencies, developers, or groups to adapt it for their own use.
---

repository: AdrianaMusic/design-system

repo_name: AdrianaMusic/design-system


text:
---
title: Trip Attack
state: draft
---
When you hit a creature with a weapon attack, you can expend one superiority die to attempt to knock the target down. You add the superiority die to the attack's damage roll, and if the target is Large or smaller, it must make a Strength saving throw.

repository: badooga/compendium

path: /_notes/manoeuvres/Trip Attack.md

filename: Trip Attack.md

repo_name: badooga/compendium


text:
---
layout: post
title: "Production Kubernetes with DevOps Part 2 - The Ingress"
categories: junk
author: "Pubudu Welagedara"
meta: "DevOps"
comments: true
---

I am going to deploy everything in one Cluster since Kubernetes Clusters are expensive. Then I am going to move the LDAP into that cluster since it is not required to have it outside now.

## Prerequisites

### Create a Static IP

```bash
gcloud auth login

gcloud config set project <project id>
```

Then connect to the cluster using the connection string given.
```bash
gcloud container clusters get-credentials kube-main --zone us-central1-a --project kubefire-209619
```

Then create the Global IP
```bash
gcloud compute addresses create devops-tools-ip --global

gcloud compute addresses describe devops-tools-ip --global
```

### Add the DNS Entries

{% assign image = "do1.png" %}
{% assign alt = "DevOps" %}
{% include srcset.html %}

## Deploy the Ingress

Now go ahead and deploy the Ingress. I am not using SSL for now.

{% assign image = "do2.png" %}
{% assign alt = "DevOps" %}
{% include srcset.html %}

## Cleanup

```
gcloud compute addresses delete devops-tools-ip --global
```

## ToDos
 - Add the default backend

## References
 - [GCloud Ingress Reference](https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip)
 - [Name based Virtual Hosting](https://kubernetes.io/docs/concepts/services-networking/ingress/#name-based-virtual-hosting)
 - [Default HTTP Backend](https://github.com/smpio/kube-default-http-backend)


text:
---
layout: post
title: "Picking your brain: Cold call email etiquette"
date: 2015-07-20 07:20:09
summary: I get a LOT of emails from people asking to "pick my brain." Here's how I wish people would approach this.
categories: networking mentorship advice
---

I love meeting with people.  Coffee chats with different people is so fun!

But, unfortunately, I don't scale well.  After talking with my friend
[Rebecca Garcia](http://www.geekgirlweb.com/), I realized I wasn't alone in
this.  As you get more involved with work and the community, a coffee chat isn't
just a one-off, it's a constant stream of requests.

I've slowly gotten better at handling them, I use [Assistant.to](http://www.assistant.to/)
to schedule things and try to only meet with people a max of twice a week, but
that's created *quite* the backlog.

So, this is a post for you, the meeting requester, and for the benefit of you
and myself/anyone busy to whom you might be reaching out.  This is how to ask a
busy person for a coffee chat.

### 1. Be specific.

Don't just email someone saying, "Hey, I saw you once, and I'd like to pick your
brain! I'm free tomorrow."

I don't know what you mean.

You could want to literally pick pieces of their brain out for all I know.  Not
to be dramatic.  But you didn't specify.

Make sure you say *how* you met the person you're contacting, and why you're
contacting them.  Simple stuff.  Big impact.

### 2. Be flexible.

You're asking the person to meet you and take time out of their day.  So, you
should work with their schedule!  Give them several options, let them pick a
location (or pick one that's convenient for them), give them an out, and don't
be sad if they aren't able to meet with you for a while (if at all).

At that point, just email them your questions or whatever, and have hope that
they'll email you back!

### 3. Don't be afraid to ping them if they haven't gotten back to you in a week.

  I personally really
appreciate pings.  It's a friendly reminder, and it puts you back on someone's
radar.  Now, if the busy person doesn't get back to you after a couple pings,
don't fret, and again, don't be sad.  They're busy for a reason!  There's plenty
of other fish in the sea.  And people in the world, if fish isn't your thing.

### 4. Be prepared.

If you're nonspecific in your email, that can be frustrating for a busy person
among their mass amounts of meet up requests.  Outside of being specific, be
prepared.  If they reply back with questions about what you mean, or with
details, be ready to give that information up right away.  This is both for your
benefit and theirs, because you want to get the most out of it, and they want to
give relevant, efficient information and advice!

### That's it.

A *much* better email is something like, "hey, I'm interested in how you got to
where you are today. I'm a ________ at ________, I met you at/read about you in
________ and I just have a few questions about what classes I should take/which
role I should follow/how I should format my resume.  Could we meet for coffee?
My treat!  I'm pretty free over the next few weeks, Tuesdays and Thursdays are
best!  I know you're a busy bee, so I can work around your schedule.
Let me know, and see you soon."

This is clear, it gives options for dates, it gives an out, and it lets the
person know what you want from them.  If I got this email, I would meet with you.

Good luck!


text:
## E_MISSING_CLDR

Thrown when any required CLDR item is NOT found.

Error object:

| Attribute | Value |
| --- | --- |
| code | `E_MISSING_CLDR` |
| path | Missing CLDR item path |

repository: radville/map-demo

gha_language: JavaScript

repo_name: radville/map-demo


text:
---
title: "Testers Tools"
collection: qatools
type: "Handy Testing Tools for Quality Engineers"
permalink: /qatools/test-automation-tools
date: 2018-04-29
location: "Mumbai, India"
---

Handy Tools for Quality Engineers
======
- FireBug, FirePath / chropath - For Locator extractions
- Postman / DHC/ RestClient - For Various Rest API Tests
- SQL Workbench/ dBeaver - For database validations
- Jmeter - Load test Creation
- IntelliJ/ Eclipse - IDE for various automation tests scripts
- LightHouse - Front-end Performance, SEO, UX, Accessibility Audits
- [Tenon](http://tenon.io) - For Accessibility Tests
- W3C html / css validator - For HTML and CSS validation tests
- Sonar/ ESLint - For Semantics, Code Vulnerabilities
- OWASP - ZAP - Spider SCAN for TOP10 OWASP vulnerabilities
- Putty, MSTSC - Remote server/machine connectivity
- WinSCP, FileZilla - For validate server files, log files, etc.
- Jira, Rally, TFS, ALM - For Project Management, Defect tracking, Agile Practices.


text:
# Peachili

This programming language is inspired Golang strongly.

## usage

```
$ peachili compile <peachili-file> # generate an assembly-file for X86_64
$ peachili compile <peachili-file> # generate an assembly-file for aarch64
```

## [Documents](https://github.com/Drumato/peachili/blob/master/docs/main.md)

## Run all tests

### Unit tests

```
cargo test
```

### Integration tests on x86_64

```
cargo build
./x64_test.sh
```

### Integration tests on Arm64

```
cargo build
./aarch64_test.sh
```

## Debug

```
# x64

# aarch64
qemu-aarch64-static -g <port> a.out
gdb-multiarch a.out
target remote :<port>
```

repository: Drumato/peachili

gha_language: Rust

repo_name: Drumato/peachili


text:
---
slug: doo-and-figwheel
date: 2015-09-22
title: "ClojureScript Testing with Doo and Figwheel"
author: "Torsten Uhlmann"
tags: []
categories: []
description:
banner: banner.jpg
bannerCredit: "Photo by Torsten Uhlmann"
---

These days I was playing with strategies to run
ClojureScript tests. Information about this particular topic is not too easy to find, so here's a short list of posts I found particularly useful:

* [How can you test ClojureScript applications and libraries?](http://www.lispcast.com/testing-clojurescript)
* [Testing ClojureScript code with cljs.test with phantomjs and figwheel](https://nvbn.github.io/2015/06/08/cljs-test/)

The Unit test library used in these cases is `cljs.test`.
One way to run the tests is to add a test and runner configuration to your `project.clj` (see the second link for an example).

Another way is to use [lein-doo](https://github.com/bensu/doo) as a way to configure all kinds of different test runners, like Karma, SlimerJs or PhantomJs, and also watch your source directories and run tests on file change.

I was playing with that Doo configuration in my project and ran into a strange error.

My project uses a figwheel configuration, and the funny thing is, figwheel barks on this piece of Doo config:

`:main 'example.runner`

with the exception:

```
java.lang.ClassCastException:
clojure.lang.PersistentList cannot be cast to
clojure.lang.Named
```

To get around this problem, write the Doo config like this:

`:main "example.runner"`

Figwheel works again and even Doo still works :)

Hope this might help somebody...


text:
# Chat
A simple TCP based console chat server written in C.

## Project Structure

The make command generates 3 executable files:

- Server: Starts the chat server. In this terminal you can see when new clients connect/disconnect to the server.
- Client: Starts a new client. In this terminal you can see messages sent by other clients.
- Client_Messages: In this terminal you can write and send messages to other clients.

## Considerations

- You should run the Server program first and only once.
- You can run the Client program as many times as clients you want to have.
- For each Client program you run you should also run one Client_Messages program in the same computer.
- You can run the all the executables in the same computer, or run the Server program in one computer and the Client programs in another one. You can also run all the Client programs in different computers.
- All computers must be inside the same LAN.

## Build and Run Instructions

1. Open a terminal.
2. In your server PC open /etc/services: `sudo nano /etc/services`
3. Add a new TCP network service. Create a row indicating service name, port, type of protocol, a brief description and save changes. For example: `test-sockets  15556/tcp  #Test for TCP chat application`
4. Clone repository: `git clone https://github.com/marianodato/Chat.git`
5. Change to project directory: `cd Chat`
6. Change the ip address and port in Client_Socket.c with your server pc ip and the port number you added in /etc/services.
7. Change the service name in Server.c with the service name you added in /etc/services.
8. Change MAX_CLIENTS variable in Server.h with the maximum amount of clients you want the server to support.
9. Run makefile: `make`
10. Run the Server program: `./Server`
11. In the same or in other computers run the Client programs: Both `./Client` and `./Client_Messages` in the same PC.

repository: marianodato/Chat

repo_name: marianodato/Chat


text:
# GeekSpeak-Show-Timer

During the recording of GeekSpeak, we need to plan for 2 inexactly placed breaks and a precise show length.  This is the timer we use to make that as easy as possible.

=======

## Known issues:
- Autolayout constraints of master view (slide over view) are non-existent.
- Pause and Next buttons need to be graphic buttons for instant clarity.

## Todo:

- GeekSpeakButton UIButton subclass
	- use outline art
	- fill outline art on touch
	- animate radiating rings after sucessful touch
- Create animating buttons for start/pause and Next Segment buttons
- Create graphics for Add & Remove seconds buttons.
- Add GameKit discovery and networking between timers on multiple devices
- Animations:
	On section3 warning & alarm color, animate visibility of each ring radiating out (to catch everyone's eye)
- images need to be scaled and optimized for: @1x,@2x,@3x
- ~~Add color change to last segment ring at 2 minutes and a third color change at 30 seconds.~~ (done)
- ~~When 1st and 2nd segment finish, indicate time is still counting by creating a 5 degree gap that continues rotation at the same pace.~~ (done)
- ~~Fix labels under show times to scale based on text width~~ (done)
- ~~Showtime should always count up.~~ (done)
- ~~Section time should always count down. Except when all three segments are completed~~ (done)
- When show is done (three segments are completed)
	- ~~timer labels change to GeekSpeakBlue~~ (done)
	- Add and remove buttons should be disabled
	- Next Segment button animates away (reset button animates it back)
- Abstract out the hard coded definition of 3 segments with one minute breaks, into a definition file. Auto generate views based on this definition, allowing multiple show formats
- ~~Add test mode in settings app to switch between demo/testing and live show timer~~ (done)
- ~~Add keyArchivers to save state between launches~~ (done)
- BUG: track down archiving state bug when timer is running and state preservation kicks in
- BUG: Nav button shows 'Hide' (not 'Show Timer') on start up

## Versions:
# v0.1
_(this version is tagged but, the tag isn't showing up in github for some reason)_

This is the initial version of the timer and used on the 8/3/15 recording of the show (aired 8/8/15).  It is useable, but truely a beta release.  Many views don't have any layout code and you can get the app in an unusable state on iPhones. [Here is a video demo of this version](https://www.youtube.com/watch?v=kwDyj1H7LJw)

repository: lyle/GeekSpeak-Show-Timer

repo_name: lyle/GeekSpeak-Show-Timer


text:
# react-native-aspnet-membership

## Introduction

[TBD]

## Usage

[TBD]

## Contributing

Bug reports and pull requests are welcome on GitHub. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org/) code of conduct.

1. Fork it (https://github.com/99xt/react-native-boilerplate-with-authentication)
2. Create your feature branch (<tt>git checkout -b my-new-feature</tt>)
3. Commit your changes (<tt>git commit -am 'Add some feature'</tt>)
4. Push to the branch (<tt>git push origin my-new-feature</tt>)
5. Create a new Pull Request

## License

Copyright (c) 2016

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


text:
# VbTrackConfiguration

## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**trackIndex** | **Integer** | Track index. |  [optional]
**speakerName** | **String** | Speaker name. Defaults to \"Speaker\". If this attribute is specified, all channels are mixed into a single one. |  [optional]
**speakerRole** | **String** | A description of the speaker role. Only used as metadata. |  [optional]
**channels** | [**java.util.List<VbChannelConfiguration>**](VbChannelConfiguration.md) | List of channels to process, mutually exclusive with \"speakerName\" |  [optional]

repository: voicebase/voicebase-java-samples

repo_name: voicebase/voicebase-java-samples


text:
---
title: Retries
weight: 100
description: Implement upstream retries when experiencing transient network errors
---

Specifies the retry policy for the route where you can say for a specific error condition how many times to retry and
for how long to try.

* `retryOn` : specifies the condition under which to retry the forward request to the upstream. Same as [Envoy x-envoy-retry-on](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on).
* `numRetries` : (default: 1) optional attribute that specifies the allowed number of retries.
* `perTryTimeout` : optional attribute that specifies the timeout per retry attempt. Is of type [Google.Protobuf.WellKnownTypes.Duration](https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/duration).

{{< highlight yaml "hl_lines=20-23" >}}
apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: 'default'
  namespace: 'gloo-system'
spec:
  virtualHost:
    domains:
    - '*'
    routes:
    - matchers:
       - prefix: '/petstore'
      routeAction:
        single:
          upstream:
            name: 'default-petstore-8080'
            namespace: 'gloo-system'
      options:
        retries:
          retryOn: 'connect-failure'
          numRetries: 3
          perTryTimeout: '5s'
{{< /highlight >}}

repository: solo-io/gloo

gha_language: Go

repo_name: solo-io/gloo


text:
# EDLine-py
Forked from [this](https://github.com/HanjieLuo/EDLine_parallel)

![test image](./data/result.png)

## Update
- added python binding
- added web [api](api/readme.md)
- install with *cmake*

## Requirements ##
The code is tested on Debian GNU/Linux 10. It requires the following tools and libraries: *python3.8*, *pybind11*, *Numpy*, *OpenCV 3.4.13*.

## Installation ##
in the project root, execute the following command in your terminal
```bash
pip install .
```

## Usage ##
```python
import cv2
import edline
# args: kernel:int
#       sigma:float
#       gradient_threshold:float
#       anchor_threshold:float
#       scan_interval:int
#       min_line_len:int
#       line_fit_err_threshold:float
ed = edline.EDLine(5, 1.0, 30, 5, 2, 25, 1.8)
lines = edline.LineList()
img = cv2.imread(SOME_IMAGE, 0)  # 0: gray scale
ed.detect(img, lines, False)
for l in lines:
    print(l.endpoint)  # [x1, y1, x2, y2]
```

Test:

```bash
PYTHONPATH=. python test/demo.py
```

## Known issues ##
*OpenCV* that is installed with `pip` may not work with this repo.
install with *CMake* instead:
```
# assuming numpy has been installed
git clone -b 3.4.13 https://github.com/opencv/opencv.git
cd opencv
mkdir build
cd build
cmake .. -DPYTHON3_EXECUTABLE=`which python`
make -j`nproc`
make install
```