text:
# Catstamp

Thus named for one of my more common uses of `cat`, notetaking.
The idea is from a program that already exists, but that I couldn't remember what it was called.

The following use of `cat` is a reasonable method for capturing notes.
(Use Control+D (^D) to send an EOF and exit.)

```
$ cat >> somefile.txt
this is a test
oh, yeah.  the test.
oops.  forgot.
^D
$ cat somefile.txt
this is a test.
oh, yeah.  the test.
oops.  forgot.
$
```

This, though, will also timestamp each captured line.

```
$ catstamp >> somefile.txt
this is a test
oh, yeah.  the test.
oops.  forgot.
^D
$ cat somefile.txt
2018-01-22 19:24:10	this is a test
2018-01-24 02:03:05	oh, yeah.  the test.
2018-01-24 02:03:06	oops.  forgot.
$
```

## License

MIT License

Copyright (c) 2018 Luke Fritz

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

repository: ldfritz/catstamp

repo_name: ldfritz/catstamp


text:
---
layout: simple
---

Agile Let Us Down

    Scrum (or XP or Kanban or just Agile) has failed us. We still have crunch mode, death marches, and buggy software.

    Fortunately writing clean, tested code that you enjoy isn't a function of process. It's a function of technical skill and you DON'T have to ask for permisssion.

Start Doing Real World TDD Now!

    Eric Smith (me) is a Polyglot developer, trainer, and writer. I'm currently working on the The Guide to Real World TDD featuring real world problems for full stack developers.

By day I'm the Director of Training Services at 8th Light, by night I'm the proud dad of 5 kids and happily married to a QA Engineer.

Follow me on twitter, and anyplace you find the tag paytonrules it's probably me.

New Here?

How to Learn TDD

The Evidence For TDD

The One Practice To Adopt Today

The Latest Stuff

      {% for post in site.posts limit:4 %}

        {{ post.date | date_to_string }} »

          {{ post.title }}

      {% endfor %}

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# Android Retrofit

`Retrofit` is a library built by the `Squareup` team that let's you perform async HTTP requests easily.

You will need to define a web service location based on a Java Interface (this is not mandatory).

The "good" thing about `Retrofit` is that can handle network requests without creating any can of Thread or AsyncTask.

In this repository I perform a `GET HTTP Request` to the `Typicode Fake server` for testing purposes only. Check the `build.gradle` file for the custom implementations of the dependencies used, including `retrofit` and `gson` for parsing requested data into Java Objects.

## Background Threads

Any software naturally runs code instructions one at a time, these ones are run in sequence. But modern software has the capability to take advantage of every aspect of the hardware specs, especially CPU's. Actually nowadays any computer without multiple CPUs it's pretty much considered old.

So, getting back to the first idea, once a task or instruction is executed, the CPU can't do anything else, this is called a blocking call.

## Multithreading

This term takes place directly for computers with multiple CPUs (Cores) when the CPU it's able to generate multiple Threads that can be executed concurrently. That's the case for modern Android devices, almost all of them have multiple cores in its configuration.

The main Thread is used for UI related activities, this is mandatory, and another operations are handled or should be done with the help of the threading techniques, this is because the any demanding operation can freeze the UI, the same happens with network-related operations.

## Credits

 - [David E Lares](https://twitter.com/davidlares3)

## License

 - [MIT](https://opensource.org/licenses/MIT)

repository: davidlares/retrofit

repo_name: davidlares/retrofit


text:
# word ladder II

Given two words (beginWord and endWord), and a dictionary's word list, find all shortest transformation sequence(s) from beginWord to endWord, such that:

Only one letter can be changed at a time
Each transformed word must exist in the word list. Note that beginWord is not a transformed word.
## Note:

Return an empty list if there is no such transformation sequence.
All words have the same length.
All words contain only lowercase alphabetic characters.
You may assume no duplicates in the word list.
You may assume beginWord and endWord are non-empty and are not the same.

## Example 1:
```
Input:
beginWord = "hit",
endWord = "cog",
wordList = ["hot","dot","dog","lot","log","cog"]

Output:
[
  ["hit","hot","dot","dog","cog"],
  ["hit","hot","lot","log","cog"]
]
```
Example 2:
```
Input:
beginWord = "hit"
endWord = "cog"
wordList = ["hot","dot","dog","lot","log"]

Output: []
```

Explanation: The endWord "cog" is not in wordList, therefore no possible transformation.

```python
class Solution:
    def findLadders(self, beginWord, endWord, wordList):
        """
        :type beginWord: str
        :type endWord: str
        :type wordList: List[str]
        :rtype: List[List[str]]
        """
        words = set(wordList)
        if endWord not in wordList:
            return []

        words.add(beginWord)
        graph = collections.defaultdict(set)
        if not self.bfs(beginWord, endWord, graph, words):
            return []

        paths, path = [], [beginWord]
        self.dfs(graph, endWord, paths, path)
        return paths

    def bfs(self, beginWord, endWord, graph, words):
        n = len(beginWord)
        levels = [set([beginWord]), set([endWord])]

        while levels[0] and levels[1]:
            idx = 0 if len(levels[0]) < len(levels[1]) else 1
            preLevel = levels[idx]
            levels[idx] = set()
            isFound = False

            for word in preLevel:
                words.remove(word)
            for word in preLevel:
                for i in range(n):
                    prefix, suffix = word[:i], word[i + 1:]
                    for c in 'abcdefghijklmnopqrstuvwxyz':
                        if c == word[i]:
                            continue
                        w = prefix + c + suffix
                        if w in words:
                            if w in levels[1 - idx]:
                                isFound = True
                            levels[idx].add(w)
                            if idx == 0:
                                graph[word].add(w)
                            else:
                                graph[w].add(word)
            if isFound:
                return True
        return False

    def dfs(self, graph, endWord, paths, path):
        if path[-1] == endWord:
            paths.append(list(path))
        for neighbor in graph[path[-1]]:
            path.append(neighbor)
            self.dfs(graph, endWord, paths, path)
            path.pop()
```

再来一个速度超级快的算法：

```python
class Solution:
    def findLadders(self, beginWord, endWord, wordList):
        """
        :type beginWord: str
        :type endWord: str
        :type wordList: List[str]
        :rtype: List[List[str]]
        """
        if endWord not in wordList or not endWord or not beginWord:
            return []
        wordList = set(wordList)
        forward, backward = {beginWord}, {endWord}
        direction = 1
        parents = collections.defaultdict(set)
        while forward and backward:
            if len(forward) > len(backward):
                forward, backward = backward, forward
                # We need to trace the direction in order to distinguish the parents
                direction *= -1

            # The new set of words which will be forward in the next iteration
            next_foward = set()

            # Because all words in forward will be modified by one character
            wordList -= forward
            for word in forward:
                for i in range(len(word)):
                    first, second = word[:i], word[i+1:]
                    for ch in string.ascii_lowercase:
                        combined_word = first + ch + second
                        if combined_word in wordList:
                            next_foward.add(combined_word)
                            # Because at the last part, we find parents by indexing dictionary from endWord
                            # So when direction == 1, the combined_word is the key
                            # otherwise, the combined_word should be the value of dictionary.
                            if direction == 1:
                                parents[combined_word].add(word)
                            else:
                                parents[word].add(combined_word)

            # next_foward and backward are always in different direction,
            # so if they have common elements we find a path.
            # We check and return this function inside is because this problem finds the all shortest paths
            if next_foward & backward:
                # Starting from the endWord, we find its parent and append to results
                # And do this until we reach the beginWord
                results = [[endWord]]
                while results[0][0] != beginWord:
                    results = [ [parent] + result for result in results for parent in parents[result[0]] ]
                return results
            forward = next_foward
        return []

```

repository: hanleilei/leetcode

gha_license_id: MIT

repo_name: hanleilei/leetcode


text:
## Introduction

Recruiters in the analytics/data science industry expect you to know at least two algorithms: Linear Regression and Logistic Regression. I believe you should have in-depth understanding of these algorithms. Let me tell you why.

Due to their ease of interpretation, consultancy firms use these algorithms extensively. Startups are also catching up fast. As a result, in an analytics interview, most of the questions come from linear and Logistic Regression.

In this article, you'll learn about Logistic Regression in detail.  Believe me, Logistic Regression isn't easy to master. It does follow some assumptions like Linear Regression. But its method of calculating model fit and evaluation metrics is entirely different from Linear/Multiple regression.

But, don't worry! After you finish this tutorial, you'll become confident enough to explain Logistic Regression to your friends and even colleagues. Alongside theory, you'll also learn to implement Logistic Regression on a data set. I'll use R Language. In addition, we'll also look at various types of Logistic Regression methods.

**Note:** You should know basic algebra (elementary level). Also, if you are new to regression, I suggest you read how [Linear Regression](http://blog.hackerearth.com/beginners-guide-regression-analysis-plot-interpretations) works first.

[![machinelearning](http://blog.hackerearth.com/wp-content/uploads/2017/05/6.jpg)](https://www.hackerearth.com/challenge/competitive/machine-learning-indiahacks-2017/?source=practice)

## Table of Contents

1. What is Logistic Regression ?
2. What are the types of Logistic Regression techniques ?
3. How does Logistic Regression work ?
4. How can you evaluate Logistic Regression's model fit and accuracy ?
5. Practical - Who survived on the Titanic ?

## What is Logistic Regression ?

Many a time, situations arise where the dependent variable isn't normally distributed; i.e., the assumption of normality is violated. For example, think of a problem when the dependent variable is binary (Male/Female). Will you still use Multiple Regression? Of course not! Why? We'll look at it below.

Let's take a peek into the history of data analysis.

So, until 1972, people didn't know how to analyze data which has a non-normal error distribution in the dependent variable. Then, in 1972, came a breakthrough by [John Nelder](https://en.wikipedia.org/wiki/John_Nelder) and [Robert Wedderburn](https://en.wikipedia.org/wiki/Robert_Wedderburn_(statistician)) in the form of **Generalized Linear Models**. I'm sure you would be familiar with the term. Now, let's understand it in detail.

Generalized Linear Models are an extension of the linear model framework, which includes dependent variables which are non-normal also. In general, they possess three characteristics:

1. These models comprise a linear combination of input features.
2. The mean of the response variable is related to the linear combination of input features via a link function.
3. The response variable is considered to have an underlying probability distribution belonging to the family of exponential distributions such as binomial distribution, Poisson distribution, or Gaussian distribution. Practically, binomial distribution is used when the response variable is binary. Poisson distribution is used when the response variable represents count. And, Gaussian distribution is used when the response variable is continuous.

Logistic Regression belongs to the family of generalized linear models. It is a binary classification algorithm used when the response variable is dichotomous (1 or 0). Inherently, it returns the set of probabilities of target class. But, we can also obtain response labels using a probability threshold value. Following are the assumptions made by Logistic Regression:

1. The response variable must follow a binomial distribution.
2. Logistic Regression assumes a linear relationship between the independent variables and the link function (logit).
3. The dependent variable should have mutually exclusive and exhaustive categories.

In R, we use `glm()` function to apply Logistic Regression. In Python, we use `sklearn.linear_model` function to import and use Logistic Regression.

**Note:** We don't use Linear Regression for binary classification because its linear function results in probabilities outside [0,1] interval, thereby making them invalid predictions.

## What are the types of Logistic Regression techniques ?

Logistic Regression isn't just limited to solving binary classification problems. To solve problems that have multiple classes, we can use extensions of Logistic Regression, which includes **Multinomial Logistic Regression** and **Ordinal Logistic Regression**. Let's get their basic idea:

**1. Multinomial Logistic Regression:** Let's say our target variable has K = 4 classes. This technique handles the multi-class problem by fitting K-1 independent binary logistic classifier model. For doing this, it randomly chooses one target class as the reference class and fits K-1 regression models that compare each of the remaining classes to the reference class.

Due to its restrictive nature, it isn't used widely because it does not scale very well in the presence of a large number of target classes. In addition, since it builds K - 1 models, we would require a much larger data set to achieve reasonable accuracy.

**2. Ordinal Logistic Regression:** This technique is used when the target variable is ordinal in nature. Let's say, we want to predict years of work experience (1,2,3,4,5, etc). So, there exists an order in the value, i.e., 5>4>3>2>1. Unlike a multinomial model, when we train K -1 models, Ordinal Logistic Regression builds a single model with multiple threshold values.

If we have K classes, the model will require K -1 threshold or cutoff points. Also, it makes an imperative assumption of proportional odds. The assumption says that on a logit (S shape) scale, all of the thresholds lie on a straight line.

**Note:** Logistic Regression is not a great choice to solve multi-class problems. But, it's good to be aware of its types. In this tutorial we'll focus on Logistic Regression for binary classification task.

## How does Logistic Regression work?

Now comes the interesting part!

As we know, Logistic Regression assumes that the dependent (or response) variable follows a binomial distribution. Now, you may wonder, what is binomial distribution? Binomial distribution can be identified by the following characteristics:

1. There must be a fixed number of trials denoted by **n**, i.e. in the data set, there must be a fixed number of rows.
2. Each trial can have only two outcomes; i.e., the response variable can have only two unique categories.
3. The outcome of each trial must be independent of each other; i.e., the unique levels of the response variable must be independent of each other.
4. The probability of success (p) and failure (q) should be the same for each trial.

Let's understand how Logistic Regression works. For Linear Regression, where the output is a linear combination of input feature(s), we write the equation as:

```
                            `Y = βo + β1X + ∈`
```

In Logistic Regression, we use the same equation but with some modifications made to `Y`. Let's reiterate a fact about Logistic Regression: we calculate probabilities. And, probabilities always lie between 0 and 1. In other words, we can say:

1. The response value must be positive.
2. It should be lower than 1.

First, we'll meet the above two criteria. We know the exponential of any value is always a positive number. And, any `number` divided by `number + 1` will always be lower than 1. Let's implement these two findings:

![img](http://blog.hackerearth.com/wp-content/uploads/2017/04/logistic.jpg)

This is the logistic function.

Now we are convinced that the probability value will always lie between 0 and 1. To determine the link function, follow the algebraic calculations carefully. `P(Y=1|X)` can be read as "probability that Y =1 given some value for x." Y can take only two values, 1 or 0. For ease of calculation, let's rewrite `P(Y=1|X)` as `p(X)`.

[![logistic regression equation derivation](http://blog.hackerearth.com/wp-content/uploads/2017/01/equateimage-e1483685096494.png)](http://blog.hackerearth.com/wp-content/uploads/2017/01/equateimage.png)

As you might recognize, the right side of the (immediate) equation above depicts the linear combination of independent variables. The left side is known as the **log - odds** or **odds ratio** or **logit** function and is the link function for Logistic Regression. This link function follows a sigmoid (shown below) function which limits its range of probabilities between 0 and 1.

[![SigmoidPlot logistic function](http://blog.hackerearth.com/wp-content/uploads/2017/01/SigmoidPlot1.png)](http://blog.hackerearth.com/wp-content/uploads/2017/01/SigmoidPlot1.png) Until here, I hope you've understood how we derive the equation of Logistic Regression. But how is it interpreted?

We can interpret the above equation as, a unit increase in variable x results in multiplying the odds ratio by **ε to power β**. In other words, the regression coefficients explain the change in log(odds) in the response for a unit change in predictor. However, since the relationship between p(X) and `X` is not straight line, a unit change in input feature doesn't really affect the model output directly but it affects the odds ratio.

This is contradictory to Linear Regression where, regardless of the value of input feature, the regression coefficient always represents a fixed increase/decrease in the model output per unit increase in the input feature.

In Multiple Regression, we use the **Ordinary Least Square (OLS)** method to determine the best coefficients to attain good model fit. In Logistic Regression, we use **maximum likelihood method** to determine the best coefficients and eventually a good model fit.

Maximum likelihood works like this: It tries to find the value of coefficients (βo,β1) such that the predicted probabilities are as close to the observed probabilities as possible. In other words, for a binary classification (1/0), maximum likelihood will try to find values of βo and β1 such that the resultant probabilities are closest to either 1 or 0. The likelihood function is written as

[![img](http://blog.hackerearth.com/wp-content/uploads/2017/01/myloglike-e1483528776450.png)](http://blog.hackerearth.com/wp-content/uploads/2017/01/myloglike.png)

## How can you evaluate Logistic Regression model fit and accuracy ?

In Linear Regression, we check adjusted R², F Statistics, MAE, and RMSE to evaluate model fit and accuracy. But, Logistic Regression employs all different sets of metrics. Here, we deal with probabilities and categorical values. Following are the evaluation metrics used for Logistic Regression:

### 1. Akaike Information Criteria (AIC)

You can look at AIC as counterpart of adjusted r square in multiple regression. It's an important indicator of model fit. It follows the rule: Smaller the better. AIC penalizes increasing number of coefficients in the model. In other words, adding more variables to the model wouldn't let AIC increase. It helps to avoid overfitting.

Looking at the AIC metric of one model wouldn't really help. It is more useful in comparing models (model selection). So, build 2 or 3 Logistic Regression models and compare their AIC. The model with the lowest AIC will be relatively better.

### 2. Null Deviance and Residual Deviance

Deviance of an observation is computed as -2 times log likelihood of that observation. The importance of deviance can be further understood using its types: Null and Residual Deviance. Null deviance is calculated from the model with no features, i.e.,only intercept. The null model predicts class via a constant probability.

Residual deviance is calculated from the model having all the features.On comarison with Linear Regression, think of residual deviance as residual sum of square (RSS) and null deviance as total sum of squares (TSS). The larger the difference between null and residual deviance, better the model.

Also, you can use these metrics to compared multiple models: whichever model has a lower null deviance, means that the model explains deviance pretty well, and is a better model. Also, lower the residual deviance, better the model. Practically, AIC is always given preference above deviance to evaluate model fit.

### 3. Confusion Matrix

Confusion matrix is the most crucial metric commonly used to evaluate classification models. It's quite confusing but make sure you understand it by heart. If you still don't understand anything, ask me in comments. The skeleton of a confusion matrix looks like this:

[![confusion matrix logistic regression](http://blog.hackerearth.com/wp-content/uploads/2017/01/myprobbb.jpg)](http://blog.hackerearth.com/wp-content/uploads/2017/01/myprobbb.jpg)

As you can see, the confusion matrix avoids "confusion" by measuring the actual and predicted values in a tabular format. In table above, Positive class = 1 and Negative class = 0. Following are the metrics we can derive from a confusion matrix:

**Accuracy** - It determines the overall predicted accuracy of the model. It is calculated as `Accuracy  = (True Positives + True Negatives)/(True Positives + True Negatives + False Positives + False Negatives)`

**True Positive Rate (TPR)** - It indicates how many positive values, out of all the positive values, have been **correctly predicted**. The formula to calculate the true positive rate is `(TP/TP + FN)`. Also, `TPR =  1 - False Negative Rate`. It is also known as **Sensitivity** or **Recall**.

**False Positive Rate (FPR)** - It indicates how many negative values, out of all the negative values, have been **incorrectly predicted**. The formula to calculate the false positive rate is `(FP/FP + TN)`. Also, `FPR = 1 - True Negative Rate`.

**True Negative Rate (TNR)** - It indicates how many negative values, out of all the negative values, have been **correctly predicted**. The formula to calculate the true negative rate is (`TN/TN + FP)`. It is also known as **Specificity**.

**False Negative Rate (FNR)** - It indicates how many positive values, out of all the positive values, have been incorrectly predicted. The formula to calculate false negative rate is `(FN/FN + TP)`.

**Precision:** It indicates how many values, out of all the predicted positive values, are actually positive. It is formulated as:`(TP / TP + FP)`. **F Score:** F score is the harmonic mean of precision and recall. It lies between 0 and 1. Higher the value, better the model. It is formulated as `2((precision*recall) / (precision+recall))`.

### 4. Receiver Operator Characteristic (ROC)

ROC determines the accuracy of a classification model at a user defined threshold value. It determines the model's accuracy using Area Under Curve (AUC). The area under the curve (AUC), also referred to as index of accuracy (A) or concordant index, represents the performance of the ROC curve. Higher the area, better the model. ROC is plotted between True Positive Rate (Y axis) and False Positive Rate (X Axis). In this plot, our aim is to push the red curve (shown below) toward 1 (left corner) and maximize the area under curve. Higher the curve, better the model. The yellow line represents the ROC curve at 0.5 threshold. At this point, sensitivity = specificity.

[![roc curve logistic regression](http://blog.hackerearth.com/wp-content/uploads/2017/01/roc.gif)](http://blog.hackerearth.com/wp-content/uploads/2017/01/roc.gif)

## Practical - Who survived the Titanic disaster?

For illustration, we'll be working on one of the most popular data sets in machine learning: **Titanic**. It's fairly small in size and a variety of variables will give us enough space for creative feature engineering and model building. This data set has been taken from [Kaggle](https://www.kaggle.com/c/titanic/data). I hope you know that model building is the last stage in machine learning. Therefore, we'll be doing quick data exploration, pre-processing, and feature engineering before implementing Logistic Regression. For your convenience, the data can downloaded from [here](http://blog.hackerearth.com/wp-content/uploads/2017/01/Titanic_Data.zip).

```
#set working directory
> path <- "~Manish/Data/Titanic"
> setwd(path)

#load libraries and data
> library (data.table)
> library (plyr)
> library (stringr)
> train <- fread("train.csv",na.strings = c(""," ",NA,"NA"))
> test <- fread("test.csv",na.strings = c(""," ",NA,"NA"))
```

After we've loaded the data in R, we'll quickly look at the variables.

```
> head(train)
> head(test)
> str(train)
```

This data set comprises a mix of character and numeric variables. Among these variables,

```
Survived
```

 is the dependent variable. To get a deeper understanding of these variables, let's do a quick data exploration. Discoveries from this step will help us determine what to do next:

```
#check missing values
> colSums(is.na(train))
> colSums(is.na(test))
```

We see that variable

```
Age
```

 and

```
Cabin
```

 have missing values. Interestingly, the variable

```
Fare
```

 has one missing value only in the test set. Let's start exploring each variable individually. For numeric variables, we'll use the

```
summary
```

function. For character / factor variables, we'll use table for exploration:

```
#Quick Data Exploration
> summary(train$Age)
> summary(test$Age)

> train[,.N/nrow(train),Pclass]
> test[,.N/nrow(test),Pclass]

> train [,.N/nrow(train),Sex]
> test [,.N/nrow(test),Sex]

> train [,.N/nrow(train),SibSp]
> test [,.N/nrow(test),SibSp]

> train [,.N/nrow(train),Parch]
> test [,.N/nrow(test),Parch] #extra 9

> summary(train$Fare)
> summary(test$Fare)

> train [,.N/nrow(train),Cabin]
> test [,.N/nrow(test),Cabin]

> train [,.N/nrow(train),Embarked]
> test [,.N/nrow(test),Embarked]
```

Following are the insights we can derive from the data exploration above:

1. The variable `Fare` is skewed (right) in nature. We'll have to log transform it such that it resembles a normal distribution.
2. The variable `Parch` has one extra level (9) in the test set as compared to the train set. We'll have to combine it with its mode value.

A smart way to make modifications in train and test data is by combining them. This way, you'll save yourself from writing some extra lines of code. I suggest you follow every line of code carefully and simultaneously check how every line affects the data.

```
#combine data
> alldata <- rbind(train,test,fill=TRUE)
```

After looking at the data, I could figure out to create some new variables. Didn't you? If not, I request you to look deeper into the data; check the distribution of the dependent variable with predictor variables. You'd discover various interesting insights about probability of surviving passengers.

For now, we'll create two new variables. First, title of the passengers. This information can be extracted from `Name`. Second, I suspect that `Ticket` notation could give us some information. For example, some ticket notation starts with alpha numeric, while others only have numbers. We'll capture this trend using a binary coded variable.

```
#New Variables
#Extract passengers title
> alldata [,title := strsplit(Name,split = "[,.]")]
> alldata [,title := ldply(.data = title,.fun = function(x) x[2])]
> alldata [,title := str_trim(title,side = "left")]

#combine titles
> alldata [,title := replace(title, which(title %in% c("Capt","Col","Don","Jonkheer","Major","Rev","Sir")), "Mr"),by=title]
> alldata [,title := replace(title, which(title %in% c("Lady","Mlle","Mme","Ms","the Countess","Dr","Dona")),"Mrs"),by=title]

#ticket binary coding
> alldata [,abs_col := strsplit(x = Ticket,split = " ")]
> alldata [,abs_col := ldply(.data = abs_col,.fun = function(x)length(x))]
> alldata [,abs_col := ifelse(abs_col > 1,1,0)]
```

Next, we'll impute missing values, transform

```
Fare
```

 variable and remove an extra level from

```
Parch
```

 variable. This will make our data ready for machine learning.

```
#Impute Age with Median
> for(i in "Age")
        set(alldata,i = which(is.na(alldata[[i]])),j=i,value = median(alldata$Age,na.rm = T))

#Remove rows containing NA from Embarked
> alldata <- alldata[!is.na(Embarked)]

#Impute Fare with Median
> for(i in "Fare")
      set(alldata,i = which(is.na(alldata[[i]])),j=i,value = median(alldata$Fare,na.rm = T))

#Replace missing values in Cabin with "Miss"
> alldata [is.na(Cabin),Cabin := "Miss"]

#Log Transform Fare
> alldata$Fare <- log(alldata$Fare + 1)

#Impute Parch 9 to 0
> alldata [Parch == 9L, Parch := 0]
```

The method of using for - set loop for imputing missing values works blazing fast on large data sets. In our case, the data set is small, hence it's difficult to note the difference. Now, our data set is ready. Let's implement Logistic Regression and check our model's accuracy.

```
#Collect train and test
> train <- alldata[!(is.na(Survived))]
> train [,Survived := as.factor(Survived)]

> test <- alldata[is.na(Survived)]
> test [,Survived := NULL]

#Logistic Regression
> model <- glm(Survived ~ ., family = binomial(link = 'logit'), data = train[,-c("PassengerId","Name","Ticket")])
> summary(model)
```

In R, you can implement Logistic Regression using the

```
glm
```

 function. Now, let's understand and interpret the crucial aspects of summary:

1. The glm function internally encodes categorical variables into n - 1 distinct levels.
2. Estimate represents the regression coefficients value. Here, the regression coefficients explain the change in log(odds) of the response variable for one unit change in the predictor variable.
3. Std. Error represents the standard error associated with the regression coefficients.
4. z value is analogous to t-statistics in multiple regression output. z value > 2 implies the corresponding variable is significant.
5. p value determines the probability of significance of predictor variables. With 95% confidence level, a variable having p < 0.05 is considered an important predictor. The same can be inferred by observing stars against p value.

In addition, we can also perform an ANOVA Chi-square test to check the overall effect of variables on the dependent variable.

```
#run anova
> anova(model, test = 'Chisq')
```

Following are the insights we can collect for the output above:

1. In the presence of other variables, variables such as Parch, Cabin, Embarked, and abs_col are not significant. We'll try building another model without including them.
2. The AIC value of this model is 883.79.

Let's create another model and try to achieve a lower AIC value.

```
> model2 <- glm(Survived ~ Pclass + Sex + Age + SibSp + Fare + title, data = train,family = binomial(link="logit"))
> summary(model2)
```

As you can see, we've achieved a lower AIC value and a better model. Also, we can compare both the models using the ANOVA test. Let's say our null hypothesis is that second model is better than the first model. p < 0.05 would reject our hypothesis and in case p > 0.05, we'll fail to reject the null hypothesis.

```
#compare two models
> anova(model,model2,test = "Chisq")
```

With p > 0.05, this ANOVA test also corroborates the fact that the second model is better than first model. Let's predict on unseen data now. Since, we can't evaluate a model's performance on test data locally, we'll divide the train set and use model 2 for prediction.

```
#partition and create training, testing data
> library(caret)
> split <- createDataPartition(y = train$Survived,p = 0.6,list = FALSE)

> new_train <- train[split]
> new_test <- train[-split]

#model training and prediction
> log_model <- glm(Survived ~ Pclass + Sex + Age + SibSp + Fare + title, data = new_train[,-c("PassengerId","Name","Ticket")],family = binomial(link="logit"))
> log_predict <- predict(log_model,newdata = new_test,type = "response")
> log_predict <- ifelse(log_predict > 0.5,1,0)
```

For now, I've set the probability threshold value as 0.5. Let's get the flavor of our model's accuracy. We'll use AUC-ROC score to determine model fit. Higher the score, better the model. You can also use confusion matrix to determine accuracy using

```
confusionMatrix
```

 function from caret package.

```
#plot ROC
> library(ROCR)
> library(Metrics)
> pr <- prediction(log_predict,new_test$Survived)
> perf <- performance(pr,measure = "tpr",x.measure = "fpr")
> plot(perf) > auc(new_test$Survived,log_predict) #0.76343
```

Our AUC score is 0.763. As said above, in ROC plot, we always try to move up and top left corner. From this plot, we can interpret that the model is predicting more negative values incorrectly. To move up, let's increase our threshold value to 0.6 and check the model's performance.

```
> log_predict <- predict(log_model,newdata = new_test,type = "response")
> log_predict <- ifelse(log_predict > 0.6,1,0)

> pr <- prediction(log_predict,new_test$Survived)
> perf <- performance(pr,measure = "tpr",x.measure = "fpr")
> plot(perf)
> auc(new_test$Survived,log_predict)` ` #0.8008
```

![auc plot probability threshold logistic regression](http://blog.hackerearth.com/wp-content/uploads/2017/01/Rplot2.png)

Now, our AUC has increased to 0.80 along with a slight uplift in the ROC curve. With this, we've reached to the end of this tutorial. You can try and test AUC value for other values of probability threshold as well. From here, I would request you go ahead and test your model on the original test set, upload your solution and check your kaggle rank. Moving beyond Logistic Regression, you can further improve your model's accuracy using tree-based algorithms such as [Random Forest](http://blog.hackerearth.com/practical-tutorial-random-forest-parameter-tuning-r) or [XGBoost](http://blog.hackerearth.com/beginners-tutorial-on-xgboost-parameter-tuning-r). The complete code for this tutorial is also available on [Github](https://github.com/saraswatmks/Machine-Learning-in-R/blob/master/Logistic_Regression.R).

## Summary

This tutorial is meant to help people understand and implement Logistic Regression in R. Understanding Logistic Regression has its own challenges. No doubt, it is similar to Multiple Regression but differs in the way a response variable is predicted or evaluated. This tutorial is more than just machine learning. In the practical section, we also became familiar with important steps of data cleaning, pre-processing, imputation, and feature engineering.

While working on any classification problem, I would advise you to build your first model as Logistic Regression. The reason being that you might score a surprising accuracy even better than non-linear methods. If it's otherwise, you'd learn that the given data set would be better handled with non-linear methods, and you can use Logistic Regression's accuracy as your benchmark score.

While reading and practicing this tutorial, if there is anything you don't understand, don't hesitate to drop in your comments below!

https://www.hackerearth.com/practice/machine-learning/machine-learning-algorithms/logistic-regression-analysis-r/tutorial/

url: https://raw.githubusercontent.com/johnnymck/MoocX/52c8450ff7ecc8450a8adc2457233d5777a3d5bb/misc/hackerearth/ml/3MLAlgorithms/2Practical Guide to Logistic Regression Analysis in R.md

repository: johnnymck/MoocX

repo_name: johnnymck/MoocX


text:
AndroidxTreeView
====================

### Recent changes

- Migrate to androidx.
- Buildable in recent AndroidStudio (3.6.3) in 2020.
- Migrate to ViewBinding and DataBinding.
- Make a lot of Classes fluent setable.
- Update some dependencies

### Description

Tree view implementation for androidx.

This is a successor of the great work of Bogdan Melnychuk.

You can find the original project here: https://github.com/bmelnychuk/AndroidTreeView

repository: cdot/AndroidSquirrel

repo_name: cdot/AndroidSquirrel


text:
# Bank Tech Test

### About
Tech test to practice OOD, TDD and tech tests in week 10 of MA.

Clone the repo. Then in the Javascript console run:

```Javascript Console

account = new Account

deposit_transaction = {
     debit: 0,
     credit: 1000,
     date: "12/02/2012",
   }

withdraw_transaction = {
     debit: 500,
     credit: 0,
     date: "12/02/2012",
   }

account.transaction(deposit_transaction)
account.transaction(withdraw_transaction)
account.printTransactionHistory()
```

#### To run the tests

Open SpecRunner.html in a browser.

repository: EOSullivanBerlin/BankTechTest

repo_name: EOSullivanBerlin/BankTechTest


text:
# Jupyter notebooks as Markdown documents, Python or R scripts

[![Pypi](https://img.shields.io/pypi/v/jupytext.svg)](https://pypi.python.org/pypi/jupytext)
[![Pypi](https://img.shields.io/pypi/l/jupytext.svg)](https://pypi.python.org/pypi/jupytext)
[![Build Status](https://travis-ci.com/mwouts/jupytext.svg?branch=master)](https://travis-ci.com/mwouts/jupytext)
[![codecov.io](https://codecov.io/github/mwouts/jupytext/coverage.svg?branch=master)](https://codecov.io/github/mwouts/jupytext?branch=master)
![pylint Score](https://mperlet.github.io/pybadge/badges/9.9.svg)
[![pyversions](https://img.shields.io/pypi/pyversions/jupytext.svg)](https://pypi.python.org/pypi/jupytext)
[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/mwouts/jupytext/master?filepath=demo)

You've always wanted to
* edit Jupyter notebooks in your favorite editor?
* have Jupyter notebooks under version control?
* *collaborate* on Jupyter notebooks using standard (text-only) merge tools?

## Supported formats

The `jupytext` package allows you to open and edit, in Jupyter,
- Python and R scripts ( extensions `.py` and `.R`)
- Markdown documents (extension `.md`)
- R Markdown documents (extension `.Rmd`).

Obviously these documents can also be edited outside of Jupyter. You will find it useful to refactor your notebook as a mere python script in a real IDE. If you are working on a documentation and you prefer the Markdown format, you will be able to use both Jupyter and your specialized Markdown editor. Also, any of these text format is very convenient for versioning the notebook!

Reloading the updated document in Jupyter is just a matter of reloading the corresponding page in the browser. Refreshing preserves the python variables. Outputs are not stored in the text representation, so we recommend using [paired notebooks](#paired-notebooks) to also preserve the outputs.

| Format       | Extension          | text editor | git friendly | preserve output |
| ------------ | ------------------ | ----------- | ------------ | --------------- |
| jupyter notebook | `.ipynb`       |             |              | ✔               |
| script/markdown  | `.py`/`.R`/`.md`/`.Rmd` | ✔  | ✔            |                 |
| [paired notebook](#paired-notebooks)  | (`.py`/`.R`/`.md`/`.Rmd`) + `.ipynb` | ✔ | ✔ | ✔ |

## Try it!

- See how notebooks are represented as text in our [demo](https://github.com/mwouts/jupytext/tree/master/demo) folder.
- Open these text notebooks in Jupyter on [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/mwouts/jupytext/master?filepath=demo)! And run then like you would do for any classical notebook.
- You may also open arbitrary python scripts like `Matplotlib example.py` (run it).
- Feel free to explore the package files with Jupyter (open `README.md` at the project root).
- Check by yourself that outputs and variables are preserved, and inputs are updated, when a [paired notebook](#paired-notebooks) is modified outside of Jupyter (this is `Paired Jupyter notebook and python script.ipynb`).

## Installation

To open `.py`, `.R`, `.md` and `.Rmd` files as notebooks in Jupyter, use our `ContentsManager`. To do so:
- generate a Jupyter config, if you don't have one yet, with `jupyter notebook --generate-config`
- edit the config and include the following:
```python
c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
```

Then, make sure you have the `jupytext` package up-to-date and restart jupyter, i.e. run
```bash
pip install jupytext --upgrade
jupyter notebook
```

## Screenshots

The animated GIF below demonstrates the same notebook with three different extensions
- Original `.ipynb` file, in JSON format (5.5MB). Not adapted to text editors, but great for use in Jupyter, and also a perfect support for sharing your work. GitHub has full support for displaying notebooks and their outputs.
- Paired python script (`.py`, 2.2KB). An excellent format for developing Jupyter notebooks with complex code: code is easily refactored. Navigating through code and documentation is easier than in Jupyter. Step by step execution and breakpoints are accessible. Notebooks as python scripts are, in our opinion, the best candidate for versioning and collaborating on Jupyter notebooks - for sure you already have lots of practice on collaborating on simple scripts with Git, right? And when you're done with developping, you just need to refresh the notebook in Jupyter, run it all, save and share!
- Paired markdown document (`.md`, 2.2KB). An excellent format for developing documentation with few code samples.

![](https://raw.githubusercontent.com/mwouts/jupytext/master/img/jupyter_python_markdown.gif)

## Round-trip conversion

Round-trip conversion is safe! A few hundred tests help guarantee this.
- Script to Jupyter notebook, to script again is identity. If you
associate a Jupyter kernel with your notebook, that information will go to
a yaml header at the top of your script.
- Markdown to Jupyter notebook, to Markdown again is identity.
- Jupyter to script, then back to Jupyter again preserves source and metadata.
- Jupyter to Markdown, and Jupyter again preserves source and metadata (cell metadata available only for R Markdown). Note that Markdown cells with two consecutive blank lines will be split into multiple cells (as the two blank line pattern is used to separate cells).

## Paired notebooks

The idea of paired notebooks is to store a `.ipynb` file alongside the text-only version. This lets us get the best of both worlds: a text-only document to put under version control, and an easily sharable notebook which stores the outputs.

To enable paired notebooks, add a `jupytext_formats` entry to the notebook metadata with *Edit/Edit Notebook Metadata* in Jupyter's menu:
```
{
  "jupytext_formats": "ipynb,py",
  "kernelspec": {
    "name": "python3",
    (...)
  },
  "language_info": {
    (...)
  }
}
```

When you save the notebook, both the Jupyter notebook and the python scripts are updated. You can edit the text version
and then get the updated version in Jupyter by refreshing your browser (you may want to deactivate Jupyter's autosave with `%autosave 0`).

Accepted formats are: `ipynb`, `md`, `Rmd`, `py` and `R`. In case you want multiple text extensions, please note that the
order matters: the first non-`ipynb` extension
is the one used as the reference source for notebook inputs when you open the `ipynb` file.

Finally, it is also possible to pair every notebook with a text representation. If you add
```python
c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
c.ContentsManager.default_jupytext_formats = "ipynb,py" # or "ipynb,nb.py" # or "ipynb,md" # or "ipynb,Rmd"
```
to your Jupyter configuration file, then *every* Jupyter notebook that you save will have a companion `.py` (`.nb.py`, `.md`, or `.Rmd`) notebook. And every `.py` (`.nb.py`, `.md`, or `.Rmd`) notebook will have a companion `.ipynb` notebook.

## Command line conversion

The package provides a `jupytext` script for command line conversion between the various notebook extensions:

```bash
jupytext notebook.ipynb md --test          # Test round trip conversion
jupytext notebook.ipynb md                 # display the markdown version on screen

jupytext notebook.ipynb .md                # create a notebook.md file
jupytext notebook.ipynb .py                # create a notebook.py file
jupytext notebook.ipynb notebook.py        # create a notebook.py file

jupytext notebook.md .ipynb                # overwrite notebook.ipynb (remove outputs)
jupytext notebook.md .ipynb --update       # update notebook.ipynb (preserve outputs)

jupytext notebook1.md notebook2.py .ipynb  # overwrite notebook1.ipynb notebook2.ipynb
```

## Format specifications

### Markdown and R markdown

Our implementation for Jupyter notebooks as Markdown or R Markdown documents is straightforward:
- A YAML header contains the notebook metadata (Jupyter kernel, etc)
- Markdown cells are inserted verbatim, and separated with two blank lines
- Code and raw cells start with triple backticks collated with cell language, and end with triple backticks. Cell metadata are available in the [R Markdown format](https://rmarkdown.rstudio.com/authoring_quick_tour.html).

### R scripts

Implement these [specifications](https://rmarkdown.rstudio.com/articles_report_from_r_script.html):
- Jupyter metadata in YAML format, in a `#' `-escaped header
- Markdown cells are commented with `#' `
- Code cells are exported verbatim. Cell metadata are signalled with `#+`. Cells end with a blank line, an explicit start of cell marker, or a Markdown comment.

### Python scripts

We wanted to represent Jupyter notebooks with the least explicit markers possible. The rationale for that is to allow **arbitrary** python files to open as Jupyter notebooks, even files which were never prepared to become a notebook. Precisely:
- Jupyter metadata go to an escaped YAML header
- Markdown cells are commented with `# `, and separated with a blank line
- Code cells are exported verbatim (except for Jupyter magics, which are escaped), and separated with blank lines. Code cells are reconstructed from consistent python paragraphs (no function, class or multiline comment will be broken). A start-of-cell delimiter `# + {}` is used for cells that have explicit metadata (inside the curly bracket, in JSON format), and for cells that include blank lines (outside of functions, classes, etc). The end of cell delimiter is `# -`, and is omitted when followed by another explicit start of cell marker.

## Will my notebook really run in an IDE?

Well, that's what we expect. There's however a big difference between the python environments between Jupyter and Python IDEs, or RStudio: in the IDE code is executed with `python` rather than `ipython`. For this reason, `jupytext` escapes Jupyter magics found in your notebook. Comment a magic with `#noescape` on the same line to avoid escaping. User defined magics can be escaped with `#escape` (magics are not escaped in the plain Markdown representation).

Also, you may want some cells to be active only in the Python, or R Markdown representation. For this, use the `active` cell metadata. Set `"active": "ipynb"` if you want that cell to be active only in the Jupyter notebook. And `"active": "py"` if you want it to be active only in the Python script. And `"active": "ipynb,py"` if you want it to be active in both, but not in the R Markdown representation...

## I like this, how can I contribute?

Your feedback is precious to us: please let us know how we can improve `jupytext`. With enough feedback we will be able to transition from the current beta phase to a stable phase. Thanks for staring the project on GitHub. Sharing it is also very helpful! By the way: stay tuned for announcements and demos on<EMAIL_ADDRESS>and [twitter](https://twitter.com/marcwouts)!

repository: dldx/jupytext

repo_name: dldx/jupytext


text:
# domain-user-acct-mgmt
User account management scripts for AD and AAD

## Offboard-User.ps1
### An simple Powershell-based method for disabling user accounts
Prerequisites to run Offboard-User.ps1:
1. Download the Offboard-User.ps1 script and save to the AD server that also has ADSync installed.
2. Ensure you have the following modules installed, you may be prompted to install during the first run if you don't have them installed:
    - AzureAD
    - ExchangeOnlineManagement
    - ADSync
    - ActiveDirectory
3. Ensure you have WMF 5.1 installed. This will require a reboot if not already installed.

Steps to run:
1. Open Powershell as a user that has permission to make changes to ActiveDirectory and initiate a sync to AzureAD.
2. Change your directory to wherever the script is saved, for example the desktop of the user you're logged in as: `cd $env:userprofile\Desktop` or another example, `C:\Scripts\domain-user-acct-mgmt` would be: `cd C:\Scripts\domain-user-acct-mgmt`
3. Launch the script with the following command: `.\Offboard-User.ps1`
4. If you didn't satisify the prerequisites above, you may be prompted to install some additional modules and restart the script.
5. The first page you should see assuming prerequisites are satisfied should be a **Search for user by** page where you can enter the number of how you'd like to query for the user to disable.
6. You'll be prompted to search by whichever option you selected. Enter a query and we'll return a list of users that match your criteria. If only 1 user matches, then we'll confirm we're thinking of the same user.
7. You'll be prompted if you'd like to change the user's OU upon disabling. If so then:
    1. You'll hit Y to confirm.
    2. Search for the name of the OU. If you are looking for **Disabled Users**, type that in or part of that name like **disable**.
    3. Any OUs matching the search will be returned. Confirm the OU you would like to move the user to.
8. Once confirmed, we'll disable the user, change passwords, remove from groups except for the users' primary and if ADSync is configured for group based sync, we'll exclude that as well. If you're moving the user to another OU, that happens right before an ADSync kicks off.
9. Next, you'll be prompted to select the Office 365/Azure AD environment - select Commerical unless you are confident you need to connect to another environment. If you need another environment, please create an issue so it can be added.
10. Sign in with AzureAD/Office 365 credentials.
11. You should see that the user will be removed from Exchange Online only distribution groups, mailbox converted to a shared mailbox, AAD tokens are revoked, and licenses are removed from the user.
12. Once completed, you should get a list of licenses that can be reclaimed.

![Offboard-UserDemo](docs/assets/Offboard-UserDemo.gif)

repository: KSMC-TS/domain-user-acct-mgmt

branch_name: refs/heads/main

gha_language: PowerShell

repo_name: KSMC-TS/domain-user-acct-mgmt


text:
Include Assets extension for the HTML Webpack Plugin
========================================
[![npm version](https://badge.fury.io/js/html-webpack-include-assets-plugin.svg)](https://badge.fury.io/js/html-webpack-include-assets-plugin) [![Build Status](https://travis-ci.org/jharris4/html-webpack-include-assets-plugin.svg?branch=master)](https://travis-ci.org/jharris4/html-webpack-include-assets-plugin) [![js-semistandard-style](https://img.shields.io/badge/code%20style-semistandard-brightgreen.svg?style=flat-square)](https://github.com/Flet/semistandard)

Enhances [html-webpack-plugin](https://github.com/ampedandwired/html-webpack-plugin)
functionality by allowing you to specify js or css assets to be included.

Prior Version
------------

- `html-webpack-include-assets-plugin` version `2.x` and above require ** Node >= 8.6 **
- For older version of Node, please install version `1.x` and consult the [old readme](https://github.com/jharris4/html-webpack-include-assets-plugin/blob/master/README.V1.md)

Motivation
------------

When using a plugin such as [copy-webpack-plugin](https://github.com/kevlened/copy-webpack-plugin) you may have assets output to your build directory that are not detected/output by the html-webpack-plugin.

This plugin plugins gives you the tools to fix that and also lets you inject the webpack `publicPath` or compilation `hash` into your assets paths if you so choose.

Installation
------------
You must be running webpack on node 8.x or higher

Install the plugin with npm:
```shell
$ npm install --save-dev html-webpack-include-assets-plugin
```

Basic Usage
-----------
Require the plugin in your webpack config:

```javascript
var HtmlWebpackIncludeAssetsPlugin = require('html-webpack-include-assets-plugin');
```

Add the plugin to your webpack config:

```javascript
output: {
  publicPath: '/abc/'
},
plugins: [
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({ assets: ['a.js', 'b.css'], append: true })
]
```

Which will generate html like this:

```html
<head>
  <!-- other head content -->
  <link rel="stylesheet" href="/abc/b.css"/>
</head>
<body>
  <!-- other body content -->
  <script type="text/javascript" src="/abc/a.js"></script>
</body>
```

Options
-------
The available options are:

- `jsExtensions:string|array` - The file extensions to use when attempting to determine if an `assets` object is a `script`.

- `cssExtensions:string|array` - The file extensions to use when attempting to determine if an `assets` object is a `link`.

- `append:boolean` - Use `true` to `append` assets or `false` to `prepend` assets.

- `publicPath`: `boolean` or `string`, or `function(path:string, publicPath:string) => any:string` - (**default** `true`)

  * Determines hether the assets should be prepended with webpack's public path or a custom publicPath (string or function).

  * A value of `false` may be used to disable prefixing with webpack's publicPath, or a value like `myPublicPath/` may be used to prefix all assets with the given string. Default is `true`.

- `hash`: `boolean` or `function(path: string, hash: string) => any: string` - (**default** `false`)

  Specifying whether the assets should be appended with webpack's compilation hash. This is useful for cache busting. Default is `false`.

- `files`: `string` or `array` - (**default** `[]`)

  Files that the assets will be added to. Using this option can be necessary if you are using multiple instances of the this plugin or `html-webpack-plugin` plugin instances.

  By default the assets will be included in all files. If files are defined, the assets will only be included in specified file globs (uses the [minimatch](https://github.com/isaacs/minimatch) package).

- `links:string|array|object` - (**default** `[]`) - Shortcut to add assets that are all `type` `css`. Will be added **after** any `assets` values.

- `scripts:string|array|object` - (**default** `[]`) - Shortcut to add assets that are all `type` `js`. Will be added **after** any `assets` values.

- `assets:string|array|object` - (**default** `[]`)

  Assets that will be output into your html-webpack-plugin template.

  - To specify just one asset, simply pass a string or object.
  - To specify multiple, pass an array of strings or objects.

  Assets that do not have a `type` attempt to infer it from the asset `path` using the `jsExtensions` and `cssExtensions` options values.

  This plugin will throw an error if it cannot determine the type of any asset, we can specify asset as `objects` to fix that.

  ```javascript
  const oldAsset = 'abc'; // change this

  const newAsset = { // to this
    path: 'abc',
    type: 'css'
  }
  ```

  Each `asset object` may have the following properties:
-----
  - `path:string` (**required**) - The asset path.
  - `type:string` (optional) - For assets where the type is unknown, this can be set to either of: `['css', 'js']`.
  - `glob:string` and `globPath:string` (**must be together**) - Lets you use a [glob](https://github.com/isaacs/node-glob) to insert multiple assets from the `globPath`.
  - `attributes:object` (optional) - The attributes to be injected into the html tags. Some attributes are filtered by `html-webpack-plugin` (especially for script tags). To work **requires** that you set the `html-webpack-plugin` options to: `{ inject: true }`.
  - `assetPath:string` (optional) - property may be used to specify a source path to be added as an entry to `html-webpack-plugin`. This can be useful as it will trigger a recompilation after the assets have changed when using `webpack-dev-server` or `webpack-dev-middleware` in development mode.
  - `publicPath:boolean|function(path:string, publicPath:string) => any:string` (optional) - Controls whether the webpack `publicPath` will be injected into the asset path. `true` mean always. `false` means never. `function` means manual, `undefined` means use global settings.
  - `hash:boolean|function(path:string, hash:string) => any:string` (optional) - Controls whether the webpack `compilation hash` will be injected into the asset path. `true` mean always. `false` means never. `function` means manual, `undefined` means use global settings.

-----

Examples
-------

_____

Using `HtmlWebpackIncludeAssetsPlugin` and `CopyWebpackPlugin` to include assets to `html-webpack-plugin` template :

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['css/bootstrap.min.css', 'css/bootstrap-theme.min.css'],
    append: false
  })
]
```

_____

Appending and prepending at the same time :

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['css/bootstrap.min.css', 'css/bootstrap-theme.min.css'],
    append: false
  }),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['css/custom.css'],
    append: true
  })
]
```

_____

Using custom `jsExtensions` :

```javascript
plugins: [
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['dist/output.js', 'lib/content.jsx'],
    append: false,
    jsExtensions: ['.js', '.jsx']
  })
]
```

_____

Using custom `publicPath` :

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['css/bootstrap.min.css', 'css/bootstrap-theme.min.css'],
    append: false,
    publicPath: 'myPublicPath/'
  })
]
```

_____

Or to include assets without prepending the `publicPath`:

```javascript
plugins: [
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['css/no-public-path.min.css', 'http://some.domain.com.js'],
    append: false,
    publicPath: false
  })
]
```

_____

Manually specifying asset types :

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: [
      '/css/bootstrap.min.css',
      '/css/bootstrap-theme.min.css',
      { path: 'https://fonts.googleapis.com/css?family=Material+Icons', type: 'css' }
    ],
    append: false,
    publicPath: ''
  })
]
```

_____

Adding custom attributes to asset tags :

The bootstrap-theme link tag will be given an id="bootstrapTheme" attribute.

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: [
      '/css/bootstrap.min.css',
      { path: '/css/bootstrap-theme.min.css', attributes: { id: 'bootstrapTheme' } }
    ],
    append: false,
    publicPath: ''
  })
]
```

_____

Using `hash` option :

When the hash option is set to `true`, asset paths will be appended with a hash query parameter (`?hash=<the_hash>`)

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: ['css/bootstrap.min.css', 'css/bootstrap-theme.min.css'],
    append: false,
    hash: true
  })
]
```

_____

When the hash option is set to a `function`, asset paths will be replaced with the result of executing that function

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'somepath/somejsfile.js', to: 'js/somejsfile.[hash].js' },
    { from: 'somepath/somecssfile.css', to: 'css/somecssfile.[hash].css' }
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: [{ path: 'js', glob: '*.js', globPath: 'somepath' }],
    assets: [{ path: 'css', glob: '*.css', globPath: 'somepath' }],
    append: false,
    hash: function(assetName, hash) {
      assetName = assetName.replace(/\.js$/, '.' + hash + '.js');
      assetName = assetName.replace(/\.css$/, '.' + hash + '.css');
      return assetName;
    }
  })
]
```

_____

Specifying specific `files`

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin({
    filename: 'a/index.html'
  }),
  new HtmlWebpackPlugin({
    filename: 'b/index.html'
  }),
  new HtmlWebpackIncludeAssetsPlugin({
    files: ['a/**/*.html'],
    assets: ['css/a.css'],
    append: true
  }),
  new HtmlWebpackIncludeAssetsPlugin({
    files: ['b/**/*.html'],
    assets: ['css/b.css'],
    append: true
  })
]
```

_____

Specifying assets usings a `glob`

Note that since `copy-webpack-plugin` does not actually copy the files to webpack's output directory until *after* `html-webpack-plugin` has completed, it is necessary to use the `globPath` to retrieve filename matches relative to the original location of any such files.

```javascript
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: [{ path: 'css', glob: '*.css', globPath: 'node_modules/bootstrap/dist/css/' }],
    append: true
  })
]
```

_____

Specifying `links` (a shortcut for specifying assets of `type` `css`)

```javascript
output: {
  publicPath: '/my-public-path/'
},
plugins: [
  new CopyWebpackPlugin([
    { from: 'node_modules/bootstrap/dist/css', to: 'css/'},
    { from: 'node_modules/bootstrap/dist/fonts', to: 'fonts/'}
  ]),
  new HtmlWebpackPlugin(),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: [],
    append: true,
    links: [
      {
        href: 'asset/path',
        attributes: {
          rel: 'icon'
        }
      },
      {
        href: '/absolute/asset/path',
        asset: false,
        attributes: {
          rel: 'manifest'
        }
      }
    ]
  })
]
```

Will append the following link elements into the index template html

```html
<head>
  <!-- previous header content -->
  <link rel="icon" href="/my-public-path/asset/path"/>
  <link rel="manifest" href="/absolute/asset/path"/>
</head>
```

Note that the second link's href was not prefixed with the webpack `publicPath` because `csAsset.asset` was set to `false`.

_____

Caveats
-------

Some users have encountered issues with plugin ordering.

- It is advisable to always place any `HtmlWebpackPlugin` plugins **before** any `HtmlWebpackIncludeAssetsPlugin` plugins in your webpack config.

This plugin has only been tested with **two instances** in one webpack config, where one had `option.append: false` and the other had `option.append: true`.

- It is **not recommended to use more than one instance of this plugin** in one webpack config unless using the above configuration.

Changing `HtmlWebpackPlugin.options.inject` from its **default value** may cause **issues**.

- This plugin **requires** `HtmlWebpackPlugin.options.inject` to be `true` (it defaults to true if undefined) for attribute injection to work.

If you setup your webpack config to have `HtmlWebpackPlugin.options.inject: false` like this:

```javascript
output: {
  publicPath: '/the-public-path/'
},
plugins: [
  new HtmlWebpackPlugin({ inject: false }),
  new HtmlWebpackIncludeAssetsPlugin({
    assets: [{ path: 'css/bootstrap-theme.min.css', attributes: { id: 'bootstrapTheme' } }],
    links: [{ href: 'the-ref', attributes: { rel: 'icon' } }],
    append: true
  })
]
```

You will need to add the following to your *template* `index.html` to get assets to be **generated**:

```html
<head>
  <!-- other head content -->
  <% for (var cssIndex = 0; cssIndex < htmlWebpackPlugin.files.css.length; cssIndex++) { %>
    <link rel="stylesheet" href="<%= htmlWebpackPlugin.files.css[cssIndex] %>">
  <% } %>
</head>
<body>
  <!-- other body content -->
  <% for (var jsIndex = 0; jsIndex < htmlWebpackPlugin.files.js.length; jsIndex++) { %>
    <script src="<%= htmlWebpackPlugin.files.js[jsIndex] %>"></script>
  <% } %>
</body>
```

Using the (lodash) `template syntax` like this for css and js files is necessary when you turn injection off.

But, the `template syntax` does not allow injection of more than `one attribute value`.

This means it will **generate** an `index.html` that looks like this:

```html
<head>
  <link rel="stylesheet" href="/the-public-path/css/bootstrap-theme.min.css">
  <link rel="stylesheet" href="/the-public-path/the-ref">
</head>
```

None of the `link` elements have any of the `attributes` we specified for the `assets` or `links`.

This is because `HtmlWebpackPlugin.options.inject` needs to be set to `true` for `attributes` injection to work.


text:
# Connect to PostgreSQL

When using Nanobox, projects are meant to be both portable and secure. In order for your project to be both, you should use environment variables in your PostgreSQL connections.

## Use Environment Variables to Connect
In your app's connection to PostgreSQL, use the auto-generated environment variables.

```yaml
# ruby database.yml example

production:
  adapter: postgresql
  encoding: unicode
  database: <%= ENV['DATA_DB_NAME'] %>
  pool: 5
  username: <%= ENV['DATA_DB_USER'] %>
  password: <%= ENV['DATA_DB_PASS'] %>
```

## Auto-Generated Environment Variables
When data components are provisioned with Nanobox, unique connection credentials are generated along with environment variables for each. Secure connections to PostgreSQL require a host, port, database name, user, and password.

**Port**
The port will always be the default PostgreSQL port `5432`.

**Name**
The database name is provided by Nanobox and will always be `gonano`.

#### Variable Naming Convention
Environment variable names/keys are generated using the ID of the PostgreSQL component in your boxfile.yml - data.postgres, data.whatever, etc. Letters are capitalized and special characters are replaced with underscores. For example:

```yaml
data.postgres => DATA_POSTGRES
data.db       => DATA_DB
data.cthulu   => DATA_CTHULU
```

Using this convention, environment variables for each required credential are generated.

```yaml
# Component ID
data.postgres

# Auto-Generated Environment Variables
DATA_POSTGRES_HOST
DATA_POSTGRES_USER
DATA_POSTGRES_PASS
```

## Viewing Environment Variables
To view the environment variables in your app, you can use the `evar ls` command.

```bash
# View environment variables in local environment
nanobox evar ls local
```

repository: nanobox-io/nanobox-guides

gha_language: Pug

repo_name: nanobox-io/nanobox-guides


text:
# saslprep

This package provides a common lisp implementation of unicode stringprep functions d Annex #15 found at [http://www.unicode.org/reports/tr15/tr15-22.html](http://www.unicode.org/reports/tr15/tr15-22.html) as well as saslprep and stringprep functions required by  [https://tools.ietf.org/html/rfc5802](https://tools.ietf.org/html/rfc5802) and
found in:

  * [https://tools.ietf.org/html/rfc4013](https://tools.ietf.org/html/rfc4013) (2005) SASLprep
  * [https://tools.ietf.org/html/rfc3454](https://tools.ietf.org/html/rfc3454) (2002) Stringprep

This is a fork of a subset of work done by Takeru Ohta in 2010. Future work is intended to provide support for:

# Implementation Notes

# Usage
It has one major exported function:

  * (saslprep-normalize (str)

Saslprep-normalize example
```common-lisp
    (saslprep-normalize "some-string-here")

```

Explanation of what saslprep-normalize adds to the process RFC 4013 and RFC 3454:

1. First map non-ascii space characters to space #x0020
2. Then delete characters that are commonly mapped to nothing
3. KC Normalization as described in RFC 3454 Table 4
4. Generate an error if there are any prohibited input characters

    * Non-ascii space characters (RFC 3454 Table c.1.2)
    * Ascii control characters (RFC 3454 Table c.2.1)
    * Non-ascii control characters (RFC 3454 Table c.2.2)
    * Private use characters (RFC 3454 Table c.3)
    * Non-character code points (RFC 3454 Table c.4)
    * Surrogate code points (RFC 3454 Table c.5)
    * Inappropriate for plain text characters (RFC 3454 Table c.6)
    * Inappropriate for canonical representation characters (RFC 3454 Table, C.7)
    * Change display properties or deprecated characters (RFC 3454 Table, C.8)
    * Tagging characters (RFC 3454 Table, C.9)

5. Check for bidirectional strings (RFC 3454 Table Section 6) and ensure that there are no conflicting directions.

# To Do list
  * Support for RFC 8264 [https://tools.ietf.org/html/rfc8264](https://tools.ietf.org/html/rfc8264) (2017)
  * Support for RFC 7564 [https://tools.ietf.org/html/rfc7564](https://tools.ietf.org/html/rfc7564) (2015)
  * Review test failures in ABCL and Allegro
  * All the things I do not understand yet
  * Optimization?

# Data Files
  * UnicodeData.txt was downloaded from [http://www.unicode.org/Public/UNIDATA/UnicodeData.txt](http://www.unicode.org/Public/UNIDATA/UnicodeData.txt)
  * The file CompositionExclusions.txt was downloaded from [http://www.unicode.org/Public/UNIDATA/CompositionExclusions.txt](http://www.unicode.org/Public/UNIDATA/CompositionExclusions.txt)
  * The file DerivedNormalizationProps.txt was downloaded from [http://www.unicode.org/Public/UNIDATA/DerivedNormalizationProps.txt](http://www.unicode.org/Public/UNIDATA/DerivedNormalizationProps.txt)
  * The test file NormalizationTest.txt was downloaded from [http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt](http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt)

# Other References
  * [https://tools.ietf.org/html/rfc5802](https://tools.ietf.org/html/rfc5802)
  * [https://tools.ietf.org/html/rfc4013](https://tools.ietf.org/html/rfc4013)
  * [https://tools.ietf.org/html/rfc3454#section-3.1](https://tools.ietf.org/html/rfc3454#section-3.1)
  * [http://www.unicode.org/reports/tr15/#References](http://www.unicode.org/reports/tr15/#References)
  * [https://www.unicode.org/reports/tr41/tr41-24.html](https://www.unicode.org/reports/tr41/tr41-24.html)
  * [https://www.unicode.org/charts/normalization/](https://www.unicode.org/charts/normalization/)
  * [https://towardsdatascience.com/difference-between-nfd-nfc-nfkd-and-nfkc-explained-with-python-code-e2631f96ae6c](https://towardsdatascience.com/difference-between-nfd-nfc-nfkd-and-nfkc-explained-with-python-code-e2631f96ae6c)
  * [https://en.wikipedia.org/wiki/Unicode_equivalence](https://en.wikipedia.org/wiki/Unicode_equivalence)
  * [http://www.unicode.org/faq/normalization.html](http://www.unicode.org/faq/normalization.html)
  * [https://github.com/edicl/cl-unicode/blob/master/specials.lisp](https://github.com/edicl/cl-unicode/blob/master/specials.lisp)
  * [https://perldoc.perl.org/Unicode/Normalize.html](https://perldoc.perl.org/Unicode/Normalize.html)
  * [https://www.mkssoftware.com/docs/perl/lib/Unicode/Normalize.asp](https://www.mkssoftware.com/docs/perl/lib/Unicode/Normalize.asp)
  * [https://github.com/Wisdom/Awesome-Unicode](https://github.com/Wisdom/Awesome-Unicode)
  * [https://www.postgresql.org/docs/current/multibyte.html](https://www.postgresql.org/docs/current/multibyte.html)

repository: sabracrolleton/saslprep

repo_name: sabracrolleton/saslprep


text:
# Part 1

**Searching for words across files**

    - The best way to search for words in multiple files was using the grep command

    - The user needs to cd into the location of the files they are wanting to search

    - Next the user will use the command grep -l <word you are searching for> f*

    - -l prints the file(s) containing the word searched for

    - f* tells the command to search for the word whose file names begin with an f ex. file00000, file00102, etc.

**Moving files**

    - The command I used to move the files matching my search was mv

    - mv $(grep -l <word> f*) ../<word>

    - The above command while place all file instances in the directory I choose at the end of the command

    - ../<word> and word is considered my directory in this example

    - I could also list out the exact location of the directory as well

# Part 2

**Task 1**

    - The command beginning with the word sort, allows me to place my list in alphabetical order

    - Even if the words repeat it will list all instances before continuing to the next set of words in order

    - The > allows me to create a new .txt file with my new sorted list of words

    - The command I ran in my bash terminal was

        sort shuffled-words.txt > shuffled-words-sorted.txt

**Task 2**

    - As listed in the previous task, the sort command sorts my list in alphabetical order

    - The list I am using is a rand-words.txt

    - I use the uniq -c command after typing the file I am wanting to use

    - The uniq -c counts my words

    - -nr will make my list sort numerically in a reverse fashion for example:

        dog 10
        cat 5
        bird 3

    - head -10 will list my top 10 words in reverse numerical order with how many times each of those words occured

    - Lastly as listed in task 1, the > allows me to create a new .txt file with my new sorted list of words

    - The command I ran was

        sort rand-words.txt | uniq -c | sort -nr |head -10 > common-words.txt

**Task 3**

    - This task is very similar to the previous task only using numbers in a .txt file

    - I ran this command once using only the -n and it gave me a list of 10 numbers but only showing that each of them occured once

    - Be sure to use the -nr portion so that you get the numerically reverse sorted listed of most occurences of each number

    - The command I ran was

        sort rand-numbers.txt | uniq -c | sort -nr | head -10 > common-numbers.txt

**Task 4**

    - The previous command of

        sort rand-numbers-2.txt | uniq -c | sort -nr| head -10 > common-2.txt

        Has the same functionality and allows for me to check on the pairs of numbers

    The only difference I have between task 3 and 4 is the .txt file to pull info from and to send the new info to

**Task 5**

    - I was unable to sort out by columns as listed in the instructions

        1, 2, 3
        2, 3, 4
        3, 4, 5
        1, 3, 4
        1, 4, 5
        1, 3, 5
        2, 4, 5

    - I was able to create a list of the top 10 of each set of five numbers like in task 4


text:
# Tracking Consent

### Tracking consent values

To be compliant with the GDPR regulation, the SDK requires the tracking consent value at initialization.
The `trackingConsent` can be one of the following values:

1. `TrackingConsent.pending` - the SDK starts collecting and batching the data but does not send it to Datadog. The SDK waits for the new tracking consent value to decide what to do with the batched data.
2. `TrackingConsent.granted` - the SDK starts collecting the data and sends it to Datadog.
3. `TrackingConsent.notGranted` - the SDK does not collect any data: logs, traces, and RUM events are not sent to Datadog.

### Updating the tracking consent at runtime

To change the tracking consent value after the SDK is initialized, use the `Datadog.set(trackingConsent:)` API call.
The SDK changes its behavior according to the new value. For example, if the current tracking consent is `.pending`:

- if changed to `.granted`, the SDK will send all current and future data to Datadog;
- if changed to `.notGranted`, the SDK will wipe all current data and will not collect any future data.

repository: iosovo/dd-sdk-ios

gha_language: Swift

gha_license_id: NOASSERTION

repo_name: iosovo/dd-sdk-ios


text:
# Stop Dataflow job

Quick snippet to showcase how to stop a Dataflow job programmatically

## Quickstart

Install the following dependencies:
``` bash
pip install --upgrade google-api-python-client oauth2client
```

Modify the parameters in the script:
``` python
# change the parameters accordingly
project='PROJECT_ID'
job='JOB_ID'
location='REGION'
```

and run the script:
``` bash
python stop.py
```

Tested with:
* `google-api-python-client==1.7.4`
* `oauth2client==4.1.2`

## Example

Dataflow jobs can be stopped through the UI or using the `gcloud` [command](https://cloud.google.com/sdk/gcloud/reference/dataflow/jobs/cancel). However, if we inspect the Dataflow API, we can see that the jobs REST [collections](https://cloud.google.com/dataflow/docs/reference/rest/) do not have a `cancel` method.

If we trace the aforementioned command by running:
``` bash
gcloud dataflow jobs cancel <JOB_ID> --log-http
```

We'll see the following output snippets:
``` bash
==== request start ====
uri: https://dataflow.googleapis.com/v1b3/projects/<PROJECT_ID>/locations/<REGION>/jobs/<JOB_ID>?alt=json
method: PUT
...
== body start ==
{"requestedState": "JOB_STATE_CANCELLED"}
== body end ==
```

Basically, we update the job with a request to [`projects.locations.jobs.update`](https://cloud.google.com/dataflow/docs/reference/rest/v1b3/projects.locations.jobs/update) specifying that the desired state is `JOB_STATE_CANCELLED`.

Translating this into Python code:
``` python
result = dataflow.projects().locations().jobs().update(
	projectId=project,
	jobId=job,
	location=location,
	body=state
).execute()
```
where `state` is `{"requestedState": "JOB_STATE_CANCELLED"}`.

## License

These examples are provided under the Apache License 2.0.

## Issues

Report any issue to the GitHub issue tracker.


text:
# Learning Sparse Sharing Architectures for Multiple Tasks

This is a Pytorch implementation for [Learning Sparse Sharing Architectures for Multiple Tasks](https://arxiv.org/pdf/1911.05034.pdf), accepted by AAAI 2020 (Oral).

Illustration of our approach to learn sparse sharing architectures. Gray squares are the parameters of a base network. Orange squares represent the shared parameters, while blue and green ones represent private parameters of task 1 and task 2 respectively.

## Requirements

- pytorch == 1.2.0
- fastNLP == 0.5.0

## Data
There are three experiment settings in our original paper:
- `conll03`: Conll2003 (POS + Chunk + NER)
- `ontonotes`: OntoNotes 5.0 (POS + Chunk + NER)
- `ptb`: PTB (POS) + Conll2000 (Chunk) + Conll2003 (NER)

In this instruction, we give an example to reproduce the experiment on `conll03`.

To process the dataset, run the following code.
And the processed data will be saved at `./data` folder.
```shell script
python prepare-data.py \
  --type conll03 \
  --pos /path/to/pos \
  --chunk /path/to/chunk \
  --ner /path/to/ner
```

## Trainig
There are two kinds of training in our experiments:

- `single`: Learning sparse sharing architecture, generating subsets for each task.
- `mtl`: Training subsets in parallel.

#### Single task pruning
The first step is to generate subnets for each task. This is done by running:
```shell script
bash run_conll03.sh single
```
We suggest generate subsets with Multi-Task Warmup, by runing:
```shell script
bash run_conll03.sh single /path/to/mtl-checkpoints
```

The generated subsets are stored in the folders named after numbers under `./exp/conll03/single/cp/`,
The subsets' are named after `[#pruning]_[nonzero param percent].th`.
You can choose different subsets for each task by editing the numbers `MASK_LIST` in the script `prepare-masks.sh`,
where each number represents the number of pruning, `#pruning`.
After editing the numbers, run:
```shell script
bash prepare-masks.sh conll03
```
to prepare the masks and initial checkpoints for multi-task training.

#### MTL training
Run the following script for multi-task training with different subsets in parallel:
```shell script
bash run_conll03.sh mtl
```
**Note**: before running another MTL training, edit the `NAME` variable in `run_conll03.sh` to specify a different name for the training run.
Otherwise, the previous experiment results may be overwritten.

#### Evaluation
When the training finished, the model will be evaluated on the test sets automatically.

#### More
You can also directly run the python file `train_single.py` and `train_mtl.py` with different arguments to gain more flexibility.
And use the argument `-h` to see the full arguments descriptions.

## Citation

```
@inproceedings{sun2020sparsing,
  author = {Tianxiang Sun and Yunfan Shao and Xiaonan Li and Pengfei Liu and Hang Yan and Xipeng Qiu and Xuanjing Huang},
  title = {Learning Sparse Sharing Architectures for Multiple Tasks},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://arxiv.org/abs/1911.05034}
}
```

## Acknowledgments
[fastNLP](https://github.com/fastnlp/fastNLP)


text:
# blynkapi

This is a simple blynk HTTP/HTTPS API wrapper.

### Introduction
This library created for simple using [Blynk API](http://docs.blynkapi.apiary.io/#reference/0/pin-actions) and manage your project via python. It's simple one class lib which used `urllib2` for requests.

You can also:
  - Read/write pin value
  - Check hardware network status
  - Check application network status
  - Send push notification
  - Send email
  - Get pin history data
  - Get QR for project cloning
  - Get project info
  - Query API

### Installation

Install via pip
```bash
pip install blynkapi
```
after installation import it in your code
```python
from blynkapi import Blynk
```

Or simple copy `blynkapi/Blynk.py` to your main python program dir and import it
```python
from Blynk import *
```

### Using
```python
# if you install it from pip, else use `from Blynk import *`
from blynkapi import Blynk
# vars
auth_token = "sdjflksjflkdsjfkldsjfkldfkldjflk"

# create objects
room_light = Blynk(auth_token, pin = "V3")
kitchen_light = Blynk(auth_token, pin = "V4")
house_door = Blynk(auth_token, pin = "V5")
amp_power = Blynk(auth_token, pin = "V6")

# get current status
res = room_light.get_val()
print res

# set pin value (one)
amp_power.set_val(["120"])

# set pin value to 1
room_light.on()
# set pin value to 0
room_light.off()
```

### Available methods

All methods return values from API or error if it fixed. For detailed description of returning values and scheme read official [Blynk API guide] (http://docs.blynkapi.apiary.io/#reference)

Creating main object
```python
Blynk(token, server, protocol, port, pin, value)
```
  - `token` - Your project token
  - `server` - Blynk server for API requests, default "blynk-cloud.com"
  - `protocol` - http or https, default "http"
  - `port` - Your Blynk server API port, default "8080"
  - `pin` - Pin for working with, default "None"
  - `value` - Value for method get_val, default "None"

To turn pin on (set val "1" to pin)
```python
on()
```

To turn pin off (set val "0" to pin)
```python
off()
```

To set your value for pin
```python
set_val(value)
```
  - `value` - Custom value in list ["x"] or multiple values in list ["x", "y"]. [Details] (http://docs.blynkapi.apiary.io/#reference/0/write-pin-value-via-put/write-pin-value-via-put)

To get data from pin
```python
get_val()
```

Send push notification
```python
push(value)
```
  - `value` - String no more than 255 chars.

Send email
```python
email(to, title, subj)
```
  - `to` - String
  - `title` - String
  - `subj` - String

Check hardware status (connection to server)
```python
hw_status()
```

Check app status (connection to server)
```python
app_status()
```

Get pin history
```python
history()
```

Get QR image
```python
qr()
```

Get project info For details see [docs] (http://docs.blynkapi.apiary.io/#reference/0/get-project/get-project)
```python
get_project()
```

Query API. For details see [docs] (http://docs.blynkapi.apiary.io/#reference/0/query-api/query-api)
```python
query_api(groupBy, aggregation, pin, value)
```
  - `groupBy` - String
  - `aggregation` - String
  - `pin` - String
  - `value` - String

### Todos

 - Write Tests

License
----

MIT

repository: microplc/blynkapi

repo_name: microplc/blynkapi


text:
# Magic 8 Ball

A magic 8 ball iOS app with motion detection.

[![demo](https://github.com/DudkinAS/Magic-8-Ball-iOS-app/blob/master/img/demo.png?raw=true)](img/demo.png)

## LICENSE
code share under [MIT](LICENSE) License
all picture assets provided by [London App Brewery](https://www.londonappbrewery.com)


text:
# How to run via Docker

1. Make sure you are in the `docker` group.
Execute `sudo usermod -a -G docker $USER` and reboot.
2. From the root project directory execute
`./gradlew build && docker-compose build --no-cache && docker-compose up -d cassandra-test && docker-compose up ignite-cassandra-client
`
That should build the project and create a corresponding
image from that. Then a Cassandra instance should be started and
after that a client instance should be brought up.
The client will try to connect to Cassandra and eventually
you should see that client has successfully put and gotten value from Cassandra
`org.testprojects.ignite.ignte_cassandra_client.EntityWithMapField{attrs={1=one, 2=two}}`

# How to run from host with a custom Cassandra installation

1. Add your Cassandra contact point if required in [CassandraConfig#getDataSource](src/main/java/org/testprojects/ignite/ignte_cassandra_client/CassandraConfig.java)
2. Run [IgniteServerRunner](src/main/java/org/testprojects/ignite/ignte_cassandra_client/IgniteServerRunner.java)


text:
# Low Power Temperature and Humidity Sensor
Has the schematics and code to create a battery power temperature and humidity sensor.
Based on the design from the [Home Circuits Blog](https://homecircuits.eu/blog/battery-powered-esp8266-iot-logger/).

## Setup Arduino Uno
The Arduino IDE can be used to program the ATtiny13.
I followed the instructions on [this instructable](https://www.instructables.com/id/Updated-Guide-on-How-to-Program-an-Attiny13-or-13a/)
as they worked better for me than what was on the blog.

### Connect the Uno and ATtiny
Here is the breadboard layout.
![Programmer](images/ATtiny13 Programmer_Schematic.png)
The Pinout follows this table

| Arduino Uno Pin  | ATtiny Pin | Type  |
|------------------|------------| ------|
| 5v               | Pin 8      | Vcc   |
| GND              | Pin 4      | GND   |
| Pin 13           | Pin 7      | SCK   |
| Pin 12           | Pin 6      | MISO  |
| Pin 11           | Pin 5      | MOSI  |
| Pin 10           | Pin 1      | RESET |

This figure shows the pinout of the ATtiny13
![ATtiny13 Pinout](images/ATtiny13.jpg)

### Setting Up the Arduino As ISP (In-System Programmer)
Arduino IDE already contains the program that needs to be loaded to Arduino board, so then it acts the programmer for other boards.
Navigate to Files-Examples-ArduinoISP and click on ArduinoISP.
Configure Bard type to Arduino UNO, select the correct Serial Port and Upload the sketch as any other sketch to Arduino board.
At this point Aruduino board is prepared to be programmer for other microcontrolers.

### Downloading the Attiny13(a) Core Files
MicroCore requires Arduino IDE version 1.6.13 or greater.

* Open the Arduino IDE.
* Open the File > Preferences menu item.
* Enter the following URL in Additional Boards Manager URLs:
https://raw.githubusercontent.com/sleemanj/optiboot/master/dists/package_gogo_diy_attiny_index.json
* Open the Tools > Board > Boards Manager... menu item.
* Wait for the platform indexes to finish downloading.
* Scroll down until you see the DIY ATtiny and click on it.
* Click Install.
* After installation is complete close the Boards Manager window.

### Burning the Bootloader to the Attiny
Next, we want to add the Bootloader to the ATtiny
* Go to Tools-Board and select ATtiny13.
* After selecting ATtiny13, navigate to Tools-Processor Version and select ATtiny13a.
* It is not necessary to change any other settings
* Change the programmer to be set as "Arduino as ISP"
* Finally, click the Burn Bootloader button at the bottom of the tools drop-down menu.

This is a figure of the settings
![ArduinoIDE](images/ArduinoIDE.png)

### Programming the Chip
This is the last part for setting up the ATtiny.
I took the code from the blog and use the Arduino IDE to program the chip.
The code can be found [here](https://gitlab.com/snippets/30510).
A few changes to the code have been made and uploaded to this repository [here](attiny/attiny.ino).

repository: JimInCO/low-power-temp-humidity

repo_name: JimInCO/low-power-temp-humidity


text:
---
layout: post
title: First vim Blog Entry
---

Blogging from a "blog" alias i set up in ~/.inputrc to vi a file in /User/dickie/Documents/rljhaines.github.io/_posts/`date +%Y-%m-%d`.md. I used the Mac (not MAC, that's Media Access Control) GitHub application to get git to my user directory Documents folder prior to that. I also set my borne shell to use vi through a .bashrc file i found online at github of all places. I will note it soon. I put it in the .inputrc file, however.

I also found sample .bashrc and .bash_profile (whatever that is) here: http://tldp.org/LDP/abs/html/sample-bashrc.html. I will check them out later.

Ha! it was and .inputrc file and it is here: https://gist.github.com/gregorynicholas/1812027.

Earlier I took a brief look at https://try.github.io/levels/1/challenges/10 and
am hoping to post this entry with however much ado is required... Hoping not muc
h :-p

Uh oh, now I can't open this file for editing... need to figure that out first!!
! Ah ha! typo in path, trying again!!!

This pushed to github, but did not push through to the blog. For now I hope this modification will allow it to push through (vi Jekyll i believe) to rljhaines.github.io.

Okay, that didn't work. I will give the file a name suffixed to the current date moniker.

Okay, that worked. Did I not read carefully enough to know the naming requirement for Jekyll? Hmmmmm....

Now I will see if these changes I'm now entering into this properly-named post will push through to the blog!

Sure enough!!! git add, commit, push gets the job done :-)


text:
# 4.0.0
* Added: support for MongoDB 3.6
* Changed: config file format, see new example-config.js for reference. Use `aliases` to keep old URLs working

# 3.4.0
* Changed: design rework
* Added: select field in find()
* Added: copy query button
* Changed: improved field suggestions
* Changed: respect date, id and binary display settings on "Show content"

# 3.3.0
* Added: auto completion \o/

# 3.2.0
* Added: prompt mode and API

# 3.1.0
* Fixed: context menu sometimes not showing
* Added: support for mongodb 3.2 new aggregation stages

# 3.0.0

## Breaking changes
* Changed: use `PM2` instead of `forever` in `npm start`
* Changed: dropped support for node v0.10

## Non-breaking changes
* Added: option to display object ids as timestamps
* Added: plot data (beta) (supported chart types: area, bar, column, line, histogram)
* Fixed: [#19](https://github.com/clubedaentrega/node-mongo-admin/issues/19)
* Fixed: remove some clutter from the UI

# 2.1.1
* Fixed: `json.preParse`

# 2.1.0
* Added: support for GeoJSON
* Added: support for count queries
* Changed: custom input elements

# 2.0.0
* Added: support for multiple users in basic http auth
* Added: allow a users to use only some connections

## Breaking changes
`basicAuth` field in config is now an array. If you're using basic auth, change from:

```js
basicAuth: {
	user: 'x',
	password: 'pass'
}
```

to

```js
basicAuth: [{
	user: 'x',
	password: 'pass'
}]
```


text:
# Includes Parameters
Documentation on parameters used for includes throughout the site. For information on includes see the [Jekyll docs](https://jekyllrb.com/docs/includes/).

## cite-page
Displays the cite this page button.
`{%- include cite-page.html classes="page-highlights__col page-higlights__cite" -%}`

- `classes`: Secondary classes to attach to the component container

## download-pdf
Displays the link to download an associated PDF if there is one for a post.
`{%- include download-pdf.html classes="page-highlights__col page-higlights__download" -%}`

- `classes`: Secondary classes to attach to the component container

## img-caption
Displays the caption and source for a photo.
`{%- include img-caption.html tag='p' desc=page.image_caption source=page.image_credit -%}`

- `tag`: The HTML tag to use. Default is `figcaption`
- `desc`: The image caption
- `source`: The image source (optional)

## page-header-background-img
Used by the `page-header` include to capture a `style` attribute for the page header's background image.
`{% include page-header-background-img.html image_url=page.image %}`

- `image_url`: The URL to the image to display

## post-block
`{% include post-block.html hide_excerpt=true show_image=true %}`

- `hide_excerpt`: Set to `true` to hide the post excerpt
- `show_image`: Set to `true` to show the post thumbnail
- `class`: Add additional classes to block. Useful for specifying a featured post
- `num_themes`: Leave blank to show all themes, otherwise set to number of themes to display. Parameter is passed to the `themes-list` include.

## post-meta-authors
`{% include post-meta-authors.html authors=page.authors %}`

- `authors`: Set to `page.authors` in page-header (post-meta-authors will rely on post.authors by default)

## themes-list
`{%- include themes-list.html num_themes='1' -%}`
- `num_themes`: Leave blank to show all themes, otherwise set to number of themes to display

repository: CSIS-iLab/health-commission

gha_language: HTML

repo_name: CSIS-iLab/health-commission


text:
# Meeting Minutes

Jeanne pointed out some data problems. She noted that there were something English sentences where there were meant to be Setswana translations.
She also noticed a couple of messy tokens.

Jade's suggestion: For now, sample about 100 of them and see if it occurs. If not, then it's probably small enough to be ignorable
Idea: Someone find a language detector for South African languages at least (I think Bernhard Duvenhage built one).

Benji brought up the idea of data version control. Conclusion: the test sets should go into github - which is already current ways of working

Abdullah asked what the "4000" number is for the BPE tokenization.
Jade: It's the number of BPE tokens one should create.

Vukosi reported that he is asking for HANSARDS for multiple languages but apparently they only translate from other languages to english and not vice versa. Jade recommended SaDiLAR data for Xitsonga.

Alp joined to find out if we'd heard from EthioNLP. We hadn't so Jade is chasing up.
Alp also wanted to know about running the notebook on a server. Jade recommended we download it as a .py and run that instead.

There was a question about how incorporating external data fits into the project. The answer: Please do! In fact, it is encouraged. Note: It MUST just be open source.

Blessing asked a question about website scraping. Vukosi highlighted that news websites have started changing terms and conditions to not allow scraping. Blessing suggested he could try contact the website to ask for permission - Vukosi offered to read over the request/letter to increase the possibility of it getting accepted.

There was a bug in the most recent version of the notebook. It's fixed now.
A note on validation_freq based on screen-sharing session with Abdullah, quoted from JoeyNMT documentation:

```
How often should I validate?

Depends on the size of your data. For most use-cases you want to validate at least once per epoch. Say you have 100k training examples and train with mini-batches of size 20, then you should set validation_freq to 5000 (100k/20) to validate once per epoch.
```

Jade worked with Musie to get Amharic PR in - first official result which can be used as an example for future PRs. Screen sharing sessions via zoom worked really well

repository: Niger-Volta-LTI/masakhane

gha_language: Jupyter Notebook

repo_name: Niger-Volta-LTI/masakhane