text:
# Pine

[![Build Status](https://travis-ci.org/thomas-crane/pine.svg?branch=master)](https://travis-ci.org/thomas-crane/pine)

Pine is a general purpose, type checked language which is heavily inspired by Rust.
Pine compiles to Node.js style JavaScript.

## Contents

+ [Foreword](#foreword)
+ [Install](#install)
  + [From GitHub](#from-github)
+ [The CLI](#the-cli)
  + [`print <filename>`](#print-filename)
  + [`compile <filename>`](#compile-filename)
+ [Syntax and examples](#syntax-and-examples)
  + [Variables and if statements](#variables-and-if-statements)
  + [Functions](#functions)
  + [Custom types](#custom-types)
+ [Building](#building)
+ [Testing and contributing](#testing-and-contributing)

## Foreword

This project is still in early development. Many features may be buggy or not yet implemented.

## Install

### From GitHub

```bash
git clone https://github.com/thomas-crane/pine.git
cd pine
npm link
```

## The CLI

There are several CLI commands available which can be used to compile and inspect Pine code.

The current supported arguments are

### `print <filename>`

Prints the AST representation of the Pine code in the given file.

### `compile <filename>`

Compiles the given Pine file to JavaScript. **This is not yet implemented.**

## Syntax and examples

These examples are also available in the `examples/` directory.
A full description of the [grammar in EBNF form](/docs/grammar.ebnf) is available in the `docs/` directory.

### Variables and if statements

```pine
num x = 20;
str test;

if x > 15 {
  test = 'x > 15';
} else if x < 10 {
  test = 'x < 10';
} else {
  test = '15 > x > 10';
}
```

If statements can also be used as expressions, so the following code is equivalent

```pine
str test = if x > 15 {
  'x > 15'
} else if x < 10 {
  'x < 10'
} else {
  '15 > x > 15'
}
```

### Functions

```pine
fn fib(num n) -> num {
  if n <= 1 {
    1
  } else {
    fib(n - 1) + fib(n - 2)
  }
}

fib(10);
```

### Custom types

```pine
# A type which represents a coordinate on a grid.
type Point
  | num x
  | num y
  ;

# Methods which are available on Point type.
type Point has {

  # Static method.
  fn new(num x, num y) -> Point {
    Point {
      x = x,
      y = y,
    }
  }

  # Instance method.
  fn scale(self, num factor) -> Point {
    Point::new(self:x * factor, self:y * factor)
  }
}
```

## Building

If you want to create a custom fork of the Pine compiler, you will need to rebuild the project when you make changes. The `package.json` has 3 scripts which are used for building.

+ `npm run clean` - deletes the build directory (`dist/`)
+ `npm run compile` - compiles the TypeScript.
+ `npm run build` - Runs `clean`, then `compile`.

Most of the time, `build` is the script that you want to run. If it is your first time building however, `compile` will be a bit faster.

## Testing and contributing

If you wish to contribute to the Pine compiler, feel free to create a pull request. When contributing, it is important to contribute tested code. To run existing tests, use the command

```bash
npm test
```

This will run the unit tests located in the `test/` dir.

When adding a feature, make sure the new feature has tests. Even if some of them do not pass.

There is an additional npm command which can be run to ensure your code follows the project's style guidelines.

```bash
npm run lint
```

This command runs tslint. You should make sure this passes, as pull requests that do not pass this command will be rejected.

repository: thomas-crane/pine

repo_name: thomas-crane/pine


text:
---
layout: page
title: Java Programming FAQ
permalink: /faq/java/
---

Major Changes:
roboRIO has a Java 8 SE Embedded JVM
Official Support for the Eclipse IDE
0 based port numbers

---

##Getting Started With Java Quick Guide

1. Wire control system according to official wire diagram. See the [Electrical FAQ](/RoboRio/faq/electrical/) for more info.
2. Flash and image roboRIO to Latest versions. See the [roboRIO FAQ](/RoboRio/faq/roborio/) for more info.
3. Install JVM on roboRIO. See question below.
4. Install 32-bit Java 8 Development Kit on Development Computer (http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html)
Note: Mac Users must install 64-bit JDK
5. Download 32-bit Eclipse IDE for Java Development on Development Computer (install 32-bit even if you are on 64-bit OS)  [Luna](https://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/lunasr1)
6. Extract eclipse to your desired location.
7. Open Eclipse and install Official FRC Plugins [plugin URL Released at KickOff] ()
8. Create New WPI lib Java Project File->New->Other->WPI Robot Project
9. Enter team number and project name, Select either "Command Based" or "Iterative Robot" frameworks
10. Manually add JDK to "Build Path" for your Robot Project (Right Click Project -> Build Path -> etc..)
11. Deploy code to robot.

---

##How do I install the Java Virtual Machine on the roboRIO

Installing the JVM for Embedded Devices, is not the same as installing the JVM on a desktop. Oracle, the developers of the Java Virtual Machine for Embedded Devices decided to release the JVM with a special installation procedure. The user must download the JVM to a host computer and run a program which builds an embedded JVM for your target with all of the configurations you desire. Then, you transfer the built JVM to your target (embedded device).

FIRST has created an easy-to-follow process that does all of the configurations for you, and runs the commands to build the JVM for you so that you do not need to be concerned with the details. Follow their directions below. If you wish to do it on your own, I am working on posting the manual steps required.

###Official FIRST Procedure
Ensure roboRIO is flashed and imaged to latest versions then continue with [Offical screensteps directions](https://wpilib.screenstepslive.com/s/4485/m/13809/l/243933-installing-java-8-on-the-roborio-java-only)

###Manual Procedure
1. Create an account on Oracles Website [oracle](https://login.oracle.com/mysso/signon.jsp)
2. Download the Java SE 8 Embedded JVM for ARMv7 Linux VFP, SoftFP ABI, Little Endian [ARM JVM](http://www.oracle.com/technetwork/java/embedded/embedded-se/downloads/javase-embedded-downloads-2209751.html)
3. Use 7-zip to untar the download on your host machine

TO BE CONTINUED

---

##Can I see live Program Output in Eclipse?

Yes. Use the RIOLog Viewer. Official steps here [Screensteps Live](https://wpilib.screenstepslive.com/s/4485/m/13810/l/284333-using-riolog-to-view-console-output)

To find the RIOLog window, in eclipse, go to Window, Show View, Other and search for RIOlog and you'll see it. Once you add it, it will stay around between eclipse restarts.

---

##Can I use an older version of Eclipse like Kelper with the plugins?
Yes, but only with a Java v7 or earlier JDK. There is a ANT build bug in older version of eclipse where the default ANT build version that ship with the older eclipse versions do not recognize Java v8.

See the bug report here: [bug](https://wiki.eclipse.org/Ant/Java8)

We recommend using Eclipse Luna where ever possible, with JDK v8, but if you must use Kepler, downgrade the JVM to v7 or run the fix noted in the bug report to use JVM v8 with Eclipse Kepler.

---

##When I deploy code, where is it saved on the roboRIO?

User programs are downloaded to /home/lvuser directory

---

##When I open eclipse I get an error stating "Unable to find JVM"

This problem is most likely caused by not having a JVM or JDK installed on your computer. Please install a JVM on your programming computer in accordance with step 4 of the quick start guide above.

---

##When I Deploy My Java Program I get an "Unable to find Javac Message"

If you get a message similar to the one below:

```
   Unable to find a javac compiler;
   com.sun.tools.javac.Main is not on the classpath.
   Perhaps JAVA_HOME does not point to the JDK
```

It means you haven't set up your project build path with a Java Development Kit (JDK).

---

##When I deploy I get "Remote command failed with exit status 1" error

If you get an error similar to this after line `cmd : test -d /usr/local/frc/JRE`

<img src = "../../Images/eclipseerror/jvmmissingerror.png">

The most likely cause is your roboRIO does not have the Java JVM installed. Please be sure to follow the steps exactly above to install the JVM on the roboRIO.

If you believe you installed the JVM but are still receiving this error, please ensure that the JVM is in the right location and that that the bin file has the executable attribute set.

1. Use "find" to determine where the JVM is located. Run `find / -name *jar*` in the roboRIO terminal. The `jar` application must be in the location `/usr/local/frc/JRE/bin`
2. Once the JVM is in the right folder location ensure the execute attribute is set `chmod +x /usr/local/frc/JRE/bin/*`

---

##After I deploy code from eclipse, the DS still shows no Robot Code.

This most likely means your code is crashing upon startup. Check for any runtime errors, like null pointers, or other exceptions. See the [roboRIO FAQ](/RoboRio/faq/roborio) for more details on what this status light means.

---

##Can you tell me whats new between 2014 Java and 2015 Java WPILib

- This list is a work in progress and will be provided soon

---

##Can I install the WPILib Eclipse plugins on a computer that does not have connection to the internet?

Yes. You can install it as a local plugin, but you will need to use a computer with internet connection to download it first.

###On Computer with Internet connection

1. Open a web browser and navigate to the URL where the Eclipse plugins are kept
2.  Download the artifacts.jar file to the computer and save it to the folder you created
4. Download the contents.jar file to the computer and save it to the folder you created
5. Create a folder called "features" inside of the folder you created in step 2.
6. Create a folder called "plugins" inside of the folder you created in step 2.
7. In your web browser click on the features link of the WPI plugins website, and download the latest features.jars (should be 3 files). You can use the site.xml site to determine which is the latest, or just look for the last modified date. Save this file inside of the features folder you created.
8. In your web browser click on the plugins link of the WPI plugins website, and download the latest plugins.jars (should be 4 files). You can use the site.xml site to determine which is the latest, or just look for the last modified date. Save this file inside of the plugins folder you created.
9. Save this folder and all of its contents to portable media like a flash drive

###On computer with Eclipse wishing to install WPI plugins.

1. Open Eclipse
2. Transfer folder to computer previously downloaded containing the plug-ins
3. Navigate to help->install new software
4. Click Add
5. Name it whatever you like i.e. WPIPluginsV20
6. Select local and navigate to the folder
7. Click OK
8. Install the Java or C++ plugins

This will allow you to install the plugins locally, and you can use the same folder to install to multiple computers.

license: MIT

repository: KHEngineering/RoboRio

branch_name: refs/heads/gh-pages

extension: md

gha_language: CSS

license_type: permissive

repo_name: KHEngineering/RoboRio


text:
# Mac OS X setup
This repository is probably not very interesting to anyone but me. It's all about the things that need to be done to get a fresh install of Mac OS X configured to my liking. I'm writing all of this down here so that I do not have to think too much when reinstalling the system.

## System Preferences
- General
  - Show scroll bars
     - :white_check_mark: Always
  - Default web browser: Firefox Developer Edition
- Keyboard
  - Keyboard
    - Modifier Keys
      - Caps Lock: No Action
      - Option: Command (if using an Apple Keyboard)
      - Command: Option (if using an Apple Keyboard)
  - Shortcuts
    - Select the previous input source Shift + Command + L
    - :white_check_mark: Use keyboard navigation to move focus between controls
  - Input Sources
    - Polish Pro
    - [USGerman Keyboard Layout](https://hci.rwth-aachen.de/usgermankeyboard)
- Mission Control
  - Hot Corners
    - Top Right: Desktop
    - Bottom Right: Mission Control
    - Top Left: Application Windows
- Date & Time
  - Clock
    - :white_check_mark: Show date
- Dock & Menu Bar
  - :x: Show recent applications in Dock
  - :white_check_mark: Bluetooth - Show in Menu Bar
  - :white_check_mark: Fast User Switching - Show in Menu Bar
  - :white_check_mark: Battery - Show Percentage
  - :x: Spotlight - Show in Menu Bar
- Display
  - Night shift
    - 08:00 - 18:00

## Finder defaults

```bash
defaults write com.apple.finder AppleShowAllFiles YES
defaults write com.apple.finder CreateDesktop false
killall Finder
```

## Command Line Tools

```bash
xcode-select --install
```

## Install stuff

```bash
./install.sh
```

## Configure manually

- Git
  - Add user.name and user.email to global config
  - Set `~/.gitignore` as `core.excludesfile`
- Alfred
  - Activate Powerpack
  - Default Results - Setup fallback results
    - Remove Google, put DuckDuckGo on top
  - Web Search
    - `dict` - `http://www.dict.cc/?s={query}`
    - `diki` - `https://www.diki.pl/slownik-angielskiego?q={query}`
    - `dikig` - `https://www.diki.pl/slownik-niemieckiego?q={query}`
    - `collins` - `https://www.collinsdictionary.com/dictionary/german-english/{query}`
    - `conj` - `http://conjugator.reverso.net/conjugation-german-verb-{query}.html`
- iTerm 2
  - General
    - :x: Copy to pasteboard on selection
  - Profiles
    - Colors: Solarized Dark preset
    - Text: 13pt
    - Terminal: :white_check_mark: Unlimited scrollback
  - Pointer
    - :x: Middle button single click - Paste from clipboard
- RubyMine
  - Editor
    - General
      - :white_check_mark: Use soft wraps in editor
      - :white_check_mark: Ensure line feed at file end on Save
      - :white_check_mark: Highlight current scope
      - :x: Copy as rich text by default
      - Appearance
        - :white_check_mark: Show whitespaces
      - Editor Tabs
        - :white_check_mark: Mark modified tabs with asterisk
        - :x: Show tabs in single row
    - Code Style
      - Other File Types
        - Tab size: 2, Indent: 2
  - Tools
    - Terminal
      - :x: Copy to clipboard on selection
      - :x: Paste on middle mouse button click
- Firefox
  - Install extensions:
    - 1password
    - AdBlock Plus
    - DuckDuckGo Privacy Essentials
- Carbon Copy Cloner

repository: angelikatyborska/mac-os-x-setup

branch_name: refs/heads/master

repo_name: angelikatyborska/mac-os-x-setup


text:
# wintersmith

This example uses Metalsmith to emulate a [Wintersmith](http://wintersmith.io) static site. To test it out yourself just run:

```bash
npm install
npm start
```

View it in your browser:

```bash
npm run serve
```

repository: metalsmith/metalsmith

gha_language: JavaScript

gha_license_id: MIT

repo_name: metalsmith/metalsmith


text:
# Common actions

## Frontend / AngularJS
Roughly in order of most to least common:
- To add a new route/page: `yo mean-seed` and select `ng-route` then follow the prompts

- To add a common component / module (directive or service), do **one** of:
	1. Use the [angular-module generator](https://github.com/jackrabbitsgroup/generator-angular-module) to make a Bower component then include it.
		- These are SEPARATE, version controlled repos and only the final compiled *.min.js file is included here in this project. So all building and testing is done separately and then when the module is built, add it to `bower.json` and run `bower install`. This is the preferred method for all directives IF they're common enough to warrant other people wanting to use this module AND you're ready and willing to maintain (since it will be out in the wild now, publicly available!).
		- Make sure to include / reference your new files (.js and optionally .less) in `buildfilesModules.json`
	2. OR create a new folder and add your new files to `modules/directives` or `modules/services`. Almost always namespace your module into the pre-existing "dtv" (for directives) or "svc" (for services) angular modules. This makes it so you shouldn't have to update `common/app.js` to add this new module.
		- If you're creating a new angular module, add it in to `common/js/app.js`. Though in general try to group modules together - i.e. in "dtv" or "svc" modules. Should virtually NEVER have a module with just one file in it; if that's the case just group it with something else or put in the "dtv" or "svc" modules.
		- Add the new module as a *FOLDER* and that folder should have:
			- The *actual file* (directive or service) itself.
			- A *"test"* folder or file with .spec.js test(s) for this directive/service.
			- A .less file(s) [for directives only - services should not touch the DOM].
		- Make sure to include / reference your new files (.js and optionally .less) in `buildfilesModules.json`

## Backend / Node.js
- Add a new api route group. To add an individual api call to an existing group, just edit the files below accordingly.
	1. Create a new folder in `app/modules/controllers` with the name of your new api endpoint namespace and fill it with the following files. In general, just copy an existing route and then edit the files to fit your new route.
		1. [your-module].api.js		The rpc route setup (endpoints, parameters, function calls)
		2. [your-module].js		The controller/model file that interacts with the database and actually does the work
		3. [your-module].test.js		Jasmine-node tests for these new routes
	2. `require` your new api file and set up the router endpoints in `app/routes/api/index.js`.
	3. Update `app/test/all.spec.js` to include and run your new tests. Typically, 3 additions need to be made:
		1. `require` [your-module].test.js file at the top.
		2. Call it in the `initModules` function. (Pass in the db, api, and any other modules you need access in your test file.)
		3. Actually run your tests. In general, these should have a `run` function that returns a promise when all tests are complete.
	4. Add an html link to `app/routes/api/rpc/api-help.html` to add your api to the reference. (The links are somewhere around line 365.)

- Add a new service or library of functions. Do **one** of:
	1. Build publicly - create a npm module and publish it then include it via `npm install`
	2. Build locally
		1. Create a new folder in `app/modules/services` with your file.
		2. `require` your new module in `app/routes/api/index.js` and pass it in to the any controller modules that need to use it.

- Add a new database collection
	1. Update `app/db_schema.json` with the new collection. Fields are technically optional but highly recommended: we use pure mongo-db-native, so a strict schema is NOT enforced. Fields listed here are just for documentation purposes.

## TODO
- create yeoman subgenerators for all common actions
	- frontend/angular
		- new local angular module (directive / service)
	- backend/node
		- new api route/controller
		- new local service

repository: adamrights/generator-mean-seed

repo_name: adamrights/generator-mean-seed


text:
---
layout:     post
title:      When are Neural Networks more powerful than Neural Tangent Kernels?
date:       2021-03-25 14:00:00
author:     Yu Bai, Minshuo Chen, Jason D. Lee
visible:    True
---

The empirical success of deep learning has posed significant challenges to machine learning theory: Why can we efficiently train neural networks with gradient descent despite its highly non-convex optimization landscape? Why do over-parametrized networks generalize well? The recently proposed Neural Tangent Kernel (NTK) theory offers a powerful framework for understanding these, but yet still comes with its limitations.

In this blog post, we explore how to analyze wide neural networks beyond the NTK theory, based on our recent [Beyond Linearization paper](https://arxiv.org/abs/1910.01619) and follow-up [paper on understanding hierarchical learning](https://arxiv.org/abs/2006.13436). (This blog post is also cross-posted at the [Salesforce Research blog](https://blog.einstein.ai/beyond-ntk/).)

### Neural Tangent Kernels
The Neural Tangent Kernel (NTK) is a recently proposed theoretical framework for establishing provable convergence and generalization guarantees for wide (over-parametrized) neural networks [(Jacot et al. 2018)](https://arxiv.org/abs/1806.07572). Roughly speaking, the NTK theory shows that

* A sufficiently wide neural network trains like a linearized model governed by the derivative of the network with respect to its parameters.
* At the infinite-width limit, this linearized model becomes a kernel predictor with the Neural Tangent Kernel (the NTK).

Consequently, a wide neural network trained with small learning rate converges to 0 training loss and generalize as well as the infinite-width kernel predictor. For a detailed introduction to the NTK, please refer to the earlier [blog post](http://www.offconvex.org/2019/10/03/NTK/) by Wei and Simon.

### Does NTK fully explain the success of neural networks?
Although the NTK yields powerful theoretical results, it turns out that real-world deep learning *do not operate in the NTK regime*:

* Empirically, infinite-width NTK kernel predictors perform slightly worse (though competitive) than fully trained neural networks on benchmark tasks such as CIFAR-10 [(Arora et al. 2019b)](https://arxiv.org/abs/1904.11955). For finite width networks in practice, this gap is even more profound, as we see in Figure 1: The linearized network is a rather poor approximation of the fully trained network at practical optimization setups such as large initial learning rate [(Bai et al. 2020)](https://arxiv.org/abs/2002.04010).
* Theoretically, the NTK has poor *sample complexity for learning certain simple functions*. Though the NTK is a universal kernel that can interpolate any finite, non-degenerate training dataset [(Du et al. 2018](https://arxiv.org/abs/1810.02054)[, 2019)](https://arxiv.org/abs/1811.03804), the test error of this kernel predictor scales with the RKHS norm of the ground truth function. For certain non-smooth but simple functions such as a single ReLU, this norm can be exponentially large in the feature dimension [(Yehudai & Shamir 2019)](https://arxiv.org/abs/1904.00687). Consequently, NTK analyses yield poor sample complexity upper bounds for learning such functions, whereas empirically neural nets only require a mild sample size [(Livni et al. 2014)](https://arxiv.org/abs/1410.1141).

Figure 1.
Linearized model does not closely approximate the training trajectory of neural networks with practical optimization setups, whereas higher order Taylor models offer a substantially better approximation.

These gaps urge us to ask the following
> **Question**: How can we theoretically study neural networks beyond the NTK regime? Can we prove that neural networks outperform the NTK on certain learning tasks?

The key technical question here is to mathematically understand neural networks operating *outside of the NTK regime*.

## Higher-order Taylor expansion
Our main tool for going beyond the NTK is the *Taylor expansion*. Consider a two-layer neural network with $m$ neurons, where we only train the "bottom" nonlinear layer $W$:

$$
f_{W_0 + W}(x) = \frac{1}{\sqrt{m}} \sum_{r=1}^m a_r \sigma( (w_{0,r} + w_r)^\top x).
$$

(Here, $W_0+W$ is an $m\times d$ weight matrix, where $W_0$ denotes the random initialization and $W$ denotes the trainable "movement" matrix initialized at zero). For small enough $W$, we can perform a Taylor expansion of the network around $W_0$ and get

$$
f_{W_0+W}(x) = \frac{1}{\sqrt{m}} \sum_{r=1}^m a_r \sigma(w_{0,r}^\top x) + \sum_{k=1}^\infty \frac{1}{\sqrt{m}} \sum_{r=1}^m a_r \frac{\sigma^{(k)} (w_{0,r}^\top x)}{k!} (w_r^\top x)^k
$$

Let us denote the $k$-th order term as $ f^{(k)}_{W_0, W}$, and rewrite this as

$$
f_{W_0+W}(x) = f^{(0)}_{W_0}(x) + \sum_{k=1}^\infty f^{(k)}_{W_0, W}(x).
$$

Above, term $f^{(k)}$ is a $k$-th order polynomial of the trainable parameter $W$. For the moment assume that $f^{(0)}(x)=0$ (this can be achieved via techniques such as the symmetric initialization).

The key insight of the NTK theory can be described as the following **linearized approximation** property
> For small enough $W$, the neural network $f_{W_0,W}$ is closely approximated by the linear model $f^{(1)}$.

Towards moving beyond the linearized approximation, in our [Beyond Linearization paper](https://arxiv.org/abs/1910.01619), we start by asking
> Why just $f^{(1)}$? Can we also utilize the higher-order term in the Taylor series such as $f^{(2)}$?

At first sight, this seems rather unlikely, as in Taylor expansions we always expect the linear term $f^{(1)}$ to dominate the whole expansion and have a larger magnitude than $f^{(2)}$ (and subsequent terms).

### “Killing" the NTK term by randomized coupling
We bring forward the idea of *randomization*, which helps us escape the "domination" of $f^{(1)}$ and couple neural networks with their quadratic Taylor expansion term $f^{(2)}$. This idea appeared first in [Allen-Zhu et al. (2018)](https://arxiv.org/abs/1811.04918) for analyzing three-layer networks, and as we will show also applies to two-layer networks in a perhaps more intuitive fashion.

Let us now assign each weight movement $w_r$ with a *random sign* $s_r\in\\{\pm 1\\}$, and consider the randomized weights $\\{s_rw_r\\}$. The random signs satisfy the following basic properties:

$$
E[s_r]=0 \quad {\rm and} \quad s_r^2 \equiv 1.
$$

Therefore, let $SW\in\mathbb{R}^{m\times d}$ denote the randomized weight matrix, we can compare the first and second order terms in the Taylor expansion at $SW$:

$$
E_{S} \left[f^{(1)}_{W_0, SW}(x)\right] = E_{S} \left[ \frac{1}{\sqrt{m}}\sum_{r\le m} a_r \sigma'(w_{0,r}^\top x) (s_rw_r^\top x) \right] = 0,
$$

whereas

$$
f^{(2)}_{W_0, SW}(x) = \frac{1}{\sqrt{m}}\sum_{r\le m} a_r \frac{\sigma^{(2)}(w_{0,r}^\top x)}{2} (s_rw_r^\top x)^2 = \frac{1}{\sqrt{m}}\sum_{r\le m} a_r \frac{\sigma^{(2)}(w_{0,r}^\top x)}{2} (w_r^\top x)^2 = f^{(2)}_{W_0, W}(x).
$$

Observe that the sign randomization keeps the quadratic term $f^{(2)}$ unchanged, but "kills" the linear term $f^{(1)}$ in expectation!

If we train such a randomized network with freshly sampled signs $S$ at each iteration, the linear term $f^{(1)}$ will keep oscillating around zero and does not have any power in fitting the data, whereas the quadratic term is not affected at all and thus becomes the leading force for fitting the data. (The keen reader may notice that this randomization is similar to Dropout, with the key difference being that we randomize the weight *movement* matrix, whereas vanilla Dropout randomizes the weight matrix itself.)

Figure 2.
The NTK regime operates in the "NTK ball" where the network is approximately equal to the linear term. The quadratic regime operates in a larger ball where the network is approximately equal to the sum of first two terms, but the linear term dominates and can blow up at large width. Our randomized coupling technique resolves this by introducing the random sign matrix that in expectation "kills" the linear term but always preserves the quadratic term.

Our first result shows that networks with sign randomization can still be efficiently optimized, despite its now non-convex optimization landscape:
> **Theorem**: Any escaping-saddle algorithm (e.g. noisy SGD) on the regularized loss function $E_S[L(W_0+SW)]+R(W)$, with freshly sampled sign $S=S_t$ per iteration, can find the global minimum in polynomial time.

The proof builds on the quadratic approximation $E_S[f]\approx f^{(2)}$ and recent understandings on neural networks with quadratic activation, e.g. [Soltanolkotabi et al. (2017)](https://arxiv.org/abs/1707.04926) & [Du and Lee (2018)](https://arxiv.org/abs/1803.01206).

### Generalization and sample complexity: Case study on learning low-rank polynomials
We next study the generalization of these networks in the context of learning *low-rank degree-$p$ polynomials*:

$$
f_\star(x) = \sum_{s=1}^{r_\star} \alpha_s (\beta_s^\top x)^{p_s}, \quad |\alpha_s|\le 1,\|(\beta_s^\top x)^{p_s}\|_{L_2} \le 1, p_s\le p \quad \textrm{for all } s.
$$

We are specifically interested in the case where $r_\star$ is small (e.g. $O(1)$), so that $y$ only depends on the projection of $x$ on a few directions. This for example captures teacher networks with polynomial activation of bounded degree and analytic activation (approximately), as well as constant depth teacher networks with polynomial activations.

For the NTK, the sample complexity of learning polynomials have been studied extensively in [(Arora et al. 2019a)](https://arxiv.org/abs/1901.08584), [(Ghorbani et al. 2019)](https://arxiv.org/abs/1904.12191), and many concurrent work. Combined, they showed that the sample complexity for learning degree-$p$ polynomials is $\Theta(d^p)$, with matching lower and upper bounds:
> **Theorem (NTK)** : Suppose $x$ is uniformly distributed on the sphere, then the NTK requires $O(d^p)$ samples in order to achieve a small test error for learning any degree-$p$ polynomial, and there is a matching lower bound of $\Omega(d^p)$ for any inner-product kernel method.

In our [Beyond Linearization paper](https://arxiv.org/abs/1910.0161), we show that the quadratic Taylor model achieves an improved sample complexity of  $\tilde{O}(d^{p-1})$ with isotropic inputs:
> **Theorem (Quadratic Model)**: For mildly isotropic input distributions, the two-layer quadratic Taylor model (or two-layer NN with sign randomization) only requires $\tilde{O}({\rm poly}(r_\star, p)d^{p-1})$ samples in order to achieve a small test error for learning a low-rank degree-$p$ polynomial.

In our [follow-up paper on understanding hierarchical learning](https://arxiv.org/abs/2006.13436), we further design a "hierarchical learner" using a specific three-layer network, and show the following
> **Theorem (Three-layer hierarchical model)**: Under mild input distribution assumptions, a three-layer network with a fixed representation layer of width $D=d^{p/2}$ and a trainable quadratic Taylor layer can achieve a small test error using only $\tilde{O}({\rm poly}(r_\star, p)d^{p/2})$ samples.

When $r_\star,p=O(1)$, the quadratic Taylor model can improve over the NTK by a multiplicative factor of $d$, and we can further get a substantially larger improvement of $d^{p/2}$ by using the three-layer hierarchical learner. Here we briefly discuss the proof intuitions, and refer the reader to our papers for more details.

* **Generalization bounds**: We show that, while the NTK and quadratic Taylor model expresses functions using similar random feature constructions, their generalization depends differently on the norm of the input. In the NTK, the generalization depends on the L2 norm of the features (as well as the weights), whereas generalization of the quadratic Taylor model depends on the operator norm of the input matrix features $\frac{1}{n}\sum x_ix_i^\top$ times the nuclear norm of $\sum w_rw_r^\top$. It turns out that this decomposition can match the one given by the NTK (it is never worse), and in addition be better by a factor of $O(\sqrt{d})$ if the input distribution is mildly isotropic so that $\\|\frac{1}{n}\sum x_ix_i^\top\\|_{\rm op} \le 1/\sqrt{d} \cdot \max \\|x_i\\|_2^2$, leading to the $O(d)$ improvement in the sample complexity.

* **Hierarchical learning**: The key intuition behind the hierarchical learner is that we can utilize the $O(d)$ sample complexity gain to its fullest, by applying quadratic Taylor model to not the input $x$, but a feature representation $h(x)\in \mathbb{R}^D$ where $D\gg d$. This yields a gain as long as $h$ is rich enough to express $f_\star$ and also isotropic enough to let the operator norm $\\|\frac{1}{n}\sum h(x_i)h(x_i)^\top\\|_{\rm op}$ be nice. In particular, for learning degree-$p$ polynomials, the best we can do is to choose $D=d^{p/2}$, leading to a sample complexity saving of $\tilde{O}(D)=\tilde{O}(d^{p/2})$.

### Concluding thoughts
In this post, we explored higher-order Taylor expansions (in particular the quadratic expansion) as an approach to deep learning theory beyond the NTK regime. The Taylorization approach has several advantages:

* Non-convex but benign optimization landscape;
* Provable generalization benefits over NTKs;
* Ability of modeling hierarchical learning;
* Convenient API for expeirmentation (cf. the [Neural Tangents](https://github.com/google/neural-tangents) package and the [Taylorized training](https://arxiv.org/abs/2002.04010) paper).

We believe these advantages make the Taylor expansion a powerful tool for deep learning theory, and our results are just a beginning. We also remark that there are other theoretical frameworks such as the [Neural Tangent Hierarchy](https://arxiv.org/abs/1909.08156) or the [Mean-Field Theory](https://arxiv.org/abs/1804.06561) that go beyond the NTK with their own advantages in various angles, but without computational efficiency guarantees. See the [slides](https://jasondlee88.github.io/slides/beyond_ntk.pdf) for more on going beyond NTK. Making progress on any of these directions (or coming up with new ones) would be an exciting direction for future work.


text:
# Intro
ebs_tools is a simple tools for doing AWS EBS snapshots and also copy and clean them
up.

# Installation

```
gem install ebs_tools
```

# Usage

# Contributing to ebs_tools

* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
* Fork the project.
* Start a feature/bugfix branch.
* Commit and push until you are happy with your contribution.
* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

# Copyright

- Author: Kostiantyn Lysenko<EMAIL_ADDRESS>
```text
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```


text:
# GOTPIDLoop
This library implements a general purpose PID loop. Tuning, deadband, output action and output limit functionality exists. The loop processing frequency can also be set.

## LOOP EXECUTION
Loop control action will be processed at the time interval specified when the PID Loop is instaniated in milliSeconds (mS). It is important to implement the PID Loop as follows:
1. Instaniate the PID Loop with a interval time in milliseconds (mS).
2. The loop tuning and other parameters setup function should be called during the setup() using the following functions.
    setTuningParameters(float p, float i, float d, bool action);
    setDeadband(float db);
    setOutputLimited(float opMin, float opMax);
3. Call the reset() function after setting all the values.
4. The execute() function should be called in the loop() function. PID control action will only be calculated when the set time interval is complete.

## SETPOINT AND PROCESS VARIABLE
The Set Point (setpoint) and Process Variable (processVariable) properties should set set prior to calling the execute function.

## MANUAL CONTROL AND OUTPUT
After the execute function has been called the Output (output) property can be directly read.
At any time the output can be placed in Manual mode and the output set manually using the following functions. When the loop is in manual mode the automated adjustment of the output property is disabled.
setManualMode();
setAutoMode();
setManualValue(float opValue);

Additionally a reset() function is included to allow Integral Windup to be zeroed where required.

repository: Grumpy-Old-Tech/GOTPIDLoop

repo_name: Grumpy-Old-Tech/GOTPIDLoop


text:
# 🦟 Hoverfly

A simple container to run [hoverfly](https://hoverfly.readthedocs.io/en/latest/pages/reference/api/api.html).

## Usage

### Running in OpenShift

[OpenShift Applier](https://github.com/redhat-cop/openshift-applier)

Run the following to pull in applier:

```
ansible-galaxy install -r requirements.yml -p galaxy -f
```

Now, deploy to your openshift cluster:

```
oc login <dev cluster>
ansible-playbook -i .applier/ galaxy/openshift-applier/playbooks/openshift-cluster-seed.yml
```

### Uploading Simulation Files in OpenShift

`$ curl -X PUT <hoverfly-admin-route>/api/v2/simulation --upload-file <file-name>`

See [the docs](https://docs.hoverfly.io/en/latest/pages/reference/api/api.html) for detail

### Running and Uploading in Docker

Running in standalone docker is not an objective for this container. Look at [this issue](https://github.com/SpectoLabs/hoverfly/issues/675) in hoverfly or just [run the binary locally](https://docs.hoverfly.io/en/latest/pages/introduction/downloadinstallation.html) without a container. It's a Go binary, so it's easy.

### Runtime Environment Variables

- `RUN_AS_WEBSERVER`: if set to any value, the hoverfly will start as a webserver. defaults to running as a proxy

repository: redhat-cop/containers-quickstarts

gha_language: Shell

repo_name: redhat-cop/containers-quickstarts


text:
---
title: "Box"
description: "Rclone docs for Box"
versionIntroduced: "v1.38"
---

# {{< icon "fa fa-archive" >}} Box

Paths are specified as `remote:path`

Paths may be as deep as required, e.g. `remote:directory/subdirectory`.

The initial setup for Box involves getting a token from Box which you
can do either in your browser, or with a config.json downloaded from Box
to use JWT authentication.  `rclone config` walks you through it.

## Configuration

Here is an example of how to make a remote called `remote`.  First run:

     rclone config

This will guide you through an interactive setup process:

```
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> remote
Type of storage to configure.
Choose a number from below, or type in your own value
[snip]
XX / Box
   \ "box"
[snip]
Storage> box
Box App Client Id - leave blank normally.
client_id>
Box App Client Secret - leave blank normally.
client_secret>
Box App config.json location
Leave blank normally.
Enter a string value. Press Enter for the default ("").
box_config_file>
Box App Primary Access Token
Leave blank normally.
Enter a string value. Press Enter for the default ("").
access_token>

Enter a string value. Press Enter for the default ("user").
Choose a number from below, or type in your own value
 1 / Rclone should act on behalf of a user
   \ "user"
 2 / Rclone should act on behalf of a service account
   \ "enterprise"
box_sub_type>
Remote config
Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes
n) No
y/n> y
If your browser doesn't open automatically go to the following link: http://<IP_ADDRESS>:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
--------------------
[remote]
client_id =
client_secret =
token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"XXX"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
```

See the [remote setup docs](/remote_setup/) for how to set it up on a
machine with no Internet browser available.

Note that rclone runs a webserver on your local machine to collect the
token as returned from Box. This only runs from the moment it opens
your browser to the moment you get back the verification code.  This
is on `http://<IP_ADDRESS>:53682/` and this it may require you to unblock
it temporarily if you are running a host firewall.

Once configured you can then use `rclone` like this,

List directories in top level of your Box

    rclone lsd remote:

List all the files in your Box

    rclone ls remote:

To copy a local directory to an Box directory called backup

    rclone copy /home/source remote:backup

### Using rclone with an Enterprise account with SSO

If you have an "Enterprise" account type with Box with single sign on
(SSO), you need to create a password to use Box with rclone. This can
be done at your Enterprise Box account by going to Settings, "Account"
Tab, and then set the password in the "Authentication" field.

Once you have done this, you can setup your Enterprise Box account
using the same procedure detailed above in the, using the password you
have just set.

### Invalid refresh token

According to the [box docs](https://developer.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-tokens):

> Each refresh_token is valid for one use in 60 days.

This means that if you

  * Don't use the box remote for 60 days
  * Copy the config file with a box refresh token in and use it in two places
  * Get an error on a token refresh

then rclone will return an error which includes the text `Invalid
refresh token`.

To fix this you will need to use oauth2 again to update the refresh
token.  You can use the methods in [the remote setup
docs](/remote_setup/), bearing in mind that if you use the copy the
config file method, you should not use that remote on the computer you
did the authentication on.

Here is how to do it.

```
$ rclone config
Current remotes:

Name                 Type
====                 ====
remote               box

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> e
Choose a number from below, or type in an existing value
 1 > remote
remote> remote
--------------------
[remote]
type = box
token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2017-07-08T23:40:08.059167677+01:00"}
--------------------
Edit remote
Value "client_id" = ""
Edit? (y/n)>
y) Yes
n) No
y/n> n
Value "client_secret" = ""
Edit? (y/n)>
y) Yes
n) No
y/n> n
Remote config
Already have a token - refresh?
y) Yes
n) No
y/n> y
Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes
n) No
y/n> y
If your browser doesn't open automatically go to the following link: http://<IP_ADDRESS>:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
--------------------
[remote]
type = box
token = {"access_token":"YYY","token_type":"bearer","refresh_token":"YYY","expiry":"2017-07-23T12:22:29.259137901+01:00"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
```

### Modified time and hashes

Box allows modification times to be set on objects accurate to 1
second.  These will be used to detect whether objects need syncing or
not.

Box supports SHA1 type hashes, so you can use the `--checksum`
flag.

### Restricted filename characters

In addition to the [default restricted characters set](/overview/#restricted-characters)
the following characters are also replaced:

| Character | Value | Replacement |
| --------- |:-----:|:-----------:|
| \         | 0x5C  | ＼           |

File names can also not end with the following characters.
These only get replaced if they are the last character in the name:

| Character | Value | Replacement |
| --------- |:-----:|:-----------:|
| SP        | 0x20  | ␠           |

Invalid UTF-8 bytes will also be [replaced](/overview/#invalid-utf8),
as they can't be used in JSON strings.

### Transfers

For files above 50 MiB rclone will use a chunked transfer.  Rclone will
upload up to `--transfers` chunks at the same time (shared among all
the multipart uploads).  Chunks are buffered in memory and are
normally 8 MiB so increasing `--transfers` will increase memory use.

### Deleting files

Depending on the enterprise settings for your user, the item will
either be actually deleted from Box or moved to the trash.

Emptying the trash is supported via the rclone however cleanup command
however this deletes every trashed file and folder individually so it
may take a very long time.
Emptying the trash via the  WebUI does not have this limitation
so it is advised to empty the trash via the WebUI.

### Root folder ID

You can set the `root_folder_id` for rclone.  This is the directory
(identified by its `Folder ID`) that rclone considers to be the root
of your Box drive.

Normally you will leave this blank and rclone will determine the
correct root to use itself.

However you can set this to restrict rclone to a specific folder
hierarchy.

In order to do this you will have to find the `Folder ID` of the
directory you wish rclone to display.  This will be the last segment
of the URL when you open the relevant folder in the Box web
interface.

So if the folder you want rclone to use has a URL which looks like
`https://app.box.com/folder/11xxxxxxxxx8`
in the browser, then you use `11xxxxxxxxx8` as
the `root_folder_id` in the config.

{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/box/box.go then run make backenddocs" >}}
### Standard options

Here are the Standard options specific to box (Box).

#### --box-client-id

OAuth Client Id.

Leave blank normally.

Properties:

- Config:      client_id
- Env Var:     RCLONE_BOX_CLIENT_ID
- Type:        string
- Required:    false

#### --box-client-secret

OAuth Client Secret.

Leave blank normally.

Properties:

- Config:      client_secret
- Env Var:     RCLONE_BOX_CLIENT_SECRET
- Type:        string
- Required:    false

#### --box-box-config-file

Box App config.json location

Leave blank normally.

Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.

Properties:

- Config:      box_config_file
- Env Var:     RCLONE_BOX_BOX_CONFIG_FILE
- Type:        string
- Required:    false

#### --box-access-token

Box App Primary Access Token

Leave blank normally.

Properties:

- Config:      access_token
- Env Var:     RCLONE_BOX_ACCESS_TOKEN
- Type:        string
- Required:    false

#### --box-box-sub-type

Properties:

- Config:      box_sub_type
- Env Var:     RCLONE_BOX_BOX_SUB_TYPE
- Type:        string
- Default:     "user"
- Examples:
    - "user"
        - Rclone should act on behalf of a user.
    - "enterprise"
        - Rclone should act on behalf of a service account.

### Advanced options

Here are the Advanced options specific to box (Box).

#### --box-token

OAuth Access Token as a JSON blob.

Properties:

- Config:      token
- Env Var:     RCLONE_BOX_TOKEN
- Type:        string
- Required:    false

#### --box-auth-url

Auth server URL.

Leave blank to use the provider defaults.

Properties:

- Config:      auth_url
- Env Var:     RCLONE_BOX_AUTH_URL
- Type:        string
- Required:    false

#### --box-token-url

Token server url.

Leave blank to use the provider defaults.

Properties:

- Config:      token_url
- Env Var:     RCLONE_BOX_TOKEN_URL
- Type:        string
- Required:    false

#### --box-root-folder-id

Fill in for rclone to use a non root folder as its starting point.

Properties:

- Config:      root_folder_id
- Env Var:     RCLONE_BOX_ROOT_FOLDER_ID
- Type:        string
- Default:     "0"

#### --box-upload-cutoff

Cutoff for switching to multipart upload (>= 50 MiB).

Properties:

- Config:      upload_cutoff
- Env Var:     RCLONE_BOX_UPLOAD_CUTOFF
- Type:        SizeSuffix
- Default:     50Mi

#### --box-commit-retries

Max number of times to try committing a multipart file.

Properties:

- Config:      commit_retries
- Env Var:     RCLONE_BOX_COMMIT_RETRIES
- Type:        int
- Default:     100

#### --box-list-chunk

Size of listing chunk 1-1000.

Properties:

- Config:      list_chunk
- Env Var:     RCLONE_BOX_LIST_CHUNK
- Type:        int
- Default:     1000

#### --box-owned-by

Only show items owned by the login (email address) passed in.

Properties:

- Config:      owned_by
- Env Var:     RCLONE_BOX_OWNED_BY
- Type:        string
- Required:    false

#### --box-encoding

The encoding for the backend.

See the [encoding section in the overview](/overview/#encoding) for more info.

Properties:

- Config:      encoding
- Env Var:     RCLONE_BOX_ENCODING
- Type:        MultiEncoder
- Default:     Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot

{{< rem autogenerated options stop >}}

## Limitations

Note that Box is case insensitive so you can't have a file called
"Hello.doc" and one called "hello.doc".

Box file names can't have the `\` character in.  rclone maps this to
and from an identical looking unicode equivalent `＼` (U+FF3C Fullwidth
Reverse Solidus).

Box only supports filenames up to 255 characters in length.

Box has [API rate limits](https://developer.box.com/guides/api-calls/permissions-and-errors/rate-limits/) that sometimes reduce the speed of rclone.

`rclone about` is not supported by the Box backend. Backends without
this capability cannot determine free space for an rclone mount or
use policy `mfs` (most free space) as a member of an rclone union
remote.

See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/)

## Get your own Box App ID

Here is how to create your own Box App ID for rclone:

1. Go to the [Box Developer Console](https://app.box.com/developers/console)
and login, then click `My Apps` on the sidebar. Click `Create New App`
and select `Custom App`.

2. In the first screen on the box that pops up, you can pretty much enter
whatever you want. The `App Name` can be whatever. For `Purpose` choose
automation to avoid having to fill out anything else. Click `Next`.

3. In the second screen of the creation screen, select
`User Authentication (OAuth 2.0)`. Then click `Create App`.

4. You should now be on the `Configuration` tab of your new app. If not,
click on it at the top of the webpage. Copy down `Client ID`
and `Client Secret`, you'll need those for rclone.

5. Under "OAuth 2.0 Redirect URI", add `http://<IP_ADDRESS>:53682/`

6. For `Application Scopes`, select `Read all files and folders stored in Box`
and `Write all files and folders stored in box` (assuming you want to do both).
Leave others unchecked. Click `Save Changes` at the top right.

repository: rclone/rclone

gha_language: Go

repo_name: rclone/rclone


text:
---
layout: post
title: "ABY3: A Mixed Protocol Framework for Machine Learning"
excerpt_separator:
categories:
  - Publications
tags:
  - MPC
  - Honest Majority
  - semi-honest
  - Secret Sharing
  - Garbled Circuit
  - Malicious
  - Machine Learning
  - Payman Mohassel
  - CCS
---

Payman Mohassel & Peter Rindal ~ eprint/2018/403 ~ CCS'18

Machine learning is widely used to produce models for a range of applications and is increasingly offered as a service by major technology companies. However, the required massive data collection raises privacy concerns during both training and prediction stages.

In this paper, we design and implement a general framework for privacy-preserving machine learning and use it to obtain new solutions for training linear regression, logistic regression and neural network models. Our protocols are in a three-server model wherein data owners secret share their data among three servers who train and evaluate models on the joint data using three-party computation (3PC).

Our main contribution is a new and complete framework (ABY3) for efficiently switching back and forth between arithmetic, binary, and Yao 3PC which is of independent interest. Many of the conversions are based on new techniques that are designed and optimized for the first time in this paper. We also propose new techniques for fixed-point multiplication of shared decimal values that extends beyond the three-party case, and customized protocols for evaluating piecewise polynomial functions. We design variants of each building block that is secure against _malicious adversaries_ who deviate arbitrarily.

We implement our system in C++. Our protocols are up to _four orders of magnitude_ faster than the best prior work, hence significantly reducing the gap between privacy-preserving and plaintext training.


text:
Virtual Environment Operator
===========

Isolate kubernetes pod communication into virtual groups according to pod label, with route fallback support.

![isolation](https://virtual-environment.oss-cn-zhangjiakou.aliyuncs.com/image/diagram-en-us.jpg)

Check ☞[document](https://alibaba.github.io/virtual-environment/#/en-us/)☜ for more information.

[中文介绍](./README_CN.md)

## Instruction

This project inspired by the "project environment" practice widely used in Alibaba Group,
witch allow developers create virtual testing environment cheaply and quickly by reusing many service instances from
another testing environment.

```
           +----------+   +----------+   +----------+
ParentEnv  | ServiceA |   | ServiceB |   | ServiceC |
           +----------+   +-^------+-+   +----------+
                            |      |
           +-----------+    |      |    +-----------+
SubEnv    -> ServiceA' +----+      +----> ServiceC' |
           +-----------+                +-----------+
```

This is a simple example of 2 virtual environments. A ParentEnv contains instances of all the 3 services,
while a SubEnv inherited from ParentEnv only contains instance of ServiceA and ServiceC (record as `ServiceA'` and `ServiceC'`).

The arrow shows a call sequence "ServiceA->ServiceB->ServiceC", when the request comes from instance `ServiceA'`,
it should firstly fallback to hit `ServiceB` in the ParentEnv, and finally turn to `ServiceC'` in SubEnv.

## How it works

In this implementation, a pod label is used to identify the name and inherit relationship of pods.
Service mesh (currently only support Istio) rules will be dynamically generated to fit the isolation requirement.

For example, the identify label key is set as `virtual-env`, and symbol `.` is used to split inherit level. Then,

- All pods contain label `virtual-env: dev` and `virtual-env: dev.proj1` would logically belong to two virtual environments named `dev` and `dev.proj1`
- In any condition, when traffic from `dev.proj1` virtual environment visit a service which has no pod available in that environment, pod from the upper level (i.e. `dev`) virtual environment would response
- Even the traffic have been passed to a upper level pod, in the following sequence, if target service has pods in the original environment (i.e. `dev.proj1`), those pods should respond for the call
- If the virtual environment label contains multiple inherit symbol, e.g. `dev.proj1.user1`, then the fallback logic should goes up through the hierarchy, until find matched pod or reach the most top level

## Limitation

This project currently only support [Istio](http://istio.io) service mesh control plane, the limitations are mostly come with Istio.

- Only support HTTP protocol
- Only support service discovery via kubernetes internal DNS, framework with external service discovery (e.g. Dubbo, SpringCloud) would not work
- The application MUST pass down a HTTP header contains original traffic environment name by itself. This can be done by OpenTracing's baggage, or implement manually

## Support

Contact us with DingTalk:


text:
[![Actions Status](https://github.com/UziTech/jasmine-expect-count/workflows/CI/badge.svg)](https://github.com/UziTech/jasmine-expect-count/actions)

# Jasmine Expect Count

Lets you specify the number of expects expected to be called.

## Installing

Save `jasmine-expect-count` in devDependencies

```sh
npm install jasmine-expect-count --save-dev
```

then require it

```js
require("jasmine-expect-count");
```

before your tests.

## Using

Jasmine Expect Count allows you to specify the number of expects to be called in an `it`.

```js
it("should call expect 3 times", function () {
  jasmine.expectCount(3);

  expect(1).toBe(1);

  const p = new Promise(function (resolve) {
    setTimeout(function () {
      expect(2).toBe(2);
      resolve();
    }, 1);
  });

  expect(3).toBe(3);

  // if returning p is forgotten only 2 expects will be called.
  return p;
});
```

repository: UziTech/jasmine-expect-count

repo_name: UziTech/jasmine-expect-count


text:
# Scala.js typings for double-hmac

Typings are for version 1.0

## Library description:
Compare two Buffers using Double HMAC to protect against timing attacks

|                    |                 |
| ------------------ | :-------------: |
| Full name          | double-hmac |
| Keywords           | double, hmac, constant, time, comparison, timing, attack, crypto, compare, comparison, equality |
| # releases         | 1 |
| # dependents       | 0.0 |
| # downloads        | 11.666666666666666 |
| # stars            | 7.0 |

## Links
- [Homepage](https://github.com/emilbayes/double-hmac#readme)
- [Bugs](https://github.com/emilbayes/double-hmac/issues)
- [Repository](https://github.com/emilbayes/double-hmac)
- [Npm](https://www.npmjs.com/package/double-hmac)

## Note
This library has been generated from typescript code from [DefinitelyTyped](https://definitelytyped.org).

Provided with :purple_heart: from [ScalablyTyped](https://github.com/oyvindberg/ScalablyTyped)

## Usage
See [the main readme](../../readme.md) for instructions.

repository: ScalablyTyped/Distribution

repo_name: ScalablyTyped/Distribution


text:
## Hyperic HQ SpringBoot Plugin

This project is a Hyperic HQ plugin for monitoring [Springboot](http://http://projects.spring.io/spring-boot/)

Source code is available at [github.com/mlmeyers/hyperic_springboot](https://github.com/mlmeyers/hyperic_springboot)

### Platform Support

Only supports MongoDB running on Windows since it uses a powershell to invoke a rest request to collect the metrics from Actuator.

### Auto-Discovery

The SpringBoot server process (java.exe) is auto-discovered.

### Metrics

The following metrics are available:

* All actuator metrics
* All server process (java.exe) metrics are also available - CPU Utilization, Resident Memory, etc.

### Log File Tracking

Messages are optionally reported from springboot.log and can be filtered by
regex include/exclude.

### Config File Tracking

The SpringBoot config file can be monitored for changes.

### Control Actions

None yet.

### Dependencies

Actuator enabled for SpringBoot Application +
Powershell scripts must enabled to run on the agents host - Set-ExecutionPolicy RemoteSigned +
Requires Powershell version 3+ on Agent Host

### SpringBoot Version Support

Tested on Server 2012 R2 with SpringBoot 1.2

### Hyperic HQ Version Support

Tested with [Hyperic HQ](http://www.hyperic.com/) version 5.4.8

### Install

#### Server Installation
* Copy the file springboot-plugin.xml into the following folder under the server installation
<pre>
hq-plugins
</pre>

### Developers
To test any changes prior to deploying

    java -jar hq-pdk.jar -p springboot -t "SpringBoot 1.2" -m metric

### Author

Matthew Meyers<EMAIL_ADDRESS>:: @mlmeyers


text:
---
title: Philosophy
layout: post
---

> Philosophy is a wide world of thought and discussion, ranging from metaphysics to epistemology to political theory to ethics and beyond

Some people find that Philosophy are scary & messy stuffs. But to me, Philosophy are principles of the World we live. The better you know about it, the better life you have.

Here are my 2 cents about Philosophy. Have a good read!

<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vTjrqEqGb6JMazIZqwX6i-qq2VHhOO-3dgkchm2EO9ZTaLjcVf7T-1TQfb9CDvcz-Pde2oCv46DkmDS/embed?start=false&loop=false&delayms=3000" frameborder="0" width="750" height="450" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>


text:
# cars

Simple `car` react game made by Jakub Vonsovsky in 2017. Feel free to copy, edit or even take over the world with a fork.

![Car game](cars.png)

### Installation

`npm install`

### Run

`npm start`

repository: vonsovsky/cars

repo_name: vonsovsky/cars


text:
# Microhalo models
Tools for predicting the formation and evolution of dark matter halos, particularly at the smallest scales.

Requires: numpy and scipy.

## halo_formation.py

Samples a halo population using the method of [arXiv:1905.05766](https://arxiv.org/abs/1905.05766) (modified based on Appendix A of [arXiv:1910.08553](https://arxiv.org/abs/1910.08553)). Density peaks are sampled from a matter power spectrum, and the structure of each peak predicts the structure of the resulting halo.

**These predictions are very accurate when the matter power spectrum exhibits a steep boost at some small scale** (for example, due to early matter domination, axion PQ symmetry breaking, or particle production during inflation, among other cosmological scenarios). **They fare less well with flatter power spectra**, like standard CDM, for which (a) halo mergers are much more important and (b) smooth accretion may be rapid enough to drive internal halo evolution.

## tidal_evolution.py

Predicts the evolution of a subhalo due to a host halo's tidal forces using the model described in [arXiv:1906.10690](https://arxiv.org/abs/1906.10690) (with an addition based on Appendix C of [arXiv:1910.08553](https://arxiv.org/abs/1910.08553)).

In order to model the range of scales relevant to microhalo scenarios, these tidal evolution predictions have been validated across a much broader range of subhalo-host systems and timescales than are ordinarily seen in cosmological simulations. The J factor evolution is the most tightly tuned, whereas there is more scatter from the r_max and v_max predictions.

## stellar_encounters.py

Predicts the evolution of a halo due to an arbitrary sequence of encounters with point objects, such as stars. The model is described in [arXiv:1907.13133](https://arxiv.org/abs/1907.13133) (with a refinement to the postencounter halo density profile given in Appendix B of [arXiv:2109.2109.03240](https://arxiv.org/abs/2109.03240)).

# Examples

1. [extragalactic-annihilation](examples/extragalactic-annihilation): Computes the factor by which microhalos arising due to early matter domination boost the extragalactic annihilation signal.
2. [annihilation-suppression](examples/annihilation-suppression): Integrates the aggregate factor by which tidal evolution scales the annihilation rate within subhalos of a set internal density inside a set host halo.
3. [pulsar-timing](examples/pulsar-timing): Generates a sample of halos in the solar vicinity for the purpose of pulsar timing observations.

# Planned additions

1. A predictive model of the outcomes of mergers between microhalos.
2. For certain power spectra, smooth accretion may be rapid enough to drive internal halo evolution. Explore this effect.
3. A more general model of subhalo tidal evolution would be nice.

repository: LBJ-Wade/microhalo-models

branch_name: refs/heads/main

repo_name: LBJ-Wade/microhalo-models


text:
# Example communication using react and firebase
This project is about sending and reading data(string) from firebase

## Getting Started
To use this project follow this steps.

### Installing

01. Clone repository

    ```markdown
    $ git clone https://github.com/wit03/basiccommunication
    ```

02. Install packages

    ```markdown
    $ npm install
    ...
    ```
03. Development

    ```markdown
    $ cd basiccommunication
    $ npm start
    ...
    ```


text:
---
layout: post
title: Rust Installation
tags:
- Rust
- Programming
- Coding
---

This must be the easiest language installation I have had to do to date. Not that using homebrew is difficult, but some beginners don't love the command line.

If you go to the main page for the language, [rust-lang.org](https://rust-lang.org). You it will attempt to auto-detect the right installer for you and you can just click the "Install" button.

If you are downloading Rust to install on a different machine, or want control over which version you are getting, you can go to the [downloads](https://www.rust-lang.org/downloads.html) page instead.

Once it is downloaded, open the package and run through the wizard. On a mac, you will need to open it using the override method, by option-clicking on it or right click and choose open.

Once it finishes going through the wizard, you are ready to go!

You can confirm rust is indeed installed by opening terminal and running:

{% highlight bash %}
  $ rustc --version
  rustc 1.3.0 (9a92aaf19 2015-09-15)
{% endhighlight %}

As long as you get something returned like the second line (depending on your version), you should be good to go!


text:
---
layout: post
title: "Commit bccg1aea: Preemptively Structuring the Chaos"
category: "Rovani in C♯"
tags:
---

I have found that it is better to put structure in place around a project before going hog wild on implementation. While I recognize that some practices grow organically, a healthy amount of structure up-front can save a project from technical bankruptcy down the line. The two tools that I use early and often are interfaces and contract classes.

### Interfaces and Contract Classes

There are a collection of interfaces that I regularly utilize throughout most projects.

- ICreated
  - VigilUser CreatedBy
  - DateTime CreatedOn
- IModified : ICreated
  - VigilUser ModifiedBy
  - DateTime? ModifiedOn
  - bool MarkModified(VigilUser, DateTime)
- IDeleted
  - VigilUser DeletedBy
  - DateTime? DeletedOn
  - bool MarkDeleted(VigilUser, DateTime)
- IOrdered
  - int Ordinal
- IEffective
  - DateTime EffectiveOn
  - bool SetEffectiveOn(DateTime)
- IEffectiveRange : IEffective
  - DateTime EffectiveUntil
  - bool SetEffectiveRange(DateTime, DateTime)

### The Obvious — ICreated, IModified, IDeleted, IOrdered

These four interfaces serve to ensure that all properties are identically named throughout the solution. They also serve as a guide to future developers about what restrictions should be placed on the class. Every class that persists data to storage should implement ICreated — everything was created by someone and at some time.  An object that doesn?t implement IModified should never actually be modified. A class with IDeleted should not have the records actually removed from storage, but just fill in these fields to flag them as deleted. IOrdered means we know to set a default sort on the Ordinal field.

### Instead of Deleting It — Effective It!

For values where we need historical data to be easily accessible, such as changing schedules or payment information, an interface is created to track the EffectiveOn date. Starting at that specific date and time, the new record becomes effective. There needs to be some way to group the different records, which is usually performed with a header type table. By definition, there can only be one effective record at any given date.

An example of its use is storing the payment information for a recurring transaction. The historical data for payment information is useful when looking into the past, and frequently the patron may want to change the payment option effective some future date. However, at no point will two or more payment methods be active. By implementing the IEffective and ICreated, the details of who made the change (and when) fully replace any functionality that might be lost by not implementing IModified.

### Need Multiple, Overlapping Effectives — Range It!

When multiple records need to have their history saved or allow for changes effective in the future, the IEffectiveRange comes into utility. For IEffective records, the end of their effectiveness is the beginning of the next record?s effective date. However, for classes where there can be multiple records that come and go, an explicit range is required. This is useful for line items on a gift ? do see when any given detail was the authoritative record, especially useful after the gift has been posted and receipted.

### Contract Classes For

I battled back and forth with whether to include the shell for code contracts with every interface I write, or to only build it out when needed. I quickly found that I was including, at minimum, a contract invariant method on each interface, so I decided to just make it a standard part of every interface. Even if the Contract Class contains no Contracts, at least the shell is there, and I can easily add them later, as needed.

A quick standard I have put together is that all Contract Classes are contained in the same file as the interface, in a child namespace of ?Contracts,? and named after the interface. Keeping the classes as internal and abstract means it can only be called within the assembly,

### Minimum Viable Framework

I finally feel that I am at a stage where I have the minimum of a framework on which I can start to build an actual product. From here, I will slowly add and expand modules, slowly drifting towards the user interface, and tool with refactoring, templating, scripting, and finding other ways that I can reduce the complexity of complex tasks.

> Added generic interfaces, and began cleaning up some Code Analysis rules from the ?All Rules? playlist.
>
> —[Commit 4010e840a8b168a1ab65466fa5c61cc342b56d8e]()

> Added new Code Analysis rule sets to eliminate rules that I would be ignoring anyway.
> Resolved several Code Contracts and Code Analysis issues.
>
> —[Commit 2b1ff8a55711cb585750d4241df9c97895b7edfc]()

> Decided I didn?t like having fields for the context, testuser, and now. Put them back as variables in each test method, and added a ContractVerification(false) attribute to the whole class. I think I?m going to find myself adding that attribute to a lot of testing classes.
>
> —[Commit df9e61081f22471546dd1fd1132c8ecfecc3a26c]()

> Added Contract Classes for each of the Interfaces.
> Created the Identity and TypeBase classes to expand the foundation of classes before working on POCO?s.
> Need to add more unit tests to fill out the Code Coverage.
>
> —[Commit 716da101f6da28158ab0ed52aaba037b4c027676]()

> Added ExcludeFromCodeCoverage attributes to the ObjectInvariant methods in the interface contract classes.
> Added tests for the TypeBase class (via inheritance through a TestTypeBase class).
> Filled out the ChangeLogTests tests.
>
> —[Commit bcc51aeaa46dc8780ffcd9c33f88d8ecb1c2fffd]()

gha_language: HTML


text:
---
layout: post
title: 'Notes: k8s python dynamic client examples, Red Hat OpenShift Administration I (DO280)#5'
description: "rough notes of what things I did today"
category: upstream-contribution
tags: [kubernetes]
comments: false
---

May 05, 2021

Same as yesterday, I spent most of my day continuing with the **Red Hat OpenShift Administration I (DO280)** virtual instructor led training (VILT). It's going wonderful. :)

The topics from today's class were around configuring OpenShift networking for applications.
- I learnt how OpenShift exposes the conatiner applications to external networks using ingress & routes. I learnt about different types of routes (secure & unsecure) that could be created under the OpenShift clusters:
    - `Edge routes` ~ where TLS termination occurs at the router, between the external client & the application running in the cluster container
     ![edge-routes](https://user-images.githubusercontent.com/30499743/117184624-8af7fc80-adf6-11eb-9bc5-1487c24a4ca6.png)
    - `Pass-through routes` ~ where encrypted traffic is sent straight to the destination pod, without router providing TLS termination.The certificate is mounted on the container side using secrets. Currently, this is the only mode that supports mutual authentication between application & client that access it.
     ![passthrough-routes](https://user-images.githubusercontent.com/30499743/117184744-b1b63300-adf6-11eb-9e22-8927bfdb1395.png)
    - `Re-encrypted routes` ~ (similar to `edge-routes`) where router terminates TLS connection from the client, with one certificate & then re-encrypt its connection to the container application endpoint once again, with different (or similar) certificate. So, this essentially provides full path encryption.

- I also learnt how I can verify which part of network paths is TLS encrypted, from the **external client**, through the **router**, to all the way to the **container applications**.

- And lastly, two more concepts, **[Security context constraints (CSS)](https://docs.openshift.com/container-platform/4.7/authentication/managing-security-context-constraints.html)** **[Network Policies](https://docs.openshift.com/container-platform/4.6/networking/network_policy/about-network-policy.html)**. (Well, this is something I'm still struggling to properly understand, so will spend some more time on it, later)

---

***Some of the updates from the Kubernetes upstream contributions,***

- Re [PR (add examples to demonstrate the usage of dynamic client #1448)](https://github.com/kubernetes-client/python/pull/1448), it was on hold from being merged due to `strategic-merge-patch` throwing `unsupportedMediaType` error. This got merged today. The reviewers confirmed that `strategic-merge-patch` is not supported currently on custom resources by the upstream kubernetes project. More precise explanation:

  > native k8s resources like "deployment" support strategic-merge-patch. CR does not currently have that support. The support needs to be on the k8s server side, not on the python client side.

- Last week, I've been going through some of the open feature requests on the k8s python client project. I wasn't sure if I understood the requirements properly, so, yersterday I reached out to one of the project lead. I volunteered to pick up this one issue, [Drop Python 2 support #1413](https://github.com/kubernetes-client/python/issues/1413). I asked them to provide me some pointers on how to implement this. They've very kindly updated the issue with all the information. So, yes, I've assigned it to myself. Will start working on this after the upcoming release of the python-client. :)

---

***And the last update from today is quite special.***

I submitted a talk proposal for PyCon India, 2021 (again). Here is the [link to the proposal](https://in.pycon.org/cfp/2021/proposals/getting-started-with-kubernetes-python-client~axknq/).

The special part is not submitting the proposal, but seeing how much I've grown up, since the last time I submitted one (which is the last year itself).

Look at my talk proposal [video from last time](https://www.youtube.com/watch?v=1FOecXfqFdk) and the [one from this time](https://www.youtube.com/watch?v=AphSNKf-GPs), the confidence, I've build up in myself is the special part. I'm so happy for the current myself. The earlier me looks troubled (and I know I was) :)

So, yea, that's all from today. Good night!


text:
---
layout: post
title: GRE over IPSec
categories: [Network Security]
tags: [netsec]
description: This is a IPSec VPN Tunnel over GRE tunnel
---

| IP Header | ESP | IP Header | GRE | IP Header | TCP/UDP | Data |

* Compared to regular IPSec, we waste 20bytes (IP Header) and 4bytes (GRE)
* To save 20bytes (IP Header) , we can make use of the Transport Mode which will just COPY the IP header over instead of creating new IP Header
* To save even the the extra 4 bytes (GRE), make the tunnel mode as IPSEC
  * Command:
    ```
    (config-if)# tunnel mode ipsec ipv4
    ```

* Different way of giving ISAKMP KEY is using KEY RING
  * Command:
    {% highlight text %}
    * (config)# crypto keyring <_____>
      * (config-keyring)# preshared key address <peer ip><mask> key 0 <key>
    * (config)# crypto isakmp-profile <_____>
      * (isakmp-profile)# keyring <_____>
      * (isakmp-profile)# match identity address <peer address>
    * (config)# crypto ipsec-profile <______>
      * (ipsec-profile)# set isakmp-profile <____>
    * (config)# interface tunnel 0
      * (config-if)# tunnel protection ipsec profile <_____>
  {% endhighlight %}

The GRE tunnel keepalive mechanism is slightly different than for Ethernet or serial interfaces. It gives the ability for one side to originate and receive keepalive packets to and from a remote router even if the remote router does not support GRE keepalives. Since GRE is a packet tunneling mechanism for tunneling IP inside IP, a GRE IP tunnel packet can be built inside another GRE IP tunnel packet. For GRE keepalives, the sender pre-builds the keepalive response packet inside the original keepalive request packet so that the remote end only needs to do standard GRE decapsulation of the outer GRE IP header and then forward the inner IP GRE packet. This mechanism causes the keepalive response to forward out the physical interface rather than the tunnel interface. This means that the GRE keepalive response packet is not affected by any output features on the tunnel interface, such as 'tunnel protection ...', QoS, and so forth. ).

## GRE OVER IPSEC

{% highlight text %}
R1#
crypto ipsec transform-set tset esp-3des esp-md5-hmac

crypto ipsec profile IPSEC-PROFILE                                       //for tunnel protection
set transform-set tset

interface Tunnel1
tunnel protection ipsec profile IPSEC-PROFILE
R2#
crypto ipsec transform-set tset esp-3des esp-md5-hmac

crypto ipsec profile IPSEC-PROFILE
set transform-set tset

interface Tunnel1
tunnel protection ipsec profile IPSEC-PROFILE
{% endhighlight %}

After IPSEC profile is applied to tunnel0 all data will be protected

![b]({{site.url}}/img/b.png)

GRE without IPSEC in transport mode

![c]({{site.url}}/img/c.png)
![d]({{site.url}}/img/d.png)

GRE with IPSEC in transport mode

![e]({{site.url}}/img/e.png)
![f]({{site.url}}/img/f.png)

Verification:
{% highlight text %}
Sh crypto isakmp sa
Sh crypto isakmp sa  details
Sh crypto ipsec sa
Sh inter tunnel 0
Sh ip route
R1#
crypto ipsec transform-set tset esp-3des esp-md5-hmac

crypto ipsec profile IPSEC-PROFILE                                       //for tunnel protection
set transform-set tset

interface Tunnel1
tunnel protection ipsec profile IPSEC-PROFILE
R2#
crypto ipsec transform-set tset esp-3des esp-md5-hmac

crypto ipsec profile IPSEC-PROFILE
set transform-set tset

interface Tunnel1
tunnel protection ipsec profile IPSEC-PROFILE
{% endhighlight %}

After IPSEC profile is applied to tunnel0 all data will be protected

![g]({{site.url}}/img/g.png)

## Task

![h]({{site.url}}/img/h.png)

Step 1: No Access List
Step 2: ISAKMP Policy
* (config)# crypto isakmp policy 10
    * (config-policy)# authentication pre-share
Step 3: ISAKMP Key
* (config)#crypto isakmp key 0 cisco address <IP_ADDRESS>
Step 4: IPSEC Transform Set
* (config)# crypto ipsec transform-set GSET esp-3des esp-sha-hmac
Step 5: IPSEC Profile
* (config)# crypto ipsec profile GRE_IPSEC
    * (config-profile)# set transform-set GSET
Step 6: Apply to the interface
* (config)# interface tunnel 0
    * (config-if)# tunnel protection ipsec profile GRE_IPSEC

GRE Over IPsec with AH - Transport Mode

![i]({{site.url}}/img/i.png)

GRE Over IPsec with AH - Tunnel Mode
![j]({{site.url}}/img/j.png)

repository: themayursinha/blog

gha_license_id: NOASSERTION

repo_name: themayursinha/blog


text:
---
title: Pre-class on "Version Control" - January 14 19:00
layout: post
---

We will meet for the first pre-class on **January 14 at 19h** in order to build up some basic knowledge before actual
classes start.

Official classes by Neil Gershenfeld will start **on Jan 28 2015**.

The theme will be Version Control systems such as GIT and Mercurial.

[Lesson notes](http://fabacademy.fablabcascina.org/classes/pre-01/)

[Hangout link](https://plus.google.com/u/0/events/cti4kabfq0nffbdk4tqfrvb9cv4)

extension: markdown


text:
# LegendaryCoin (LGD) - PoS/PoW hybrid. #
![alt tag](http://i.imgur.com/v3kCA1R.png)

LegendaryCoin combines the great features from and is a descendent of:
   - Philosopherstone (Gui Enhancements)
   - DiamondCoin (several algorithms fixed and improved)
   - Novacoin/PPCoin (Hybrid Pow/PoS)
   - Luckycoin (random blocks)
   - Florincoin (transaction message)

General Info / Features:

   - Data Port: 8877 (18877 Testnet)
   - RPC Port: 7777 (17777 Testnet)
   - Transaction fee destruction
   - 2 minute block time
   - 6 confirmations for transactions
   - 50 confirmations for minted blocks
   - Transaction messaging

POW Block Rewards:

   - 1st 3600 Blocks (5 days) double payout
   - 7 Coins per block
   - 14 Coin Superblock every hour
   - 56 Coin Superblock every day
   - 10000000 Max Coins
   - Payout will be halved every 64,800 blocks (3 months)

POS Rewards:

   - LGD will generate if held and not moved for 5-15 days.
   - Annual stake rate of 50%
   - Aprox. 3.44% monthly


text:
# hackers and painters
## paul graham
### [paulgraham,com](http://www.paulgraham.com/hp.html)

---
summary

---

you cannot ignore the *act of making* inherent to "computer science" (even though the author doesn't like that term)

hackers are the ones who "who are trying to write interesting software, and for whom computers are just a medium of expression, as concrete is for architects or paint for painters". indeed the *what* to do and *how* to do it are very much intertwined in cs-as-hacking

his concept of the **beautiful**: "to make subtle tweaks to something that already exists, or to combine existing ideas in a slightly new way" (WRITERLY TEXT? NON AUTHOR?)

the eternal test of beauty, though, is time (that's one way to see it?)

he has quite a fantasized version of what painters and architects do. "You should figure out programs as you're writing them, just as writers and painters and architects do."

programming languages are for "thinking through" programs, not carving them in stone (he doesn't like static typing), but how about production software? that's the reality of it.

could we actually have multiple phases in the writing of programs? moving between hacker and engineer?

in large companies, hackers (programmers) are seen as the grunt technicians, the code-monkeys. which asks the question of agency in such places, and in such books like *software development* and *clean code*

"Writing novels doesn't pay as well as writing ad copy for garbage disposals. And hacking programming languages doesn't pay as well as figuring out how to connect some company's legacy database to their Web server."

cf. line 14: learning to hack/paint/architect is *iterative* and *by examples*

one standard he adheres to is **make the program very short**

one way to combine modularity and authorship: "The right way to collaborate, I think, is to divide projects into sharply defined modules, each with a definite owner, and with interfaces between them that are as carefully designed and, if possible, as articulated as programming languages."

he cites **empathy** as the greatest quality a hacker can have because:

---

Part of what software has to do is explain itself. So to write good software you have to understand how little users understand.

---

repository: periode/thesis

repo_name: periode/thesis


text:
---
title: Connect to Metamask to authenticate and display the user's wallet address and possessed DEV
date: 2021-08-26
permalink: /{{ locale }}/learning/learning/authenticate-and-display/index.html
eleventyNavigation:
  key: authenticate-and-display
  parent: learning
  order: 4200
  title: Hands-on 1
---

### Hands-on 1

First, I will introduce the finished product to be created this time. Please access the following URL
https://27v7x.csb.app/

This application is an application that connects to Metamask, authenticates, and displays the user's wallet address and possessed DEV.

### [Explanation of operation]

Please access the following URL and open CodeSandbox.
https://codesandbox.io/s/login-with-wallet-build-27v7x

Select src / index.ts from Explore on the left to display the source code.

If you scroll to the bottom, you will find ①.
From now on, I will code from ① to ⑨.

①: Here, the link between the click event of the button and the `clickLoginButton` method, and the code of the processing when the button is pressed are written.
(2): The `clickLoginButton` method accesses Metamask, performs authentication processing, and displays the user's wallet address and possessed DEV. It is a method that does up to.
③: From ③ to ⑦, judge the status of Metamask to determine if you are logged in. The following is defined as the state of Metamask

| Login judgment | MetaMask state                           | Explanation                                                                                |
| -------------- | :--------------------------------------- | :------------------------------------------------------------------------------------------------ |
| Not logged in  | Metamask not installed                   | The browser does not have Metamask installed                                                      |
| Not logged in  | Not connected to Metamask                | Website is not allowed to connect with Metamask                                                   |
| Not logged in  | Not logged in with Metamask              | Not logged in on Metamask                                                                         |
| Not logged in  | Metamask connection network is different | The network to which Metamask connects is different from the network your website expects         |
| Login          | Not applicable to the above conditions   | You can get the wallet address from Metamask and connect to the network that the website expects. |

In ③, it is judged that "Metamask is not installed". Scroll to find the `isMetamask` function. Code the `isMetamask` function as follows:

```tsx
function isMetamask(): boolean {
	return !!window.ethereum && !!window.ethereum.isMetaMask
}
```

When metamask is installed in your browser, `ethereum` will be added to the` window` object. Using this as a judgment material, it returns whether "Metamask is not installed".

④: Next, determine "Not connected to Metamask". Scroll down to find the `connectMetaMask` function. Code the `connectMetaMask` function as follows:

```tsx
async function connectMetaMask(): Promise<boolean> {
	try {
		await window.ethereum.request({ method: 'eth_requestAccounts' })
	} catch (e) {
		if (e.code === 4001) {
			return false
		}
	}
	return true
}
```

To determine "not connected to Metamask", query the address of Ethereum from Metamask. At that time, if the connection between the website and Metamask cannot be established, such as when the user refuses to connect with Metamask, a 4001 code error will be returned, so this is used for judgment.
refer to: https://docs.metamask.io/guide/rpc-api.html#permissions

⑤: Next, determine "not logged in with Metamask". Scroll down to find the `isMetaMaskLogin` function. Code the `isMetaMaskLogin` function as follows:

```tsx
async function isMetaMaskLogin(): Promise<boolean> {
	return !!(await getAccount())
}
```

The `isMetaMaskLogin` function calls the` getAccount` function. The `getAccount` function gets the wallet address from Metamask and returns it.
Code the `getAccount` function as follows:

```tsx
async function getAccount() {
	const accounts = (await window.ethereum.request({
		method: 'eth_accounts',
	})) as string[]

	return accounts[0]
}
```

⑥: Next, determine "the state where the connection network of Metamask is different". Scroll down to find the `isMainNet` function. Code the `isRopsten` function as follows:

```tsx
function isRopsten() {
	return parseInt(window.ethereum.chainId) === 3
}
```

`window.ethereum.childId` returns the following` CHAIN_ID` depending on the network you are connecting to

| Network | CHAIN_ID | Explanation                             |
| :------ | -------: | :-------------------------------------- |
| Mainnet |        1 | Ethereum production environment network |
| Ropsten |        3 | Ethereum Test Network (PoW)             |
| Rinkeby |        4 | Ethereum Test Network (PoA)             |

refer to: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-155.md

⑦: If the checks from ③ to ⑥ do not catch the item, it is considered as "logged in". After this, we will code the post-login process.

⑧: Next, get the user's wallet address and display it. Scroll to find ⑧ and code as follows.

```tsx
const walletAddress = await getAccount()
const addressElement = document.getElementById('address')
addressElement.innerText = walletAddress
```

The `getAccount` function created in ⑤ is used to get the wallet address.

⑨: Next, acquire the user's possessed DEV and display it. Scroll to find the `getBalanceOfDEV` function. Code the `getBalanceOfDEV` function as follows:

```tsx
import { addresses, contractFactory } from '@devprotocol/dev-kit'
import { BigNumber } from '@ethersproject/bignumber'
import Web3 from 'web3'

async function getBalanceOfDEV(walletAddress: string) {
	const provider = new Web3(window.ethereum)
	const client = contractFactory(provider.currentProvider)

	const registryContract = client.registry(addresses.eth.ropsten.registry)

	const addressDEV = await registryContract.token()

	const amountBigNumber = BigNumber.from(
		await client.dev(addressDEV).balanceOf(walletAddress)
	)

	const amount = amountBigNumber.div('1000000000000000000').toString()
	return amount
}
```

> - Add `await` when calling` getBalanceOfDEV`

`contractFactory` is a function declared in` @ devprotocol / dev-kit`, and you can create a `client` by passing the` currentProvider` of `Web3` as an argument. You can use this `client` to operate the Dev Protocol. This time, we are using `client` to get the number of possessed DEVs.

```tsx
const registryContract = client.registry(addresses.eth.ropsten.registry)
```

Dev Protocol contracts are distributed by type, and the address of each contract is managed by `registryContract`. In the above part, we are getting the `registryContract` of the` ropsten` environment. DevProtocol has `main` and` ropsten` environments. In the above part, we are getting the registryContract of the ropsten environment. DevProtocol has main and ropsten environments.

```tsx
const addressDEV = await registryContract.token()
```

In this hands-on, we want to display the number of DEVs we have, so we are returning the address of the `token` contract that handles DEV tokens.

```tsx
const amountBigNumber = BigNumber.from(
	await client.dev(addressDEV).balanceOf(walletAddress)
)
```

The `dev` function is a method for contracts related to DEV tokens. By setting the contract address of the DEV token in the `dev` function and specifying the wallet address of the target person in the `balanceOf` method, the number of DEVs possessed by the target person is returned.

In addition to the `dev` function,` client` has the following functions.

```tsx
export type DevkitContract = {
	readonly allocator: ReturnType<typeof createAllocatorContract>
	readonly market: ReturnType<typeof createMarketContract>
	readonly property: ReturnType<typeof createPropertyContract>
	readonly propertyFactory: ReturnType<typeof createPropertyFactoryContract>
	readonly lockup: ReturnType<typeof createLockupContract>
	readonly withdraw: ReturnType<typeof createWithdrawContract>
	readonly dev: ReturnType<typeof createDevContract>
	readonly registry: ReturnType<typeof createRegistryContract>
	readonly policy: ReturnType<typeof createPolicyContract>
	readonly policyGroup: ReturnType<typeof createPolicyGroupContract>
	readonly metrics: ReturnType<typeof createMetricsContract>
	readonly policyFactory: ReturnType<typeof createPolicyFactoryContract>
}
```

Codebase: https://github.com/dev-protocol/dev-kit-js/blob/372c762539855b794af2e3df5774061d640f61d0/lib/contract.ts

In addition to the `balanceOf` function, the` dev` function has the following functions.

```tsx
export type DevContract = {
	readonly totalSupply: () => Promise<string>
	readonly balanceOf: (address: string) => Promise<string>
	readonly transfer: (to: string, value: string) => Promise<boolean>
	readonly allowance: (from: string, to: string) => Promise<string>
	readonly approve: (to: string, value: string) => Promise<boolean>
	readonly transferFrom: (
		from: string,
		to: string,
		value: string
	) => Promise<boolean>
	readonly name: () => Promise<string>
	readonly symbol: () => Promise<string>
	readonly decimals: () => Promise<string>
	readonly deposit: (to: string, value: string) => Promise<boolean>
	readonly contract: () => Contract
}
```

Codebase: https://github.com/dev-protocol/dev-kit-js/blob/372c762539855b794af2e3df5774061d640f61d0/lib/dev/index.ts

```tsx
const amount = amountBigNumber.div('1000000000000000000').toString()
```

Since the decimal point of the ERC20 token is 18 digits, it is divided to match the decimal point.

This is the end of coding. Let's check the operation.
When you can log in and your wallet address and the number of DEVs you have are displayed on the screen, you are done.

filename: hands-on 1.md


text:
---
layout: post
title:  "Golang read file line by line - 3 Simple ways"
author: Pramod
categories: [Interviews, Golang]
image: assets/images/golang.jpg
beforetoc: "3 simple ways to read file line by line in Golang"
toc: true
promote: true
---

Welcome back to our Golang tutorial series! In our previous post, we covered 3 simple ways to get the first character of
a string in Golang. If you missed that post, you can check it out here: [how to get first character of a string in golang](https://tipseason.com/how-to-get-first-character-in-string-golang/).

In this post, we'll be discussing another important topic - how to read a file line by line in Golang. Reading a file
line by line is a common task in programming, and Golang provides several ways to accomplish this. We'll be exploring 3
simple methods using the "bufio", "ioutil", and "os" packages.

Let's dive in!

## 1: Using ioutil ReadFile

Here's an example code snippet that demonstrates how to read a file line by line using the "ioutil" package:

The second method uses the "ioutil" package to read a file line by line. This method reads the entire contents of a file
into memory using the "ioutil.ReadFile" function. The contents of the file are returned as a byte slice, which can be
converted to a string using the "string" function. The string can then be split into lines using the "strings.Split"
function with the "\n" delimiter. This method is simple and straightforward, but it may not be suitable for large files
that cannot be read entirely into memory.

```
package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "strings"
)

func main() {
    data, err := ioutil.ReadFile("example.txt")
    if err != nil {
        log.Fatal(err)
    }

    lines := strings.Split(string(data), "\n")
    for _, line := range lines {
        fmt.Println(line)
    }
}

```

In this example, we first read the entire file "example.txt" into memory using the "ioutil.ReadFile" function. We then
split the byte slice into lines using the "strings.Split" function with the "\n" delimiter and print each line using
the "fmt.Println" function.

## 2: Using bufio Scanner

The first method uses the "bufio" package to read a file line by line. This method creates a new scanner using the "
bufio.NewScanner" function, which takes a file as an argument. The scanner provides a convenient way to read the
contents of a file line by line using the "scanner.Scan" method. This method returns a boolean value indicating whether
a line was successfully scanned or not, and the line itself can be obtained using the "scanner.Text" method. If an error
occurs during scanning, the "scanner.Err" method can be used to retrieve it.

Here's an example code snippet that demonstrates how to read a file line by line using the "bufio" package:

```
package main

import (
    "bufio"
    "fmt"
    "log"
    "os"
)

func main() {
    file, err := os.Open("example.txt")
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        line := scanner.Text()
        fmt.Println(line)
    }

    if err := scanner.Err(); err != nil {
        log.Fatal(err)
    }
}

```

In this example, we first open the file "example.txt" using the "os.Open" function and create a new scanner using the "
bufio.NewScanner" function. We then use a for loop to read each line of the file using the "scanner.Scan" method and
print it using the "fmt.Println" function. Finally, we check for any errors using the "scanner.Err" method.

## 3: Using os Open and bufio NewReader

The third method uses the "os" and "bufio" packages to read a file line by line. This method opens the file using the
"os.Open" function and creates a new buffered reader using the "bufio.NewReader" function. The reader provides a
convenient way to read the contents of a file line by line using the "reader.ReadString" method. This method reads a
string up to the specified delimiter, which in this case is the newline character '\n'. If an error occurs during
reading, the loop can be exited. This method is also simple and straightforward, and it is suitable for reading large
files since it reads the file contents in chunks rather than reading the entire file into memory.

Here's an example code snippet that demonstrates how to read a file line by line using the "os" and "bufio" packages:
```
package main

import (
    "bufio"
    "fmt"
    "log"
    "os"
)

func main() {
    file, err := os.Open("example.txt")
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    reader := bufio.NewReader(file)
    for {
        line, err := reader.ReadString('\n')
        if err != nil {
            break
        }
        fmt.Print(line)
    }
}

```

In this example, we first open the file "example.txt" using the "os.Open" function and create a new buffered reader
using the "bufio.NewReader" function. We then use a for loop to read each line of the file using the "reader.ReadString"
method and print it using the "fmt.Print" function. Finally, we check for any errors and break out of the loop if the
end of the file is reached.

### Conclusion

In conclusion, reading a file line by line is an essential task in Golang programming, and there are several ways to
accomplish this. In this post, we discussed 3 simple methods using the "bufio", "ioutil", and "os" packages. Depending
on the specific requirements of your project, one method may be more suitable than the others.

We hope you found this post informative and useful. If you'd like to receive more Golang tutorials and tips like this
one, be sure to subscribe to our blog. Thanks for reading!