text:
# Controllers

Igo uses [Express 4](http://expressjs.com/) as a web framework for the Controller layer.

## Routes

Routes are defined in the `/app/routes.js` file.

See the routing documentation here: See documentation here: http://expressjs.com/en/guide/routing.html

```js
module.exports.init = function(app, controllers) {
  // welcome
  app.get ('/',   controllers.WelcomeController.index);
};
```

## Middlewares configuration

### Compression

```js
var compression       = require('compression');
app.use(compression());
```

### Static files

All files located in `/public` are served statically with `express.static()`.

```js
app.use(express.static('public'));
```

### Cookies and Sessions

Cookies are signed with the [cookie-parser](https://github.com/expressjs/cookie-parser) module.

Sessions are encoded, signed and stored in cookies with [cookie-session](https://github.com/expressjs/cookie-session).

It is highly recommended configure your own secret keys in `/app/config.js`.

```js
config.signedCookiesSecret = 'abcdefghijklmnopqrstuvwxyz';
config.cookieSessionConfig = {
  name:   'app',
  keys:   [ 'aaaaaaaaaaa' ]
  maxAge: 24 * 60 * 60 * 1000 // 24 hours
};
```

### Multipart data

Multipart data is parsed with [multiparty](https://github.com/pillarjs/multiparty). The fields and the files are put in `req.body` and `req.files`, and ready to be used.

### Validation

Request validation is made with [express-validator](https://github.com/ctavan/express-validator).

### Flash Scope

The Flash scope allows data to be stored in session during only 2 requests. Very useful when performing redirects.

The Flash scope is a middleware as simple as this:

```js
module.exports = function(req, res, next) {
  req.session.flash = req.session.flash || {};
  res.locals.flash  = req.session.flash;
  if (req.method === 'GET') {
    // clear flash scope
    req.session.flash = {};
  }
  req.flash = function(key, value) {
    req.session.flash[key] = value;
  };
  next();
};
```

repository: derniercri/igo

repo_name: derniercri/igo


text:
# L_0 Regularizer, l0_layer and l0_dense:
## Introduction:
- L_0 regularization has only zero value at the origin and holds fixed otherwise. The regularization form can discard unnecessary tensor by forcing them to zero while keeping others untouched at the same time. However it is not directly applicable since there is no gradient almost everywhere and the most informative point is not differentiable.
- We implement an approximate version of L_0 regularization from [Louizos et al. 2017](https://arxiv.org/abs/1712.01312).
- We provide examples (l0_dense and l0_layer) to demonstrate how to incoporate L_0 regularization into a model since L_0 regularization induces an architectural change (probabilistic mask creation) therefore cannot be applied directly as any other L^p regularization

## Detailed Introduction:
- the program is organized as the following:
	- **l0_computation -> l0_regularizer -> l0_dense/l0_layer**
	- **l0_computation** defines the computational mechanisms of l0 regularization on a tensors during which a masked conditional tensor is created and will replace the original tensor for model building. For details please refer to [Louizos et al. 2017](https://arxiv.org/abs/1712.01312). **l0_computation** is not recommended for users for common applications
- **l0_regularizer**:
	- adpated from the structure of tf.contrib.layers.l2_regularizer
	- it is intended to develop into a format that can be applicable to any architecture
	- inputs: scale, scope=None
	- outputs: the (scaled) regularization loss
- **l0_dense**
	- adapted from the structure of tf.layers.dense
	- almost the same as the original dense function.
	- Except an additional argument is_training is added for tagging training/prediction status of the weights/bias
	- the losses created after applying l0_regularizer can be obtained by calling _tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)_
- **l0_layer**
	- inherited from the base.Layer class and structured much like tf.layers.dropout
	- input format is assumed to be [?, D] or [D, ?]
	- inputs: input_tensor, reg_const, training, seed, name
	- outputs: the masked layer activity
	- the losses created after applying l0_regularizer can be obtained by calling _tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)_

## Requirements:
- python 3.6
- tensorflow 1.7

## Installation and Usage:
- python3 -m pip install --index-url https://test.pypi.org/simple/l0-regularization
- In the code (also for sanity check):
    - from l0_regularization.l0_layer import l0_layer
    - from l0_regularization.l0_dense import l0_dense
- Checkout the demo_with_autoencoder.ipynb for an application of our module to autoencoders

## Note:
- The masks are created in l0_computation and are retrieved at the l0_dense.call method.
- The PyPI page for the project https://test.pypi.org/project/l0-regularization/

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# homebridge-gpio-electic-rim-lock
Homebridge plugin to open electric rim locks via Raspberry Pi GPIO pins.

## Circuit

This plugin assumes that you are using a Raspberry Pi to directly control your electric rim locks. Electric rim locks usually have a switch on the wall that you can push to open the door. On my model, this is just a very simple switch that completes a 12vac circuit. The button must be pressed for about a second before the door will open. In order for this to be an effective electric rim locks opener, you need a relay that will perform the duty of the button.

![Schematic](https://github.com/roberto-montanari/homebridge-gpio-electic-rim-lock/blob/master/images/schematic.png?raw=true)

## Installation

Install this plugin using:
```
sudo npm install -g homebridge-gpio-electic-rim-lock --unsafe-perm
```

## Configuration

You will need to add the following accessory configuration to the Homebridge [config.json](https://github.com/nfarina/homebridge/blob/master/config-sample.json).

Configuration sample:

```JSON
{
    "bridge": {
        "name": "Raspberry Pi 2",
        "username": "CC:22:3D:E3:CE:32",
        "port": 51826,
        "pin": "031-45-154"
    },

    "accessories": [
        {
                "accessory": "Tiro",
                "name": "Door",
                "pin": 12,
                "duration": 500
        }
    ],

    "platforms": []
}

```

Fields:

* name - Can be anything (required).
* pin - The physical GPIO pin number that controls the relay (required).
* duratin - Number of milliseconds to trigger the relay. Defaults to 1000 millseconds (1 second) if not specified.

gha_license_id: MIT


text:
# d3scription

D3 tooltip description following mouse cursor. With window edge collision solved.

![screenshot](media/screenshot.png)

This plugin tries to be as universal and simple as possible.
You can install it using your favorite dependency manager - doesn't matter if it is [NPM](http://npmjs.org/) or [bower](https://bower.io/).
You can also **directly download distribution** if you do not use any of these. It supports many module systems (commonjs, AMD, TypeScript).
You can easily customize look in your css and overwrite all default values by providing your own.

# Instalation

## NPM

This will install npm package with **commonjs** module system:

```
npm install d3scription --save
```

## Bower

This will install bower package with **AMD** module and **global `d3scription` variable**:

```
bower install d3scription --save
```

## Direct download

If you do not want to use any dependency manager in your project you can simply download [distribution](dist/d3scription.js)
and place it in your project. This is exactly the same file as installed using bower. It supports **AMD modules or global `d3scription` function**.

```
curl https://raw.githubusercontent.com/GlobalWebIndex/d3scription/master/dist/d3scription.js > d3scription.js
```

# Usage
You can use this plugin with both **JavaScript and TypeScript**. Please see [demos](demo/) for some example usage.

## JavaScript

Assume you have d3scription.js file already loaded in browser together with d3.js or AMD manager is used you're able to use this plugin in similar fashion.
Don't forget, that this plugin itself is dependent on D3 so you need to include d3 before you load d3scription!

```js
// define some example data
var data = [
    {
        x: 50,
        y: 100,
        desc: 'We are your friends...'
    },
    {
        x: 200,
        y: 50,
        desc: 'I love it!'
    }
]

// select SVG element and create main group
var el = d3.select('svg')
    .append('g');

// setup plugin with description getter function and default options.
// if you want to overwrite default z-index, offset or class name please see `api:setup:options section`
var tipFactory = d3scription(function(d) { return d.desc; });

// create tip for our main group element
var tip = tipFactory()
    .element(el);

// set data to circles
var circles = el.selectAll('.circle')
    .data(data);

// draw circles
var cEnter = circles.enter()
    .append('circle')
    .attr('class', 'circle')
    .attr('r', 30)
    .attr('cx', function(d) { return d.x; })
    .attr('cy', function(d) { return d.y; })
    // setup show and hide action on mouse events
    .on('mouseover', tip.show)
    .on('mouseout', tip.hide);
```

## Typescript

Usage with typescript is quite simmilar as usage with any other module system.

```typescript
import d3scription from 'd3scription';

interface Data {
    x : number;
    y : number;
    desc : string;
}

// define some example data
const data : Data[] = [
    {
        x: 50,
        y: 100,
        desc: 'We are your friends...'
    },
    {
        x: 200,
        y: 50,
        desc: 'I love it!'
    }
]

// select SVG element and create main group
const el = d3.select('#first-example')
    .append('g');

// setup plugin with description getter function and default options.
// if you want to overwrite default z-index, offset or class name please see `api:setup:options section`
const tipFactory = d3scription((d : Data) => d.desc);

// create tip for our main group element
const tip = tipFactory()
    .element(el);

// set data to circles
const circles = el.selectAll('.circle')
    .data(data);

// draw circles
const cEnter = circles.enter()
    .append('circle')
    .attr('class', 'circle')
    .attr('r', 30)
    .attr('cx', d => d.x)
    .attr('cy', d => d.y)
    // setup show and hide action on mouse events
    .on('mouseover', tip.show)
    .on('mouseout', tip.hide);
```

# Styles

You can import or copy [default styles](css/style.css) to your project if you want.
Anyway as you can see writing your own styles for tooltip is really simple.

# API

Api is design as three steps process to make reuse and flow as clean as possible.
These steps are `setup` > `initialize` > `use`. Folloving paragraphs describes each step individually.

## Setup

First you need to setup new Tip factory (you can think about it as alternative to subclassing base class with necessary options even it's not really uses classes at all).
This is done using the only public function provided by d3scription out of the box - `de3scription([contentGetter], [optionalOptions])`.

This function accept two parameters - `contentGetter` and `options`.

**Content getter** is just regular function which takes `data` and returns `string` content of tip.
This works the same way as many d3 methods like `.attr('x', [getter])` or `.style('fill', [getter])`.

### Options

Are used for overwriting default settings of plugin. It's a **hash** where every key is optional.
For typescript plugin exports this interface under name `Options`.
There are all possible values:

* `zIndex` sets css z-index of tip element (default is 100).
* `class` set custom class name for element (default is `d3scription-tip`).
* `offset["top"]` top offset of tip (default is `-10`).
* `offset["left"]` left offset of tip (default is `10`).

You can create one settuped tipFactory and reuse it on multiple places or create as many of these factories as you wish.

### Example

```js
function getDescription(data) {
  return data.toolTipText;
}
var options = {
  zIndex: 1000,
  class: 'my-tooltip',
  offset: {
    top: 20,
    left: 20
  }
}
var myD3scription = d3Scription(getDescription, options);
var defaultD3scription = d3scription(getDescription);
```

## Factory

Settup function return another function which takes **element** in which tip should be active.
This is element in which tip will follow mouse. This function also creates final tip.
You can create many tips from one factory using this function.

### Example

```js
var element = d3.select('svg').append('g');
var tip = myD3scription().element(element); // this function was returned by setup
```

## Tip

Finally you'll have a tip instance returned by from factory (after passing element).
Now you can call methods on this tip as you wish. Most common case is to `show(data)` and `hide()`
on mouse events. you can also take advantage of `element(element)` for changing orginal element of tip
or using `destroy()`.

## Tip API

* `show(data)` sets content of tip and displays it.
* `hide()` hides tip
* `element(element)` sets new element for tip
* `destroy()` removes tip from dom

## Basic Example of usage

```js
// define circle elements and bind data for them
var circles = el.selectAll('.circle')
    .data(data);

// create circles and add tips actions to them
var cEnter = circles.enter()
    .append('circle')
    .attr('class', 'circle')
    .attr('r', 30)
    .attr('cx', function(d) { return d.x; })
    .attr('cy', function(d) { return d.y; })
    // show tip on mouseover
    // NOTE: will use binded data for setting content of tip
    .on('mouseover', tip.show)
    // hide tip on mouseout event
    .on('mouseout', tip.hide);
```
# Changelog

- v1.0.1 - update position on show (fixes situations where `show` is called but `mouseMove` event doesn't happen)
- v1.0.0 - improve API
- v0.0.2 - window edge collision detections
- v0.0.1 - initial release

# Building localy

All PRs are welcome! This is why we tried to made whole development setup as simple as possible.

Clone repository:

```
git clone git@github.com:GlobalWebIndex/d3scription.git
```

Install all dependecies via NPM and Bower:

```
npm install && bower install
```

The main file you want to work with is [index.ts](index.ts). There are also [demo examples](demo/) where you can play with plugin.

## Building project

There are few handy npm scripts already settuped up.

If you want to run project as watch run:

```
npm run watch
```

you can then open [demo/index.html](examples) in browser. Project will be compiled in background if you made any change to source.

For building distribution please run following.

```
npm run compile
```

This will compile all versions of plugin (including commonjs for usage with NPM).

# License

MIT

## About GlobalWebIndex

![globalwebindex](https://pbs.twimg.com/profile_images/468332770624679937/wd2TMi0i_400x400.png)

d3scription is maintained by GlobalWebIndex Ltd.

See more about us at [www.globalwebindex.net](https://www.globalwebindex.net).

gha_language: TypeScript


text:
[TOC]

# Overview

- [Docker misconceptions](https://devopsu.com/blog/docker-misconceptions/)

Docker is an open-source project that automates **the deployment of
applications inside software containers**, by providing an additional
layer of abstraction
    - Deploys applications in a sandbox (containers) to run on the host
    operating system e.g. Linux.

Docker uses **resource isolation** features of the Linux kernel such as
[cgroups](https://en.wikipedia.org/wiki/Cgroups) and **kernel
namespaces** to allow independent "containers" to run within a single
Linux instance, avoiding the overhead of starting an maintaining virtual
machines.

The Linux kernel's support for namespaces mostly isolates an
application's view of the operating environment, including process
trees, network, user IDs and mounted file systems, while the kernel's
cgroups provide resource isolation, including the CPU, memory, block I/O
and network.

Since version 0.9, Docker includes the **libcontainer** library as its
own way to directly use virtualization facilities provided by the Linux
kernel, in addition to using abstracted virtualization interfaces via
[libvirt](https://en.wikipedia.org/wiki/Libvirt),
[LXC](https://en.wikipedia.org/wiki/LXC) (Linux Containers) and
[systemd-nspawn](https://en.wikipedia.org/wiki/Systemd-nspawn).

>Docker is a tool that can package an **application and its
>dependencies** in a **virtual container** that can run on any Linux
>server.

## History

**Solomon Hykes** started Docker as an internal project within dotCloud.
Docker represents an evolution of dotCloud's proprietary technology.

Docker was released as open source in March 2013. On March 13, 2014,
with the release of version 0.9, Docker dropped LXC as the default
execution environment and replaced it with its own libcontainer library
written in **Go** language.

# Versions

## Bundle

- https://docs.docker.com/docker-for-mac/docker-toolbox/
- Docker Desktop (newer version)
    + `docker` and `docker-compose`
    + only one virtual machine
    + there are STABLE and EDGE versions
- Docker Toolbox
    + `docker-machine` => can create multiple virtual machine, routing,
    etc.
    + older than Desktop version

## Specific Tools an Their Functions

- `docker`: Docker Engine
    + builds images and creates containers
- `docker-compose`
    + manages multiple containers
    + creates a network to allow containers communicate
- `docker-machine`
    + creates multiple virtual machines => swarms (multiple containers
    can run on one virtual machine)
- Docker Hub
    + a registry stores all the Docker images
    + similar like GitHub for Docker images
- Kubernetes
    + an operating system of machines (virtual/real)
    + manages and schedules containers on machines

# Tutorials

## Use Cases of Docker

- Trying out new technologies and don't want to mess up your environment
- Working or sharing with other people
    + It also includes the situation in which you move to a new machine,
    and you don't want to reinstall all development tools again.

## General Steps

0. `mkdir apps`
1. `cd apps`
2. `mkdir <container1_name>`
    + e.g., `mkdir website`
3. `mkdir <container2_name>`
    + e.g., `mkdir database`
4. Creating a `Dockerfile` for each container
    + Writing a blueprint for your environment
    + If you don't want to modify the base image from Docker Hub, you
    can use `docker-compose.yml` without `Dockerfile`, but if you want
    to modify something in the base image, you should use `Dockerfile`
    to do that.
5. `vim docker-compose.yml`
    + Specifying configurations for the container: build, command,
    environment, ports, working_dir, depends_on, volumes, etc.
    + Coordinating containers if you have more than one container
6. Using `docker-compose` to manage the services/containers
    + Creating containers from images and start it: `docker-compose up`
    + Start all services and detach from the terminal; services run in
    background: `docker-compose start` (no creating containers)
    + Stop all services: `docker-compose stop`
    + Launch a specific service: `docker-compose up <service_name>`
    + Restart a single service: `docker-compose restart <name>`
    + Log a service: `docker-compose logs -f <name>`
    + SSH to a service container: `docker-compose exec <name> bash`
7. (Optional) Write a shell script if the command to run
   `docker-compose` is long to type

- `Dockerfile` example

```Dockerfile
FROM ruby:2.3.3
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
RUN mkdir /app
WORKDIR /app
```

- `docker-compose.yml` example

```yaml
version: '3'

services:
  njs1:
    build: ./njs1
    command: sh -c "npm install && npm start"
    environment:
      - NODE_ENV=development
      - PORT=7000
    ports:
      - '7000:7000'
    working_dir: /root/njs1
    volumes:
      - ./njs1:/root/njs1:cached # <--- This will map ./njs1 to /root/njs1 inside the container.

  njs2:
    image: node:12.3-alpine
    command: sh -c "npm install && npm start"
    environment:
      - NODE_ENV=development
      - PORT=8000
    ports:
      - '8000:8000'
    working_dir: /root/njs2
    volumes:
      - ./njs2:/root/njs2:cached # <--- This will map ./njs2 to /root/njs2 inside the container.

  py1:
    image: python:3-stretch
    command: sh -c "pip install -r requirements.txt && python -m server"
    environment:
      - PORT=9000
      - FLASK_ENV=development
    ports:
      - '9000:9000'
    working_dir: /root/py1
    volumes:
      - ./py1:/root/py1:cached # <--- This will map ./py1 to /root/py1 inside the container.

  go1:
    image: golang:1.12-alpine
    command: sh -c "go run ."
    environment:
      - PORT=5000
    ports:
      - '5000:5000'
    working_dir: /root/go1
    volumes:
      - ./go1:/root/go1:cached # <--- This will map ./py1 to /root/py1 inside the container.
```

Links
- https://blog.atulr.com/docker-local-environment/
- NodeJS: https://tutorialedge.net/docker/working-with-docker-nodejs/
- https://www.netguru.com/codestories/efficient-way-to-use-docker-in-development

# Networking

TODO

# File System Sharing

TODO

# Develop with Docker

## Best Practices

### How to keep images small?

- Start with an appropriate base image. For instance, if you need a JDK,
  consider basing your image on the official `openjdk` image, rather
  than starting with a generic `ubuntu` image and installing `openjdk`
  as part of the Dockerfile.
- Use multistage builds. For instance, you can use the `maven` image to
  build your Java application, then reset to the `tomcat` image and copy
  the Java artifacts into the correct location to deploy your app, all
  in the same Dockerfile. This means that your final image doesn’t
  include all of the libraries and dependencies pulled in by the build,
  but only the artifacts and the environment needed to run them.
    + If you need to use a version of Docker that does not include
    multistage builds, try to reduce the number of layers in your image
    by minimizing the number of separate `RUN` commands in your
    Dockerfile. You can do this by consolidating multiple commands into
    a single `RUN` line and using your shell’s mechanisms to combine them
    together. Consider the following two fragments. The first creates
    two layers in the image, while the second only creates one.

```docker
RUN apt-get -y update
RUN apt-get install -y python
===
RUN apt-get -y update && apt-get install -y python
```

- If you have multiple images with a lot in common, consider creating
  your own base image with the shared components, and basing your unique
  images on that. Docker only needs to load the common layers once, and
  they are cached. This means that your derivative images use memory on
  the Docker host more efficiently and load more quickly.
    + https://docs.docker.com/develop/develop-images/baseimages/
- To keep your production image lean but allow for debugging, consider
  using the production image as the base image for the debug
  image. Additional testing or debugging tooling can be added on top of
  the production image.
- When building images, always tag them with useful tags which codify
  version information, intended destination (`prod` or` test`, for
  instance), stability, or other information that is useful when
  deploying the application in different environments. Do not rely on
  the automatically-created `latest` tag.

### How to persist application data? How to store information?

- Using volumes or bind mounts. **Avoid** storing in the container.
    + volumes: https://docs.docker.com/storage/volumes/
    + bind mounts: https://docs.docker.com/storage/bind-mounts/
- Manage sensitive data with Docker secrets
    + https://docs.docker.com/engine/swarm/secrets/
- Store configuration data using Docker Configs
    + https://docs.docker.com/engine/swarm/configs/

### Use Continuous Integration/Continuous Deployment

TODO

### Differences in Development and Production Environments

| Development                                                        | Production                                                                                |
|--------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Use bind mounts to give your container access to your source code. | Use volumes to store container data.                                                                                |
| Use Docker Desktop for Mac or Docker Desktop for Windows.          | Use Docker Engine, if possible with userns mapping for greater isolation of Docker processes from host processes.                                                                                |
| Don’t worry about time drift.                                      | Always run an NTP client on the Docker host and within each container process and sync them all to the same NTP server. If you use swarm services, also ensure that each Docker node syncs its clocks to the same time source as the containers. |

## Build Images

- Docker builds images automatically by reading instrutions from a
  `Dockerfile`.
- A Docker image consists of read-only layers each of which represents a
  Dockerfile instruction.
    + The layers are stacked and each one is a delta of the changes from
    the previous layer.

```docker
FROM ubuntu:18.04
COPY . /app
RUN make /app
CMD python /app/app.py
```

- When you run an image and generate a container, you add a new WRITABLE
  layer (the “container layer”) on top of the underlying layers. All
  changes made to the running container, such as writing new files,
  modifying existing files, and deleting files, are written to this thin
  writable container layer.

### Build Context

- `docker build`: when enter this command
    + the current directory = build context
    + `Dockerfile` in the current directory is used

### Multi-stage builds

- https://docs.docker.com/develop/develop-images/multistage-build/

## Manage Images

- Docker Hub
    + `docker login`
    + `docker search`
    + `docker pull`
    + `docker push`

# Docker internal

Docker implements a **high-level API** to provide lightweight containers
that run processes in isolation. Building on top of facilities provided
by the Linux kernel (primarily **cgroups** and **namespaces**).

A Docker container, as opposed to a traditional virtual machine, does
not require or include a separate operating system. Instead, it relies
on the **kernel's functionality** and uses **resource isolation** (CPU,
memory, block I/O, network, etc.) and **separate namespaces** to isolate
the application's view of the operating system.

Docker accesses the Linux kernel's virtualization features either
directly through the provided **libcontainer** library or indirectly via
**libvirt**, **LXC** or **systemd-nspawn**.

By using containers, resources can be isolated, services restricted, and
processes provisioned to have an almost completely private view of the
operating system with their own process ID space, file system structure,
and network interfaces. Multiple containers share the same kernel, but
each container can be constrained to only use a defined amount of
resources such as CPU, memory and I/O.

## How is this different from Virtual Machines?

### Virtual Machines Architecture

![Virtual Machines Architecture](../graphic/docker/virtual-machine-arch.png)

### Docker Engine

![Docker Engine](../graphic/docker/docker-engine.png)

## Operating-system-level virtualization

Operating-system-level virtualization is a server virtualization method
where the kernel of an operating system allows for multiple isolated
user space instances, instead of just one. Such instances (often called
containers, virtualization engines, virtual private servers, or jails)
may look and feel like a real server from the point of view of its
owners and users.

### Uses

Commonly used in virtual hosting environments.

Separating several applications to separate containers for improved
security, hardware independence, and added resource management features.

### Overhead

Operating-system-level virtualization usually imposes little to no
overhead, because programs in virtual partitions use the operating
system's normal system call interface and do not need to be subjected to
emulation or be run in an intermediate virtual machine, as is the case
with whole-system virtualizers (such as **VMWare**,
[QEMU](https://en.wikipedia.org/wiki/QEMU) or
[Hyper-V](https://en.wikipedia.org/wiki/Hyper-V)) and paravitualizers
(such as [Xen](https://en.wikipedia.org/wiki/Xen) or
[UML](https://en.wikipedia.org/wiki/User-mode_Linux))

### Flexibility

Operating-system-level virtualization is not as flexibility as other
virtualization approaches since it cannot host a guest operating system
different from the host one, or a different guest kernel. For example,
with Linux, different distributions are fine, but other operating
systems such as Windows cannot be hosted.

## Docker Engine (docker command)

The Docker Engine consists of two parts: a **daemon**, a **server
process** that manages all the containers; a **client**, which acts as a
remote control for the daemon.

You can think of **containers as a process in a box**. The box contains
everything the process might need, so it has the filesystem, system
libraries, shell and such, but by default none of these are running. You
`start` a container by running a process in it.

# Docker Engine Basic (docker command)

- [Docker cheatsheet and best practices](http://zeroturnaround.com/rebellabs/docker-commands-and-best-practices-cheat-sheet/)

![Docker cheat sheet](../graphic/docker/docker-cheat-sheet-rebellabs.png)

`# docker`: show help for docker commands

`# docker version`: show version of docker client and server

`# docker search <string>`: search container image from Docker Hub Registry

`# docker pull <username>/<container>`: download container image (it will download multiple layers for docker)

`# docker run <username>/<container> echo "hello world"`: run process `echo "hello world"` in container

`# docker run learn/tutorial apt-get install -y ping`: install package `ping` in container but not yet saved

`# docker commit [-author=""] [-m="message commit"] <container id> <new repository name>`: save change of container to new repository (container id get from command `# docker ps -l`)

`# `
`# docker ps`: list container running

`# docker exec -it <id process> bash` : spawn new bash shell for container with ID

`# docker exec -it <name container> bash`: spawn new bash shell for container with name

`# docker stop container <container>`

# Docker Machine (docker-machine command)

- `$ docker-machine create --driver virtualbox default`: create a new virtual machine
- `$ eval $(docker-machine env default)`: connect the shell to the virtual machine
- `$ docker-machine start default`: start the virtual machine
- `$ docker-machine stop default`: stop the virtual machine

# Docker Compose (docker-compose command)

## Create docker-compose.yml

```yaml
version: '3.4'
services:
  dev:
    image: ethanlee/cs61c-env
    security_opt:
      - seccomp:unconfined
    volumes:
      - .:/cs61c
```

## Usage

- `docker-compose run dev bash`: run this command at the directory
  contains the docker-compose.yml to run a bash shell.

# Tips and Tricks

## Running GUI apps in Docker containers

-<EMAIL_ADDRESS>- Install an X Server
    + Mac OS: XQuartz (brew cask install XQuartz)
- Allow remote connections to the X server
- Disable access control of the X server
    + `xhost + <local_ip>`
- Export DISPLAY environment variable
    + `export DISPLAY=<local_ip>:0`
- Run the image
    + `docker run --rm -i -e DISPLAY=<local_ip>:0 sontran/firefox`
    + You can build the image by yourself. Install an application on an
    OS such as ubuntu or alpine.

## Add a new volume in existing containers

- https://stackoverflow.com/questions/28302178/how-can-i-add-a-volume-to-an-existing-docker-container

```bash
$ docker ps  -a
CONTAINER ID        IMAGE                 COMMAND                  CREATED              STATUS                          PORTS               NAMES
    5a8f89adeead        ubuntu:14.04          "/bin/bash"              About a minute ago   Exited (0) About a minute ago                       agitated_newton

$ docker commit 5a8f89adeead newimagename

$ docker run -ti -v "$PWD/dir1":/dir1 -v "$PWD/dir2":/dir2 newimagename /bin/bash
```

## Pair programming with Docker, SSH and TMUX

- Create an container supports ssh and tmux for pair programming.


text:
# UseBaseClassRule

A sample project to use IBLinter `use_base_class_rule`.

If labels in `xib` or `storyboard` don't use custom class defined in `.iblinter.yml` as base classes, Xcode reports warnings.

![sample](https://raw.githubusercontent.com/masamichiueta/UseBaseClassRule/master/sample.png)

repository: masamichiueta/UseBaseClassRule

repo_name: masamichiueta/UseBaseClassRule


text:
# Kits

Permissions-based player kits for Fabric Servers.

[![Release](https://img.shields.io/github/v/release/John-Paul-R/kits?style=for-the-badge&include_prereleases&sort=semver)][modrinth:files]

Kits is a **Minecraft mod** for [Fabric][fabric] servers that adds configurable,
permissions-based player kits. This allows server owners to easily set up
batches of items that their players can claim, depending on their rank.
Additionally, kits can have a predefined cooldown, so that players do not get to
use them infinitely.

**Compatible with LuckPerms**

## Usage

### Adding Kits

Command: `/kit add [kit_name] [cooldown_milliseconds]`

Requires Permission: `kits.manage`

To add a kit, simply place the items you would like to be included in this kit
anywhere in your inventory and run the command `/kit add`.

### Claiming Kits

Command: `/kit claim [kit_name]`

Requires Permission: `kits.claim.[kit_name]`

For players, claiming kits is simple. They simply type `/kit claim [kit_name]`.
If they have permissions to use the specified kit, and that kit is not on
cooldown for them, they receive the kit in their inventory (or on the ground if
their inventory is full.)

Brigadier suggestions are enabled, and will only suggest kits the player has
permissions for.

### Removing Kits

Command: `/kit remove [kit_name]`

Requires Permission: `kits.manage`

To remove a kit, simply type `/kit remove [kit_name]`. This *irreversibly and
completely* **deletes** the specified kit from the server, for all users.

## Advanced

### Files

If, for whatever reason, you wish to configure kits manually, instead of using
the built-in commands, you can do so by modifying the kit files directly.

Kit files are stored in the directory `config/kits` in Minecraft's NBT format.
You can modify and create such files with a tool like [NBTExplorer](nbtexp).

This allows you to modify kit contents or cooldowns after creation, or even add
entirely new kits.

---

## Contributing

Thank you for considering contributing to Kits! You can do so on the project's
[GitHub page][github].

## Licence

Kits is open-sourced software licenced under the [MIT license][licence].

## Discord

Questions? Contact me in [my Discord server][discord].

[nbtexp]: https://github.com/jaquadro/NBTExplorer
[github]: https://github.com/John-Paul-R/kits
[curseforge]: https://curseforge.com/minecraft/mc-mods/kits
[curseforge:files]: https://www.curseforge.com/minecraft/mc-mods/kits/files/all
[modrinth:files]: https://modrinth.com/mod/kits/versions
[fabric]: https://fabricmc.net/
[licence]: https://cdn.modrinth.com/licenses/mit.txt
[minecraft]: https://minecraft.net/
[releases]: https://github.com/John-Paul-R/kits/releases
[security]: .github/SECURITY.md
[discord]: https://discord.jpcode.dev/

repository: John-Paul-R/kits

gha_language: Java

repo_name: John-Paul-R/kits


text:
# Confidential Identity Library

This library implements the Confidential Identity Library protocol, as described
[here][wiki_main_design]. The cryptographic building blocks are described
[here][wiki_crypto_design].

Refer to the README.md file in the root of the repository for build
instructions.

The library provides the following APIs.
- Rust library: current directory
- FFI Wrappers: [./ffi][ffi]
- WASM library: [./wasm][wasm]
- CLI: [./cli][cli]

## Library Description

CDD Claims and Scope Claims are types of claims that can be attached to an investor's identity on the PolyMesh.
CDD claims are registered by a trusted CDD provider using [cdd-register-did][cdd-register-did] API, and Scope
claims are created and added to the chain by the investor using [add_claim][add-claim] API.

CDD claims are generated by a PolyMesh trusted CDD provider during the CDD process (see
[cdd-register-did][cdd-register-did]), and privately link an investor's unique id (`investor_unique_id`, which
is kept secret between the investor and CDD provider) to their on-chain identity (`investor_did`, the serialized
[IdentityId][IdentityId] of the investor). In order to achieve this, the CDD provider has to generate a CDD Id,
include it in their claim, and submit it to the PolyMesh chain.
Note that every time an investor requests a new identity from a CDD provider, the same unique identity (which is
obtained from the Polymath Unique Identity System (PUIS)) must be used to create the CDD claim, even though the
investor's DID will be different. A CDD claim has the following format.

```json
{
  "investor_did":[32_bytes_array],
  "investor_unique_id":[16_bytes_array]
}
```

Scope Claims are generated by the investors and reference a specific scope (e.g. asset); see [add_claim][add-claim].
The investor must generate this claim for each identity that they wish to use to invest in a particular asset
(i.e. scope). Currently `scope_did` is set to be the asset's ticker name (a 12 bytes string, e.g. `AAPL`).
The Scope Claim has the following format.

```json
{
  "scope_did":[12_bytes_array],
  "investor_unique_id":[16_bytes_array]
}
```

[cdd-register-did]: https://docs.polymesh.live/pallet_identity/enum.Call.html#variant.cdd_register_did
[IdentityId]: https://docs.polymesh.live/polymesh_primitives/identity_id/struct.IdentityId.html
[add-claim]: https://docs.polymesh.live/pallet_identity/enum.Call.html#variant.add_claim

[wiki_main_design]: https://polymath.atlassian.net/wiki/spaces/PC/pages/172523576/Asset+Granularity+Unique+Identity
[wiki_crypto_design]: https://polymath.atlassian.net/wiki/spaces/CE/pages/202571817/Claim+Proof+Prototype
[ffi]: /confidential-identity/ffi/
[wasm]: /confidential-identity/wasm/
[cli]: /confidential-identity/clil/

gha_language: Rust


text:
# WMI Admin Template [![Tweet]

Please help us on. Thanks in advance!

This is meant to be the UX game changer. Pure & transparent code is devoid of redundant components, so the app is light enough to offer ultimate user experience. This means mobile devices also, where the navigation is just as easy and intuitive as on a desktop or laptop. The CoreUI Layout API lets you customize your project for almost any device – be it Mobile, Web or WebApp.

## Table of Contents

* [Versions](#versions)
* [Installation](#installation)
* [Usage](#Usage)
* [Features](#Features)
* [Creators](#creators)
* [Community](#community)
* [Community Projects](#community-projects)
* [License](#license)

## Versions

WMI is built on top of Bootstrap 4.

## Installation

``` bash
# clone the repo
$ git clone https://github.com/jyyblue/wmi.git

# install app's dependencies
$ composer install

# install app's dependencies
$ npm install

```
### If you choice to use MySQL

Copy file ".env.example", and change its name to ".env".
Then in file ".env" complete this database configuration:
* DB_CONNECTION=mysql
* DB_HOST=<IP_ADDRESS>
* DB_PORT=3306
* DB_DATABASE=laravel
* DB_USERNAME=root
* DB_PASSWORD=

### Set APP_URL

> If your project url looks like: example.com/sub-folder
Then go to `wmi/.env`
And modify this line:

* APP_URL =

To make it look like this:

* APP_URL = http://localhost:8000

### Next step

``` bash
# in your app directory
# generate laravel APP_KEY
$ php artisan key:generate

# run database migration and seed
$ php artisan migrate:refresh --seed

# generate mixing
$ npm run dev

# and repeat generate mixing
$ npm run dev
```

## Usage

``` bash
# start local server
$ php artisan serve

# test
$ php vendor/bin/phpunit
```

Open your browser with address: [localhost:8000](localhost:8000)
Click "Notes" on topbar menu and log in with credentials:

* E-mail<EMAIL_ADDRESS>* Password: _password_

This user has roles: _user_ and _admin_
* Role _user_ is required for **notes** management.
* Role _admin_ is required for **users** management.

---

### How to add a link to the sidebar:

> Instructions for CoreUI Free Laravel admin template only. _Pro and Vue.js versions have separate instructions._

#### To add a __link__ to the sidebar - modify seeds file:
`wmi/database/seeds/MenusTableSeeder.php`

In `run()` function - add `insertLink()`:
```php
$id = $this->insertLink( $rolesString, $visibleName, $href, $iconString);
```
* `$rolesString` - a string with list of user roles this menu element will be available, ex. `"guest,user,admin"`
* `$visibleName` - a string caption visible in sidebar
* `$href` - a href, ex. `/homepage` or `http://localhost:8000/dashboard`
* `$iconString` - a string containing valid CoreUI Icon name (kebab-case), ex. `cui-speedometer` or `cui-star`

To add a __title__ to the sidebar - use function `insertTitle()`:
```php
$id = $this->insertTitle( $rolesString, $title );
```
* `$rolesString` - a string with list of user roles this menu element will be available, ex. `"guest,user,admin"`
* `$title` - a string caption visible in sidebar

To add a __dropdown__ menu to the sidebar - use function `beginDropdown()`:
```php
$id = $this->beginDropdown( $rolesString, $visibleName, $iconString);
```
* `$rolesString` - a string with list of user roles this menu element will be available, ex. `"guest,user,admin"`
* `$visibleName` - a string caption visible in sidebar
* `$iconString` - a string containing valid CoreUI icon name (kebab-case). For example: `cui-speedometer` or `cui-star`

To end dropdown section - use function `endDropdown()`.

To add __link__ to __dropdown__ call function `insertLink()` between function calls `beginDropdown()` and `endDropdown()`.
Example:
```php
$id = $this->beginDropdown('guest,user,admin', 'Some dropdown', 'cui-puzzle');
$id = $this->insertLink('guest,user,admin', 'Dropdown name', 'http://example.com');
$this->endDropdown();
```

__IMPORTANT__ - At the end of `run()` function, call `joinAllByTransaction()` function:
```php
$this->joinAllByTransaction();
```

Once done with seeds file edit, __run__:
``` bash
$ php artisan migrate:refresh --seed
# This command also rollbacks database and migrates it again.
```

## Features

### Table of contents:
* [Notes](#notes)
* [Users](#users)
* [Management of menus](#menu-management)
* [Manage menu items](#Manage-menu-items)
* [Role management](#Role-management)
* [Management of the media](#Manage-media)
* [BREAD](#BREAD-system)
* [Email Templates](#E-mail-Templates)

#### Notes
It is an example of data presentation in a pagination table, and CRUD functionality.

#### Users
It is a simple example of how to manage registered users.

#### Menu management
It is a system that allows you to create a new menu and edit existing menus. To place a new menu named "new" in any view use this code:
```php
        <?php
            use the App\MenuBuilder\FreelyPositionedMenus;
            if(isset($appMenus['new'])){
                FreelyPositionedMenus::render( $appMenus['new'] , '', 'your-css-class-of-ul-element');
            }
        ?>
```
#### Manage menu items
Allows you to add, edit and delete menu items.
To add a new menu item to the menu you must:
* specify to which menu you are adding the item,
* specify the roles of users for whom the item will be visible
* name the item
* type ( to choice: link, title and dropdown)
* Href, the address to which the item is to refer,
* Parent (To nest the item inside dropdown)
* CORUI icons

#### Role management
Allows you to create, edit, delete and reorder user roles.
When a user has more than one role, the highest hierarchical role is used to create a menu for him.

#### Manage media
It allows to:
* Create virtual media folders.
* Send media to applications.
* Move media between folders,
* Cut images,

#### BREAD system
BREAD stands for: browse, read, edit, add, delete.
Our BREAD system allows you to easily and quickly generate for any table, from the database, a simple BREAD.
To create a new BREAD just enter a table name from the database.  Then enter a name for the form. Enter the number of rows in the browse table. Choose if you want the browse table to contain buttons: "show", "edit", "add", "delete".
Assign roles for users who will be able to use the ready BREAD.
Then complete each column of the table separately:
* the column name visible to the user,
* the input type for the column,
The last step is to select the appropriate checkboxes:
* browse (allows to display the column in the data table),
* read (allows you to display the column in the show view,)
* edit (enables column editing)
* add (allows you to complete the column data when adding a record. Required if the column is not nullable).
It is also possible to handle relationships with another table.
If the column is a foreign key, it should be specified: in the field "Optional relation table name" - table name to which the foreign key refers, in the "Optional column name in relation table - to print" field - the name of the column that is in the relation table to be displayed. Finally, select one of the two "field types" that relate to the relation: 'relation select' or 'relation radio'.

#### E-mail Templates
It is an example of managing e-mail templates. Allows you to create, edit and delete templates. It also allows you to send an E-mail to a selected address.

## Creators

**Xiang Ming**

* <https://twitter.com/lukaszholeczek>

repository: jyyblue/wmi

gha_language: HTML

repo_name: jyyblue/wmi


text:
# apikube
This repo contains code for Creating, Updating and Deleting Deployment, Service and Ingress using client-go.

## Commands
First clone the repo and run the command `go install`

Then run the following commands:
```
apikube create //creates a deployment
apikube expose //exposes the deployment
apikube scale // scale the deployment
apikube ingress // creates a ingress
apikube delete // deletes all
```

repository: faem/apikube

repo_name: faem/apikube


text:
# Methodological Guidelines and Meta-Science
<standard name="Meta Science">

A paper that analyses an issue of research methodology or makes recommendations for conducting research

## Application

This standard applies to papers that provide analysis of one or more methodological issues, or advice concerning some aspect of research.

- may or may not include primary or secondary empirical data or analysis.
- may consider philosophical or practical issues
- may simultaneously be a methodology paper and an empirical study, to which another standard also applies; for example, if a paper reports a case study and then gives advice about a methodological issue illuminated by the case study, consider both this standard and the **Case Study Standard**.

## Specific Attributes

### Essential Attributes
<checklist name="Essential">

<results>

- [ ] presents information that is useful for other researchers
- [ ] makes recommendations for future research
- [ ] presents clear, valid arguments supporting recommendations

<other>

</checklist>

### Desirable Attributes
<checklist name="Desirable">

- [ ] synthesizes related work from reference disciplines
- [ ] provides insight specifically for software engineering; goes beyond summarizing methodological guidance from existing works or reference disciplines;
- [ ] results integrated back into prior theory or research
- [ ] develops helpful artifacts (e.g. checklists, templates, tests, tools, sets of criteria)
  </checklist>

### Extraordinary Attributes
<checklist name="Extraordinary">

- [ ] includes an empirical study (e.g. a systematic literature review) that motivates the analysis of guidance
- [ ] quantitative simulation illustrating methodological issues
  </checklist>

## General Criteria

- **comprehensiveness** of analysis or guidance provided
- **usefulness** to the research community
- quality of **argumentation** supporting analysis of guidance
- degree of **integration** with previous work, both in software engineering and in reference disciplines

## Antipatterns

- overreaching; informal logical fallacies (e.g. straw man argument, appeal to popularity, shifting the burden of proof)[1](myfootnote1)
- discussing an issue without clear conclusions; failing to provide clear guidelines
- attacking individual studies or researchers; hypothetical examples should be used to avoid engendering animosity

## Invalid Criticisms

- Guidelines are not based on empirical evidence. Empirically testing meta-scientific propositions is typically impractical or impossible. Reviewers should evaluate the face validity, comprehensiveness and usefulness of the guidelines. It is not appropriate to reject methodological guidelines over lack of empirical support.

## Notes

Because metascientific claims often cannot be justified empirically:

- reviewers of methodology papers must themselves be experts in the methodology, so that they can evaluate the reasonableness of the guidelines
- reviewers should be more pedantic in critiquing the discussion and guidelines than for an empirical paper

## Exemplars

Natalia Juristo and Ana M. Moreno. 2001. Basics of Software Engineering Experimentation._ Springer Science & Business Media.

Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering.

Barbara Kitchenham, Lesley Pickard, and Shari Lawrence Pfleeger. 1995. Case studies for method and tool evaluation. _IEEE software._ 12, 4 (1995), 52-62.

Paul Ralph. 2019. Toward methodological guidelines for process theories and taxonomies in software engineering. _IEEE Transactions on Software Engineering_. 45, 7 (Jan. 2018), 712-735.

Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering. _Empirical software engineering._ 14, 2, article 131.

Miroslaw Staron. 2019. Action Research in Software Engineering: Theory and Applications. In _International Conference on Current Trends in Theory and Practice of Informatics_. 39-49.

Klaas-Jan Stol, Paul Ralph, and Brian Fitzgerald. 2016. Grounded theory in software engineering research: a critical review and guidelines. In _Proceedings of the 38th International Conference on Software Engineering (ICSE '16)_. Association for Computing Machinery, New York, NY, USA, 120–131. DOI:10.1145/2884781.2884833

Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2012. _Experimentation in Software Engineering_. Springer Science & Business Media.

Helen Sharp, Yvonne Dittrich, and Cleidson RB De Souza. 2016. "The role of ethnographic studies in empirical software engineering." _IEEE Transactions on Software Engineering_ 42, 8: 786-804.

Yvonne Dittrich, Kari Rönkkö, Jeanette Eriksson, Christina Hansson, & Olle Lindeberg (2008). Cooperative method development. _Empirical Software Engineering,_ 13, 3, 231–260.

---
[1](#myfootnote1)citing seminal works is not the “appeal to authority” fallacy

</standard>

repository: whasselbring/EmpiricalStandards

gha_language: JavaScript

repo_name: whasselbring/EmpiricalStandards


text:
# Badhamiopsis ainoae (Yamash.) T.E. Brooks & H.W. Keller, 1976 SPECIES

#### Status
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Mycologia 68(4): 836 (1976)

#### Original name
Badhamia ainoae Yamash., 1936

### Remarks
null

repository: mdoering/backbone

repo_name: mdoering/backbone


text:
# Configuration

Various settings can be retrieved post login.

<aside class="notice">
Certain functionality needs to be able to be enabled for a user for example the Agent Terminal, Forex, dependent on Feature Flags.
</aside>

## Fetch configuration for user

```shell
curl "https://<IP_ADDRESS>.xip.io/api/v1/mobile/users/d19bff36-4733-11e5-946b-9ba904d8238e/config"
  -H "Authorization: Token token=YOURTOKEN"
  -H "Content-Type: application/json"
```

> The above command returns JSON structured like this:

```json
{
    "status": "ok",
    "details": {
        "features": {
            "agentterminal": true,
            "forex": false,
            "prepaid": true,
            "prepaid_airtime": true,
            "prepaid_electricity": false
        }
    }
}
```

This endpoint retrieves the configuration for the specific user.

### HTTP Request

`GET https://<IP_ADDRESS>.xip.io/api/v1/mobile/users/<USER>/config`

### URL Parameters

Parameter | Description
--------- | -----------
USER | The UUID of the user whose config you would like to retrieve

### Response Result Set

Parameter | Type | Description
--------- | ---- | -----------
features.agentterminal | boolean | Whether user should have access to the agent terminal (being added in a later build of the app)
features.forex | boolean | Whether user should have to the forex functionality
features.prepaid | boolean | Whether user should have access to the prepaid items
features.prepaid_airtime | boolean | Whether user should have access to buying prepaid airtime
features.prepaid_electricity | boolean | Whether user should have access to buying prepaid electricity

repository: jacques/plutus-mobile-app-api-docs

gha_license_id: NOASSERTION

repo_name: jacques/plutus-mobile-app-api-docs


text:
# trim-call

Drops undefined arguments from the end of a function call.

Useful if you’re forwarding arguments from one function to another, but the second function has behavior which is affected by the presence of an explicit `undefined` argument.

Accepts an arguments list, just like [`Function.prototype.call()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/call). If you want to provide an argument array instead, use the [`trim-apply`](https://github.com/lamansky/trim-apply) module.

## Installation

Requires [Node.js](https://nodejs.org/) 6.0.0 or above.

```bash
npm i trim-call
```

## API

The module exports a function (`trimCall()`) that has one other function attached to it as a method (`trimCall.new()`).

### `trimCall()`

#### Parameters

1. `fn` (function): The function to call.
2. `thisArg` (any): The value of `this` while the function is being called.
3. Variadic: `...args` (one or more of: any): The arguments for the function call. Any `undefined` arguments at the end will be dropped.

#### Return Value

The return value of `fn` when called with `thisArg` and `args`.

### `trimCall.new()`

#### Parameters

1. `Cls` (class): The class whose constructor you want to call.
2. Variadic: `...args` (one or more of: any): The arguments for the constructor call. Any `undefined` arguments at the end will be dropped.

#### Return Value

A new instance of `Cls` constructed with `args`.

## Examples

```javascript
const trimCall = require('trim-call')

f1('test')

function f1 (a, b) {
  trimCall(f2, this, a, b)
}

function f2 () {
  arguments.length // 1
}
```

Because of `trimCall()`, the `f2()` function only receives one argument.

Here is the above example repeated _without_ `trimCall()`:

```javascript
f1('test')

function f1 (a, b) {
  f2.call(this, a, b)
}

function f2 () {
  arguments.length // 2
}
```

Without `trimCall()`, the undefined `b` argument of `f1()` becomes an explicit second argument for `f2()`.

## Related

* [trim-apply](https://github.com/lamansky/trim-apply)

repository: lamansky/trim-call

repo_name: lamansky/trim-call


text:
# Build VisualStudio solution GitHub Action
Composite GitHub action to restore NuGet packages and build a Visual Studio sln file

# Inputs
> `solution-file` (required string)

The path of the solution file to build relative to the repository root

branch_name: refs/heads/main


text:
---
uid: custom-stream-processing
title: Custom stream processing
---

# Custom Stream Processing

While the processing vocabulary of Akka Streams is quite rich (see the [Streams Cookbook](xref:streams-cookbook) for examples) it is sometimes necessary to define new transformation stages either because some functionality is missing from the stock operations, or for performance reasons. In this part we show how to build custom processing stages and graph junctions of various kinds.

> [!NOTE]
> A custom graph stage should not be the first tool you reach for, defining graphs using flows and the graph DSL is in general easier and does to a larger extent protect you from mistakes that might be easy to make with a custom `GraphStage`

## Custom Processing with GraphStage

The `GraphStage` abstraction can be used to create arbitrary graph processing stages with any number of input or output ports. It is a counterpart of the `GraphDSL.Create()` method which creates new stream processing stages by composing others. Where `GraphStage` differs is that it creates a stage that is itself not divisible into smaller ones, and allows state to be maintained inside it in a safe way.

As a first motivating example, we will build a new `Source` that will simply emit numbers from 1 until it is cancelled. To start, we need to define the "interface" of our stage, which is called shape in Akka Streams terminology (this is explained in more detail in the section [Modularity, Composition and Hierarchy](xref:streams-modularity)). This is how this looks like:

```csharp
using Akka.Streams.Stage;

class NumbersSource : GraphStage<SourceShape<int>>
{
    //this is where the actual (possibly statefull) logic will live
    private sealed class Logic : GraphStageLogic
    {
        public Logic(NumbersSource source) : base(source.Shape)
        {
        }
    }

    // Define the (sole) output port of this stage
    public Outlet<int> Out { get; } = new Outlet<int>("NumbersSource");

    // Define the shape of this tage, which is SourceShape with the port we defined above
    public override SourceShape<int> Shape => new SourceShape<int>(Out);

    //this is where the actual logic will be created
    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

As you see, in itself the `GraphStage` only defines the ports of this stage and a shape that contains the ports. It also has, a currently unimplemented method called `CreateLogic`. If you recall, stages are reusable in multiple materializations, each resulting in a different executing entity. In the case of `GraphStage` the actual running logic is modelled as an instance of a `GraphStageLogic` which will be created by the materializer by calling the `CreateLogic` method. In other words, all we need to do is to create a suitable logic that will emit the numbers we want.

> [!NOTE]
> It is very important to keep the `GraphStage` object itself immutable and reusable. All mutable state needs to be confined to the `GraphStageLogic` that is created for every materialization.

In order to emit from a `Source` in a backpressured stream one needs first to have demand from downstream. To receive the necessary events one needs to register a callback with the output port (`Outlet`). This callback will receive events related to the lifecycle of the port. In our case we need to override `onPull` which indicates that we are free to emit a single element. There is another callback, `onDownstreamFinish` which is called if the downstream cancelled. Since the default behavior of that callback is to stop the stage, we don't need to override it. In the `onPull` callback we will simply emit the next number. This is how it looks like in the end:

```csharp
using Akka.Streams.Stage;

class NumbersSource : GraphStage<SourceShape<int>>
{
    //this is where the actual (possibly statefull) logic will live
    private sealed class Logic : GraphStageLogic
    {
        // All state MUST be inside the GraphStageLogic,
        // never inside the enclosing GraphStage.
        // This state is safe to access and modify from all the
        // callbacks that are provided by GraphStageLogic and the
        // registered handlers.
        private int _counter = 1;

        public Logic(NumbersSource source) : base(source.Shape)
        {
            SetHandler(source.Out, onPull: () => Push(source.Out, _counter++));
        }
    }

    // Define the (sole) output port of this stage
    public Outlet<int> Out { get; } = new Outlet<int>("NumbersSource");

    // Define the shape of this stage, which is SourceShape with the port we defined above
    public override SourceShape<int> Shape => new SourceShape<int>(Out);

    //this is where the actual logic will be created
    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

Instances of the above `GraphStage` are subclasses of `Graph<SourceShape<int>, NotUsed>` which means that they are already usable in many situations, but do not provide the DSL methods we usually have for other `Source`s. In order to convert this `Graph`to a proper `Source` we need to wrap it using `Source.FromGraph` (see [Modularity, Composition and Hierarchy](xref:streams-modularity) for more details about graphs and DSLs). Now we can use the source as any other built-in one:

```csharp
// A GraphStage is a proper Graph, just like what GraphDSL.Create would return
var sourceGraph = new NumbersSource();

// Create a Source from the Graph to access the DSL
var mySource = Source.FromGraph(sourceGraph);

// Returns 55
var result1Task = mySource.Take(10).RunAggregate(0, (sum, next) => sum + next, materializer);

// Returns 5050
var result2Task = mySource.Take(100).RunAggregate(0, (sum, next) => sum + next, materializer);
```

### Port States, InHandler and OutHandler

In order to interact with a port (`Inlet` or `Outlet`) of the stage we need to be able to receive events and generate new events belonging to the port. From the `GraphStageLogic` the following operations are available on an output port:

* `Push(out,elem)` pushes an element to the output port. Only possible after the port has been pulled by downstream.
* `Complete(out)` closes the output port normally.
* `Fail(out,exception)` closes the port with a failure signal.

The events corresponding to an *output* port can be received in an `Action` registered to the output port using `SetHandler(out, action)`. This handler has two callbacks:

* `onPull` is called when the output port is ready to emit the next element, `Push(out, elem)` is now allowed to be called on this port.
* `onDownstreamFinish` is called once the downstream has cancelled and no longer allows messages to be pushed to it. No more `onPull` will arrive after this event. If not overridden this will default to stopping the stage.

Also, there are two query methods available for output ports:

* `IsAvailable(out)` returns true if the port can be pushed
* `IsClosed(out)` returns true if the port is closed. At this point the port can not be pushed and will not be pulled anymore.

The relationship of the above operations, events and queries are summarized in the state machine below. Green shows the initial state while orange indicates the end state. If an operation is not listed for a state, then it is invalid to call it while the port is in that state. If an event is not listed for a state, then that event cannot happen in that state.

![outport transitions1](/images/outport_transitions1.png)

The following operations are available for *input* ports:

* `Pull(in)` requests a new element from an input port. This is only possible after the port has been pushed by upstream.
* `Grab(in)` acquires the element that has been received during an `onPush` It cannot be called again until the port is pushed again by the upstream.
* `Cancel(in)` closes the input port.

The events corresponding to an *input* port can be received in an `Action` registered to the input port using `setHandler(in, action)`. This handler has three callbacks:

* `onPush` is called when the output port has now a new element. Now it is possible to acquire this element using `Grab(in)` and/or call `Pull(in)` on the port to request the next element. It is not mandatory to grab the element, but if it is pulled while the element has not been grabbed it will drop the buffered element.
* `onUpstreamFinish` is called once the upstream has completed and no longer can be pulled for new elements. No more `onPush` will arrive after this event. If not overridden this will default to stopping the stage.
* `onUpstreamFailure` is called if the upstream failed with an exception and no longer can be pulled for new elements. No more `onPush` will arrive after this event. If not overridden this will default to failing the stage.

Also, there are three query methods available for input ports:

* `IsAvailable(in)` returns true if the port can be grabbed.
* `HasBeenPulled(in)` returns true if the port has been already pulled. Calling `Pull(in)` in this state is illegal.
* `IsClosed(in)` returns true if the port is closed. At this point the port can not be pulled and will not be pushed anymore.

The relationship of the above operations, events and queries are summarized in the state machine below. Green shows the initial state while orange indicates the end state. If an operation is not listed for a state, then it is invalid to call it while the port is in that state. If an event is not listed for a state, then that event cannot happen in that state.

![Inport transitions](/images/inport_transitions1.png)

Finally, there are two methods available for convenience to complete the stage and all of its ports:

* `CompleteStage()` is equivalent to closing all output ports and cancelling all input ports.
* `FailStage(exception)` is equivalent to failing all output ports and cancelling all input ports.

In some cases it is inconvenient and error prone to react on the regular state machine events with the signal based API described above. For those cases there is a API which allows for a more declarative sequencing of actions which will greatly simplify some use cases at the cost of some extra allocations. The difference between the two APIs could be described as that the first one is signal driven from the outside, while this API is more active and drives its surroundings.

The operations of this part of the :class:`GraphStage` API are:

* `Emit(out, elem)` and `EmitMultiple(out, Enumerable(elem1, elem2))` replaces the `OutHandler` with a handler that emits one or more elements when there is demand, and then reinstalls the current handlers
* `Read(in, andThen, onClose)` and `ReadMany(in, n, andThen, onClose)` replaces the `InHandler` with a handler that reads one or more elements as they are pushed and allows the handler to react once the requested number of elements has been read.
* `AbortEmitting(out)` and `AbortReading(in)` which will cancel an ongoing emit or read

Note that since the above methods are implemented by temporarily replacing the handlers of the stage you should never call `SetHandler` while they are running `Emit` or `Read` as that interferes with how they are implemented. The following methods are safe to call after invoking `Emit` and `Read` (and will lead to actually running the operation when those are done): `Complete(out)`, `CompleteStage()`, `Emit`, `EmitMultiple`, `AbortEmitting()` and `AbortReading()`

An example of how this API simplifies a stage can be found below in the second version of the Duplicator.

### Custom Linear Processing Stages Using GraphStage

Graph stages allows for custom linear processing stages through letting them have one input and one output and using `FlowShape` as their shape.

Such a stage can be illustrated as a box with two flows as it is seen in the illustration below. Demand flowing upstream leading to elements flowing downstream.

![graph stage conceptual](/images/graph_stage_conceptual1.png)

To illustrate these concepts we create a small `GraphStage` that implements the `Map` transformation.

![graph stage map](/images/graph_stage_map1.png)

Map calls `Push(out)` from the `onPush` handler and it also calls `Pull()` from the `onPull` handler resulting in the conceptual wiring above, and fully expressed in code below:

```csharp
class Map<TIn, TOut> : GraphStage<FlowShape<TIn, TOut>>
{
    private sealed class Logic : GraphStageLogic
    {
        public Logic(Map<TIn, TOut> map) : base(map.Shape)
        {
            SetHandler(map.In, onPush: () => Push(map.Out, map._func(Grab(map.In))));
            SetHandler(map.Out, onPull: ()=> Pull(map.In));
        }
    }

    private readonly Func<TIn, TOut> _func;

    public Map(Func<TIn, TOut> func)
    {
        _func = func;
        Shape = new FlowShape<TIn, TOut>(In, Out);
    }

    public Inlet<TIn> In { get; } = new Inlet<TIn>("Map.in");

    public Outlet<TOut> Out { get; } = new Outlet<TOut>("Map.out");

    public override FlowShape<TIn, TOut> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

Map is a typical example of a one-to-one transformation of a stream where demand is passed along upstream elements passed on downstream.

To demonstrate a many-to-one stage we will implement filter. The conceptual wiring of `Filter` looks like this:

![Graph stage filter](/images/graph_stage_filter1.png)

As we see above, if the given predicate matches the current element we are propagating it downwards, otherwise we return the "ball" to our upstream so that we get the new element. This is achieved by modifying the map example by adding a conditional in the `onPush` handler and decide between a `Pull(in)` or `Push(out)` call (and of course not having a mapping f function).

```csharp
class Filter<T> : GraphStage<FlowShape<T, T>>
{
    private sealed class Logic : GraphStageLogic
    {
        public Logic(Filter<T> filter) : base(filter.Shape)
        {
            SetHandler(filter.In, onPush: () =>
            {
                var element = Grab(filter.In);
                if(filter._predicate(element))
                    Push(filter.Out, element);
                else
                    Pull(filter.In);
            });

            SetHandler(filter.Out, onPull: ()=> Pull(filter.In));
        }
    }

    private readonly Predicate<T> _predicate;

    public Filter(Predicate<T> predicate)
    {
        _predicate = predicate;
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("Filter.in");

    public Outlet<T> Out { get; } = new Outlet<T>("Filter.out");

    public override FlowShape<T, T> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

To complete the picture we define a one-to-many transformation as the next step. We chose a straightforward example stage that emits every upstream element twice downstream. The conceptual wiring of this stage looks like this:

![Graph stage duplicate](/images/graph_stage_duplicate1.png)

This is a stage that has state: an option with the last element it has seen indicating if it has duplicated this last element already or not. We must also make sure to emit the extra element if the upstream completes.

```csharp
class Duplicator<T> : GraphStage<FlowShape<T, T>>
{
    private sealed class Logic : GraphStageLogic
    {
        Option<T> _lastElement = Option<T>.None;

        public Logic(Duplicator<T> duplicator) : base(duplicator.Shape)
        {
            SetHandler(duplicator.In,
                onPush: () =>
                {
                    var element = Grab(duplicator.In);
                    _lastElement = element;
                    Push(duplicator.Out, element);
                },
                onUpstreamFinish: () =>
                {
                    if(_lastElement.HasValue)
                        Emit(duplicator.Out, _lastElement.Value);

                    Complete(duplicator.Out);
                });

            SetHandler(duplicator.Out, onPull: () =>
            {
                if (_lastElement.HasValue)
                {
                    Push(duplicator.Out, _lastElement.Value);
                    _lastElement = Option<T>.None;
                }
                else
                    Pull(duplicator.In);
            });
        }
    }

    public Duplicator(Predicate<T> predicate)
    {
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("Duplicator.in");

    public Outlet<T> Out { get; } = new Outlet<T>("Duplicator.out");

    public override FlowShape<T, T> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

In this case a pull from downstream might be consumed by the stage itself rather than passed along upstream as the stage might contain an element it wants to push. Note that we also need to handle the case where the upstream closes while the stage still has elements it wants to push downstream. This is done by overriding onUpstreamFinish in the InHandler and provide custom logic that should happen when the upstream has been finished.

This example can be simplified by replacing the usage of a mutable state with calls to `EmitMultiple` which will replace the handlers, emit each of multiple elements and then reinstate the original handlers:

```csharp
class Duplicator<T> : GraphStage<FlowShape<T, T>>
{
    private sealed class Logic : GraphStageLogic
    {
        public Logic(Duplicator<T> duplicator) : base(duplicator.Shape)
        {

            SetHandler(duplicator.In, onPush: () =>
            {
                var element = Grab(duplicator.In);
                // this will temporarily suspend this handler until the two elems
                // are emitted and then reinstates it
                EmitMultiple(duplicator.Out, new[] {element, element});
            });

            SetHandler(duplicator.Out, onPull: () => Pull(duplicator.In));
        }
    }

    public Duplicator(Predicate<T> predicate)
    {
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("Duplicator.in");

    public Outlet<T> Out { get; } = new Outlet<T>("Duplicator.out");

    public override FlowShape<T, T> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

Finally, to demonstrate all of the stages above, we put them together into a processing chain, which conceptually would correspond to the following structure:

![Graph stage chain](/images/graph_stage_chain1.png)

In code this is only a few lines, using the `Via` use our custom stages in a stream:

```csharp
var resultTask = Source.From(new [] {1,2,3,4,5})
        .Via(new Filter<int>(n => n%2 ==0))
        .Via(new Duplicator<int>())
        .Via(new Map<int, int>(n=>n/2))
        .RunAggregate(0, (sum, next) => sum + next, materializer);
```

If we attempt to draw the sequence of events, it shows that there is one "event token" in circulation in a potential chain of stages, just like our conceptual "railroad tracks" representation predicts.

![Graph stage tracks](/images/graph_stage_tracks_11.png)

### Completion

Completion handling usually (but not exclusively) comes into the picture when processing stages need to emit a few more elements after their upstream source has been completed. We have seen an example of this in our first `Duplicator` implementation where the last element needs to be doubled even after the upstream neighbor stage has been completed. This can be done by overriding the `onUpstreamFinish` callback in `SetHandler(in, action)`.

Stages by default automatically stop once all of their ports (input and output) have been closed externally or internally. It is possible to opt out from this behavior by invoking `SetKeepGoing(true)` (which is not supported from the stage's constructor and usually done in `PreStart`). In this case the stage **must** be explicitly closed by calling `CompleteStage()` or `FailStage(exception)`. This feature carries the risk of leaking streams and actors, therefore it should be used with care.

### Logging Inside GraphStages

Logging debug or other important information in your stages is often a very good idea, especially when developing
more advances stages which may need to be debugged at some point.

The `Log` property is provided to enable you to easily obtain a `LoggingAdapter`
inside of a `GraphStage` as long as the `Materializer` you're using is able to provide you with a logger.
In that sense, it serves a very similar purpose as `ActorLogging` does for Actors.

> [!NOTE]
> Please note that you can always simply use a logging library directly inside a Stage.
Make sure to use an asynchronous appender however, to not accidentally block the stage when writing to files etc.

The stage gets access to the `Log` property which it can safely use from any ``GraphStage`` callbacks:

```csharp
private sealed class RandomLettersSource : GraphStage<SourceShape<string>>
{
    #region internal classes

    private sealed class Logic : GraphStageLogic
    {
        public Logic(RandomLettersSource stage) : base(stage.Shape)
        {
            SetHandler(stage.Out, onPull: () =>
            {
                var c = NextChar(); // ASCII lower case letters

                Log.Debug($"Randomly generated: {c}");

                Push(stage.Out, c.ToString());
            });
        }

        private static char NextChar() => (char) ThreadLocalRandom.Current.Next('a', 'z'1);
    }

    #endregion

    public RandomLettersSource()
    {
        Shape = new SourceShape<string>(Out);
    }

    private Outlet<string> Out { get; } = new Outlet<string>("RandomLettersSource.out");

    public override SourceShape<string> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}

[Fact]
public void A_GraphStageLogic_must_support_logging_in_custom_graphstage()
{
    const int n = 10;
    EventFilter.Debug(start: "Randomly generated").Expect(n, () =>
    {
        Source.FromGraph(new RandomLettersSource())
            .Take(n)
            .RunWith(Sink.Ignore<string>(), Materializer)
            .Wait(TimeSpan.FromSeconds(3));
    });
}
```

> [!NOTE]
> **SPI Note:** If you're implementing a Materializer, you can add this ability to your materializer by implementing
`IMaterializerLoggingProvider` in your `Materializer`.

### Using Timers

It is possible to use timers in `GraphStages` by using `TimerGraphStageLogic` as the base class for the returned logic. Timers can be scheduled by calling one of `ScheduleOnce(key,delay)`, `SchedulePeriodically(key,period)` or `SchedulePeriodicallyWithInitialDelay(key,delay,period)` and passing an object as a key for that timer (can be any object, for example a String). The `OnTimer(key)` method needs to be overridden and it will be called once the timer of key fires. It is possible to cancel a timer using `CancelTimer(key)` and check the status of a timer with `IsTimerActive(key)`. Timers will be automatically cleaned up when the stage completes.

Timers can not be scheduled from the constructor of the logic, but it is possible to schedule them from the `PreStart()` lifecycle hook.

In this sample the stage toggles between open and closed, where open means no elements are passed through. The stage starts out as closed but as soon as an element is pushed downstream the gate becomes open for a duration of time during which it will consume and drop upstream messages:

```csharp
class TimedGate<T> : GraphStage<FlowShape<T, T>>
{
    private readonly TimeSpan _silencePeriod;

    private sealed class Logic : TimerGraphStageLogic
    {
        private bool _open;

        public Logic(TimedGate<T> gate) : base(gate.Shape)
        {
            SetHandler(gate.In, onPush: () =>
            {
                var element = Grab(gate.In);
                if (_open)
                    Pull(gate.In);
                else
                {
                    Push(gate.Out, element);
                    _open = true;
                    ScheduleOnce("Close", gate._silencePeriod);
                }
            });

            SetHandler(gate.Out, onPull: () => Pull(gate.In));
        }

        protected internal override void OnTimer(object timerKey) => _open = false;
    }

    public TimedGate(TimeSpan silencePeriod)
    {
        _silencePeriod = silencePeriod;
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("TimedGate.in");

    public Outlet<T> Out { get; } = new Outlet<T>("TimedGate.out");

    public override FlowShape<T, T> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

### Using Asynchronous Side-Channels

In order to receive asynchronous events that are not arriving as stream elements (for example a completion of a task or a callback from a 3rd party API) one must acquire a `AsyncCallback` by calling `GetAsyncCallback()` from the stage logic. The method `GetAsyncCallback` takes as a parameter a callback that will be called once the asynchronous event fires. It is important to **not call the callback directly**, instead, the external API must `invoke` the returned `Action`. The execution engine will take care of calling the provided callback in a thread-safe way. The callback can safely access the state of the `GraphStageLogic` implementation.

Sharing the `AsyncCallback` from the constructor risks race conditions, therefore it is recommended to use the `PreStart()` lifecycle hook instead.

This example shows an asynchronous side channel graph stage that starts dropping elements when a future completes:

```csharp
class KillSwitch<T> : GraphStage<FlowShape<T, T>>
{
    private sealed class Logic : GraphStageLogic
    {
        private readonly KillSwitch<T> _killSwitch;

        public Logic(KillSwitch<T> killSwitch) : base(killSwitch.Shape)
        {
            _killSwitch = killSwitch;

            SetHandler(killSwitch.In, onPush: () => Push(killSwitch.Out, Grab(killSwitch.In)));
            SetHandler(killSwitch.Out, onPull: () => Pull(killSwitch.In));
        }

        public override void PreStart()
        {
            var callback = GetAsyncCallback(CompleteStage);
            _killSwitch._killSwitch.ContinueWith(_ => callback());
        }
    }

    private readonly Task _killSwitch;

    public KillSwitch(Task killSwitch)
    {
        _killSwitch = killSwitch;
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("KillSwitch.in");

    public Outlet<T> Out { get; } = new Outlet<T>("KillSwitch.out");

    public override FlowShape<T, T> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

### Integration with Actors

**This section is a stub and will be extended in the next release This is an experimental feature***

It is possible to acquire an ActorRef that can be addressed from the outside of the stage, similarly how `AsyncCallback` allows injecting asynchronous events into a stage logic. This reference can be obtained by calling `GetStageActorRef(receive)` passing in a function that takes a `Tuple` of the sender `IActorRef` and the received message. This reference can be used to watch other actors by calling its `Watch(ref)` or `Unwatch(ref)` methods. The reference can be also watched by external actors. The current limitations of this `IActorRef` are:

* they are not location transparent, they cannot be accessed via remoting.
* they cannot be returned as materialized values.
* they cannot be accessed from the constructor of the `GraphStageLogic`, but they can be accessed from the `PreStart()` method.

### Custom Materialized Values

Custom stages can return materialized values instead of `NotUsed` by inheriting from `GraphStageWithMaterializedValue` instead of the simpler `GraphStage`. The difference is that in this case the method `CreateLogicAndMaterializedValue(inheritedAttributes)` needs to be overridden, and in addition to the stage logic the materialized value must be provided

> [!WARNING]
> There is no built-in synchronization of accessing this value from both of the thread where the logic runs and the thread that got hold of the materialized value. It is the responsibility of the programmer to add the necessary (non-blocking) synchronization and visibility guarantees to this shared object.

In this sample the materialized value is a task containing the first element to go through the stream:

```csharp
class FirstValue<T> : GraphStageWithMaterializedValue<FlowShape<T, T>, Task<T>>
{
    private sealed class Logic : GraphStageLogic
    {
        public Logic(FirstValue<T> first, TaskCompletionSource<T> completion) : base(first.Shape)
        {
            SetHandler(first.In, onPush: () =>
            {
                var element = Grab(first.In);
                completion.SetResult(element);
                Push(first.Out, element);

                // replace handler with one just forwarding
                SetHandler(first.In, onPush: () => Push(first.Out, Grab(first.In)));
            });

            SetHandler(first.Out, onPull: () => Pull(first.In));
        }
    }

    public FirstValue( )
    {
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("FirstValue.in");

    public Outlet<T> Out { get; } = new Outlet<T>("FirstValue.out");

    public override FlowShape<T, T> Shape { get; }

    public override ILogicAndMaterializedValue<Task<T>> CreateLogicAndMaterializedValue(Attributes inheritedAttributes)
    {
        var completion = new TaskCompletionSource<T>();
        var logic = new Logic(this, completion);
        return new LogicAndMaterializedValue<Task<T>>(logic, completion.Task);
    }
}
```

## Using Attributes to Affect the Behavior of a Stage

> [!NOTE]
> This section is a stub and will be extended in the next release.

Stages can access the `Attributes` object created by the materializer. This contains all the applied (inherited) attributes applying to the stage, ordered from least specific (outermost) towards the most specific (innermost) attribute. It is the responsibility of the stage to decide how to reconcile this inheritance chain to a final effective decision.

See [Modularity, Composition and Hierarchy](xref:streams-modularity) for an explanation on how attributes work.

### Rate Decoupled Graph Stages

Sometimes it is desirable to decouple the rate of the upstream and downstream of a stage, synchronizing only when needed.

This is achieved in the model by representing a `GraphStage` as a *boundary* between two regions where the demand sent upstream is decoupled from the demand that arrives from downstream. One immediate consequence of this difference is that an `onPush` call does not always lead to calling `Push` and an `onPull` call does not always lead to calling `Pull`.

One of the important use-case for this is to build buffer-like entities, that allow independent progress of upstream and downstream stages when the buffer is not full or empty, and slowing down the appropriate side if the buffer becomes empty or full.

The next diagram illustrates the event sequence for a buffer with capacity of two elements in a setting where the downstream demand is slow to start and the buffer will fill up with upstream elements before any demand is seen from downstream.

![Graph stage detached](/images/graph_stage_detached_tracks_11.png)

Another scenario would be where the demand from downstream starts coming in before any element is pushed into the buffer stage.

![graph stage detached](/images/graph_stage_detached_tracks_21.png)

The first difference we can notice is that our `Buffer` stage is automatically pulling its upstream on initialization. The buffer has demand for up to two elements without any downstream demand.

The following code example demonstrates a buffer class corresponding to the message sequence chart above.

```csharp
class TwoBuffer<T> : GraphStage<FlowShape<T, T>>
{
    private sealed class Logic : GraphStageLogic
    {
        private readonly TwoBuffer<T> _buffer;
        private readonly Queue<T> _queue;
        private bool _downstreamWaiting = false;

        public Logic(TwoBuffer<T> buffer) : base(buffer.Shape)
        {
            _buffer = buffer;
            _queue = new Queue<T>();

            SetHandler(buffer.In, OnPush, OnUpstreamFinish);
            SetHandler(buffer.Out, OnPull);
        }

        private bool BufferFull => _queue.Count == 2;

        private void OnPush()
        {
            var element = Grab(_buffer.In);
            _queue.Enqueue(element);
            if (_downstreamWaiting)
            {
                _downstreamWaiting = false;
                var bufferedElement = _queue.Dequeue();
                Push(_buffer.Out, bufferedElement);
            }
            if(!BufferFull)
                Pull(_buffer.In);
        }

        private void OnUpstreamFinish()
        {
            if (_queue.Count != 0)
            {
                // emit the rest if possible
                EmitMultiple(_buffer.Out, _queue);
            }

            CompleteStage();
        }

        private void OnPull()
        {
            if (_queue.Count == 0)
                _downstreamWaiting = true;
            else
            {
                var element = _queue.Dequeue();
                Push(_buffer.Out, element);
            }

            if(!BufferFull && !HasBeenPulled(_buffer.In))
                Pull(_buffer.In);
        }
    }

    public TwoBuffer()
    {
        Shape = new FlowShape<T, T>(In, Out);
    }

    public Inlet<T> In { get; } = new Inlet<T>("TwoBuffer.in");

    public Outlet<T> Out { get; } = new Outlet<T>("TwoBuffer.out");

    public override FlowShape<T, T> Shape { get; }

    protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) => new Logic(this);
}
```

## Thread Safety of Custom Processing Stages

**All of the above custom stages (linear or graph) provide a few simple guarantees that implementors can rely on.**

* The callbacks exposed by all of these classes are never called concurrently.
* The state encapsulated by these classes can be safely modified from the provided callbacks, without any further synchronization.

In essence, the above guarantees are similar to what `Actor`'s provide, if one thinks of the state of a custom stage as state of an actor, and the callbacks as the `receive` block of the actor.

> [!WARNING]
> It is **not** safe to access the state of any custom stage outside of the callbacks that it provides, just like it is unsafe to access the state of an actor from the outside. This means that Future callbacks should not close over internal state of custom stages because such access can be concurrent with the provided callbacks, leading to undefined behavior.

## Resources and the Stage Lifecycle

If a stage manages a resource with a lifecycle, for example objects that need to be shutdown when they are not
used anymore it is important to make sure this will happen in all circumstances when the stage shuts down.

Cleaning up resources should be done in `GraphStageLogic.PostStop` and not in the `InHandler` and `OutHandler`
callbacks. The reason for this is that when the stage itself completes or is failed there is no signal from the upstreams
for the downstreams. Even for stages that do not complete or fail in this manner, this can happen when the
`Materializer` is shutdown or the `ActorSystem` is terminated while a stream is still running, what is called an
"abrupt termination".

## Extending Flow Combinators with Custom Operators

The most general way of extending any `Source`, `Flow` or `SubFlow` (e.g. from `GroupBy`) is demonstrated above: create a graph of flow-shape like the `Filter` example given above and use the `.Via(...)` combinator to integrate it into your stream topology. This works with all `IFlow` sub-types, including the ports that you connect with the graph DSL.

Advanced .Net users may wonder whether it is possible to write extension methods that enrich `IFlow` to allow nicer syntax. The short answer is that .Net does not support this in a fully generic fashion, the problem is that it is impossible to abstract over the kind of stream that is being extended because Source, Flow and SubFlow differ in the number and kind of their type parameters.

A lot simpler is the task of just adding an extension method to `Source` and `Flow` as shown below:

```csharp
public static class Extensions
{
    public static Flow<TIn, TOut, TMat> Filter<TIn, TOut, TMat>(this Flow<TIn, TOut, TMat> flow,
        Predicate<TOut> predicate) => flow.Via(new Filter<TOut>(predicate));

    public static Source<TOut, TMat> Filter<TOut, TMat>(this Source<TOut, TMat> source,
        Predicate<TOut> predicate) => source.Via(new Filter<TOut>(predicate));
}
```

you can then replace the `.Via` call from above with the extension method:

```csharp
var resultTask = Source.From(new [] {1,2,3,4,5})
    .Filter(n => n % 2 == 0)
    .Via(new Duplicator<int>())
    .Via(new Map<int, int>(n=>n/2))
    .RunAggregate(0, (sum, next) => sum + next, materializer);
```

If you try to write this for `SubFlow`, though, you will run into the same issue as when trying to unify the two solutions above, only on a higher level (the type constructors needed for that unification would have rank two, meaning that some of their type arguments are type constructors themselves when trying to extend the solution shown in the linked sketch the author encountered such a density of compiler StackOverflowErrors and IDE failures that he gave up).

branch_name: refs/heads/dev

gha_language: C#


text:
# ploodood
> Generates words that "sound like real words".

Original work by [The Ploodood!](http://www.ploodood.net/)

I use it to generate moderately strong passwords.

## Install
Install globally with `pip`:

```bash
sudo pip install ploodood
```

or with [pipsi](https://github.com/mitsuhiko/pipsi):

```bash
pipsi install ploodood
```

## Usage
Now that ploodood is installed globally, run `ploodood` to use the CLI.

You can change some options, just look at `ploodood --help`.

Example: `ploodood --minlen 8 --wordcount 5`

## Author
**Matteo Giordano**

* [github/malteo](https://github.com/malteo)
* [twitter/malteo](https://twitter.com/malteo)

## License
Copyright (c) 2015 Matteo Giordano
Released under the MIT license

repository: malteo/ploodood

repo_name: malteo/ploodood


text:
# Deep Q-Network: Banana Problem
Deep Q-Network for solving the banana problem.

## Project Details

The banana environment is a Unity environment that consists of an agent in a room with blue and yellow bananas. The goal is to navigate and collect as many yellow bananas as possible.

- State space: 37 dimensions including velocity and ray-based perception of objects ahead of the agent.
- Action space: Move forward, backward, left, or right.
- Rewards: +1 (collect yellow banana), -1 (collect blue banana).

The environment is considered solved when the agent achieves an average reward of at least +13 over the past 100 episodes.

The code in this project is based heavily off the code from the Udacity Deep Reinforcement Learning Project 1 [base](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation) and Deep Q-Network [solution](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation).

## Getting Started

Follow the instructions at the Udacity [Deep Reinforcement Learning](https://github.com/udacity/deep-reinforcement-learning) repository for general insturctions on setting up the environment. Specific instructions for installing and downloading required files for this project are at located in [Project 1](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation).

## Instructions

Run `dqn/banana.ipynb` to train the agent and visualize the scores over time. The logic for the agent and neural network are in `dgn/dqn_agent.py` and `dqn/qnetwork.py`, respectively. There is also a replay buffer defined at `dqn/replaybuffer.py`. Finally, the `dqn` funciton in `dqn/deepqnetwork.py` manages the training for the agent. The model weights for the successful agent are saved in `dqn/checkpoint.pth`.

repository: ajkeith/banana-dqn

repo_name: ajkeith/banana-dqn