text:
---
layout: post
title: "OsERdj7"
description: ""
category: genes
tags: [seed, salt, quality, prolamin, endosperm, ER quality control]
---

* **Information**
    + Symbol: OsERdj7
    + MSU: [LOC_Os12g15590](http://rice.uga.edu/cgi-bin/ORF_infopage.cgi?orf=LOC_Os12g15590)
    + RAPdb: [Os12g0258200](https://rapdb.dna.affrc.go.jp/locus/?name=Os12g0258200)

* **Publication**
    + [OsERdj7 is an ER-resident J-protein involved in ER quality control in rice endosperm.](http://www.ncbi.nlm.nih.gov/pubmed?term=OsERdj7 is an ER-resident J-protein involved in ER quality control in rice endosperm.%5BTitle%5D), 2019, J Plant Physiol.

* **Genbank accession number**

* **Key message**
    + Next, to investigate its role during seed maturation, expression of OsERdj7 was specifically downregulated using RNA interference (RNAi) under the control of the endosperm-specific 16<U+2009>kDa prolamin promoter in transgenic rice
    + Due to PDIL2-3 and OsHard3 co-immunoprecipitating with OsERdj7 in rice protoplasts, this result implicates OsERdj7 in the translocation of some seed proteins within the ER lumen and in the degradation of misfolded or unfolded proteins
    + When the microsomal fraction of maturing seeds was treated with alkaline, high salt or detergent compounds, OsERdj7 was solubilized, even in alkaline and high salt environments, indicating that it is not tightly integrated in the ER membrane
    + OsERdj7 is an ER-resident J-protein involved in ER quality control in rice endosperm.

* **Connection**
    + __HSP70__, __OsERdj7__, [OsERdj7 is an ER-resident J-protein involved in ER quality control in rice endosperm.](http://www.ncbi.nlm.nih.gov/pubmed?term=OsERdj7 is an ER-resident J-protein involved in ER quality control in rice endosperm.%5BTitle%5D), OsERdj7 is one of six endoplasmic reticulum (ER)-resident J-domain-containing proteins (J-proteins) encoded by the rice genome that acts as a co-chaperone for Hsp70 and is characterized by the presence of two transmembrane domains

[//]: # * **Key figures**


text:
## Coding Standards

This document provides general guidance for developing code for the project.
The rules in this document guide us in writing high-quality code that will
allow us to scale the project and ensure that the code base remains readable and
maintainable.

### Follow the LLVM and Facebook coding standards

Use the Facebook coding standards when writing c++ code. The Facebook coding
standards are almost identical to the LLVM coding standards, except for lower
case variable naming and the underscore suffix. The LLVM coding standards also
covers topics that are compiler specific:

  '''
  http://llvm.org/docs/CodingStandards.html
  '''

The compiler uses the LLVM data structures, described in the links below. The
LLVM data structures and utilities are efficient, well documented and battle
hardened.

  '''
  http://llvm.org/docs/ProgrammersManual.html
  '''

The project uses a reasonable subset of C++11. Just like LLVM, the project does
not use exceptions and RTTI.

### Small incremental changes

The project is developed using small incremental changes. These changes can be
small bug fixes or minor tweaks. Other times, these changes are small steps
along the path to reaching larger stated goals. Long-term development branches
suffer from many problems, including the lack of visibility, difficulty of code
review, lack of testing of the branch and merge difficulty. Commits that go into
the project need to be reviewable. This means that commits need to be relatively
small, well documented and self-contained.

### Add tests

Functional changes to the compiler need to include a testcase. Unit tests and
regression tests are critical to the qualification of the compiler. Every bug
fix needs to include a testcase.

### Format your code

We encourage the use of clang-format to enforce code style and formatting.
Commits that only change the formatting of code should go in separate commits.
This makes reviewing the code and inspecting history easier.

It's recommended to use a pre-commit hook to properly format your code prior
committing any changes. Run the following command from the root of the repo
to enable the hook:
`ln -s ../../utils/format.sh .git/hooks/pre-commit`.

### Commit messages

Here are some guidelines about the format of the commit message:

Separate the commit message into a single-line title and a separate body that
describes the change. Make the title short (80 chars) and readable.  In changes
that are restricted to a specific part of the code, include a [tag] at the start
of the line in square brackets, for example, "[docs] ... ".

If the commit fixes an issue in the bug tracking system, include a link or a
task number. When reverting a change make sure to add a short note that
describes why the patch is being reverted.

### Code review

The project relies on code review to maintain the software quality. Review other
people's changes! Anybody is allowed to review code and comment on patches.

All changes, by all developers, must be reviewed before they are committed to
the repository.

### Nits checklist

There are some common issues that we flag in PRs that can be solved
mechanically.  Going through this checklist before submitting a PR will speed up
the review process by eliminating round-trips for feedback.

* Remember copyright headers on new source files.
* Prefer `llvm::StringRef`/`ArrayRef` to `std::string`/`vector`.
* Comments are complete sentences: begin with a capital, end with a period.
* Add Doxygen comments to every new class, method, function, global, etc.
* Boolean arguments should be tagged with a /* name_of_param */ comment.
* Always clang-format!
  Use [format.sh tool](https://github.com/pytorch/glow/blob/master/utils/format.sh) with the
  `fix` argument. (The linter will catch this, but it never hurts.)

repository: pytorch/glow

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: pytorch/glow


text:
+++
title = "About Me"
description = "Juan Carrey, Principal Software Engineer @ Zartis"
date = "2019-02-28"
aliases = ["about-me","about-juan-carrey", "juan-carrey"]
author = "Juan Carrey"
+++

My name is [Juan Carrey](https://github.com/juancarrey), I am a Principal Full Stack Software Engineer at Zartis that just wants to share my thoughts around tech related stuff in general.

## Highlights

I have worked for over 11yrs on the software industry, in three different countries, my beloved Spain (Madrid & Zaragoza), UK (London) and Ireland (Cork)

I have been working mainly in Java & C# backends, along with Angular & React, but have also started lot's of small pet projects in many other languages like Python, Kotlin, Rust, Powershell, Ruby, Go, Swift, Flutter & Haskell and others that I do not even remember

I would consider myself as a true **full stack**: UX, Frontend, Backend, Architecture, CI & CD, Infrastructure as Code and bits of Operations

## Experience

I am a generalist, I have touched many things from lot's of different databases like DynamoDB, Cassandra, ArangoDB, ScillaDB, Mongo, Postgres, Mysql, Oracle, SqlServer, Graph4j, Eventstore,
to many frameworks and tools, as well as made some Devops work with full CI&CD on AWS, and even did some Alfresco and Liferay consultancy time.

I fancy architecture patterns and love to design adecuate solutions to problems, avoiding over and under engineering them, which is not easy to achieve.

license: MIT

repository: juancarrey/hgblog

repo_name: juancarrey/hgblog


text:
# JavaScript strings marking guide

The aim of the tasks is to demonstrate an understanding of the JavaScript features covered in the [Handling text — strings in JavaScript](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/First_steps/Strings) and [Useful string methods](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/First_steps/Useful_string_methods) lessons in Learn Web Development on MDN.

Note: If there is an error in your code, it will be outputted into the results panel on the page, to help you try to figure out the answer (or into the browser's JavaScript console, in the case of the downloadable version).

## Task 1

The first task asks you to store a new string in a variable to go along wth the existing one, which contains the missing half of the famous quote. Call the new variable you add `quoteEnd`.

You'll also need to escape the apostrophe in the existing string so that it doesn't throw an error.

Finally, you'll need to concatenate the two strings together, and store the result in a variable called `finalQuote`.

The answer should look something like this:

```
const quoteStart = 'Don\'t judge each day by the harvest you reap ';
const quoteEnd = 'but by the seeds that you plant.'

const finalQuote = `${quoteStart}${quoteEnd}`;
```

## Task 2

Next up, task 2 requires the student to:

* Check the length of the provided string using `.length`, and store the result in a variable called `quoteLength`.
* Find the index of the substring 'green eggs and ham' using `.indexOf` and store it in `index`.
* Use a combination of the variables you have, `.length`, and `.slice()`, to trim down the original quote to 'I do not like green eggs and ham.', and store it in `revisedQuote`.

```
const quote = 'I do not like green eggs and ham. I do not like them, Sam-I-Am.';
const substring = 'green eggs and ham';

const quoteLength = quote.length;

const index = quote.indexOf(substring);

const revisedQuote = quote.slice(0, index + substring.length + 1);
```

## Task 3

For our next string task, we return to our Green Eggs and Ham revised quote, which someone has messed up. You need to:

* Fix the casing. The best way to do this is to put it all in lower case using `.toLowerCase()`, and then put the first letter in uppercase using `replace()`, `slice()`, and `toUpperCase()`. Store the new quote in `fixedQuote`.
* Replace `green eggs and ham` with whatever food you really don't like using `replace()`.
* Add a full stop to the end of the sentence using some means.

Your code should look something like this:

```
const quote = 'I dO nOT lIke gREen eGgS anD HAM';

let fixedQuote = quote.toLowerCase();
const firstLetter = fixedQuote.slice(0,1);
fixedQuote = fixedQuote.replace(firstLetter, firstLetter.toUpperCase());

fixedQuote = fixedQuote.replace('green eggs and ham', 'pickled onions');

const finalQuote = `${fixedQuote}.`;
```

## Task 4

Our final string task looks at your ability to use template literals. Your answer is expected to take the existing string literal, `myString`, turn it into a template literal, and include four placeholders in place of the asterisks:

* The `theorem` string.
* The value of `a`.
* The value of `b`.
* The length of the hypotenuse, if `a` and `b` are the lengths of the two shortest sides of a right-angled triangle. You can use pythagoras' theorem to work this out.

Your code should look something like this:

```
const theorem = 'Pythagorean theorem';

const a = 5;
const b = 8;

const myString = `Using ${ theorem }, we can work out that that if the two shortest sides of a right-angled triangle have lengths of ${ a } and ${ b }, the length of the hypotenuse is ${ Math.sqrt((a**2) + (b**2)) }.`;
```

You could use a different form of the fourth placeholder, such as `Math.sqrt(Math.pow(a, 2) + Math.pow(b, 2))` or `${ Math.sqrt((a * a) + (b * b)) }`.

repository: mdn/learning-area

branch_name: refs/heads/main

gha_language: HTML

repo_name: mdn/learning-area


text:
openstack-ops-tools
===================
This is a repository of useful tools we use for working with openstack.

portping
========
This utility provides a ping like interface to ping tcp port services.

Usage
-----
Usage: portping [options] url|host[:port]|ip[:port]

Options:
  -h, --help   show this help message and exit
  -c COUNT     Number of pings to send
  -i INTERVAL  Ping interval
  -d           Show timings for the entire connection

Examples
--------
Ping port 80 (http) on www.yahoo.com
------------------------------------
Here we ping www.yahoo.com via http (port 80).
It is easy to see that that address has multiple hosts responding and the
latency of each request.

    $ portping www.yahoo.com
    PORTPING www.yahoo.com:80 (<IP_ADDRESS>:80).
    from www.yahoo.com:80 (<IP_ADDRESS>:80): time=2.46 ms
    from www.yahoo.com:80 (<IP_ADDRESS>:80): time=2.43 ms
    --- www.yahoo.com ping statistics ---
    2 packets transmitted, 2 received, 0% packet loss, time 1704.0ms
    rtt min/avg/max/dev = 2.43/2.44/2.46/4.00 ms
    $

Same thing using ssl
--------------------

    $ portping www.yahoo.com:443
    PORTPING www.yahoo.com:443 (<IP_ADDRESS>:443).
    from www.yahoo.com:443 (<IP_ADDRESS>:443): time=2.89 ms
    from www.yahoo.com:443 (<IP_ADDRESS>:443): time=2.81 ms
    --- www.yahoo.com ping statistics ---
    2 packets transmitted, 2 received, 0% packet loss, time 1744.0ms
    rtt min/avg/max/dev = 2.81/2.85/2.89/6.08 ms
    $

Pinging a URL instead of the port
---------------------------------
Portping also allows specifying a URL.  If a URL is specified it will also
perform an http get request and show the response.

This is useful to see when there are hosts doing unexpected things in a dns
rotation or behind a reverse proxy or vip.

In this example we specify a url of http://cnn.com/

This shows that there are two hosts responding to this request and that they
are returning different amounts of data in their responses.

    $ portping http://cnn.com/
    PORTPING cnn.com:80 (<IP_ADDRESS>:80).
    1500 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 time=87.14 ms
    1448 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 time=64.82 ms
    1500 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 time=62.98 ms
    1500 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 time=78.30 ms
    --- cnn.com ping statistics ---
    4 packets transmitted, 4 received, 0% packet loss, time 4372.0ms
    rtt min/avg/max/dev = 62.98/73.31/87.14/56.00 ms
    [dhubbard@usi-tool1:::sshmap]$

Getting a URL with timings
--------------------------
The detailed timing flag adds timings for the parts of each tcp session.

This is useful for determining what is causing latency issues or errors.

Here we are doing the previous example with detailed timings.

In the results we can see a few new things.

It is clear that the host with address <IP_ADDRESS> is taking significantly longer to establish the tcp connection and handle the http get request.

    $ portping -d http://cnn.com/
    PORTPING cnn.com:80 (<IP_ADDRESS>:80).
    1386 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 dns=0.21ms connect=68.36ms request=130.02ms all=198.73ms
    1386 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 dns=0.30ms connect=66.72ms request=101.07ms all=168.20ms
    1500 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 dns=0.30ms connect=123.94ms request=203.08ms all=327.43ms
    1386 bytes from cnn.com:80 (<IP_ADDRESS>:80):response=200 dns=0.28ms connect=68.32ms request=87.94ms all=156.69ms
    --- cnn.com ping statistics ---
    4 packets transmitted, 4 received, 0% packet loss, time 5009.0ms
    rtt min/avg/max/dev = 156.69/212.76/327.43/138.24 ms
    $

get_instance_info
=================
Generate an instance state based on information from libvirt on the hypervisors
and information from the openstack nova service.

This is useful to generate state information that can be done before and after
operational actions to see potential issues.


text:
# koa-openapi Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## 3.6.0 - 2018-12-31
### Fixed
-<EMAIL_ADDRESS>support refs in requestBody schema to both definitions and components.schemas

## 3.5.0 - 2018-12-20
### Added
- Updated openapi-framework to handle OpenAPI V3 requestBody and components.

## 3.4.0 - 2018-12-12
### Added
- Updated openapi-framework to 0.13.0 for OpenAPI V3 response application/json validation support.

## 3.3.0 - 2018-12-11
### Added
- Updated openapi-framework to 0.12.0 for OpenAPI V3 response nullable support.

## 3.2.0 - 2018-11-21
### Added
- Updated openapi-framework to 0.10.0 for OpenAPI V3 request coercion support.

## 3.1.0 - 2018-11-20
### Added
- Updated openapi-framework to 0.9.0 for OpenAPI V3 requestBody.consumes support.

## 3.0.0 - 2018-10-25
- Updated `openapi-framework` to accept `args.enableObjectCoercion`.
- express-openapi reference in README.

## 2.0.0 - 2018-10-25 (Deprecated)
_Note: This was from the previous author_

## 0.2.0 - 2018-10-25 (Deprecated)
_Note: This was from the previous author_

## 0.1.0 - 2018-10-25 (Deprecated)
_Note: This was from the previous author_

## 0.0.6 - 2018-10-19
### Fixed
- express-openapi reference in README.

## 0.0.5 - 2018-10-17
### Added
- Getting Started guide in README.

## 0.0.4 - 2018-10-12
### Fixed
- Basic usage tested.

repository: maxime-morand/open-api

gha_license_id: MIT

repo_name: maxime-morand/open-api


text:
# {{ page.title }}

## Scope of Resource Profiles

* **Location** is physical location, where service delivery occurs.
* **Organization** is an entity providing practitioner roles and/or healthcare service.
* **PractitionerRole** (provider) is a practitioner acting in a role at one location for one organisation.
* **Practitioner** is an individual that maybe acting in a practitioner role.
* **HealthcareService** describes the services delivered at one location for one organisation.

## Interactions Overview

A subset of FHIR possible interactions are required for this profile.

### Instance Level Interactions
----------

* **[read](http://hl7.org/fhir/STU3/http.html#read)** PractitionerRole, Practitioner, HealthcareService, Location, Organization, Endpoint
* **vread** Not Required
* **update** Not Required
* **patch** Not Required
* **delete** Not Required
* **history** Not Required

### Type Level Interactions
----------

* **create** Not Required
* **[search](http://hl7.org/fhir/STU3/http.html#search)** PractitionerRole, Practitioner, HealthcareService, Location, Organization, Endpoint
* **history** Not Required

### Whole System Interactions
----------

* **capabilities** Required
* **batch/transaction** Not Required
* **history** Not Required
* **search** Not required

## Type Level Search/Include

|Resource Type|Profile|Search Parameters|Includes|
|---|---|---|---|
|[Organization](#organization)| Australian Organisation Directory Entry | name, identifier ||
|[Practitioner](#practitioner)| Australian Practitioner Directory Entry | identifier, family, given, name ||
|[PractitionerRole](#practitionerrole)| Australian PractitionerRole Directory Entry | identifier, location.address-city, location.address-postalcode, location.address-state, location.near, location.near-distance, organization.identiifer, practitioner.name, practitioner.family, practitioner.given | location, organization, endpoint |
|[HealthcareService](#healthcareservice)| Australian HealthcareService Directory Entry | name, location.address-city, location.address-postalcode, location.address-state, location.near, location.near-distance, organization.identiifer | location, organization, endpoint |
|[Location](#location)| Australian Location Directory Entry | address-city, address-postalcode, address-state, near, near-distance ||
|[Endpoint](#endpoint)| Australian Endpoint Directory Entry | ||
{:.grid}

repository: umaprasath/au-fhir-pd

repo_name: umaprasath/au-fhir-pd


text:
### Installation

In order to use the workspace, we need to follow a list of steps presented below.

[![Node version](https://img.shields.io/node/v/[fgt-workspace].svg?style=flat)](http://nodejs.org/download/)

#### Step 1: Clone the workspace

```sh
$ git clone https://github.com/PharmaLedger-IMI/fgt-workspace.git
```

After the repository was cloned, you must install all the dependencies.

For the latest versions do:
```sh
$ cd fgt-workspace
#Important: If you plan to contribute to the project and/or dependecies please set DEV:true
#in the file env.json before you run the installation!
$ npm run dev-install
```

For latest 'stable' version do:
```sh
$ npm run install
```
instead.

**Note:** this command might take quite some time depending on your internet connection and you machine processing power.

#### Step 2: Launch the "server"

While in the *fgt-workspace* folder run:

```sh
$ npm run server
```

At the end of this command you get something similar to:

![alt text](resources/scr-npm-run-server.png)

#### Step 3: Build all DSUs and anchor them to the 'blockchain'.

Open a new console inside *fgt-workspace* folder and run:

```sh
# Note: Run this in a new console inside "fgt-workspace" folder
$ npm run build-all
```

### Documentation

To be able to generate the documentation for this project via

```sh
$ npm run docs
```

[draw.io](https://github.com/jgraph/drawio-desktop/releases) must be installed. Can be also obtained via

```sh
$ snap install drawio
```

in linux

after instalation if not present, add drawio to path

```shell
$ which drawio
```

add a file under ```docs/bin``` called ```drawio_exec_command.os``` containing the command/path to execute drawio

 - Linux:
    ```echo "drawio"```
 - Windows:
    ```echo "${PATH_TO_DRAW_IO}\drawio.exe"```

repository: sanaljnair/csc-workspace-main

repo_name: sanaljnair/csc-workspace-main


text:
The purpose of this document is to provide a sufficient overview of each element of the data-generator in order to be able to write and execute data-generating circus

# Contents

1. [Overview of circus elements and runtimes](#overview-of-circus)
    1. [Populations and attributes](#population-and-attributes)
    2. [Population relationships](#Population-relationships)
    3. [Data generators](#data-generators)
    4. [Stories, activity, timer and logs](#stories-activity-timer-logs)
    5. [Caveats](#caveats)
2. [Example 1: from hello world to basic CDR generator](#example1)
2. [Example 2: Point of sale simulation](#example2)
2. [Example 3: music streaming simulation](#example3)
2. [Example 4: reusability and circus persistence with DB](#example4)

##   Overview of circus elements and runtimes

The data generator provides an API to design simulation circus and execute them in order to produce simulation logs. Those logs are the generated dataset.

A data-generator circus is called a **circus.** In order to build a circus, we have to create a python programme that assemble instances of the following elements:

*   **populations**, which encapsulate acting entities of the circus (e.g. users, cars, shop, router...)
*   population **attributes**
*   population **relationship**, encapsulating one-to-one or one-to-many relationships between populations
*   **data generators**, either random or deterministic, they are the most atomic building block of generation. Once configured, they can generate an infinite stream of values, e.g. phone numbers, normally distributed numbers, coin flips...
*   **stories**, defined as a sequence of **operations**, they encapsulate some kind of behaviour. They are assembled from the elements above and produce logs when they are executed.
*   a **clock**, which maintains the virtual time during the execution of a circus. Among other things the clock defines the simulation **time step**, which is the most granular increment of time that the circus will be able to manipulate.
    to add the new item to

Each of these element is described in greater details below.

There are two main execution times related to the data-generator:

*   **circus construction time**, which relates to the execution of the python code that you write to define populations and their relationships, configure data generators, define stories, configure the clock... The result of of this execution is a simulation circus, which can optionally be persisted, and is ready to be executed.
*   **circus execution time**, which relates to whatever happens after you launch the run() operation on the circus above. In that phase, the stories you defined start being triggered along the timeline that you defined, producing logs and/or updating the state of the circus as they go.

The data generator also comes with a crude persistence mechanism, simply referred to as the **DB**, that allows to save and load most circus elements above (keep in mind some persistence have not been implemented). This mechanism is optional: you can either write some construction-time code that build a circus and execute it directly, or you have the option to persist it in DB in between.Note that all random generators must be seeded and seeds are persisted in the DB, so the pseudo-generated datasets produced by the generator are repeatable.

![](https://www.lucidchart.com/ac/confluence/placeholder.png?autoSize=1&macroId=c8c493e7-fb0b-497e-eb14-0b5a28c04c60&pageCount=1&instanceId=Confluence%3A6567865422&pages=&width=700&documentId=5fab3d9d-60c4-4e45-a91a-5979ed5cd133&align=left&type=rich&updated=1487229444761&height=500)

##  Populations and attributes

A **population** and its attribute(s) can be pictured as a dataframe where each row represent one **population member** of a certain type. For example, a circus with one million telecom subscribers could be modelled with one population called _subscriber_ having one million rows and some attributes like "name", "MSISDN", "city",...

Each population member always gets automatically assigned a unique id.

I can be misleading at first that, in the context of this generator, one population actually embodies many acting entities(population members).

##  Population relationships

Population relationship can either encapsulate the link between the instances of one or two populations, or the link between the instances of a population(members) and some other entity, not modelled inside the circus.

A relationship can be used to link population member of the same type, e.g. modelling a social network by linking human users together, or of different types, e.g. linking human users to their set of favourite location ids.

A relationship can essentially be pictured as a join table, associating a "from id" to a "to id". The value of the "from id" is always the id of a population member whereas the "to id" is either also the id of a population member, or some other unrelated id. The later case is simply a shortcut that avoid to define populations without attributes. For example, if we need to model the relationship from some population to items like SIM card but do not need to keep any SIM card attributes, we could either create an sim_card population with no attribute and use that as the "to id" of the relationship, but we could also just put the id in the relationship, without creating the population.

##  Data generators

### Generator

Generators are objects that expose a simple generate(size) method which, unsurprisingly, return a list of data point of the specified size. Here are a few of them:

*   ConstantGenerator: generates lists made of one single unique value, defined in the constructor, repeated many times
*   SequencialGenerator, generates lists of unique values using a simple incrementing counter, optionally prepended with a simple prefix
*   NumpyRandomGenerator: generates with a probability distribution family name and parameters, produces lists of random values distributed according that distribution. The generation is actually delegated to [numpy.random.RandomState](https://docs.scipy.org/doc/numpy/reference/generated/numpy.random.RandomState.html), so any family distribution supported by [numpy](http://www.numpy.org/) is available
*   FakerGenerator: similarly to the previous one: this one generates list of values by delegating to some method the [Faker](https://pypi.python.org/pypi/Faker) library, so here again, any [feature available in Faker](https://faker.readthedocs.io/en/latest/providers.html) is available here

### DependentGenerator

All the generators presented above are execution time context agnostic in the sense that they do not need to be aware of any execution time information in order to produce data. There exist cases where generating data based on execution time data is handy. For example in a circus simulating purchases of items between sellers and buyers, one could want to generate a random flag that indicates the sellers becomes worried about low stock where the probability of such flag raise would get higher as the stock get lower. This could be modelled with a random generator of boolean (aka a Bernouilli distribution), whose parameters are computed at execution time based on stock level. We'll see an example of just that in example 2 below.

There are just two DependentGenrator at the moment:

*   DependentTriggerGenerator: generates list of booleans corresponding to the example described above, by first mapping an execution time data into trigger probability
*   DependentBulkGenerator: generates lists of "bulks", i.e. list of list of values. The values themselves are generated by any of the generator above whereas the size of each "bulk" is received at execution time.

##  Stories, activity, timer and logs

**Stories** are included in a circus in order to encapsulate dynamic behaviour, typically for generating logs and/or to update the state of the circus. An story is simply made of a linear sequence of steps, called **operations**, that define what to do. For example, an story to generate a dataset of purchase logs could be made of the following operations: randomly choose a shop, randomly choose an item, lookup its price, update the monetary accounts of both buyers and sellers, and finally output a logs recording the transaction. Story could probably benefit from supporting DAGs of operations in the future, as opposed to simple sequences at the moment.

### Details of story

When building an story, one has to specify the following elements:

*   the initiating population (the buyer, in the above example)
*   activity levels, which defines how frequently an story is executed on average. There is one activity level per population member, so if the story is initiated by a population with 10000 population members, we need to specify 10000 activity levels, which is why the activity levels are specified with a generator.
*   a timer profile: a probability distribution over an arbitrary time period specifying when executions are more likely to occur over that period. Such histogram could for example span a 24h period, thus specifying which moments of the day have higher activity, or a 7 days period, allowing to have different profiles during week days and weekend, or whatever. Such timer is handy to capture the fact that purchase event can only happen during office opening hours, or phone calls are more likely in the evenings except on weekends, ...
*   a sequence of operations

In the example timer profile below, note that you can specify some periods of the day when the action is impossible, simply by setting the probability to zero:

![](https://www.lucidchart.com/ac/confluence/placeholder.png?autoSize=1&macroId=f229abd0-7e98-42d7-d668-e6df684c3343&pageCount=1&instanceId=Confluence%3A6567865422&pages=&width=700&documentId=82364925-0f97-4d5c-ad49-01281543ae37&align=left&type=rich&updated=1487234798023&height=500)

### Story execution mechanism

In order to avoid design mistakes and keep in mind the power and limitations of stories, it help to be aware of their high level execution mechanism:

*   at each clock step (say, every hour of simulation time):
    *   determine which population members will execute the story: this is influenced both by the activity levels and the timer profile. This boil down to selecting some rows in the corresponding population dataframe.
    *   for each population member, execute the sequence of operations:
        *   column-wise: all population members will first execute the first operation 1, then the second,...
        *   concurrently and without coordination: the current operation will be applied for all population members in arbitrary order within one clock step, which means that most state updating operations are subject to **race conditions**. An important exception to this is the one\_to\_one parameter of the relationship.select() operations, which prevents the same "to" side to be returned several times concurrently (e.g. this prevents a shop to sell several times the same item to different customers during one timestep)

This design allows to "vectorize" the execution in the current implementation (and potentially to parallelise it in a later one), at the price of some inconsistencies. This is not as bad as it might sound, for the following reasons:

*   in a lot of situations, only a small subset of population members are involved during any specific clock-step and thus collisions are typically rare.
*   The intend of the simulator is to generate test data and we usually do not care about generating a perfect dataset, but rather to obtain a dataset that has some chosen properties that are similar to a real one: so if monetary account of simulated users evolve according to transactions, with a erroneous glitch here and there, that's usually fine in the context of a test.

If necessary, the following two mitigations approach can reduce the inconsistencies:

*   reduce the size of the clock-step, thus removing the amount of concurrently active population members and thus the probability of collisions. This is likely to slow down the simulation though.
*   use the one\_to\_one parameter of select_one, removing some collisions
*   Add some consistency checks in the story and apply a DropRow operation in case the row contains an inconsistency. Keep in mind that there is no such thing as an acid-rollback here, so any state mutation is not undone by the drop.

### Operations

You can either code an operation from scratch to model to behaviour you need, or more likely just pick some of the ready-made operations part of the data-generator elements (see below). In both case an operation must simply respect the contract of:

*   transforming an **story_data**, which is a simply pandas dataframe whose columns are called **fields**, into an story\_data. There is one row in the story\_data per currently active population member during the current clock step. Typically an operation will add one or several fields and/ drop rows or just return it as-is
*   optionally, each operation can also emit logs, which will be automatically appended to the resulting CSV dataset by the generator. Typically only one operation outputs logs, the last one, although that is not an obligation.

![](https://www.lucidchart.com/ac/confluence/placeholder.png?autoSize=1&macroId=718e0fce-723f-4d74-800e-527d3bd2c33a&pageCount=1&instanceId=Confluence%3A6567865422&pages=&width=700&documentId=ee3a3a0d-a8a5-4a9d-833f-851e02b783af&align=left&type=rich&updated=1485341361266&height=500)

Here is an overview of the available operations, some random, some deterministic, that can be picked from the elements of your circus:

*   Population:
    *   lookup(): adds a fiels to the story_data by copying values from the population's attribute
    *   overwrite(): update the value of population attribues
    *   select\_one(): adds a field in the story\_data, filled with randomly-selected id of that population
    *   update(): add population members to that population
*   Relationship
    *   select\_one(): adds a field to the story\_data with one randomly selected "to" side of the relationship of each population member row currently active
    *   select_many(): same as select_one(), but selects several "to" sides instead of one
    *   select_all()
    *   add(), remove(): update the content of that relationship
*   Clock
    *   timestamp(): adds a field to the story_data with a randomly generated timestamp within the current clock step
*   Generator:
    *   generate(): adds a field to the story_data with randomly generated data
*   Other stories (not attached to a circus element):
    *   DropRows: remove rows from the story_data, based on some boolean condition
    *   Apply: arbitrary computation based on one or several fields and appending one or several fields
    *   Chain: this is just a composite of a sequence of stories. This is handy to be able to re-use part of the execution flow in several stories, as illustrated in example 3 below.
    *   FieldLogger: does not update the story_data, but read fields from it and generate logs with their content
*   Custom story:
    *   You can also sub-class operations.Operation, operations.SideEffectOnly or operations.AddColumns to create your own operations

See the API for the

![](https://www.lucidchart.com/ac/confluence/placeholder.png?autoSize=1&macroId=7f86ec24-e65a-4a0b-ee4a-99fce5b0347c&pageCount=1&instanceId=Confluence%3A6567865422&pages=&width=700&documentId=9744aa18-ad2d-4cd9-a77e-6ba9b6be5faa&align=left&type=rich&updated=1485345021421&height=500)

### More details on activity and timer profiles

This section provides more details on how the activity and timer profiles works behind the scene (this probably can safely be skipped during a first read).

*   Even though activity levels are configured as frequencies, since we figured that would be easier to manipulate during configuration, what is actually used by the engine is the period => what we are actually configuring for each population member is an average duration between two executions. This duration is expressed internally in number of clock steps.
*   At execution time, the circus is maintaining a counter for each population member that specifies how many more clock steps each particular population member must wait until it triggers this specific story. This counter is called the **timer** of each population member for this story. At every clock step all the positive timers are decremented by one and all the population members whose timer is zero are included in the story execution of this clock step.
*   By default, stories are eternally repeating themselves, which is implemented internally by automatically reseting the timers to some positive number after an story execution (more on how this works below).
*   You can de-activate the automatic timer reset during the configuration of an story. If you do that, the counter will reach -1 after an execution and remain to that value, which implies the populations will never be triggered by the clock (since it will never reach zero). You typically want to do that for stories that are executed as a reaction to others and not on a time basis as described here. In order to trigger the execution of an story as a reaction of another story, you can use the force\_act\_next() operation on the story, which essentially just sets the corresponding counters to zero.
*    Another operation you can do on timer is explicitly reseting the timers of some population members of some story. This is done simply by invoking the reset_timer() operation on that story. This is less often useful, thought could be desired if an story involved more than one population (e.g. a voice call) and we decide to model the fact that both A and B should have their timer reset after the story: the automatic timer reset described above would automatically reset the timer of the initiating population A, but you need to specifically invoke that operation for B.

When the timers are reset, the following steps are performed by the engine:

*   it looks up the activity level for that population member and translate it into the corresponding number of average clock steps
*   it generates a random duration with this expected duration
*   if the timer profile is flat, this duration is basically the new value of the timer. If not, we just distort the duration to land with higher probability in the moments that are more likely as defined by the timer profile.

An example might help to clarify this:

*   Assume we use the timer profile as defined by the squares below (more squares piled vertically means this time slot is more likely) and the user has an activity level of 4 times per day, i.e. an average period of  6 hours.
*   Also, assume the current time when we are resetting the timer is 8am, i.e. we are "after" the grey squares
*   We generate a new random duration with mean 6h. Say we draw a 9, which corresponds to 9 squares on that histogram
*   We simply "walk 9 squares along the histogram" and land on the 4pm time slot
*   Since 4pm is 8h later than 8am, we set the new timer counter to 8h, which is 4 clock steps (since the clock step in that example is 2h)

![](https://www.lucidchart.com/ac/confluence/placeholder.png?autoSize=1&macroId=a58d571e-8f1b-4de1-fff6-42d92aa4bf36&pageCount=1&instanceId=Confluence%3A6567865422&pages=&width=700&documentId=e5a5482c-09ac-4fc5-a2b7-43d011c61c08&align=left&type=rich&updated=1487234633579&height=500)

##  Caveats

When using the data-generator, keep in mind the following current caveats:

*   it is currently coded in python/pandas, and therefore runs on a single core in a single host. Since most of the logic is dataframe-oriented, re-coding everything (like 2000 lines of code) in Spark should be doable.
*   it can be hard to debug: since the code you write is executed to assemble stories that are themselves executed, stacktraces can sometimes appear cryptic are may require knowledge of the internals to understand what is going on. Also, it _is_ possible to run an story inside a debugger and watch intermediary states as the operations are executed, but placing breakpoint at the appropriate place can appear tricky at first.
*   it's (currently) not restartable: at the moment the DB persistence mechanism cannot be used after a circus has been started. This means you cannot generate one month of data, save the circus state, and come back later to add a couple of days. There exist a [ticket](https://realimpactanalytics.atlassian.net/browse/OASD-3001) for that feature.
*   it's a young project, lot's of effort has been put abstract away the API from the implementation and execution details, although more clean-up could be added. Once we start writing a lot of circuses of course, this clean-up will be harder.

Example 1: from hello world to basic CDR generator
==================================================

This example tries to walk you through most of the basic concepts of the data generator. An implementation of each step of this example is available on [github](https://github.com/RealImpactAnalytics/lab-data-generator/blob/develop/tests/tutorial/example1.py).

Step 1 - Hello World
--------------------

*   create one Person population of size 1000 without any attributes
*   create one clock with step 1h
*   create one story where each Person emits a "hello world" log once per hour
*   run the simulation to generate 48 hours of logs

Step 2 - Add Person attributes and log timestamp
-------------------------------------------------

*   add a first name attribute baked by a FakerGenerator
*   add a lookup operation to add the person first name as a field in the story_data of the story
*   update the story so it contains a random timestamp

Step 3 - Use a generator to generate durations at runtime
---------------------------------------------------------

So far the simulation is deterministic. Let's make it pseudo-random by generating some fields value at execution time.

*   Create a exponential generator with lambda = 1/60, we're going to use it to generate random durations, measured in seconds
*   Create a set of 1000 site ID and create a generator called locationGenerator that generate site id randomly picked from that set.
*   Update the story with those generators  to add:
    *   a "talkingDuration" field
    *   a "location" field

Step 4 - Adding a relationship to capture mobility
--------------------------------------------------

Let's assume that each user is now only allowed to be in a limited set of locations, which could reflect knowledge of "frequent locations". A simple approach for that would be to store for each user their set of allows locations and then at execution time pick one randomly. This one-to-many relationship can be captured with a relationship.

Let's first generate the set of allowed locations of each user. We'll assume for simplicity that each user has exactly 5 frequent locations.

*   Add a relationship "allowed_locations" to the Person population
*   Initialize it with the Site ID generator so each person is related to exactly 5 sites (don't bother about making the sites unique per user, just call add_relationships 5 times)
*   Update the story so now the location is dynamically sample from the allowed\_locations of the Person, via a select\_one call

Step 5 - Make some locations more likely with a weighted relationship
---------------------------------------------------------------------

*   Update the User-> Location relationship above by adding a weight  so some location are more likely than others. For example 2 locations could both have a high 30% weight (which is reminiscent of "Home" and "work") and the other 3 could share the rest

Step 6 - Add a relationship to capture a social network and use an execution runtime weight
-------------------------------------------------------------------------------------------

In the example above, the weight used in the select_one is carved into the relationship itself. In case the relationship is between populations, we can also use one property of the "to" population after of relationship. Let's illustrate that by updating the example so that a Person talks to another person of chosen within their social network, giving a higher preference to persons having a high popularity.

*   Add a Popularity attribute to each person, initialised thanks to uniform distribution over \[0, 1\]
*   Add a second relationship to the Person population called "Friends". In this case, both "from" and "to" sides of the relationship are population members of a Person, though they could be of different populations as well).
*   Use the make\_random\_bipartite_data method to initialise this relationship (do not set a weight)
*   Update the story so it now logs a "COUNTER\_PART\_NAME" field, which is the called person and is chosen at random among the active population's Friends, using a select_one.
*   Make sure the weight used in this select_one is now based on the Popularity attribute of the called person.

Note that the above is a very simplistic way of generating a social network, more appropriate methods exist in case be need to keep some clustering or other aspects in considerations.

Step 7 - Add price and currency fields with a generic Apply operator
--------------------------------------------------------------------

Finally, as an illustration of ad-hoc computed field, now that our logs look more and more like CDR's, let's compute the billed amount of each log.

*   write a python function that
    *   takes as argument a dataframe with columns locationA, locationB and  duration
    *   returns a dataframe with 2 columns: "price" and "currency". This dataframe must have the same index as the input dataframe (since that's what is used behind the scene to inject the result into the story_data)
*   update the operation with an Apply operation using the method above

Example 2: Point of sale simulation
===================================

This example is inspired from the SND circus. We simulate here customers buying items from point of sales and illustrate slightly more advanced concepts like dependentGenerators, and triggers,.. Also, in this circus we'll have several stories executed by the circus.

An implementation of each step of this example is available on [github](https://github.com/RealImpactAnalytics/lab-data-generator/blob/develop/tests/tutorial/example2.py).

Step 1- Point of sales population and stock level report
----------------------------------------------------

Here we simply initialise the first population + some basic initial logs.

*   Create PointOfSale population of size 100
*   confiture the clock with a step of 1h
*   Add the attributes administrator name, city, and company name (use a FakerGenerator for all that)

To keep things simple, we imagine each point of sale is selling just one kind of item. Each item sold and bought is explicitly modelled and identified by a unique id. We can model the point of sale stock of such items with a simple relationship:

*   Create a "items" relationship on the PointOfSale population. This relationship is currently empty.

Finally, let's add a report story that will inform us on the stock levels at the end of each day:

*   Create a new story "reportStockLevel" that triggers exactly once per day for each point of sale. This can be achieved with a ConstantDependentGenerator.
*   Add an operation to generate a timestamp field. Since we want the report at regular interval, set "random" to false so the timestamp is at exactly the same time every day. Also, you can change the log_format if required
*   Add an operation to add a "stock\_level" field: use the get\_neighbourhood_size operation of the relationship for that
*   Add a Log operation to the story to actually produce logs. You can specify the fields and order if you want, or just use default.

Execute the simulation and check everything looks ok: all reported stock sizes should be zero.

Step 2 - Initialise the stock of items
---------------------------------------

*   Create a Sequencial ItemId generator that we are going to use to generate the ids of items
*   Attach it to the circus: we'll need this same generator later on for the restock story
*   Generate a list of 100 arrays of 5 items ids with this generator inside a simple for loop.
*   Use the add_grouped operation on the "items" relationship of the PointOfSale so each population receive one list of 5 unique initial items

Execute the simulation and check everything looks ok: now all reported stock sizes should be always be 5.

Step 3 - Add an initial dumb re-stock story with a DependentGenerator
----------------------------------------------------------------------

So far we've only seen stories whose primary purpose is to generate logs. Stories can also be used to update the state of the circus, as illustrated below.

At the moment all the point of sales's stock are eternally 5. Let's create a basic "restock" story to let the point of sale population regularly buy new items from their provider. This impacts the state of the circus since it will populate new members to the "items" relationship. In order to keep things super simple, we're not modelling the distributors here: any time a point of sale is restocking, it's actually just generating new items out of the blue and adding them to its stock (and then makes Mexico pay for it).

This is not very realistic though: later on in this tutorial, we'll show how to trigger the restock story when the actual stock is getting low instead of on a timer-basis as described here.

*   Create a "restock" story initiated by the PointOfSale population. Associate it with a default daily profile and an activity level of one trigger per week. You can use a ConstantGenerator for the activity, so that each POS has the same activity level, and use the activity() method of the timer to obtain the activity level corresponding to once per week.
*   Add a lookup operation to include the administrator name in the story_data of the restock story

In order to do a restock, we'll need 2 generators, one to generate the number of items added to the stock by the story, and another to generate the ids for each of those items. Let's start with the first generator:

*   Create a stock size generator respecting the following empirical distribution (you can use a NumpyGenerator with choice method for that):
    *   5: 10%
    *   15: 20%
    *   20: 50%
    *   25: 20%
*   Add an operation to the restock story that is generating the field "restock_volume", using the generator above

So far so good. Creating a generator of item ids is easy, just re-use a sequencial generator we built before

*   retrieve the sequencial item id generator from the circus
In order to put them together, we first wrap the ItemId generator inside a DependentBulkGenerator, so that it's able to generator lists (bulks) of ids for each point of sale:

*   Wrap the ItemId inside a  DependentBulkGenerator so it now generates lists (i.e. bulks) of itemIds based on some input sizes that will be received at runtime (i.e. read from the current story_data dataframe of the story).

*   Add an operation to the "new\_items\_ids" field using this generator, specifying that the observed field is the restock_size field generated above.

Note that the description above illustrate the nominal usage of a DependentGenerator, in which the generator is bound to the some observed field of the story_data that will only be available at simulation time. In this specific case, since that field is also coming from a generator (as opposed to an population attribute or anything else), a shortcut could have been to compose the 2 generators with the flatmap operator to obtain directly one generator of list of item ids. In the suggested solution of this exercise, this is illustrated in the step 3bis).

The last thing we need to do is to add those new item\_ids to the "items" relationship of the point of sale. Example 1 showed how to add\_relationships to a relationship statically, during circus construction time. We can also do that dynamically as part of an story, which we are going to do here. Since we are adding groups of items ids to each point of sale (as opposed to just one new relationship at a time), we do this with the add_grouped operation:

*   add an operation to the restock story to add the items contained in the "new\_items\_ids" field to the stock of the corresponding point of sale.
*   Add a Log operation to keep track of each re-stock, explicitly specify the list of logged fields and do not specify the "new\_items\_ids", so that the logs do not get cluttered by huge list of ids

Step 4 - Customer populations: modelling a purchase story with select_one(pop=True)
-------------------------------------------------------------------------------

Now that we have a story for the point of sale to actually get some stock, let's add one story for selling stuff to customers

*   Create a Customer population of size 2500, add them a first and last name
*   Create a relationship called "my_items" on the Customer population. This relationship is current empty.
*   Create a purchase story initiated by the customer, in which each customer tend to buy one item per day, exclusively during office hours:
    *   In order to force stories to happen only during office hours, set the timer to WorkHoursTimerGenerator: this is a predefined timer that sets the probability of a story to 0 for hours outside of office hours and to some constant during office hours.
    *   In order to have each customer to perform on average one story per day, simply fill in the activity levels with values corresponding to one occurence per day. The hard (and unnecessary) way to do it requires to inspect the timer: since the activity levels are specified in number of occurrences per duration of the timer period, so we can look at WorkHoursTimerGenerator and see it has a period of one week and thus set the activity levels to 7. The easier to the same thing is to use the activity() helper method of the timers, which provides the correct activity level corresponding to a desired frequency.
    *   Once you know the activity level, you can plug it into the story thanks to a constant generator (which would imply that each customer would have the same activity level) or use any random generator whose expected value is set to this activity level (which would imply that some customer are more active than others, on average)

*   Add a lookup operation to the purchase include the customer first and last name as fields in the story_data
*   Add an operation to this story to randomly select one point of sale and add it to the story\_data as "point\_of\_sale" field. Use the population.select\_one()  for that purpose
*   Add another lookup operation to include the point of sale city and company name as fields in the story_data
*   Add a FieldLogger operation to log all the fields mentioned above

Note that the logic above allows any customer to buy from any point of sale across the domain. If investigating geographic metrics were relevant to our use case, we could also explicitly model the movements of customers and select a point of sale according to the site where they are at the moment of the item purchase.

Let's simulate the purchase transaction itself:

*   Use the select_one operation of the "items" relationship of the point of sales to select the item that is sold to each customer. We need to specify a few parameters here:
    *   one\_to\_one=True, meaning that even if 2 customers are buying the same shop, each one must be assigned a item from the relationship
    *   discard\_empty=True: if the shop is out of stock, the row should be evicted from the story\_data, essentially cancelling the buy story for that customer. Note that another approach here would be to retry to execute the story at the next clock step for the population member that failed to buy some item at that moment. You can achieve that by setting discard\_empty=False, in which case some rows might have empty values instead of purchased items in case the POS had nothing more to sell. You can then invoke explicitly a force\_act_next() of the same story for those populations, as described above, so the buyer tries again an maybe succeed, maybe because they pick a different POS or because that POS will have restocked some items.
    *   pop=True: when an item has been selected, it is automatically removed from the relationship of the point of sale
*   Use the "add" operation of the Customer "my_items" relationship to add the new item to their list of owned stuff
*   Add a timestamp, and log the transaction

Note: there is probably a bug here: if 10 customers buy from a shop with stock level 5, I'm not sure what will happen...

Step 5 - Refactor the restock story to trigger re-stock when stock is low with force\_act\_next()
--------------------------------------------------------------------------------------------------

As stated earlier, a caveat of the current simulation is that the re-stock stories are happening on a time-basis. In order to illustrate how to randomly or deterministically trigger an story from another story, let's refactor the circus so the customer purchase story now triggers the point of sale trigger story in case stock gets log.

*   First remove the timer gen from the re_stock story => this imply the story is currently not happening anymore
*   Add two  get\_neighbourhood\_size operations to the story: one before and one after updating the stock

The logic we'll put in place is that at the end of the purchase story, which is triggered by the customer population, we'll check the stock level of the point of sale population and, if "too low", we'll explicitly trigger the restock story of the point of sale.

*   Add an the get\_neighbourhood\_size operation to the purchase story to retrieve the stock level of the point of sale and include it as "pos\_stock\_level" field

We want to generate a "should\_restock" boolean field. One deterministic way to do this would be to write a predicate that returns true if "pos\_stock\_level" is below a certain threshold and wrap it inside an Apply operation, as already illustrated in example 1. In an attempt to illustrate something different, lets' use here a more probabilistic approach: let's convert the stock level into a probability to restock.  This is done below with a bounded\_sigmoid. We can then randomly generate boolean triggers based on those probability, which is done with a DependentTriggerGenerator.

*   Use the bounded_sigmoid factory to create a decrementing S-shape function that is at its maximum when stock level is at 2 and is at 0 at 10.
*   Use this sigmoid inside the DependentTriggerGenerator to include a "should\_restock" field in the purchase story story\_data

Now all we need is to force the execution of the restock_story when that boolean is true

*   use the force\_act\_next() operation on the restock\_story to triggered it conditioned by the story\_data field "should_restock".

Example 3: music streaming simulation
=====================================

We'll build a simulation of music listening events. This will allow us to explore more advanced concepts like re-usable set of operation stories in a Chain and using stories during the circus initialisation.

Step 1 - Create the music_repo population as a repository of song playlist
----------------------------------------------------------------------

We're going to pretend that each song belongs to only one genre. Let's create a population whose sole purpose is to contain the list of songs of each genre:

*   Create a music_repository population with size = 5 : one for each genre
*   Add a "genre_name" attribute with the values \["blues", "jazz", "electro", "pop", "rock" \]
*   Add an empty "songs" relationship to this population
*   Create a songId sequencial generator,
*   Loop over the 5 genre values and inside that loop, first generate 1000 song\_ids, with the generator above and add them with an add\_grouped() call to the "songs" relationship

Step 2 - Listener population and music_listen event
----------------------------------------------

This is a simple population with one story to generate song_listen events

*   Create a user population of size 1000, with a first and last name
*   Create a listen_events story where
    *   users only listen to songs during work hours
    *   the number of songs a user will listen to per day should be a normal distribution around 20 per day where we cut out the extremes : nobody should listen to less than 10 songs per day, and nobody should listen to more than 30 songs per day.
*   Add an operation to add the user name and last name
*   Add an operation to that story to randomly select a genre among the genres of the music_repository population
*   add a select_one() operation to select the song that is listened to
*   Add a FieldLogger to produce the log of all that

Note that the timer model here is a bit naive in the sense that music listen events time pattern are likely to be more bursty in reality than what is described above, i.e. instead of spreading the listen_event of one user over the whole day, we should create burst of event. The "state" of the population allow to model that, although that is not explored here.

Step 3 - Operation re-use with a Chain for a music_share event
--------------------------------------------------------------

*   Create another story call song_share event: modelling the fact that a song is shared by a user to another

*   Add an operation to select randomly the user the song will be shared to. Ideally we'd model a social network here, though for the sake of simplicity, just pick a random user

At this point we'd need to select a genre and then a song of that genre. Instead of copy-pasting the logic of the music_listen event, let's make it a re-usable component instead:

*   Move the 3 statements for the user name lookup + select genre + select one steps of the music event out of the story itself and inside a chain object
*   Place this chain inside the music_listen event to keep the logic as before
*   Also place it in the music_share event

Step 4 - Initialising the song details with an story Chains at build time
--------------------------------------------------------------------------

So far we've seen stories that produce logs as well as story that update the circus (or do both) at simulation time, i.e. when the simulator is running. We can also re-use all the data-generating logic of the stories before starting the circus, during its initialisation. Let's illustrating that by replacing the mere song_id we're currently using with full song details.

*   First create a Songs population with size 0
*   create the "artist\_name", "song\_genre", "title", "recording_year" and "duration" attributes
*   Keep the "songs" relationship of the music_repo population, but remove its current initialisation

We're simply going to fill-in everything we need inside the story_data of an story outside the circus, run it explicitly at initialisation time and update the "Songs" population

*   Create a python "genre" variable, set to "blues" for now
*   Create an empty Chain object called init_attribute
*   Create an artist name generator and use it as first operation of the Chain, generating the "artist_name" field
*   Repeat the last step to generate the "title", "duration" and "publication_year"
*   Use a ConstantGenerator to generate the "song_genre" field, using the value of the "genre" variable above

Since this Chain operation has been created outside the circus, as defined above it will never execute. A Chain (as any operation used in a story) is actually a python callable, so we can just invoke it on some initial story_data to generate songs. Let's do just that 5 times, once for each genre, and use the result to populate the content of the population:

*   Create a for-loop that loop over each value of the "genre" attribute of the music repository population
*   Get rid of the python "genre" variable above and move the Chain declaration inside the genre for-loop. This now iteratively declares 5 Chain, each bound to one genre, each still doing nothing so far
*   Still inside the loop, using the Story.init\story\_data constructor, build an initial story_data containing one row for each of the songs generated
*   Simply invoke the chain with this story_data, the first return value should the the generated songs
*   Use the update() method of the Songs population to add all the rows to the population (maybe some column name/removal will be required here..)
*   Also, populate the "songs" relationship for that genre in the music repository population. You can simply use add\_grouped\_relations for that.

You can also update the listen and share stories so that now we report the details of the listened and shared songs, as opposed to just the id as before.

Example 4: reusability and circus persistence with DB
=====================================================

The data generator has a very crude (CSV file based) but existing persistence functionality, which can be very handy in two situations:

1.  Meta-data reusability: some circus elements, e.g. complex random generator or population containing rich meta-data, could be re-used in several circus. We're going to illustrate this by saving the song repository in the "DB", completely un-related to the rest of that circus and showing how to load it inside the circus. It's worth noting that this mechanism also allows to load into a circus persisted data that has not been randomly generated. For example, we could imagine that we download a huge list of songs somewhere and feed them as a re-usable piece of data. Moreover, the same is true for probability distribution: it is easy to infer an empirical distribution from a client's data set, save it in the DB as a random generator and sample data from it at runtime, potentially adding some realism to a random simulation.
2.  Circus creation involving populations of large size can potentially be very time consuming (hours is not unheard of). Having the possibility to save the whole circus state, together with all its populations, attributes, relationships and generators, allows to place an important milestone between the circus creation and circus execution. A simple extension of that mechanism would allow to stop and start the circus during its execution, although that is not currently implemented.

This example is a continuation of example3.

Step 1 - Move the song repository  the DB
-----------------------------------------

*   Move the logic for the song a well as music repository creation to separate methods
*   run those methods and use the DB API to save the 2 populations with ids "music\_repository" and "songs", within a namespace called "tutorial\_example4"
*   Update Circus initialisation of example 3 to load the "musi_repository" and "songs" populations from DB instead of creating them

Step 2 - Save the circus before running it
------------------------------------------

*   At the end of the whole example 3 circus creation, use the DB API to save the whole circus to persistence. Note that stories cannot be saved in the DB, rather, they are saved "in code", err, like most dynamic behaviour I guess..
*   load the circus from persistence
*   attach the listen and share stories to it
*   execute

repository: nsutcliffe/trumania

repo_name: nsutcliffe/trumania


text:
# Delivery process

The cloud.gov team practices Scrumban, which means we practice [Kanban](http://blog.crisp.se/2009/06/26/henrikkniberg/1246053060000) around cardwall-style boards that track work from left-to-right as it is identified, prioritized, explored, delivered, and demonstrated. We augment Kanban with a subset of Scrum activities:

- a rapid (15 minutes vidconf) or asynchronous (in Slack) daily standup per theme to communicate about the status of work on the board
- bi-weekly sprint demos for stakeholders and colleagues of each theme's recent work
- bi-weekly retrospectives for each theme
- occasional higher-level planning and grooming sessions

## Kanban process

Kanban basically says cards go through a set of states, but it doesn't say much about what the cards represent or how the states are defined. Here's what they mean to us.

### Cards

Cards on our boards capture either a Feature or Story. Each type represents work to be done at a particular level of detail.

- **Features** are user-notable, program-sequenced, roadmap-worthy changes in product capabilities tracked against cadence-driven periodic milestones... big stuff that's worth planning and announcing on a regular basis. Features are fairly general, referring to stories which break down the actual changes in product behavior that we're after to get the intended benefits of the feature. Generally we do coarse analysis and UX work at the Feature level.
- **Stories** represent tactical increments of individually-valuable work deliverable by a team within a single iteration... often a single change in functionality aimed at achieving a goal for a particular kind of stakeholder, whether customer, user, or operator/admin.

We track both higher-level features and their constituent stories on the same boards.

### Column sequence
The sequence of columns goes left-to-right, but a different subset is used depending on the card type.

**Features** progress through these columns:

- Features (while implementation is in progress)
- Demo (when they are ready for widespread demonstration/release/announcement)
- Closed (when they are released and announced)

**Stories** progress through these columns:

- New (where reported issues, PRs, and newly-identified stories appear)
 - Icebox (when under consideration but not immediately scheduled)
- Backlog (when they are sequenced against stories prioritized for attention)
- Grooming (when they're being refined for implementation)
- Ready (when they're in a shovel-worthy state, just waiting for team capacity to do the work)
- In Progress (when someone is actively working on the issue)
- Awaiting Acceptance (when work is considered complete and awaiting review/merging)
- Demo (when work warrants demonstration to stakeholders and is awaiting the next sprint review)
- Closed (when work is demoed and released, and is no longer worth looking at)

### WIP Limits

If you see a number in brackets after a column name, that's a work-in-progress (WIP) limit. That means we expect to see no more than that number of cards in the column at a time. If you want to move a card right into a column that's already at its WIP limit, first figure out how to help move one of the existing cards out to the right! If the next column is also at its WIP limit, that's your cue to keep working your way over to the right, asking "how can I help?" when you encounter something being worked on for which there's space to move.

In general we use WIP limits to gate how much time and effort we put into grooming stories when we haven't got capacity to do them yet, and to gate how many things we're trying to do at once so that we focus on finishing existing work rather than starting new work. We don't want individuals to be responsible for more than one or two cards in progress at a time, and we like people to collaborate and pair up to get work done, so our WIP limit for cards actually getting development attention is set intentionally low to encourage this!

## Definition of "Done"

An agile "Definition of Done" (DoD) captures the team's agreed-upon standards for how we get work done at a consistent level of quality. Having a DoD ensures that non-functional requirements (NFRs) don't have to be re-litigated for every piece of work taken on, cards can be focused on just the relevant details, and new team members aren't surprised by assumed expectations of their colleagues.

### Column exit criteria
For cloud.gov, our DoD is broken up into [a set of statements that should be true for each card](https://github.com/18F/cg-product/blob/master/DeliveryProcess.md) before it moves to the next column on the board.

Before advancing a card from one column to the next on the board, it should meet the "exit criteria" for the current column, which are listed below.  The exit criteria for a column varies a bit when the column is used for both Features and Stories.

#### New

- Relevant points from any discussion in the comments is captured in the initial post
- Decision is made to move to the Backlog or Icebox columns, or close

#### Backlog

##### Features

- Describe what we're after, what benefits it's expected to deliver, and any relevant design/research artifacts. That's it! Any more detail than that should appear in stories.

##### Stories

- Indicate the intended benefit and who it's for in one of these forms:
 - # Story: We prefer the ["In order"](http://blog.crisp.se/2014/09/25/david-evans/as-a-i-want-so-that-considered-harmful) form (rather than the common "As a/I want/so that" form).
 - # Hypothesis: What's the lean hypothesis being tested, and how it will be validated?

#### Grooming

- Has testable/demoable Acceptance Criteria that we expect to be able to check off to help us understand when the work is done. (Try wording them as [Gherkin](https://en.wikipedia.org/wiki/Behavior-driven_development#Behavioural_specifications), and use [GFM checklists](https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments) for them)
- Discussed by the team and implementation sketched (use more checklists here)
- The scope of the work is easily doable in a few days (else split it!)
- Any authn/authz and data persistence points are discussed and requirements are addressed via Acceptance Criteria (eg "Authenticate using [18F GitHub org|login.cloud.gov]" or "Any ephemeral data resulting from usage is backed up/recoverable")
- There's a communication plan for any user-visible changes which require their attention/action

#### Ready

- No info or assistance is needed from outside the team to start work and likely finish it
- There's capacity available to work on the story (eg this column is a buffer of shovel-ready work)

#### In Progress

- Acceptance criteria are demonstrably met
- Relevant tasks complete, irrelevant checklists removed or captured on a new story
- Follows documented coding conventions
- Pair-programmed or peer-reviewed (eg use pull-requests!)
- Test coverage exists and overall coverage hasn't been reduced
- User-facing and internal operation docs have been updated
- Demoable to other people in their own time (eg staging environment, published branch)
- Any deployment is repeatable (eg at least documented to increase bus-factor beyond one) and if possible automated via CI/CD.
 - If the deployment is difficult to automate, then a story for making it automated is created at the top of `New Issues`.
- Deployment happens in the AWS GovCloud deployment (not just AWS East/West)
- [Proposed] Appropriate alerting for new stuff is set up and reporting to Riemann

#### Awaiting Acceptance

- A team-local proxy for the people the story affects reviews and approves the work as meeting acceptance criteria
- If the work has a visual aspect, post a screenshot attached for later documentation/announcement/demo purposes
- If the work as completed is obviously unscalable and will cause problems if we try, then a story for making it scalable is created at the top of `New Issues`.

#### Demo

- The card is [demonstrated to|not worth the collected attention of] colleagues and stakeholders.

#### Closed

- The work is user-visible and announceable at any time.

## Parking Lot (potential future additions)

- Minimizing user disruption
 - How do we ensure system stability for relying teams?
 - How do we avoid overhead in retraining users due to incremental release?
 - What conditions might trigger bunching stories for simultaneous release, and how would our criteria capture that status?

repository: nikzei/cg-product

repo_name: nikzei/cg-product


text:
---
layout: post
title: Session in Spring
bigimg: /img/image-header/yourself.jpeg
tags: [Spring]
---

## Table of contents
- [Given problem](#given-problem)
- [Introduction to Session in Spring](#introduction-to-session-in-spring)
- [Spring Session architecture](#spring-session-architecture)
- [Wrapping up](#wrapping-up)

## Given problem

## Introduction to Session in Spring

## Spring Session architecture

- Create a new Servlet filter.
- Add the filter to the filter chain of your servlet
- Connect the filter to the Redis connection (or an other [MapSessionRepository](https://docs.spring.io/spring-session/docs/1.0.1.RELEASE/reference/html5/#api-mapsessionrepository) backed by Hazelcast, GemFire, Coherence or any other data grid that can give us a Map reference)

## Wrapping up

Refer:

**Problems with Session**

[https://www.javacodegeeks.com/2012/07/rest-spring-security-session-problem.html](https://www.javacodegeeks.com/2012/07/rest-spring-security-session-problem.html)

[https://content.pivotal.io/blog/why-is-session-state-management-so-hard-with-spring-session-it-doesnt-have-to-be](https://content.pivotal.io/blog/why-is-session-state-management-so-hard-with-spring-session-it-doesnt-have-to-be)

[https://owasp.org/www-project-cheat-sheets/cheatsheets/Session_Management_Cheat_Sheet](https://owasp.org/www-project-cheat-sheets/cheatsheets/Session_Management_Cheat_Sheet)


text:
## 2.2.0

- Allows a reaction to be fired even if the value hasn't changed by [@amondnet](https://github.com/amondnet) in [#907](https://github.com/mobxjs/mobx.dart/pull/907)
- Adds custom `equals` for creating observables. [@amondnet](https://github.com/amondnet) in [#907](https://github.com/mobxjs/mobx.dart/pull/907)

## 2.1.4

- Allow users to bypass observability system for performance by [@fzyzcjy](https://github.com/fzyzcjy) in [#844](https://github.com/mobxjs/mobx.dart/pull/844)
- Fix Reaction lacks toString, so cannot see which reaction causes the error by [@fzyzcjy](https://github.com/fzyzcjy) in [#844](https://github.com/mobxjs/mobx.dart/pull/844)
- Add StackTrace to reactions in debug mode to easily spot which reaction it is by [@fzyzcjy](https://github.com/fzyzcjy) in [#844](https://github.com/mobxjs/mobx.dart/pull/844)

  Breaking changes:

- Avoid unnecessary observable notifications of @observable fields of Stores by [@fzyzcjy](https://github.com/fzyzcjy) in [#844](https://github.com/mobxjs/mobx.dart/pull/844)

## 2.1.2 - 2.1.3+1

- Fix tests in dart 2.19 - [@amondnet](https://github.com/amondnet)
- Dart formatting fixes
- Refactor `atom.dart` to make code cleaner by [@fzyzcjy](https://github.com/fzyzcjy) in [#854](https://github.com/mobxjs/mobx.dart/pull/854)

## 2.1.1

- Allow a custom equals parameter for ObservableStream - [@amondnet](https://github.com/amondnet)

## 2.1.0

- `ObservableSet` now uses `Set` to maintain order of iteration
- `ObservableSet.linkedHashSetFrom()` removed. Use `ObservableSet()` or `ObservableSet.of()` instead.

## 2.0.7+7

- Moved the version into its own file (`version.dart`) and exported from the main library file

## 2.0.7+6

- Fixed issue in showing issue tracker link on pub.dev

## 2.0.7+5

- revert #784 - [@amondnet](https://github.com/amondnet)

## 2.0.7+4

fixes:

- shortened `1.asObservable()` to `1.obs()` (same for boolean, double, String) - [@subzero911](https://github.com/subzero911)
- removed experimental typedefs from `2.0.7`

## 2.0.7+3

- Fixed unnecessary ActionSpyEvents being triggered for AsyncAction

## 2.0.7+1 - 2.0.7+2

- Package upgrades

## 2.0.7

- Type aliases for primitive types.\
  So instead of `Observable<int>` you can now write `ObservableInt`.
- `.asObservable()` extension for primitive types.\
  Now you can easily convert literals to observables like:
  ```dart
  var name = ''.asObservable(); // infers ObservableString
  var counter = 0.asObservable(); // infers ObservableInt
  ```
- `toggle()` method for `ObservableBool`. Lets you toggle the internal value of `ObservableBool`
  ```dart
  var lights = true.asObservable();
  lights.toggle(); // now it has a value of false
  ```
  Changes made by [@subzero911](https://github.com/subzero911)
- Allow use custom context(#770) - @amondnet

## 2.0.6 - 2.0.6+1

- Improved performance when spying is disabled. Thanks to @Ascenio.
- Package upgrades
- Adopting the recommended linting for Dart

## 2.0.5

- Fix for `ObservableStream.listen()` should also keep observable values updated (#708) - @brianrobles204
- Added missing parameter to `asyncWhen` - @easeccy

## 2.0.4

- Moved `analyzer` package to `2.0.0` - @davidmartos96
- Fixed a bug in `Computed` that was not propagating values after an exception - @brianrobles204
- Several documentations typo fixes - @sno2, @Ascenio

## 2.0.2 - 2.0.3

- Fixes #681: ObservableStream.value throws Exception when data is null
- Removes `mockito` as a dependency and using `mocktail` instead.

## 2.0.1 - 2.0.1+1

- Adds `@readonly` annotation support for creating read-only observables.

## 2.0.0

- Full support for Null Safety

## 2.0.0-nullsafety.6

- Adding support for `@readonly` annotation, as per this [issue](https://github.com/mobxjs/mobx.dart/issues/220)

## 2.0.0-nullsafety.0 - 2.0.0-nullsafety.5

- Null safe migration
- Formatting fixes to improve Pub Points

## 1.2.1+2 - 1.2.1+4

- Reformatting for improving the pub.dev score
- Regenerated the example files

## 1.2.1 - 1.2.1+1

- Improved `observe()` API for `ObservableList`. We now get better change events with accurate details about what has changed. Thanks to
  @darkstarx for the PR (#490)
- API docs update for reaction. Rewording the _debouncing_ to _throttle_, since the behavior is more like throttling.
- Syncing versions with pubspec.yaml

## 1.2.0

- Added the ability to **spy** on changes happening inside MobX. You can now setup a `spy()` to see all these changes. See [this](https://github.com/mobxjs/mobx.dart/blob/5095e966fe591d223c7730579de2f4778a7ff465/mobx_examples/lib/main.dart#L26) for an example.
- Simplified the handling of tests and coverage. We are no longer dependent on the `test_coverage` package, which was causing issues on CI.
- A custom `EqualityComparer<T>` can now be specified for `Computed<T>`. This can be passed in with the `equals` parameter in the constructor.
- Several documentation updates from various contributors.

## 1.1.0 - 1.1.1

- All exceptions caught inside MobX reactions are now reported as `MobXCaughtException`. Previously they were reported as-is and that caused issues in `flutter_mobx`, where the stack traces were not visible. The stack trace is now being captured as part of the exception, so `flutter_mobx` can show the complete trace. This helps a lot during debugging.
- `MobXException` now extends from `Error` to capture the stack trace. This is done because Dart only captures stack traces automatically for `Error` instances.
- Bringing test coverage back to 98% in `1.1.1`

## 1.0.2 - 1.0.2+2

- Added `@StoreConfig` annotation ([@hawkbee1](https://github.com/hawkbee1))
- Fixed to link to pt-BR translation on github
- Fixed anchor link inside README.md

## 1.0.1

- Fix for ObservableMap not adding null values (#417), thanks to @Vardiak

## 1.0.0

- Ready for prime time!

## 0.4.0+3 - 0.4.0+4

- Going back to original `test_coverage` package
- Some cleanups in the tests

## 0.4.0+2

- README updates
- Switching to [Github Actions](https://github.com/mobxjs/mobx.dart/actions) for all builds and publishing

## 0.4.0 - 0.4.0+1

- Removing the deprecated `authors` field from `pubspec.yaml`
- Added extension methods to create an Observable{List,Set,Map,Future,Stream} from a regular List, Set, Map, Future, Stream. Thanks
  to the work done by [Jacob Moura](https://github.com/jacobaraujo7)
- `README.md` translated to Portuguese, thanks to [Jacob Moura](https://github.com/jacobaraujo7)

## 0.3.10

- Removed `Store.dispose`
- Fixed some analyzer errors related to `unused_element`

## 0.3.9+3

- Documentation comments for many of the public methods and classes
- Package updates

## 0.3.9 - 0.3.9+1

- Added `toString` override for `MobXException`
- Added an interface `ObservableValue` that is a common interface for all observables: `Observable`, `Computed`, `ObservableFuture`, `ObservableStream`. Thanks to [@t-artikov](https://github.com/t-artikov)
- Added chat link to Discord. It is now: [![Join the chat at https://discord.gg/dNHY52k](https://img.shields.io/badge/Chat-on%20Discord-lightgrey?style=flat&logo=discord)](https://discord.gg/dNHY52k)

## 0.3.7 - 0.3.8+1

- Fixes the type resolution bug that prevented using types from packages like `dart:ui`
- Fixes the type resolution of other public `Store` classes referenced in the `@store` based generation

Thanks to [@shyndman](https://github.com/shyndman) for the tremendous work on this release.

- Added a version constant that matches the `pubspec.yaml`

## 0.3.6

- Added new way to create `Store` classes using the `@store` annotation. This will exist as an alternative to the mixin based approach we already have.
- Includes a new option for `reaction` for using a custom `EqualityComparator<T, T>`. This is useful when you want to avoid expensive reactions by plugging in a custom comparison function for the _previous_ and _next_ values of the predicate.

## 0.3.5 - 0.3.5+1

- Fixed a bug where the `ObservableFuture<T>` would not show the correct status. This was happening because of the lazy evaluation strategy. We are now being eager in creating the status and monitoring the inner `Future<T>` immediately.
- Upgraded `test_coverage`

## 0.3.3+1 - 0.3.4

- Removed the `@experimental` annotations for `Observable{Future,Stream}` and `reaction`.
- Removed the dependency on the `meta` package.
- Some formatting changes
- Reporting update or change only when the new value is different than the old value. This is mostly for observable collections like list, set, map.

## 0.3.3

- Wrapping all collection setters in conditional action wrappers. This removes the need to wrap collection mutating methods in explicit actions.

## 0.3.0 - 0.3.2+3

- API changes introduced to the `enforceActions` setting of `ReactiveConfig`. It is now called `writePolicy` and the enum `EnforceActions` has been renamed to `ReactiveWritePolicy`.
- Also introducing a `readPolicy` setting on `ReactiveConfig`. It is an enumeration with two values:
- Removing the "strict-mode" text in the exception message when the `ReactiveWritePolicy` is violated. This was a vestige from the `mobx.js` world.
- Exposing a boolean `isWithinBatch` on `ReactiveContext`, which tells if the current code is running inside a batch. This is used to conditionally apply an action-wrapper for `mobx_codegen`-generated setters.
- Introduced an action-wrapper called `conditionallyRunInAction()` that runs the given function in an action only when outside a batch.
- Increasing test coverage

```dart
enum ReactiveReadPolicy { always, never }
```

## 0.2.1+2

- Improving test coverage

## 0.2.1+1

- README updates

## 0.2.1

- An internal change was made to support the new restrictions around [non-covariant type variables in superinterfaces](https://github.com/dart-lang/language/blob/master/accepted/future-releases/contravariant-superinterface-2018/feature-specification.md). This was causing compile errors in the Flutter `beta`/`dev`/`master` channels.

## 0.2.0

- A breaking change has been introduced to the use of the `Store` type. Previously it was meant to be used as an _interface_, which has now changed to a **mixin**. Instead of doing:

```dart
abstract class UserBase implements Store {}
```

You now do:

```dart
abstract class UserBase with Store {}
```

This allows us to add more convenience methods to the `Store` mixin without causing any breaking change in the future. With the current use of the interface, this was not possible and was limiting the purpose. `Store` was just a marker interface without any core functionality. With a **mixin**, it opens up some flexibility in adding more functionality later.

- All the docs and example code have been updated to the use of the `Store` mixin.

gha_language: Dart


text:
# Postgres SPIRE 1.5.1 deployment

This configuration is an example of a SPIRE deployment for Kubernetes using
Postgres as a datastore for the SPIRE server. This configuration provides
better resiliency and allows for scaling up the number of SPIRE servers.

+ [postgres](spire-database.yaml) runs as a StatefulSet using a
  PersistentVolume.
+ The SPIRE [server](spire-server.yaml) runs as a stateless Deployment.
+ The SPIRE agent runs as a DaemonSet - note this configuration is a symlink
  to the [simple sat example](../simple_sat/spire-agent.yaml).

Both SPIRE agent and server, along with postgres, run in the **spire**
namespace, using service accounts of **spire-database**, **spire-server**, and
**spire-agent**.

Compare the [simple sat server](../simple_sat/spire-server.yaml) configuration with
this [postgres backed server](spire-server.yaml) to see the differences, which
consist of: a Deployment instead of a StatefulSet, a datastore plugin change,
an InitContainer that waits for postgres to be up, and removal of the
PersistentVolumeClaim.

The SPIRE server can be deployed either stateless or stateful, however in
either case it's imperative to configure the server appropriately to handle
failures and scalability.

+ **stateless** - To run the SPIRE server stateless (as in this example), the
  `UpstreamAuthority` plugin needs to be used.
+ **stateful** - To run the SPIRE server stateful, the directory specified in
  `data_dir` must be persistent (such as in the [simple sat example](../simple_sat)
  where a StatefulSet and PersistentVolumeClaim are used.

In this example deployment, the SPIRE server is stateless, using the example
[dummy upstream authority](https://github.com/spiffe/spire/tree/v1.5.1/conf/server).

One other **important note**: In a production environment it is very important
to use a highly available Postgres configuration, unlike this configuration
where there is only a single Postgres instance. Doing so is outside the scope
of this example - there are a number of ways to achieve this.

## Usage

### Configuration

+ Set trust_domain and the cluster name for the k8s NodeAttestor.
+ Modify the path in the *k8s-sa-cert* volume for SPIRE server as appropriate
  for your deployment - this is the certificate used to verify service accounts
  in the cluster. This example assumes minikube.
+ Replace `MYSECRET` in both **spire-database.yaml** and **spire-server.yaml**.
  This is the password for the Postgres account used by the SPIRE server's
  datastore.

### Deployment

Start the postgres StatefulSet:

```
$ kubectl apply -f spire-database.yaml
```

Start the server Deployment:

```
$ kubectl apply -f spire-server.yaml
```

Start the agent DaemonSet:

```
$ kubectl apply -f spire-agent.yaml
```

repository: spiffe/spire-examples

repo_name: spiffe/spire-examples


text:
---
page_type: sample
languages:
- json
products:
  - azure
  - azure-hdinsight
description: "Replace occurences of the following values with the correct information for your environment."
urlFragment: resource-manager-templates-sample
---

# Resource Manager templates for creating Azure HDInsight clusters using Azure Data Lake Gen2 storage

## Using the templates

Replace occurences of the following values with the correct information for your environment:

* `<SUBSCRIPTION_ID>` - replace with your Azure subscription ID
* `<PASSWORD>` - replace with your chosen password for cluster login and ssh login

## Resources

[Read the tutorial](https://docs.microsoft.com/azure/hdinsight/hdinsight-hadoop-use-data-lake-storage-gen2).


text:
---
layout: default
---

# 1996-08-22 - Re: Spamming

## Header Data

From<EMAIL_ADDRESS>(Chris J Samuelson)

To<EMAIL_ADDRESS>Message Hash: 2d2c0b5e7c11df03a6b041e58220b66cf11eff8efecc2ade99c2e29cd4d89d2b

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1996-08-22 01:48:06 UTC

Raw Date: Thu, 22 Aug 1996 09:48:06 +0800

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Chris J Samuelson)
Date: Thu, 22 Aug 1996 09:48:06 +0800
To<EMAIL_ADDRESS>Subject: Re: Spamming
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

On Wed, 21 Aug 1996 10:52:32 -0400 "James C. Sewell"<EMAIL_ADDRESS>writes:
>At 06:09 PM 8/20/96 -0700, Rich Graves wrote:
>>On Wed, 21 Aug 1996, Vipul Ved Prakash wrote:
>>
>>> I don't know if there has been much discussion on the ethics of
>spamming
>>> here? Is spamming free speech?
>>
>>Yes.
>
>No.  I think we can all (most) agree that spam-email is like
>junk-snailmail.
>In that case there are a few things to consider:
>
>  1. Junkmail requires the SENDER to pay for it, not the recipient.
>  2. Junkmail is under the full authority of the Postmaster.  If
>     they do anything illegal they have an authority to which they
>     must answer and may face criminal charges for.
>  3. You can't legally stuff mailboxes by driving around the
>neighborhood.
>     It is against the law for me to walk up to your mailbox and put
>     something in it.  Should the same be true of emailboxes?
>  4. Junkmail is usually at a lower priority than "real" mail and
>     due to costs is usually easily identifiable.  Email isn't.
>
>  And one relating to only email:
>
>     I don't want to have to spend 10 minutes letting Eudora sort
>     through my mailbox because my filter rules are so numerous and
>     complex due to me trying to block spam.
>
>  We must remember that the First Amendment does not allow us to say
>any thing at any time via any medium we choose.  There are limits
>to it, usually in the name of public safety and harassment.  There
>should be similar limits in the Internet.
>  I'm not saying we shouldn't let you tell others how get rich quick
>but that you should not be allowed to mail to<EMAIL_ADDRESS>just to tell the
>world how great we are.
>
> Jim
>
>Jim Sewell -<EMAIL_ADDRESS>  Tantalus Incorporated - Key West, FL
>
>
Why should the spammers pay for it, any more than anyone else should.
They are still
in some way paying for the E-Mail I assume, by keeping up a server or
paying someone else for the convienince of E-Mail.  If they tried to
force spammers to pay money they would have to have a way to decide if
someone was spamming.A good excuse to read E-Mail?

{% endraw %}
```

## Thread

+ Return to [August 1996](/archive/1996/08)

+ Return to<EMAIL_ADDRESS>(Chris J Samuelson)](/authors/chris230_at_juno_com_chris_j_samuelson_)"
+ Return to "["James C. Sewell" <jims@MPGN.COM>](/authors/james_c_sewell_jims_at_mpgn_com_)"
+ Return to "[Vipul Ved Prakash <vipul@pobox.com>](/authors/vipul_ved_prakash_vipul_at_pobox_com_)"

+ 1996-08-21 (Thu, 22 Aug 1996 05:35:24 +0800) - [Re: Spamming](/archive/1996/08/2366144e340a862161d015507a7f25d175774bf791b80c4e073e57c5a4783ec8) - _"James C. Sewell"<EMAIL_ADDRESS>  + 1996-08-22 (Thu, 22 Aug 1996 09:48:06 +0800) - Re: Spamming -<EMAIL_ADDRESS>(Chris J Samuelson)_
  + 1996-08-22 (Fri, 23 Aug 1996 05:46:13 +0800) - [Re: Spamming](/archive/1996/08/6f8f8e02e47af0e65d01dcfaea8436ebdbe0757998379be95f9eedb7e5d5deca) - _Vipul Ved Prakash<EMAIL_ADDRESS>


text:
# Breadcrumbs

###### Last updated Jan 9, 2019

:::

##### Example

Add `.subnav` class to the root element with an `<ol>` element with `.breadcrumb` class as a child. Each `<li>` child should have a `.breadcrumb-item` class with either a `<span>` or `<a>`

```html
<div class="subnav">
    <ol class="breadcrumb">
        <li class="breadcrumb-item">
            <a href="/foundations/breadcrumbs#">First</a>
            <i class="fa fa-chevron-right breadcrumb-arrow"></i>
        </li>
        <li class="breadcrumb-item">
            <a href="/foundations/breadcrumbs#">Second</a>
            <i class="fa fa-chevron-right breadcrumb-arrow"></i>
        </li>
        <li class="breadcrumb-item">
            <span>Third</span>
            <i class="fa fa-chevron-right breadcrumb-arrow"></i>
        </li>
    </ol>
    <div class="breadcrumb-responsive-container">
        <a class="breadcrumb-back-button">
            <i class="fa fa-chevron-left"></i>
            <span>Back</span>
        </a>
    </div>
</div>
```

:::

            First

            Second

            Third

            Back

:::

##### Responsive

On smaller phone sized devices the `.breadcrumb` element will be hidden and in its place `.breadcrumb-responsive-container` will show. This container can have anything that would be suitable for smaller screens. Responsive behavior can be disabled by adding `.disable-responsive` to each the `.breadcrumb` and `.breadcrumb-responsive-container` element
:::

branch_name: refs/heads/dev

gha_language: TypeScript