text:
---
layout: post
title: "Training Models that have Zero Likelihood"
categories: [Optimization]
image: assets/images/celeba_spread_learned_5.png
tags: [featured]
author: davidbarber
---

How can we train generative models when we cannot calculate a gradient likelihood?

A popular class of models in machine learning is the so-called generative model class with deterministic outputs. These are currently used for example in the generation of realistic images. If we represent an image with the variable $$x$$, then these models generate an image by the following process:

1. Sample $$z$$ from a $$Z$$ dimensional Gaussian (multivariate-normal) distribution.

2. Use this $$z$$ as the input to a deep neural network $$g_\theta(z)$$ whose output is the image $$x$$.

Typically the dimension of the latent variable is chosen to be much lower than the observation variable $$Z\ll X$$. This enables one to learn a low-dimensional representation of high-dimensional observed data.

We can write this mathematically as a latent variable model producing a distribution $$p(x)$$ over images $$x$$, with

$$
p(x) = \int p(x\vert{}z)p(z)dz
$$

and $$p(x\vert{}z)$$ restricted to a deterministic distribution so that

$$
p_\theta(x) = \int \delta\left(x-g_\theta(z)\right)p(z)dz,
$$

where we write $$p_\theta(x)$$ to emphasise that the model depends on the parameters $$\theta$$ of the network. These models represent a rich class of distributions (thanks to the highly non-linear neural network) and are also easy to sample from (using the above procedure).

So what's the catch? Given a set of training images, $$x^1,\ldots,x^N$$, the challenge is to learn the parameters $$\theta$$ of the network $$g_\theta$$.

{:.text-center img}

Because the latent dimension $$Z$$ is lower than the observation dimension $$X$$, the model can only generate images in a $$Z$$ dimensional manifold within the $$X$$ dimensional image space, as depicted in the figure for a latent dimension $$Z=2$$ and observed dimension $$X=3$$. That means that only images that lie on this manifold will have non-zero probability density value. For an image $$x^n$$ from the training dataset, unless it lies exactly on the manifold, the likelihood of this image $$p_\theta(x^n)$$ will be zero. This means that typically the log likelihood of the dataset

$$
L(\theta) \equiv \sum_{n=1}^N \log p_\theta(x^n)
$$

is not well defined and standard gradient based training of $$L(\theta)$$ is not possible. This is a shame because (i) many standard training techniques are based on maximum likelihood and (ii) maximum likelihood is statistically efficient, meaning that no other training method can more  accurately learn the model parameters $$\theta$$.

The inappropriateness of maximum likelihood as the training criterion for this class of models has inspired much recent work. For example the GAN objective and MMD objectives are well known approaches to training such models[^Shakir]. However, it is interesting to consider whether one can nevertheless modify the maximum likelihood criterion to make it applicable in this situation. The beauty of doing so is that we may then potentially use standard maximum likelihood style techniques (such as variational training approaches).

### The Spread Divergence
{:.no_toc}

For concreteness, we consider the Kullback-Leibler Divergence divergence between distributions $$p$$ and $$q$$

$$
KL(p\vert{}q) \equiv \int p(x)\log \frac{p(x)}{q(x)}dx
$$

Maximising the likelihood is equivalent to minimising the KL Divergence between the empirical data distribution

$$
p(x)=\frac{1}{N}\sum_{n=1}^N \delta\left(x-x^n\right)
$$

and the model $$p_\theta(x)$$, since

$$
KL(p\vert{}p_\theta) = -\frac{1}{N}\sum_{n=1}^N \log p_\theta(x^n) + const
$$

The fact that maximum likelihood training is not appropriate for deterministic output generative models is equivalent to the fact that the KL divergence (and its gradient) are not well defined for this class of model.

More generally, for distributions $$p$$ and $$q$$, the $$f$$-Divergence is defined as

$$
D_f(p\vert q) = \int f\left( \frac{p(x)}{q(x)}\right) q(x) dx
$$

where $$f(x)$$ is a convex function with $$f(1)=0$$. However, this divergence may not be defined if the supports (regions of non-zero probability) of $$p$$ and $$q$$ are different, since then the ratio $$p(x)/q(x)$$ can cause a division by zero. This is the situation when trying to define the likelihood for data that is not on the low dimensional manifold of our model.

The central issue we address therefore is how to convert a divergence which is not defined due to support mismatch into a well defined divergence.

From $$q(x)$$ and $$p(x)$$ we define new distributions $$\tilde{q}(y)$$ and $$\tilde{p}(y)$$ that have the same support. We define distributions

$$
\tilde{p}(y) = \int p(y{\mid}x)p(x)dx, \hspace{1cm} \tilde{q}(y) = \int p(y{\mid}x)q(x)dx
$$

where $$p(y{\mid}x)$$ 'spreads' the mass of $$p$$ and $$q$$ and is chosen such that $$\tilde{p}$$ and $$\tilde{q}$$ have the same support.

Consider the extreme case of two delta distributions

$$
p(x)=\delta(x-\mu_p), \quad q(x)=\delta(x-\mu_q)
$$

for which $$KL(p\vert q)$$ is not well defined. Using a Gaussian spread distribution $$p(y{\mid}x)={\mathcal{N}}(y;x,\sigma^2)$$ with mean $$x$$ and variance $$\sigma^2$$ ensures that $$\tilde{p}$$ and $$\tilde{q}$$ have common support $$\mathbb{R}$$. Then

$$
\tilde{p}(y)=\int \mathcal{N}(y;x,\sigma^2)dx\delta(x-\mu_p)=\mathcal{N}(y;\mu_p,\sigma^2)
$$

and

$$
\tilde{q}(y)=\int \mathcal{N}(y;x,\sigma^2)dx\delta(x-\mu_q)=\mathcal{N}(y;\mu_q,\sigma^2)
$$

and the KL divergence between the two becomes

$$
KL(\tilde{p}\vert \tilde{q})=\frac{1}{2\sigma^2}(\mu_p-\mu_q)^2
$$

This divergence is well defined for all values of $$\mu_p$$ and $$\mu_q$$. Indeed, this divergence has the convenient property that $$KL(\tilde{p}\vert \tilde{q})=0 \Rightarrow p=q$$. If we consider $$p$$ to be our data distribution (a single datapoint at $$\mu_p$$) and $$q$$  our model $$p_\theta$$, we can now do a modified version of maximum likelihood training to fit $$p_\theta$$ to $$p$$ -- instead of minimising $$KL(p\vert p_\theta)$$, we minimise $$KL(\tilde{p}\vert \tilde{p}_\theta)$$.

Note that, in general, we must spread both distributions $$p$$ and $$q$$ for the divergence between the spreaded distributions to be zero to imply that the original distributions are the same. In the context of maximum likelihood learning, spreading only one of these distributions will in general result in a biased estimator of the underlying model.

### Stationary Spread Divergence
{:.no_toc}

If we consider stationary spread distributions of the form $$p(y\vert x)=K(y-x)$$, for 'kernel' function $$K(x)$$. It is straightforward to show that if the kernel $$K(x)$$ has strictly positive Fourier Transform, then

$$
D_f(\tilde{p}\vert \tilde{q}) = 0 \Rightarrow p=q
$$

Interestingly, this condition on the kernel is equivalent to the condition on the kernel in the MMD framework[^Gretton], which is an alternative way to define a divergence between distributions. It is easy to show that the Gaussian distribution has strictly positive Fourier Transform and thus defines a valid spread divergence. Another useful spread distribution with this property is the Laplace distribution.

### Machine Learning Applications
{:.no_toc}

We now return to how to train models of the form

$$
p_\theta(x) = \int \delta\left(x-g_\theta(z)\right)p(z)dz,
$$

and adjust the parameters $$\theta$$ to make the model fit the data distribution

$$
p(x) = \frac{1}{N}\sum_{n=1}^N  \delta\left(x-x^n\right)
$$

Since maximum likelihood is not (in general) available for this class of models, we instead consider minimising the spread KL divergence using Gaussian spread noise

$$
KL(\tilde{p}\vert \tilde{p}_\theta) = -\int \tilde{p}(y) \log \tilde{p}_\theta(y)dy + const
$$

where the spreaded data distribution is

$$
\tilde{p}(y) = \frac{1}{N}\sum_{n=1}^N \mathcal{N}(y;x^n,\sigma^2 I_X)
$$

The objective is then simply an expectation over the log likelihood $$\log \tilde{p}_\theta(y)$$,

$$
\tilde{p}_\theta(y) = \int p(y\vert x)p_\theta(x)dx = \int \mathcal{N}(y; g_\theta(z),\sigma^2 I_X)p(z)dz
$$

which is a standard generative model with a Gaussian output distribution.  For this we may now use a variational training approach to form a lower bound on the quantity $$\log \tilde{p}_\theta(y)$$. The final objective just then requires evaluating the expectation of this bound, which can be easily approximated by sampling from the spreaded data distribution.

Overall, this is therefore a simple modification of standard Variational Autoencoder (VAE) training in which there is an additional outer loop sampling from the spreaded data distribution[^SD].

In the figure below we show how we are able to fit a deep generative 4 layer convolutional network with deterministic output to the CelebA dataset of face images. Whilst this model cannot be trained using standard VAE approaches, using the spread divergence approach and sampling from the trained model gives images of the form below

{:.text-center img}

Our aim isn't to produce the most impressive face sampler, but rather to show how one can make a fairly simple modification of the standard training algorithm to cope with deterministic outputs.

In our paper[^SD] we apply this method to show how to overcome well known problems in training deterministic Independent Components Analysis models using only a simple modification of the standard training algorithm. We also discuss how to learn the spread distribution and how this relates to other approaches such as MMD and GANs.

## Summary
{:.no_toc}

A popular class of generative deep network models cannot be trained using standard classical machine learning approaches. However, by adding 'noise' to both the model and the data in an appropriate way, one can nevertheless define an appropriate objective that is amenable to standard machine learning training approaches.

### References
{:.no_toc}

[^SD]: M. Zhang, P. Hayes, T. Bird, R. Habib, D. Barber. Spread Divergence. [arxiv.org/abs/1811.08968](https://arxiv.org/abs/1811.08968), 2018.

[^Shakir]: S. Mohamed and B. Lakshminarayanan. Learning in Implicit Generative Models. [arxiv.org/abs/1610.03483](https://arxiv.org/abs/1610.03483), 2016.

[^Gretton]: A. Gretton et al. A Kernel Two-Sample Test. Journal of Machine Learning Research, 13. 2012.


text:
# AoC
[![CI](https://github.com/xboard/AoC/workflows/CI/badge.svg)](https://github.com/xboard/AoC/actions?query=workflow%3ACI)
![Python](https://img.shields.io/badge/Python%203.8-3776AB?logo=Python&logoColor=white&style=flat)

[Advent of Code](https://adventofcode.com/) solutions in Python 3.

## Pre-requisites

- python 3.8+
- pytest 6.1+

## To run

Go to challenge year/day directory and type `python3 <script_name>.py input.txt.gz`, for instance to run solution
for day 8 of AoC 2020:
```
cd 2020/Day_8
python3 handheld_halting.py input.txt.gz
```

You can pass a path to your own input files but you must gzip it first:
```
python3 handheld_halting.py /tmp/another_input_file.txt.gz
```

### Design

Every problem solution is designed to be easy to understand, unit testable, self-contained (including unit tests),
idiomatic and to (ab)use type annotations.

### Unit tests

Install `pytest`:

```
pip install -r requirements.txt
```

To run all unit tests for all problems:
```
pytest **/*.py
```

To run unit tests for a specific problem:
```
pytest <path_to_problem>/*.py
```

license: MIT

repository: xboard/AoC

branch_name: refs/heads/main

extension: md

gha_language: Python

gha_license_id: MIT

license_type: permissive

repo_name: xboard/AoC


text:
# **rating-operator-api**

Used as the communication interface between the rating components, the `rating-operator-api` is also used to expose metrics to users.

There's two ways to access these metrics:

- By watching the results on our Grafana dashboards.
- By querying this API through the exposed endpoints.

In this document we'll explain how to use these endpoints, with examples and expected parameters.

The endpoints are ordered by their resource (namespace, pods, etc..), and the grammar is similar for each category.

There's different kind of queries and right types:

#### Access types

- **Public**, every user of the rating-operator-api will receive the same response.
- **Tenant**, each tenant will receive only data that belongs to him. Tenants that are not logged in will only have access to public data.
- **Admin**, these endpoints can only be used if the tenant is admin and provide the secret key.

#### Differents queries

- **Simple endpoints**

No parameters required, call the endpoint, get a response.

```sh
$ curl http://rating-operator-api.rating:80/namespaces
{
    "results":[{"namespace":"default", "tenant_id":"default"}],
    "total": 1
}
```

- **Endpoints with url parameters**

The endpoints using this method will be labelled `[URL]`.

We'll use `/namespaces/<namespace>/<aggregator>` as an exemple here.

In this query, the time range is handled by the **aggregator**, and the parameters are sent through the url.
```sh
# We use 'default' namespace and the 'daily' aggregator for the example.
$ curl http://rating-operator-api.rating:80/namespaces/default/daily
{
    "results": [...],
    "total": ...
}
```

- **Endpoints with time range (TR)**

The endpoints using this method will be labelled `[TR]`.

For this example, `/namespaces/<namespace>/rating` is a perfect choice.
There's no time range specified in this query, so we need to specify it using url encoded parameters.
The default value for those is from two hours to now.

```sh
$ curl http://rating-operator-api.rating:80/namespaces/default/rating?start=2021-02-05+10%3A22%3A53.604Z&end=2021-02-05+16%3A22%3A53.604Z
{
    "results": [...],
    "total": ...
}
```

- **Endpoint with payload**

The endpoints using this method will be labelled `[PL]`.

These endpoints are the hardest to query, from a user perspective.
They are generally available for resource handling, such as rules or tenant management.
Let's create a RatingRuleModel with cURL as an example:

```sh
$ curl -X POST
       -H "Content-Type: application/json"
       -d '{"name": "test","metric_name": "test","metric": "my_test_promql","timeframe": "3600s"}'
       http://rating-operator-api.rating:80/models/add
RatingRuleModel test created.
```

----

## **Endpoints**

### ***Namespaces***

**GET `/namespaces`** **Public**

Get a list of namespaces visibile by the tenant.
No parameters expected.

**GET `/namespaces/rating`** ***[TR]*** **Tenant**

Get the rating on a time period, grouped by namespaces.

Parameters expected:

- `start`
- `end`

**GET `/namespaces/total_rating`** ***[TR]*** **Tenant**

Get the **sum** of the rating on a time period, grouped by namespaces.

Parameters expected:

- `start`
- `end`

**GET `/namespaces/metrics/rating`** ***[TR]*** **Tenant**

Get the rating on a time period, grouped by namespaces and metrics.

Parameters expected:

- `start`
- `end`

**GET `/namespaces/<namespace>/<aggregator>`** ***[URL]*** **Tenant**

Get the rating for a namespace, according to the aggregator.

Available aggregator are:

- `daily`
- `weekly`
- `monthly`

If `namespace` equal "rating", then the rating is queried for all namespaces.

**GET `/namespaces/<namespace>/rating`** ***[TR] [URL]*** **Tenant**

Get the rating on a time period, for a given namespace.

Parameters expected:

- `start`
- `end`

**GET `/namespaces/<namespace>/total_rating`** ***[TR] [URL]*** **Tenant**

Get the **sum** of the rating on a time period, for a given namespace.

Parameters expected:

- `start`
- `end`

**GET `/namespaces/<namespace>/metrics/<metric>/rating`** ***[TR] [URL]*** **Tenant**

Get the rating on a time period, for givens namespace and metric.

Parameters expected:

- `start`
- `end`

**POST `/namespaces/tenant`** **[PL]** **Admin**

Associate a tenant with a namespace.

Expect a payload with:

- `namespace`
- `tenant_id`
- `token`

### ***METRICS***

**GET `/metrics`** **Public** **Tenant**

Get the list of metrics.

**GET `/metrics/<metric>/todate`** ***[URL]*** ***[TR]*** **Tenant**

Get the rating, from the start of the month to now.

Parameters expected:

- `start`
- `end`

**GET `/metrics/rating`** ***[TR]*** **Tenant**

Get the rating per metric.

Parameters expected:

- `start`
- `end`

**GET `/metrics/<metric>/max`** ***[URL]*** ***[TR]*** **Tenant**

Get the max rating for a given metric.

Parameters expected:

- `start`
- `end`

**GET `/metrics/<metric>/ratio`** ***[URL]*** ***[TR]*** **Tenant**

Get the rating as a ratio per instance, for a given metric.

Parameters expected:

- `start`
- `end`

**GET `/metrics/<metric>/<aggregator>`** ***[URL]*** **Tenant**

Get the rating for a given metric, according to the aggregator.

Available aggregator are:

- `daily`
- `weekly`
- `monthly`

**GET `/metrics/<metric>/rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the rating for a given metric.

Parameters expected:

- `start`
- `end`

**GET `/metrics/<metric>/total_rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the aggragated rating for a given metric.

Parameters expected:

- `start`
- `end`

### ***PODS***

**GET `/pods`** ***[TR]*** **Tenant**

Get the list of pods.

Parameters expected:

- `start`
- `end`

**GET `/pods/rating`** ***[TR]*** **Tenant**

Get the rating for a given pod.

Parameters expected:

- `start`
- `end`

**GET `/pods/total_rating`** ***[TR]*** **Tenant**

Get pods aggregated rating.

Parameters expected:

- `start`
- `end`

**GET `/pods/<pod>/lifetime`** ***[URL]*** **Tenant**

Get the start and last update time of a pod.

**GET `/pods/<pod>/<aggregator>`** ***[URL]*** **Tenant**

Get the pods rating by time aggregation.

Available aggregator are:

- `daily`
- `weekly`
- `monthly`

**GET `/pods/<pod>/rating`** ***[URL]*** ***[TR]*** **Tenant**

Get pods rating.

Parameters expected:

- `start`
- `end`

**GET `/pods/<pod>/total_rating`** ***[URL]*** ***[TR]*** **Tenant**

Get pods aggregated rating.

Parameters expected:

- `start`
- `end`

**GET `/pods/<pod>/metrics/<metric>/rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the rating for a given pod and metric.

Parameters expected:

- `start`
- `end`

**GET `/pods/<pod>/metrics/<metric>/total_rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the aggregated rating for a given pod and metric.

Parameters expected:

- `start`
- `end`

### ***NODE***

**GET `/nodes`** **Tenant**

Get the list of nodes.

**GET `/nodes/rating`** ***[TR]*** **Tenant**

Get nodes rating.

Parameters expected:

- `start`
- `end`

**GET `/nodes/total_rating`** ***[TR]*** **Tenant**

Get the nodes agglomerated rating.

Parameters expected:

- `start`
- `end`

**GET `/nodes/metrics/rating`** ***[TR]*** **Tenant**

Get the nodes metrics.

Parameters expected:

- `start`
- `end`

**GET `/nodes/<node>/<aggregator>`** ***[URL]*** **Tenant**

Get the nodes rating by time aggregation.

Available aggregator are:

- `daily`
- `weekly`
- `monthly`

**GET `/nodes/<node>/rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the rating for a given node.

Parameters expected:

- `start`
- `end`

**GET `/nodes/<node>/total_rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the agglomerated rating for a given node.

Parameters expected:

- `start`
- `end`

**GET `/nodes/metrics/<metric>/rating`** ***[URL]*** ***[TR]*** **Tenant**

Get the rating for a given metric.

Parameters expected:

- `start`
- `end`

## ***CONFIGS***

### ***RatingRules***

**GET `/ratingrules`** **Public**

Get the list of all RatingRules as object.

**GET `/ratingrules/list/local`** **Public**

Get the list of all the RatingRules names from the local configuration directory.

**GET `/ratingrules/list/cluster`** **Public**

Get the list of all the RatingRules names from the cluster.

**GET `/ratingrules/<timestamp>`** ***[URL]***

Get the RatingRule for a given timestamp.

**POST `/ratingrules/add`** **[PL]** **Admin**

Add a new configuration.

Expect a payload with:

- `rules`
- `metrics`
- `timestamp`

**POST `/ratingrules/update`** **[PL]** **Admin**

Update a configuration.

Expect a payload with:

- `rules`
- `metrics`
- `timestamp`

**POST `/ratingrules/delete`** **[PL]** **Admin**

Delete a configuration.

Expect a payload with:

- `timestamp`

### ***RatingRulesModels***

**GET `/models/list`** **Public**

Get the list of all the RatingRuleModels.

**GET `/models/get`** **Public**

Get a RatingRuleModel.

Expect a payload with:

- `name`

**POST `/models/add`** **[PL]** **Admin**

Add a new RatingRuleModel.

Expect a payload with:

- `name`
- `timeframe`
- `metric_name`
- `metric`

**POST `/models/update`** **[PL]** **Admin**

Update a RatingRuleModel.

Expect a payload with:

- `name`
- `timeframe`
- `metric_name`
- `metric`

**POST `/models/delete`** **[PL]** **Admin**

Delete a RatingRuleModel.

Expect a payload with:

- `name`

repository: cherrared/rating-operator

branch_name: refs/heads/master

repo_name: cherrared/rating-operator


text:
# 22852_CA2_BACK-END-DORSET

## About

CA2 Project 22852

This is a solo project which had the purpose to perform CRUD Request through NodeJS, Express on a React App
The port used : 8080

## Dependencies

- JS
- Node
- Express
- Nodemon
- Body-Parser
- MongoDB
- Axios
- Bulma
- Webpack

## Installation

Launch the Server

```bash
npm run start

```

## :briefcase: Requirements

- Install Dependencies

```bash
npm install

```

## :email: Perform CRUD REQUESTS

### DATABASE INFORMATIONS

- Database : MongoDB
- Name : imdb
- Collection : movies

- Database's properties :

```
imdb.movies {
    title : name of the movie | String
    director : name of the director | String
    year :  year of the release date | Int32
    american : american movie or not| Boolean
    poster : link of the poster | String

}
```

- Url used to Read database : '/'
- Url used to Create a movie : '/create-movie'
- Url used to Update a movie : '/edit-movie/{id}
- DELETE's request only need a button to perform

### CREATE / POST REQUEST

The POST method requests that the server accept the entity enclosed in the request as a new subordinate of the web resource identified by the URI.
To make the post request :

- On the main page, **CLICK** on *Add a new movie*
- **FILL OUT** the inputs with appropriate datas
- **SUBMIT** form with *submit* button

If successful, the console should return :

````shell script
Created in Database

````

### READ / GET REQUEST

The GET method requests a representation of the specified resource. Requests using GET should only retrieve data and should have no other effect.

To make the GET Request :

**CONNECT** on the home page : *http://localhost:8080/*

If successful, the console should return :

````shell script
[
  {
    _id: 5fd179c0ea3e08c1295b344b,
    title: 'Tenet',
    director: 'Christopher',
    year: 2020,
    poster: 'https://images-na.ssl-images-amazon.com/images/I/71Tw-XHzu1L._AC_SL1200_.jpg',
    american: false
  },
]...

````

### UPDATE / PUT REQUEST

The PUT method requests that the enclosed entity be stored under the supplied URI. If the URI refers to an already existing resource, it is modified; if the URI does not point to an existing resource, then the server can create the resource with that URI

To make the PUT Request :

- On the main page, Pick a movie and **CLICK** on *Edit* button
- **UPDATE** the inputs you want to update with appropriate datas
- **SUBMIT** form with *submit* button

If successful, the console should return :

````shell script
Updated in Databse

````

### DELETE / DELETE REQUEST

The DELETE method deletes the specified resource.

To make the DELETE Request :

- On the main page, Pick a movie and **CLICK** on *Delete* button

If successful, the console should return :

````shell script
Deleted in Database
````

## License

[ISC](https://choosealicense.com/licenses/isc/)

license: ISC


text:
title: Using Sentry to Handle Python Exceptions in Django Projects
slug: sentry-handle-exceptions-django-projects
meta: Learn to handle Python exceptions in your Django projects with the hosted Sentry service.
category: post
date: 2020-08-23
modified: 2020-08-23
newsletter: False
headerimage: /img/headers/django-sentry.jpg
headeralt: Logos for the implementations used in this blog post. Copyright their respective owners.

Web applications built in [Django](/django.html) can become sprawlingly complex over time, which is one reason why centralized error handling is important. This tutorial will guide you through adding a free, basic Sentry configuration to a new Django project.

When we're done, you will be able to view centralized error reports in the Sentry dashboard like you see in this screenshot:

## Tutorial Requirements
Throughout this tutorial we are going to use the following dependencies,
which we will install in just a moment. Make sure you also have Python 3,
[preferrably 3.7 or newer installed](https://www.python.org/downloads/),
in your environment:

We will use the following dependencies to complete this
tutorial:

* [Django](/django.html) [web framework](/web-frameworks.html),
  [version 3.1](https://www.djangoproject.com/download/)
* [sentry-sdk](https://sentry.io/for/python/),
  [version 0.16.5](https://pypi.org/project/sentry-sdk/)

All code in this blog post is available open source under the MIT license
on GitHub under the
[sentry-handle-exceptions-django-projects directory of the blog-code-examples repository](https://github.com/fullstackpython/blog-code-examples).
Use the source code as you desire for your own projects.

## Development environment configuration
Change into the directory where you keep your Python
[virtual environments](/virtual-environments-virtualenvs-venvs.html).
Create a new virtualenv for this project using the following
command.

Start the Django project by creating a new
[virtual environment](/virtual-environments-virtualenvs-venvs.html)
using the following command. I recommend using a separate directory
such as `~/venvs/` (the tilde is a shortcut for your user's `home`
directory) so that you always know where all your virtualenvs are
located.

```bash
python3 -m venv ~/venvs/djsentry
```

Activate the virtualenv with the `activate` shell script:

```bash
source ~/venvs/djsentry/bin/activate
```

After the above command is executed, the command prompt will
change so that the name of the virtualenv is prepended to the
original command prompt format, so if your prompt is simply
`$`, it will now look like the following:

```bash
(djsentry) $
```

Remember, you have to activate your virtualenv in every new terminal
window where you want to use dependencies in the virtualenv.

We can now install the [Django](https://pypi.org/project/Django/)
package into the activated but otherwise empty virtualenv.

```
pip install django==3.1 sentry-sdk==0.16.5
```

Look for output similar to the following to confirm the appropriate
packages were installed correctly from PyPI.

```
(djsentry) $ pip install django==3.1 sentry-sdk==0.16.5
Collecting django
  Downloading https://files.pythonhosted.org/packages/2b/5a/4bd5624546912082a1bd2709d0edc0685f5c7827a278d806a20cf6adea28/Django-3.1-py3-none-any.whl (7.8MB)
    100% |████████████████████████████████| 7.8MB 6.3MB/s
Collecting sentry-sdk
  Downloading https://files.pythonhosted.org/packages/f4/4c/49f899856e3a83e02bc88f2c4945aa0bda4f56b804baa9f71e6664a574a2/sentry_sdk-0.16.5-py2.py3-none-any.whl (113kB)
    100% |████████████████████████████████| 122kB 33.7MB/s
Collecting asgiref~=3.2.10 (from django)
  Using cached https://files.pythonhosted.org/packages/d5/eb/64725b25f991010307fd18a9e0c1f0e6dff2f03622fc4bcbcdb2244f60d6/asgiref-3.2.10-py3-none-any.whl
Collecting sqlparse>=0.2.2 (from django)
  Using cached https://files.pythonhosted.org/packages/85/ee/6e821932f413a5c4b76be9c5936e313e4fc626b33f16e027866e1d60f588/sqlparse-0.3.1-py2.py3-none-any.whl
Collecting pytz (from django)
  Using cached https://files.pythonhosted.org/packages/4f/a4/879454d49688e2fad93e59d7d4efda580b783c745fd2ec2a3adf87b0808d/pytz-2020.1-py2.py3-none-any.whl
Collecting urllib3>=1.10.0 (from sentry-sdk)
  Using cached https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl
Collecting certifi (from sentry-sdk)
  Using cached https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl
Installing collected packages: asgiref, sqlparse, pytz, django, urllib3, certifi, sentry-sdk
Successfully installed asgiref-3.2.10 certifi-2020.6.20 django-3.1 pytz-2020.1 sentry-sdk-0.16.5 sqlparse-0.3.1 urllib3-1.25.10

```

We can get started coding the application now that we have all of our
required dependencies installed.

## Coding the initial application
We have everything we need to start building our application.

We can use the [Django](/django.html) `django-admin` tool to create
the boilerplate code structure to get our project started.
Change into the directory where you develop your applications. For
example, I typically use `/Users/matt/devel/py/` for all of my
Python projects. Then run the following command to start a Django
project named `djsentry`:

```
django-admin.py startproject djsentry
```

Note that in this tutorial we are using the same name for both the
virtualenv and the Django project directory, but they can be
different names if you prefer that for organizing your own projects.

The `django-admin` command creates a directory named `djsentry`
along with several subdirectories that you should be familiar with
if you have previously worked with Django.

Change directories into the new project.

```
cd djsentry
```

Create a new Django app within `djsentry`.

```
python manage.py startapp errors
```

Django will generate a new folder named `errors` for the project.
We should update the URLs so the app is accessible before we write
our `views.py` code.

Open `djsentry/djsentry/urls.py`. Add the highlighted
lines so that URL resolver will check the `errors` app
for additional routes to match with URLs that are requested of
this Django application.

```python
# djsentry/djsentry/urls.py
~~from django.conf.urls import include
from django.contrib import admin
from django.urls import path

urlpatterns = [
~~    path('', include('errors.urls')),
    path('admin/', admin.site.urls),
]
```

Save `djsentry/djsentry/urls.py` and open
`djsentry/djsentry/settings.py`.
Add the `errors` app to `settings.py` by inserting
the highlighted line:

```python
# djsentry/djsentry/settings.py
# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
~~    'errors',
]
```

Make sure you change the default `DEBUG` and `SECRET_KEY`
values in `settings.py` before you deploy any code to production. Secure
your app properly with the information from the Django
[production deployment checklist](https://docs.djangoproject.com/en/stable/howto/deployment/checklist/)
so that you do not add your project to the list of hacked applications
on the web.

Save and close `settings.py`.

Next change into the `djsentry/errors` directory. Create
a new file named `urls.py` to contain routes for the `errors` app.

Add all of these lines to the empty `djsentry/errors/urls.py`
file.

```python
# djsentry/errors/urls.py
from django.conf.urls import url
from . import views

urlpatterns = [
    url(r'^$', views.errors_index, name="index"),
]
```

Save `djsentry/errors/urls.py`. Open
`djsentry/errors/views.py` to add the
following two highlighted lines. You can keep the boilerplate comment
"# Create your views here." or delete like I usually do.

```
# djsentry/errors/views.py
from django.shortcuts import render

~~def errors_index(request):
~~    return render(request, 'index.html', {})
```

Next, create a directory for your template files named `templates` under
the `djmaps/maps` app directory.

```
mkdir templates
```

Create a new file named `index.html` within
`djsentry/errors/templates` that contains the
following [Django template language](/django-templates.html) markup.

```
<!DOCTYPE html>
<html>
  <head>
    <title>First step for errors</title>
  </head>
  <body>
   <h1>Hello, world!</h1>
  </body>
</html>
```

We can test out this static page to make sure all of our code is
correct before we start adding the meat of the functionality to
the project. Change into the base directory of your Django project
where the `manage.py` file is located. Execute the development
server with the following command:

```bash
python manage.py runserver
```

The Django development server should start up with no issues other than
an unapplied migrations warning.

```
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
August 15, 2020 - 17:26:57
Django version 3.1, using settings 'djsentry.settings'
Starting development server at http://<IP_ADDRESS>:8000/
Quit the server with CONTROL-C.

```

Open a web browser and go to `localhost:8000`.

Our code works, but it sure does not do much yet. Let's add
sentry-sdk so we can understand how it works.

## Adding Sentry and the sentry-sdk library
We can now add Sentry and test it with a bunch of errors to make sure it
is working properly.

Sentry can either be [self-hosted](https://github.com/getsentry/onpremise) or
used as a cloud service through [Sentry.io](https://sentry.io). In this
tutorial we will use the cloud hosted version because it's faster than
setting up your own server as well as free for smaller projects.

Go to [Sentry.io's homepage](https://sentry.io).

Sign into your account or sign up for a new free account. You will be at
the main account dashboard after logging in or completing the Sentry sign
up process.

There are no errors logged on our account dashboard yet, which is as
expected because we have not yet connected our account to our Django
project.

Create a new Sentry Project just for this application by clicking
"Projects" in the left sidebar to go to the Projects page.

On the Projects page, click the "Create Project" button in the top right
corner of the page.

You can either choose "Django" or select "Python". I usually just choose
"Python" if I do not yet know what framework I'll be using to build my
application. Next, give your new Project a name and then press the "Create
Project" button. Our new project is ready to integrate with our Python code.

We need the unique identifier for our account and project to authorize our
Python code to send errors to this Sentry instance. The easiest way to get
what we need is to go to the
[Python+Django documentation page](https://docs.sentry.io/platforms/python/django/)
and read how to configure the SDK.

Copy the string parameter for the `init` method and set it
[as an environment variable](https://www.twilio.com/blog/2017/01/how-to-set-environment-variables.html)
rather than having it exposed in your project's code.

```bash
export SENTRY_DSN='https://yourkeygoeshere.ingest.sentry.io/project-number'
```

**Make sure to replace "yourkeygoeshere" with your own unique identifier
and "project-number" with the ID that matches the project you just
created.**

Check that the `SENTRY_DSN` is set properly in your shell using the `echo`
command:

```bash
echo $SENTRY_DSN
```

Next, update `settings.py` with the following highlighted new lines:

```python
# settings.py
~~import os
~~import sentry_sdk

from pathlib import Path
~~from sentry_sdk.integrations.django import DjangoIntegration

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve(strict=True).parent.parent
```

At the bottom of the file after the line with `STATIC_URL`, add the
Sentry configuration:

```python
STATIC_URL = '/static/'

~~sentry_sdk.init(
~~    dsn=os.getenv('SENTRY_DSN'),
~~    integrations=[DjangoIntegration()],

~~    # If you wish to associate users to errors (assuming you are using
~~    # django.contrib.auth) you may enable sending PII data.
~~    send_default_pii=True
~~)
```

Now that we have the configuration in place we can deliberately make some
errors happen to test the connection to Sentry's service.

## Testing Sentry's error catching
We'll change some of the existing code to deliberately throw exceptions
to make sure everything is working properly.

Start by opening `errors/views.py` and updating it with one new
highlighted line that will automatically throw a generic Exception
when this function is called.

```python
# djsentry/errors/views.py
from django.shortcuts import render

def errors_index(request):
~~    raise Exception('testing exception')
    return render(request, 'index.html', {})
```

Go to `localhost:8000` in your browser and you will immediately get this
exception page when running the development server:

We can also try out code that does not simply raise an exception but instead
will definitely create one when executed, like this division by zero
operation:

```python
# djsentry/errors/views.py
from django.shortcuts import render

def errors_index(request):
~~    division_by_zero = 1 / 0
    return render(request, 'index.html', {})
```

If those exceptions both appear in the Sentry dashboard like this, you're all
set:

The above exceptions were just a couple of generic ways to test that everything
is working to send error information to Sentry. This configuration will
also handle the
[many other Django exceptions](https://docs.djangoproject.com/en/stable/ref/exceptions/)
you are likely to see when building the rest of your Django project.

## Additional resources
We just finished building a Django project that uses Sentry for
centralized error handling.

Next, try out some of these other related [Django](/django.html) tutorials:

* [Tracking Daily User Data in Django with django-user-visit](/blog/track-daily-user-data-django-user-visit.html)
* [Quickly Use Bootstrap 4 in a Django Template with a CDN](/blog/bootstrap-4-django-template.html)
* [How to Add Maps to Django Web App Projects with Mapbox](/blog/maps-django-web-applications-projects-mapbox.html)

If you have questions or comments about this tutorial, please contact me
via Twitter [@fullstackpython](https://twitter.com/fullstackpython), or
on GitHub [@mattmakai](https://github.com/mattmakai).
See something wrong with this post? Fork
[this page's source on GitHub](https://github.com/mattmakai/fullstackpython.com/blob/master/content/posts/200823-sentry-handle-exceptions-django-projects.markdown)
and submit a pull request.

extension: markdown

gha_language: HTML


text:
# List

### Intro

A list component to show items and control loading status.

### Install

Register component globally via `app.use`, refer to [Component Registration](#/en-US/advanced-usage#zu-jian-zhu-ce) for more registration ways.

```js
import { createApp } from 'vue';
import { List } from 'vant';

const app = createApp();
app.use(List);
```

## Usage

### Basic Usage

```html
<van-list
  v-model:loading="loading"
  :finished="finished"
  finished-text="Finished"
  @load="onLoad"
>
  <van-cell v-for="item in list" :key="item" :title="item" />
</van-list>
```

```js
import { ref } from 'vue';

export default {
  setup() {
    const list = ref([]);
    const loading = ref(false);
    const finished = ref(false);

    const onLoad = () => {
      setTimeout(() => {
        for (let i = 0; i < 10; i++) {
          list.value.push(list.value.length + 1);
        }
        loading.value = false;

        if (list.value.length >= 40) {
          finished.value = true;
        }
      }, 1000);
    };

    return {
      list,
      onLoad,
      loading,
      finished,
    };
  },
};
```

### Error Info

```html
<van-list
  v-model:loading="loading"
  v-model:error="error"
  error-text="Request failed. Click to reload"
  @load="onLoad"
>
  <van-cell v-for="item in list" :key="item" :title="item" />
</van-list>
```

```js
import { ref } from 'vue';

export default {
  setup() {
    const list = ref([]);
    const error = ref(false);
    const loading = ref(false);
    const onLoad = () => {
      fetchSomeThing().catch(() => {
        error.value = true;
      });
    };

    return {
      list,
      error,
      onLoad,
      loading,
    };
  },
};
```

### PullRefresh

```html
<van-pull-refresh v-model="refreshing" @refresh="onRefresh">
  <van-list
    v-model:loading="loading"
    :finished="finished"
    finished-text="Finished"
    @load="onLoad"
  >
    <van-cell v-for="item in list" :key="item" :title="item" />
  </van-list>
</van-pull-refresh>
```

```js
import { ref } from 'vue';

export default {
  setup() {
    const list = ref([]);
    const loading = ref(false);
    const finished = ref(false);
    const refreshing = ref(false);

    const onLoad = () => {
      setTimeout(() => {
        if (refreshing.value) {
          list.value = [];
          refreshing.value = false;
        }

        for (let i = 0; i < 10; i++) {
          list.value.push(list.value.length + 1);
        }
        loading.value = false;

        if (list.value.length >= 40) {
          finished.value = true;
        }
      }, 1000);
    };

    const onRefresh = () => {
      finished.value = false;
      loading.value = true;
      onLoad();
    };

    return {
      list,
      onLoad,
      loading,
      finished,
      onRefresh,
      refreshing,
    };
  },
};
```

## API

### Props

| Attribute | Description | Type | Default |
| --- | --- | --- | --- |
| v-model:loading | Whether to show loading info, the `load` event will not be Emitted when loading | _boolean_ | `false` |
| v-model:error | Whether loading is error, the `load` event will be Emitted only when error text clicked | _boolean_ | `false` |
| finished | Whether loading is finished, the `load` event will not be Emitted when finished | _boolean_ | `false` |
| offset | The load event will be Emitted when the distance between the scrollbar and the bottom is less than offset | _number \| string_ | `300` |
| loading-text | Loading text | _string_ | `Loading...` |
| finished-text | Finished text | _string_ | - |
| error-text | Error loaded text | _string_ | - |
| immediate-check | Whether to check loading position immediately after mounted | _boolean_ | `true` |
| disabled | Whether to disable the load event | _boolean_ | `false` |
| direction | Scroll direction, can be set to `up` | _string_ | `down` |
| scroller `v4.6.4` | Specifies the node that needs to listen for scroll events, defaults to the nearest parent scroll node | _Element_ | - |

### Events

| Event | Description | Arguments |
| --- | --- | --- |
| load | Emitted when the distance between the scrollbar and the bottom is less than offset | - |

### Methods

Use [ref](https://vuejs.org/guide/essentials/template-refs.html) to get List instance and call instance methods.

| Name  | Description           | Attribute | Return value |
| ----- | --------------------- | --------- | ------------ |
| check | Check scroll position | -         | -            |

### Types

The component exports the following type definitions:

```ts
import type { ListProps, ListInstance, ListDirection } from 'vant';
```

`ListInstance` is the type of component instance:

```ts
import { ref } from 'vue';
import type { ListInstance } from 'vant';

const listRef = ref<ListInstance>();

listRef.value?.check();
```

### Slots

| Name     | Description          |
| -------- | -------------------- |
| default  | List content         |
| loading  | Custom loading tips  |
| finished | Custom finished tips |
| error    | Custom error tips    |

## Theming

### CSS Variables

The component provides the following CSS variables, which can be used to customize styles. Please refer to [ConfigProvider component](#/en-US/config-provider).

| Name                         | Default Value             | Description |
| ---------------------------- | ------------------------- | ----------- |
| --van-list-text-color        | _var(--van-text-color-2)_ | -           |
| --van-list-text-font-size    | _var(--van-font-size-md)_ | -           |
| --van-list-text-line-height  | _50px_                    | -           |
| --van-list-loading-icon-size | _16px_                    | -           |

## FAQ

### How does List component work?

List will listen to the scroll event of the browser and calculate the position. When the distance between the bottom of the list and the visible area is less than `offset`, the List component will trigger a `load` event.

### Why does the load event triggered immediately after mounted?

A load event will be triggered immediately to load the first screen data. This feature can be disabled by the `immediate-check` prop.

### Why does the load event triggered continuously?

If the amount of data loaded in one request is too small, the List will continue to trigger the `load` event until the content fills the screen or the data is fully loaded.

Therefore, you need to adjust the amount of data per request. Ideally, the amount of data per request should be able to fill the height of one screen.

### What is the meaning of loading and finished?

`List` has three states, understanding these states will help you use the component:

- `loading = false`: Not in loading. The component will detect whether to trigger the `load` event according to the scroll position (if the content of the list is less than one screen, it will be triggered directly).
- `loading = true`: During loading. Indicating that an request is being sent, and the `load` event will not be triggered.
- `finished = true`: Loading is complete. No `load` will not be triggered.

After each request, you need to manually set `loading` to `false`, indicating the end of loading.

### Always trigger loading after using float layout?

If you use the float layout on the content, you can add the `van-clearfix` class to the container to clear the float, so that the List can get the element position correctly.

```html
<van-list>
  <div class="van-clearfix">
    <div class="float-item" />
    <div class="float-item" />
    <div class="float-item" />
  </div>
</van-list>
```

### Always trigger loading after setting overflow on html and body?

If the `overflow-x: hidden` style is set on the html and body tags, it will cause the List to always trigger loading.

```css
html,
body {
  overflow-x: hidden;
}
```

The reason is that when an element is styled with `overflow-x: hidden`, the element's `overflow-y` will be set to `auto` by the browser, instead of the default value of `visible`, causing the List can not determine the scroll container correctly. The workaround is to remove this style, or add the `height: 100%` style to the html and body tags.

### Always trigger loading when the direction prop is set to up?

Setting the `direction` prop to up will trigger the loading of the List component when the scrollbar is at the page top.

When using this prop, it is recommended to scroll the scroll bar to the page bottom after each data request is completed.

repository: youzan/vant

gha_language: TypeScript

repo_name: youzan/vant


text:
elixir-invoice
==============

Elixir library for invoicing and calculating VAT

This package contains a module (Einvoice) and a command-line utility (Einvoice.CMD) used to calculate and format invoice data and VAT.

The main module comes with a full suite of unit tests, which you can run with `mix test`

You can compile the command-line utility with `mix escript.build` and execute with `./einvoice` When invoked without parameters, the utility will print a help message describing its intended usage and the supported parameters.


text:
---
layout: article
title: spanning-tree Reference
mode: immersive
header:
  theme: dark
article_header:
  type: overlay
  theme: dark
  background_color: '#4c4cc2'
  background_image:
    gradient: 'linear-gradient(313deg, rgba(2,0,36, .6) 0%, rgba(76,76,194, .6) 47%, rgba(0,212,255, .6) 100%)'
    src: https://github.com/alexma2344/sitio/blob/master/assets/images/rainbows.jpg?raw=true"
sharing: true
comment: true
articles:
  excerpt_type: html
tags: computer-networks stp spanning-tree
---

---

A protocol that prevents loops on LANs

## Link State

> Distance vector was used in ARPANET untill 1979, then replaced by link state.
> A main reason was that distance vector took too long to convert after a topology change (count to infinity problem).

The idea behind link state routing is fairly simple and can be stated as five
parts. Each router must do the following things to make it work:

1. Discover neighbors and learn their network addresses.
2. Set the distance or cost metric to each of its neighbors.
3. Construct a packet telling all it has learned on the previous steps.
4. Send this packet to and receive similar packets from all other routers.
5. Compute shortest path to every other router.

### Learning about neighbors

When a router is booted its first task is to learn who its neighbors are.

So it sends **HELLO** packets on each of its lines. The router on the other end is expected to reply giving its name.

When we have multiple routers on a single broadcast domain *Figure 5-11(a)*, it leads to wasteful messages.

So instead we consider the LAN as a node itself: **N** *Figure 5-11(b)*, then one "designated router" on that LAN is selected to play the role of **N**. So going from A to C will be ANC.

### Setting link costs

Link state requires each link to have a cost metric (distance), so it can find the shortest path.

#### Bandwidth

A common choice is to make the **cost** inversely proportional to the bandwidth of the link. So that higher capacity paths are better choices.

**eg** 1Gbps Ethernet may have a cost of **1**, then 100Mbps Eth a cost of **10**.

#### Delay

Then for a geographically spread out network, **Delay** is factored into the cost, so that shorter links are better choices.

To determine the delay an **ECHO** packet is sent over the line, the other side replies immediately. By meassuring the round-trip time and dividing it by two, we get an estimated delay.

If we ping from London to Buenos Aires the round trip time is 245 miliseconds, divided by two is 122 miliseconds. Quick maths.

### Building link state packets

Once the information needed for the exchange has been collected, each router will build a packet containing all the data.

Packet contains:

- Identity of sender
- Sequence number
- Age
- List of neighbors

[OSPF packet capture](https://www.cloudshark.org/captures/293956261434)

### Distributing link state packets

All routers must get all state packets quickly and reliably. If different routers have different versions of the topology, inconsistencies such as loops or unreachable destinations.

#### Flooding

We use flooding to distribute these packets, as a control meassure each packet contains the sequence number (32-bit) that is incremented for each new packet sent.

> With one link state packet per second, it would take 137 years to wrap around, so this possibility can be ignored.

When a new packet comes in, it is checked against the list of packets that arrived earlier, if new, then it is flooded again.
If is a duplicate then it is discarded.

### Computing the New Routes

When the router has accumulated a full set of link state packets, it can construct the entire network graph, because all links are represented. (Actually links are represented twice, once for each direction, and each direction may have different costs).

Compared to distance vector routing, link state routing requires more memory
and computation.

filename: 2020-04-11-spanningtree reference.md


text:
# vertica-java-udtf
Maven project example UDTF for Java.  Demonstrates some unusual applications.

## Setup, build, test
Copy your version of VerticaSDK.jar and BuildInfo.java into the project.

Edit pom.xml for Vertica version and VerticaSDK.jar path.

Buld with "mvn clean package assmebly:single"

Copy JavaFunctions.sql and the target JAR to your Vertica node.

Check JavaFunctions.sql to ensure the library name matches your target JAR and make any other adjustments to the tests.

Run "vsql -f JavaFunctions.sql" to test the library.

## Disclaimer
This is an unsupported proof of concept!

repository: bryanherger/vertica-java-udtf

repo_name: bryanherger/vertica-java-udtf


text:
# LocalEGA-inbox
OpenSSH dropbox, with credentials from CentralEGA and RabbitMQ notifications for file system events

To build the docker image, use:

	# Create the "lega" group
	getent group lega || groupadd -g 1001 lega # adjust the number if needed

	# Build the image (using the above group, for access permissions)
	make latest

You can run a test instance (not connected to a local broker) with:

	docker-compose up -d

However, you first need to get credentials from Central EGA, and add them to the compose file.

You then connect to the running instance with:

	sftp -P 2222 username@localhost # adjust the port if needed, including in your compose file

On success, you should get a prompt such as:

	Welcome to Local EGA Demo instance
	Connected to localhost.
	sftp>

repository: EGA-archive/LocalEGA-inbox

gha_language: C

repo_name: EGA-archive/LocalEGA-inbox


text:
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
### Added
- nothing

### Changed
- nothing

### Removed
- nothing

### Fixed
- nothing

## [1.0.0] - 2019-04-01
### Change
- time trackers: implemented filter for lists (breaking change)
- time bookings: implemented filter for lists (breaking change)
- time logs: implemented filter for lists (breaking change)

## [0.4.0] - 2018-11-26
### Added
- error handling for bad request

## [0.3.0] - 2018-11-23
### Added
- time trackers: implemented bulk operations
- time bookings: implemented bulk operations
- time logs: implemented bulk operations

### Changed
- renamed properties for services in HourglassClient to be more convinient

## [0.2.0] - 2018-11-23
### Added
- Contribution guidelines, code of conduct and changelog
- time trackers: all missing operations (except bulk operations)
- time bookings: all missing operations (except bulk operations)
- time logs: all missing operations (except bulk operations)
- support for logging frameworks

### Changed
- renamed all service functions because "ById" is implicit clear
- Get throws NotFoundException instead of returning null when item was not found

## [0.0.2] - 2018-11-21
### Added
- basic implementation to read trackers and logs

### Changed
- compatibility to .NET Standard 2.0 as the web request stuff in .NET Standard 1.X doesn't allow to eveluate the response status code without parsing strings.
- naming of service classes - there will be no guaranteed compatibility to previous versions before release of version 1.0.0

## [0.0.1] - 2018-11-19
### Added
- basic implementation to read timebookings.
- based on .NET Standard 1.4 to provide high compatibility.

repository: DevelappersGmbH/RedmineHourglassNet

gha_language: C#

repo_name: DevelappersGmbH/RedmineHourglassNet


text:
# Keystores

This gem provides ruby implementations of different key stores. This was primarily created to provide the ability
to use many of the good Java key stores from ruby. This gem adds the key stores to the OpenSSL module structure,
since that is where the `OpenSSL::PKCS12` keystore lives.

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'keystores'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install keystores

## Usage

The API for this gem is modeled after the Java `KeyStore` class. All of the `KeyStore` implementations provided by this
gem conform to the `Keystores::Keystore` interface.

The certificate and key objects that these keystores return and expect are `OpenSSL::X509::Certificate` and
`OpenSSL::PKey` objects, respectively.

### Supported Key Store types

#### Java Key Store (jks) format

[Detailed documentation](JAVA_KEY_STORE_README.md)

##### Reading

This gem supports reading trusted certificate entries and private key entries. It can read
and decrypt RSA, DSA, and EC keys.

Example usage:

```
require 'keystores'
keystore = OpenSSL::JKS.new

# Load can take any IO object, or a path to a file
key_store_password = 'keystores'
keystore.load('/tmp/keystore.jks', key_store_password)

certificate = keystore.get_certificate('my_certificate')
key = keystore.get_key('my_key', key_store_password)

certificate.check_private_key(key)

certificate_chain = keystore.get_certificate_chain('my_key')
```

##### Writing

This gem supports writing trusted certificate entries and private key entries. It currently supports
writing DSA, RSA, and EC private key entries.

Example usage:

```
require 'keystores'
keystore = OpenSSL::JKS.new

key = OpenSSL::PKey::RSA.new(File.read('my_key.pem'))
cert_chain = OpenSSL::X509::Certificate.new(File.read('my_cert.pem'))
private_key_password = 'key_password'

keystore.set_key_entry('my-key', key, cert_chain, private_key_password)

key_store_password = 'keystores'
keystore.store('my_keystore.jks', key_store_password)
```

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/rylarson/keystores.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

repository: neftalyluis/keystores

repo_name: neftalyluis/keystores


text:
## 2.0.0 - Initial Release
- Merge string literals - single or double quote, `+`, `.`, or nothing as concat operator
- Merges line and docblock comments (supports `//`, `#`, `*`, and `;`)
- Removes trailing commas and omits spaces when merging onto a line with an opening or closing brace (`[]`, `{}`, `()`, or `<>`)

## 2.0.1
- README update

repository: ianobermiller/join-lines-smarter

repo_name: ianobermiller/join-lines-smarter


text:
# Webpack Customizations

The stark-build package uses several Webpack plugins as well as some utils in order to leverage lots of functionality and customizations to your builds (DEV and PROD).

This is the list of utils and plugins used by the stark-build package:

## Utils

#### Support for custom base URL for assets

Due to the possibility of defining a custom base url in your _index.html_ thanks to the [BaseHrefWebpackPlugin](https://github.com/dzonatan/base-href-webpack-plugin "BaseHrefWebpackPlugin"), it is also necessary to customize the path of the different assets (images, styles, etc.).

To do this, Stark uses the same custom logic implemented in the [@angular-devkit/build-angular](https://github.com/angular/devkit/blob/fe122511feada8d8c554799171e8e43bac950416/packages/angular_devkit/build_angular/src/angular-cli-files/models/webpack-configs/styles.ts "@angular-devkit/build-angular") to append the custom base url and the deploy url so you don't need to worry about the final path for every resource.

The only thing you need to do is to configure the **baseHref** and **deployUrl** options (if needed) in the _angular.json_ file as follows:

```json
{
  ...
  "projects": {
    "yourProjectName": {
      ...
      "architect": {
        "yourTargetName": {
          ...
          "options": {
            ...
            "baseHref": "/some-url",  // default value: "/"
            "deployUrl": "/some-url" // default value: ""
          }
        }
      }
    }
  }
}
```

## Plugins

#### [BaseHrefWebpackPlugin](https://github.com/dzonatan/base-href-webpack-plugin "BaseHrefWebpackPlugin")

Allows to customize the base url in the _index.html_ via the webpack config.

In stark-build, the custom base url provided to this plugin is the one you define in the **baseHref** option of your project's _angular.json_ file:

```json
{
  ...
  "projects": {
    "yourProjectName": {
      ...
      "architect": {
        "yourTargetName": {
          ...
          "options": {
            ...
            "baseHref": "/some-url"  // default value: "/"
          }
        }
      }
    }
  }
}
```

This plugin will automatically add the base tag `<base href="<custom-base-url>">` to the _index.html_ so you don't need to add it manually yourself.

#### [DefinePlugin](https://webpack.js.org/plugins/define-plugin)

Allows to define global variables that will be available during the compilation and building of the application bundles.

In this case, stark-build provides some global variables that are available at compilation time, which means that you can implement some checks in your code and this will be analyzed when your application bundle is being built by Webpack.

The global variables available at compilation time are the following:

-   `ENV` which indicates the current environment: `"development"` or `"production"`
-   `HMR` which indicates whether the Hot Module Replacement support is enabled (true/false).

Since the Define plugin defines those variables as global, you can use them everywhere in your code like this:

```typescript
if (ENV === "development") {
	/* the code inside this block will be executed only in development */
}
```

#### [HtmlElementWebpackPlugin](https://github.com/fulls1z3/html-elements-webpack-plugin "HtmlElementWebpackPlugin")

This webpack plugin appends head elements during the creation of index.html.

To use it, you'll have to create the `index-head-config.js` file to specify the `<link>` and `<meta>` you want to include in the `<head>` section of your index.html.

Create your file at the following location:

```txt
|
+---yourApp
|  	+---config
|	|	index-head-config.js
|   ...
```

Then, declare your file as follows:

```
module.exports = {
	link: [
		{ rel: "manifest", href: "manifest.json" },
		{ rel: "shortcut icon", type: "image/x-icon", href: "favicon.ico" },
		{ rel: "apple-touch-icon", sizes: "120x120", href: "assets/images/app-icons/apple-icon-120x120.png" },
		{ rel: "icon", type: "image/png", sizes: "32x32", href: "assets/images/app-icons/favicon-32x32.png" },
		...
	],
	meta: [
		...
		{ name: "apple-mobile-web-app-capable", content: "yes" },
		{ name: "apple-mobile-web-app-status-bar-style", content: "black" },
		{ name: "apple-mobile-web-app-title", content: "template" },
		{ name: "msapplication-config", content: "none" },
		...
	]
}
```

Finally, to indicate to your index.html file that you want to use this new file,
you will have to add the following lines in your `<head>` section:

```
<head>
...
	<% if (webpackConfig.htmlElements.headTags) { %>
	<%= webpackConfig.htmlElements.headTags %>
	<% } %>
...
</head>
```

_If you do not intend to use this way of working, simply don't create this file and
don't include the check for `webpackConfig.htmlElements.headTags` in the `<head>` section of index.html._

repository: Mallikki/stark

repo_name: Mallikki/stark


text:
# Node API Boilerplate

Tried to keep this one simple. It has:

- Express with body-parser
- ES2017 and ESLint
- iSparta for CI coverage
- Pre-commit hooks
- Nodemon for `dev`
- handy `config` util built in for different environments
- logging middleware with filtering

# Test Files

Test files should sit with their source and be in the format `*.spec.js`.

You can use nock and supertest for integration testing.

# Dev Environment

`npm run dev`

# Production

Make sure you have a `dist` folder.

`npm run build`

`npm run start`

# Health and Info Endpoints

`localhost:3000/health`

`localhost:3000/info`

repository: serinth/node-api-boilerplate

gha_language: JavaScript

repo_name: serinth/node-api-boilerplate


text:
# Attention mechanism
This repository looks to cover several of "well-known" approaches to attention mechanism. Methods that are being implemented are:

1. Global Attention

Based on paper [Effective Approaches to Attention-based Neural Machine Translation](https://arxiv.org/abs/1508.04025). It's focused on applying attention to Natural Language Translation using Seq2Seq model. The data set is simple English-French translation.

2. Self Attention

TODO

3. Multi-head Attention

TODO

## Resources

TODO
[Stanford tutorial](http://nlp.seas.harvard.edu/2018/04/03/attention.html)

[Original paper](https://arxiv.org/abs/1706.03762)

repository: bvezilic/Attention

repo_name: bvezilic/Attention


text:
---
layout: post
title:  "gem5-20.1 Roadmap"
author: Jason Lowe-Power and Bobby Bruce
date:   2020-07-01
categories: project
---

After our [successful release]({% post_url 2020-05-21-gem5-20 %}) of gem5-20, it's time to start thinking about gem5's next release, gem5-20.1!

We received great feedback during the [gem5 town hall](https://www.youtube.com/watch?v=fvCXmMBblZY) held with the [gem5 workshop](http://www.gem5.org/events/isca-2020) about the new features that the community would like to see for the next release.

Based on this feedback, below are the major projects, with links to the Jira issues, that we are planning to complete for the gem5-20.1 release.
We welcome any feedback on this roadmap!
Please use the [gem5-dev mailing list](/mailing_lists) to let us know what you think!

If you would like to suggest other features/fixes to be included in gem5-20.1, please mark them as "fix version" "gem5 20.1" on Jira.
You can find all of the current issues that we are planning to complete by the 20.1 release on [this Jira page](https://gem5.atlassian.net/projects/GEM5/versions/10003) (note: you may need be logged in to see the release page).

We're planning on releasing gem5-20.1 at the end of August.
We expect to create the staging branch and close the merge window on August 24, giving everyone a little more than 6 weeks to get these major changes done!

## Continuing to define gem5's APIs

In gem5-20, we started towards defining internal APIs for gem5.
You can see the work we did in the [Doxygen documentation](http://doxygen.gem5.org/release/current/modules.html).
In gem5-20, our goal was modest.
We only added information on [`SimObject`](http://doxygen.gem5.org/release/current/group__api__simobject.html) and the classes that [`SimObject`](http://doxygen.gem5.org/release/current/group__api__simobject.html) depends on.
Specifically, we codified APIs for [`SimObject`](http://doxygen.gem5.org/release/current/group__api__simobject.html), [`Drain`](http://doxygen.gem5.org/release/current/group__api__drain.html), [`Serialize`](http://doxygen.gem5.org/release/current/group__api__serialize.html), [`EventQueue`](http://doxygen.gem5.org/release/current/group__api__eventq.html), and [Statistics](http://doxygen.gem5.org/release/current/group__api__stats.html).

To add these to the API, we have tagged the methods that are part of the API with [Doxygen groups](https://www.doxygen.nl/manual/grouping.html).
For now, we will rely on code review to catch any changes to the API.
We will strive to keep APIs constant, but in order to improve the simulator, APIs will have to change.
However, APIs will only change at "major" releases, and we will add information to the release notes with details on how to update your code if it was using the changed API.

For gem5-20.1, we will be further expanding the methods covered by the API definitions.
We will focus on the following:

- Adding more documentation to the current APIs including the statistics, drain, serialize, event queue, and `SimObject`.
- Define the "external" APIs for the event queue. These are useful to other simulators that want to hook into gem5 either by driving the event queue themselves or having gem5's event queue drive the external simulator.
- The port interface. Ports are one of the most widely used objects in gem5. They facilitate communication between all memory system objects. This interface has changed somewhat over the past few years and [the documentation](http://www.gem5.org/documentation/learning_gem5/part2/memoryobject/) hasn't kept up. We will make sure this interface is well documented.
- "Base" utility classes. There are many classes in the `base/` directory that are used throughout the codebase. We will add these as APIs to make sure these widely used interfaces don't change without documentation.

We are also considering the best way to document the input and output of the simulator.
Specifically, we believe that the output files (e.g., `stats.txt`, `config.ini`, checkpoints, etc.) should be stable across releases.
However, Doxygen is probably not the right way to document these interfaces.
We'd love to hear your ideas on how best to document and enforce these interfaces.

See [this Jira Epic](https://gem5.atlassian.net/browse/GEM5-172) to follow the current progress on this issue.

## Improve statistics

The most common feedback we received at the gem5 users town hall was that statistics were difficult to use and not flexible enough.
To this end, we'll be working to [improve the statistics](https://gem5.atlassian.net/browse/GEM5-644) in gem5-20.1.

Our first step, will be to [convert the stats for all SimObjects](https://gem5.atlassian.net/browse/GEM5-645) to be "new style" statistics.
About a year ago, we added support for [hierarchical statistics](https://gem5-review.googlesource.com/c/public/gem5/+/19368).
This change significantly improves the usability of the statistics package.
However, we decided to keep backwards compatibility, and at the time, we didn't update all of the SimObjects to use this new API.
For gem5-20.1, we will migrate all of the SimObjects to use the new style, and we will officially deprecate the old API.

The other major feedback we received about statistics was that the current output was not easily machine readable.
To this end, we will [add new output formats that are more machine readable](https://gem5.atlassian.net/browse/GEM5-646).
The major formats we're currently considering are csv, something compatible with [pandas dataframes](https://pandas.pydata.org/pandas-docs/stable/getting_started/dsintro.html#dataframe), and [protobuf](https://developers.google.com/protocol-buffers) for streaming time series stats.

In addition to the new Jira issues linked above, there is some ongoing work with [changesets currently on gerrit](https://gem5-review.googlesource.com/c/public/gem5/+/28630/) (or recently merged).

## Improve python interfaces

The Python<->C++ interface is one of gem5's best features.
However, much of this code was written 10+ years ago, and there have been many additions to this code.
This is especially true of the "default" configuration files, `se.py` and `fs.py` which have grown to be 1000s of lines of code (when the imported files are included).

For gem5-20.1, we will begin to [refactor some of this python code](https://gem5.atlassian.net/browse/GEM5-432) to improve the design, add more documentation, and make things easier to use.

For now, this will be a *new* library (`gem5` instead of `m5`) to provide backwards compatibility for all of the configs that are currently used.

We will also [develop a "unit tests" interface](https://gem5.atlassian.net/browse/GEM5-433) for testing *python* level SimObjects.
This won't be *exactly* unit tests, but will allow SimObject designers to test their objects without having to run an entire gem5 simulation.
This will require building all of gem5 for now, but we'll think about a future where this isn't required.

We will [document the Python API](https://gem5.atlassian.net/browse/GEM5-647) with [pydoc documentation](https://docs.python.org/3.8/library/pydoc.html).
This will allow us to define this API more clearly.
We will also begin to include [Python type annotations](https://docs.python.org/3/library/typing.html).
However, since type annotations require Python 3.5+, we will initially just document this in the documentation strings, and move it into the main code at a later date.

Finally, we will start developing a [pure-Python model library](https://gem5.atlassian.net/browse/GEM5-648).
This model library will allow users to write simpler Python scripts to configure the simulator.
Eventually, this library will supplant `se.py` and `fs.py`.
Additionally, it is going to be the basis of gem5's "known-good configurations" that will be publicly validated models based on *real hardware*.

## Testing

There are a number of ways we would like to improve testing for the gem5-20 release.
One of the highest priority, is [replacing kokoro with a self-hosted CI solution](https://gem5.atlassian.net/browse/GEM5-37).
The Google-hosted kokoro process has been very helpful to catch bugs.
However, it's difficult for people outside of Google to configure, and currently it uses an old VM/docker container with GCC 4.8.
Moving to a self-hosted solution will give us more flexibility and control.
It will also help us accomplish some of our other testing goals.

Another goal is to get [nightly builds](https://gem5.atlassian.net/browse/GEM5-197) up and running.
We have both "quick" and "long" regressions marked, but we currently don't run most of the "long" regressions (they take 16-24 hours).
We are planning to run these nightly, but this wasn't possible with the kokoro system.
We're also going to look into adding other kinds of tests like testing to make sure gem5 builds will all of our supported compilers and running address sanitizers and other static analysis tools.

To accomplish these goals, we believe the best way forward is to use our own self-hosted [Jenkins](https://www.jenkins.io/) instance.
For now, we will run this instance on Google's cloud infrastructure.
However, one of the benefits of Jenkins is that we can easily move it to any public cloud or local compute resources.

We'll also be working on [fixing some failing integration tests](https://gem5.atlassian.net/browse/GEM5-361), fixing some of the failing [Linux boot tests](http://www.gem5.org/documentation/benchmark_status/), and continue to work on [increasing our unittest coverage](https://gem5.atlassian.net/browse/GEM5-4).

## Full system GPU support

As discussed in the [gem5 users workshop]({% post_url 2020-06-01-towards-full %}), developers at AMD are working to make it possible to use the GPU model in full system mode in gem5.
The goal is to be able to use the GPU model *with the upstream driver* and runtimes.
Currently, the SE mode-based GPU model is tied to an old version of the RoCM stack, and by implementing this full system support we'll be able to support newer runtime versions.

The current status of this support can be found [on this Jira Epic](https://gem5.atlassian.net/browse/GEM5-195).
There's a lot of little things to get done here, but the developers are hard at work on them!

## Memory system improvements

There are a number of improvements to gem5's memory system that we are targeting for gem5-20.1 including [an NVM model](https://gem5.atlassian.net/browse/GEM5-550), [support for transactional memory](https://gem5.atlassian.net/browse/GEM5-550), and a new Ruby protocol that supports a flexible cache hierarchy.

### NVM Model

Wendy Elsasser and other developers at Arm have been working on an improved memory controller that has both DRAM and non-volatile memory attached.
More details on this development can be found in her [gem5 workshop presentation]({% post_url 2020-05-29-memory-controller %}).

There are a set of patches [on gerrit](https://gem5-review.googlesource.com/c/public/gem5/+/29027/) right now for this new support.
We fully expect these to be integrated into gem5 before the gem5-20.1 release.

You can follow the development on [this Jira issue](https://gem5.atlassian.net/browse/GEM5-550).

### HTM Model

Hardware transactional memory has moved from research to products with [Intel](https://en.wikipedia.org/wiki/Transactional_Synchronization_Extensions) and ARM supporting some form of transactional memory in their ISAs.

There is current work to support HTM in both the [ISA models](https://gem5-review.googlesource.com/c/public/gem5/+/30314) and [Ruby](https://gem5-review.googlesource.com/c/public/gem5/+/30319).
We expect that these changesets on gerrit will be merged before the release of gem5-20.1.

You can follow the progress on the [Ruby Jira issue](https://gem5.atlassian.net/browse/GEM5-587) and the [ARM TME Jira issue](https://gem5.atlassian.net/browse/GEM5-588).

### Heterogarnet (Garnet 3.0)

As described in [the gem5 workshop talk]({% post_url 2020-05-29-heterogarnet %}) there has been work to extend Garnet to support 3D integration and other heterogeneous systems.
We will be working to merge this support into the gem5-20.1 release.

## A discussion on ISA support

gem5 supports a large number of ISAs.
This is one of gem5's best features, but the support for each ISA isn't uniform.
Each ISA has varying completeness, support for the latest revisions, fidelity, system architecture support, and testing.

We are going to solicit feedback on how to proceed with our ISA support during the gem5 20.1 development cycle.

This issue is about having a conversation about removing support for some ISAs.
Whether or not we remove these ISAs is up to the community at large.

## Other goals

- [Merge the gem5 GUI](https://gem5.atlassian.net/browse/GEM5-262)
- [Add a vector co-processor for RISC-V](https://gem5.atlassian.net/browse/GEM5-618)


text:
## The Seedbox From Scratch Script
#### Current version = 2.1.9
#### Last stable version = 2.1.8

This is a script I've being working for some time now. I decided to share it here because it has some good things to add:

* A multi-user enviroment, you'll have scripts to add and delete users.
* Linux Quota, to control how much space every user can use in your box.

## Installed software
* ruTorrent 3.4 + official plugins
* rTorrent 0.9.2 or 0.9.3 (you can choose, downgrade and upgrade at any time)
* Deluge 1.3.5 or 0.9.3 (you can choose, downgrade and upgrade at any time)
* libTorrrent 0.13.2 or 0.12.9
* mktorrent
* Fail2ban - to avoid apache and ssh exploits. Fail2ban bans IPs that show malicious signs -- too many password failures, seeking for exploits, etc.
* Apache (SSL)
* OpenVPN
* PHP 5 and PHP-FPM (FastCGI to increase performance)
* Linux Quota
* SSH Server (for SSH terminal and sFTP connections)
* vsftpd (Very Secure FTP Deamon)
* IRSSI
* Webmin (use it to manage your users quota)
* sabnzbd: http://sabnzbd.org/
* Rapidleech (http://www.rapidleech.com)

## Main ruTorrent plugins
autotoolscpuload, diskspace, erasedata, extratio, extsearch, feeds, filedrop, filemanager, geoip, history, logoff, mediainfo, mediastream, rss, scheduler, screenshots, theme, trafic and unpack

## Additional ruTorrent plugins
* Autodl-IRSSI (with an updated list of trackers)
* A modified version of Diskpace to support quota (by me)
* Filemanager (modified to handle rar, zip, unzip, tar and bzip)
* Fileupload
* Fileshare Plugin (http://forums.rutorrent.org/index.php?topic=705.0)
* MediaStream (to watch your videos right from your seedbox)
* Logoff
* Theme: Oblivion

## Before installation
You need to have a "blank" server installation, if you have a Kimsufi, just do a "reinstall" on it, using Ubuntu Server 12.04.
After that access your box using a SSH client, like PuTTY.

## Warnings

####If you don't know Linux ENOUGH:

DO NOT install this script on a non OVH Host. It is doable, but you'll have to know Linux to solve some problems.

DO NOT use capital letters, all your usernames should be written in lowercase.

DO NOT upgrade anything in your box, ask in the thread before even thinking about it.

DO NOT try to reconfigure packages using other tutorials.

## How to install
Just copy and paste those commands on your terminal:

```
wget -N https://raw.github.com/Notos/seedbox-from-scratch/v2.1.9/seedbox-from-scratch.sh
time bash ~/seedbox-from-scratch.sh
```

####You must be logged as root to run this installation or use sudo on it.

## Commands
After installing you will have access to the following commands to be used directly in terminal
* createSeedboxUser
* deleteSeedboxUser
* changeUserPassword
* installRapidleech
* installOpenVPN
* installSABnzbd
* installWebmin
* installDeluge
* updategitRepository
* removeWebmin
* upgradeRTorrent
* installRTorrent
* restartSeedbox

* While executing them, if sudo is needed, they will ask for a password.

## Services
To access services installed on your new server point your browser to the following address:
```
https://<Server IP or Server Name>/seedboxInfo.php
```

####OpenVPN
To use your VPN you will need a VPN client compatible with [OpenVPN](http://openvpn.net/index.php?option=com_content&id=357), necessary files to configure your connection are in this link in your box:
```
http://<Server IP or Server Name>/rutorrent/vpn.zip` and use it in any OpenVPN client.
```

## Supported and tested servers
* Ubuntu Server 12.10.0 - 64bit (on VM environment)
* Ubuntu Server 12.04.x - 64bit (on VM environment)
* Ubuntu Server 12.04.x - 32bit (OVH's Kimsufi 2G and 16G - Precise)
* Ubuntu Server 12.04.x - 64bit (OVH's Kimsufi 2G and 16G - Precise)
* Debian 6.0.6 - 32 and 64bit (OVH's Kimsufi 2G - Squeeze)
* Debian 6.0.6 - 32 and 64bit (on VM environment)

## Quota
Quota is disabled by default in your box. To enable and use it, you'll have to open Webmin, using the address you can find in one of the tables box above this. After you sucessfully logged on Webmin, enable it by clicking

System => Disk and Network Filesystems => /home => Use Quotas? => Select "Only User" => Save

Now you'll have to configure quota for each user, doing

System => Disk Quotas => /home => <username> => Configure the "Soft kilobyte limit" => Update

As soon as you save it, your seedbox will also update the available space to all your users.

## Changelog
Take a look at seedbox-from-scratch.sh, it's all there.

## Support

There is no real support for this script, but people is talking a lot about it [here](http://www.torrent-invites.com/seedbox-tutorials/207635-seedbox-scratch-script-multi-user-quota-sabnzbd-deluge.html) and you may find solutions for your problems in the thread.

## License

Copyright (c) 2013 Notos (https://github.com/Notos/)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--> Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php

repository: Notos/seedbox-from-scratch

repo_name: Notos/seedbox-from-scratch


text:
# Table of Contents
1. [Title](#title)
2. [Description](#description)
3. [Installation](#installation)
4. [Contributing](#contributing)
5. [Test](#test)
6. [License](#license)
7. [Questions](#questions)

# The title of the project is ReadMe

# Directions on how to make a markdownfile

# The project is installed by Via Node

# The guidelines are: Add what you like

# Was a test added? Yes

# Licence used : [![made-with-Apache](https://img.shields.io/badge/Apache-License-yellow)](https://www.apache.org/licenses/LICENSE-2.0.txt)

# Questions
  * My GitHub username is <https://github.com/sbergman98>
  * Please email me at<EMAIL_ADDRESS>with anymore questions

  [Return to Top](#return)


text:
# Project 7 - WordPress Pentesting

Time spent: 5 hours spent in total

> Objective: Find, analyze, recreate, and document **five vulnerabilities** affecting an old version of WordPress

## Pentesting Report

1. Vulnerability Name or ID: Authenticated Stored Cross-Site Scripting (XSS) in YouTube URL Embeds
  - [X] Summary: An embedded YouTube link with a script in the URL creates a XSS attack
    - Vulnerability types: XSS
    - Tested in version: 4.2
    - Fixed in version: 4.2.13
  - [X] GIF Walkthrough:
  - [X] Steps to recreate: Embed a YouTube link with an onload script in a page or post. Update/post the changes, and then view the post/page. A XSS alert will appear once the page or post is opened.
  - [X] Affected source code:
    - https://github.com/WordPress/WordPress/commit/419c8d97ce8df7d5004ee0b566bc5e095f0a6ca8

2. Vulnerability Name or ID: Authenticated Shortcode Tags Cross-Site Scripting (XSS)
  - [X] Summary: A XSS attack is executed upon hovering the mouse over a caption
    - Vulnerability types: XSS
    - Tested in version: 4.2
    - Fixed in version: 4.2.5
  - [X] GIF Walkthrough:
  - [X] Steps to recreate: Embed a caption with a link that contains an onMouseOver script. Upload the post, and upon hovering over the caption a XSS alert will appear.
  - [X] Affected source code:
    - https://blog.checkpoint.com/2015/09/15/finding-vulnerabilities-in-core-wordpress-a-bug-hunters-trilogy-part-iii-ultimatum/
3. Vulnerability Name or ID: Unauthenticated Genericons Cross-Site Scripting (XSS)
  - [X] Summary: Embedding a link to 2015 genericon theme with a script in the URL allows for a XSS attack
    - Vulnerability types: XSS
    - Tested in version: 4.2
    - Fixed in version: 4.2.2
  - [X] GIF Walkthrough:
  - [X] Steps to recreate: Embed a link to a 2015 generic theme with an onerror script in the URL. Once the page is posted, a XSS alert will appear when it is opened.
  - [X] Affected source code:
    - https://codex.wordpress.org/Version_4.2.2

## Resources

- [WordPress Source Browser](https://core.trac.wordpress.org/browser/)
- [WordPress Developer Reference](https://developer.wordpress.org/reference/)

GIFs created with [LiceCap](http://www.cockos.com/licecap/).

## License

    Copyright [yyyy] [name of copyright owner]

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.


text:
# Express-React-Redux-Boilerplate

## Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

### Prerequisites

* [PostgreSQL](https://www.postgresql.org/)
* [Node.js](https://nodejs.org/en/)

### Installing

A step by step series of examples that tell you how to get a development env running from terminal.

Instructions on how to clone a repo [here](https://help.github.com/articles/cloning-a-repository/)

Clone down [this](https://github.com/goodalls/express-react-redux-boilerplate) repo.

To install the dependencies:

```
npm install
```

To fire up a development and api server:

```
npm start
```

Once the server is running, you can visit:

* Open http://localhost:3000 to view client in the browser.
* Open http://localhost:3001 for api requests.

## Running the tests

there are two automated testing systems in use in this project. The front end uses Jest/Enzyme and the back-end uses Mocha/Chai. To run the testing suite use the following script for each respectivly.

* to test the front-end with Jest/Enzyme
  * automaticaly searches the ./src folder for files with the correct extension ex. ./src/components/App/App.test.js
```
npm test
```

* to test the Back-End with Mocha and Chai
  * only uses the ./src/db/test/routes.spec.js file

```
npm run test-api
```

## Deployment

Set up to use Continuous integration with TravisCI and Heroku (needs configured to your project)

## Built With

* [create-react-app](https://github.com/facebook/create-react-app)
* [express](https://expressjs.com/)
* [knex](https://knexjs.org/)
* [Redux](https://redux.js.org/)

## Authors

* **Jeff Goodall** - *Front End Developer* - [GitHub](https://github.com/goodalls)

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

## Acknowledgments

* **Carole Wood** - *[mkibags.com](mkibags.com)*


text:
Updating process
================

* In the outer directory install latest rails, using latest ruby:

    ```
    $ rvm get stable
    $ rvm use ruby-prev.version
    $ rvm gemset delete mayak
    $ rvm install ruby-new.version
    $ rvm --default use ruby-new.version
    $ rvm gemset create mayak
    $ rvm gemset use mayak
    $ gem install bundler
    $ gem install --no-rdoc --no-ri rails -v rails.new.version
    $ rails new mayak --skip-bundle --skip-test-unit --database=postgresql
    $ mv mayak rails_new.version
    ```

* Save this directory until next update unchanged - any chages do with a copy

* Generate scaffold for test in the new directory:

    ```
    $ bin/bundle install
    $ bin/rails generate scaffold Author name:string
    $ bin/rails generate scaffold Article author:belongs_to source:references{polymorphic} title:string slug:string:index published_at:datetime:index image:string intro:text body:text seodata:text hided:boolean:index
    $ bin/rake db:drop db:create db:migrate
    ```

* Save scaffolded version for future

* Diff with previous scaffolded rails and analyse changes for update current version of Mayak

* Copy new rails files (without scaffold) to mayaknew folder

* Merge 'Gemfile' from old Mayak with updating versions of gems

* Run `bin/bundle install --without production` and generate all configs:

    ```
    bin/rails g active_admin:install
    bin/rails g kaminari:config
    bin/rails g uploader Avatar
    bin/rake sitemap:install
    bin/bundle exec wheneverize .
    bin/bundle exec mina init
    ```

* Carefully merge Mayak to new instance

* Remove old Mayak files except '.git' directory

* Move all new Mayak files to old directory with '.git' inside

* Run `bin/rake db:drop db:create db:migrate db:seed`

* Update information about software versions in the [readme file](README.md)
    and in the [admin dashboard file](app/admin/dashboard.rb).

* Check all changes using `git diff` : we need to be sure that we didn't lost anything important

* Run `bin/rails server` and check result manually

* `git add . && git commit -m "Update to Ruby new.version and Rails new.version" && git push` - Le Roi est mort, vive le Roi!

repository: dymio/mayak

gha_language: Ruby

repo_name: dymio/mayak


text:
| Supported Targets | ESP32 |
| ----------------- | ----- |

# Musical Tesla Coil firmware

This code allows direct streaming of Bluetooth audio to a Tesla Coil driver and power amplifier through pin IO32.

It is derived from bt_stream demo by Expressif, which handles the connection to the BT client and provides a stream of stereo, 16 bit PCM data.

The modified code converts the stream to mono and interpolates it to the resonant frequency of the Tesla Coil, which in my case is close to 770 kHz. The actual frequency can be fine adjusted with a potentiometer wired to the ADC1 input of the ESP32.

After interpolation, the audio sequence is reduced to a 1 bit flow using a 2nd order sigma-delta approach (function sigma_delta_encode in bt_app_core.c). Then the sequence is doubled, putting a 0 bit after each original bit.

Finally, the ~1540 kHz stream is output on a physical digital output using the I2S interface, clocked by the high accuracy APLL. A tricky routine adjust_pll allows to reconfigure the frequency on the fly when the potentiometer is adjusted, without requiring to reinitialize the I2S subsystem as standard APIs require.

Tesla Coil center frequency is defined in main.c, line 51 (long fOut = xxx).

A frequency much higher than this may not be supported without software optimizations, because the two cores are already quite loaded.

## Build instructions (Windows - similar on Linux)

Install the Expressif esp-idf environment and enter the ESP-IDF command prompt

Enter the project directory TeslaCoil_BT

give the command "idf.py build"

connect ESP32 target with USB, check the configured serial port (in my case COM13)

download code with command "idf.py flash -p COM13"

you can see debug info with command "idf.py monitor -p COM13"

all previous commands can be combined after a modification, e.g. "idf.py build flash monitor -p COM13"

Follows the original README.md from Expressif bt_stream demo
------------------------------------------------------------

ESP-IDF A2DP-SINK demo
======================

Demo of A2DP audio sink role

This is the demo of API implementing Advanced Audio Distribution Profile to receive an audio stream.

This example involves the use of Bluetooth legacy profile A2DP for audio stream reception, AVRCP for media information notifications, and I2S for audio stream output interface.

Applications such as bluetooth speakers can take advantage of this example as a reference of basic functionalities.

## How to use this example

### Hardware Required

To play the sound, there is a need of loudspeaker and possibly an external I2S codec. Otherwise the example will only show a count of audio data packets received silently. Internal DAC can be selected and in this case external I2S codec may not be needed.

For the I2S codec, pick whatever chip or board works for you; this code was written using a PCM5102 chip, but other I2S boards and chips will probably work as well. The default I2S connections are shown below, but these can be changed in menuconfig:

| ESP pin   | I2S signal   |
| :-------- | :----------- |
| GPIO22    | LRCK         |
| GPIO25    | DATA         |
| GPIO26    | BCK          |

If the internal DAC is selected, analog audio will be available on GPIO25 and GPIO26. The output resolution on these pins will always be limited to 8 bit because of the internal structure of the DACs.

### Configure the project

```
idf.py menuconfig
```

* Set the use of external I2S codec or internal DAC for audio output, and configure the output PINs under A2DP Example Configuration

* Enable Classic Bluetooth and A2DP under Component config --> Bluetooth --> Bluedroid Enable

### Build and Flash

Build the project and flash it to the board, then run monitor tool to view serial output.

```
idf.py -p PORT flash monitor
```

(To exit the serial monitor, type ``Ctrl-]``.)

## Example Output

After the program is started, the example starts inquiry scan and page scan, awaiting being discovered and connected. Other bluetooth devices such as smart phones can discover a device named "ESP_SPEAKER". A smartphone or another ESP-IDF example of A2DP source can be used to connect to the local device.

Once A2DP connection is set up, there will be a notification message with the remote device's bluetooth MAC address like the following:

```
I (106427) BT_AV: A2DP connection state: Connected, [64:a2:f9:69:57:a4]
```

If a smartphone is used to connect to local device, starting to play music with an APP will result in the transmission of audio stream. The transmitting of audio stream will be visible in the application log including a count of audio data packets, like this:

```
I (120627) BT_AV: A2DP audio state: Started
I (122697) BT_AV: Audio packet count 100
I (124697) BT_AV: Audio packet count 200
I (126697) BT_AV: Audio packet count 300
I (128697) BT_AV: Audio packet count 400

```

Also, the sound will be heard if a loudspeaker is connected and possible external I2S codec is correctly configured. For ESP32 A2DP source example, the sound is noise as the audio source generates the samples with a random sequence.

## Troubleshooting
* For current stage, the supported audio codec in ESP32 A2DP is SBC. SBC data stream is transmitted to A2DP sink and then decoded into PCM samples as output. The PCM data format is normally of 44.1kHz sampling rate, two-channel 16-bit sample stream. Other decoder configurations in ESP32 A2DP sink is supported but need additional modifications of protocol stack settings.
* As a usage limitation, ESP32 A2DP sink can support at most one connection with remote A2DP source devices. Also, A2DP sink cannot be used together with A2DP source at the same time, but can be used with other profiles such as SPP and HFP.


text:
# Optimizing Course Planning for MIT Students

⮂ Press Arrow Keys To Transition ⮂

---

![Problem](./1.png)

---

![Solution](./2.png)

---

> Whenever you are designing optimization models, it’s typically impossible to
> capture every single aspect of the problem
> <cite>Arthur Delarue </cite>

---

![Heart of FireRoad053](./3.png)

---

# ❤

---

![Features](./4.png)

---

![Example Output](./5.png)

---

![Insights](./6.png)

---

![Future Improvements](./7.png)

---

# 👏

---

# 🤗

---

# 😊

---

# 🎈

---

# 🙌

---

# 🎆

---

# 🎇

---

# 🥳

---

# 🎊

---

# The End 🎊

---