text:
# The Gossamer Protocol

## Serialization Formats

**Actions** are serialized as JSON-encoded strings inside of a **Message**.
Example message:

```json
{
  "verb": "AppendKey",
  "provider": "foo",
  "public-key": "..."
}
```

**Messages** that are signed by a **provider** become a **SignedMessage**.
Example signed message:

```json
{
  "signature": "...",
  "message": "{\"verb\":\"AppendKey\",\"provider\":\"foo\",\"public-key\":\"...\"}",
  "provider": "foo",
  "public-key": ""
}
```

Note that the outermost `public-key` is blank on the first `AppendKey`
operation for a given **provider**.

**SignedMessages** are serialized as a JSON string and stored in the ledger.

Although pretty-printing was used in the examples above, it is strictly optional.
In libgossamer, we omit pretty-printing in favor of a flat, one-line JSON string,
like so:

```json
{"signature":"...","message":"{\"verb\":\"AppendKey\",\"provider\":\"foo\",\"public-key\":\"...\"}","provider":"foo","public-key":""}
```

At the deepest level, `signature` covers the `message` contents, as a literal
string. Signatures MUST be base64url-encoded Ed25519 signatures.

Base64url is defined in [RFC 4648](https://tools.ietf.org/html/rfc4648).
Ed25519 is defined in [RFC 8032](https://tools.ietf.org/html/rfc8032).

Each ledger may impose additional internal serialization rules. Gossamer treats
these as a black box:

 * Signed messages are serialized and written to the ledger using an appropriate
   **scribe** class.
 * These entries are read from the ledger with an appropriate **source** class,
   and deserialized as a `SignedMessage` object.

## The Super Provider

Typically, providers manage their own signing keys and share the associated
verification keys with the world. Therefore, all **actions** related to any
public key or software update owned by that provider **SHOULD** be signed by that
provider's keys.

However, many systems will need an emergency "break glass" feature to allow
providers to recover from key loss or compromise. To meet this real-world
operational need, Gossamer allows a single special provider (dubbed the
**Super Provider**) the power to sign operations on behalf of other providers.

The Super Provider **MUST NOT** be a regular provider that sees routine,
non-emergency operational usage.

For example, if the entity that manages the Super Provider is FooBar LLC, which
occupies the **foo-bar** supplier, then the Super Provider **MUST NOT** be
**foo-bar**. Instead, a separate provider identity (e.g. **foo-bar-emergency**)
should be declared, with its own set of signing keys, for the sole purpose
of handling emergency situations.

The Super Provider **MAY** issue any action (`AppendKey`, `RevokeKey`,
`AppendUpdate`, `RevokeUpdate`) for any provider. However, it **CANNOT** bypass
the requirement for all transactions to be published in the append-only
cryptographic ledger.

## Actions and Validation Rules

### AppendKey

An `AppendKey` action **MUST** contain the following fields:

| Name         | Description                         |
|--------------|-------------------------------------|
| `verb`       | Must be `AppendKey`.                |
| `provider`   | Provider name.                      |
| `public-key` | Base64url-encoded verification key. |

An `AppendKey` action **MAY ALSO** Contain the following optional fields:

| Name         | Description                                                        |
|--------------|--------------------------------------------------------------------|
| `meta`       | JSON-encoded string of optional metadata (e.g. Git commit hashes). |
| `limited`    | Boolean (True = Limited key, False = Normal key). Default: False.  |
| `purpose`    | String. Used for identifying keys used for non-Gossamer purposes.  |

If the `provider` is not found in the local key store when an `AppendKey`
action is encountered, then it **MUST** be created in the local key store.

If the `provider` was found, the `SignedMessage` that encapsulates this Action
must be signed by the same provider (or the Super Provider, if applicable).

An `AppendKey` action **MAY** contain a `limited` field, which must be boolean.
If it is absent, it is implicitly false. The first `AppendKey` for a provider
**MUST NOT** have `limited` set to `TRUE`.

An `AppendKey` action **MAY** also contain a `purpose` field, which must be a
valid UTF-8 string. Any key with a `purpose` **MUST NOT** be used with any other
Gossamer action. The `purpose` field is used to identify public keys shared through
Gossamer that are only intended for non-Gossamer uses (i.e. not for software
updates).

The provider must have at least one non-limited, non-revoked key without a
`purpose` field in order to create limited keys.

**Limited keys can only be used to issue an `AppendUpdate` action.**

**Keys with a `purpose` field cannot be used in Gossamer.**

When this action is performed, it should insert a new row in a database.

### RevokeKey

A `RevokeKey` action **MUST** contain the following fields:

| Name         | Description                         |
|--------------|-------------------------------------|
| `verb`       | Must be `RevokeKey`.                |
| `provider`   | Provider name.                      |
| `public-key` | Base64url-encoded verification key. |

If the `provider` is not found in the local key store when a `RevokeKey` action
is encountered, an error **MUST** be raised. In most languages, this means
throwing an Exception.

If the `public-key` does not belong to the correct `provider` (or the Super
Provider, if applicable), an error **MUST** be raised. In most languages, this
means throwing an Exception.

The `SignedMessage` that encapsulates this Action must be signed by the provider
(or the Super Provider, if applicable).

When this action is performed, it should update an existing row in a database
to mark the row as revoked.

### AppendUpdate

An `AppendUpdate` action **MUST** contain the following fields:

| Name         | Description                              |
|--------------|------------------------------------------|
| `verb`       | Must be `AppendUpdate`.                  |
| `provider`   | Provider name.                           |
| `public-key` | Base64url-encoded verification key.      |
| `signature`  | Signature of the update file.            |
| `package`    | Name of the package (owned by provider). |
| `release`    | Version being released.                  |
| `artifact`   | Build artifact type being released.      |

An `AppendUpdate` action **MAY ALSO** Contain the following optional fields:

| Name         | Description                                                        |
|--------------|--------------------------------------------------------------------|
| `meta`       | JSON-encoded string of optional metadata (e.g. Git commit hashes). |

If the `provider` is not found in the local key store when an `AppendUpdate`
action is encountered, an error **MUST** be raised. In most languages, this
means throwing an Exception.

The `SignedMessage` that encapsulates this Action must be signed by the
correct provider (or the Super Provider, if applicable).

When this action is performed, it should insert a new row in a database.

The `artifact` is optional. It can be used to distinguish different deliverables for different
platforms or distribution channels (e.g. for `.zip` files vs `.patch` files).

### RevokeUpdate

A `RevokeUpdate` action **MUST** contain the following fields:

| Name         | Description                              |
|--------------|------------------------------------------|
| `verb`       | Must be `RevokeUpdate`.                  |
| `provider`   | Provider name.                           |
| `public-key` | Base64url-encoded verification key.      |
| `package`    | Name of the package (owned by provider). |
| `release`    | Version being released.                  |
| `artifact`   | Which build artifact type to revoke.     |

If the `provider` is not found in the local key store when an `RevokeUpdate`
action is encountered, an error **MUST** be raised. In most languages, this
means throwing an Exception.

The `SignedMessage` that encapsulates this Action must be signed by the
correct provider (or the Super Provider, if applicable).

When this action is performed, it should update an existing row in a database
to mark the row as revoked.

### AttestUpdate

A `AttestUpdate` action **MUST** contain the following fields:

| Name          | Description                              |
|---------------|------------------------------------------|
| `verb`        | Must be `AttestUpdate`.                  |
| `provider`    | Provider name.                           |
| `package`     | Name of the package (owned by provider). |
| `release`     | Version of the package in question.      |
| `artifact`    | Which build artifact type to attest.     |
| `attestor`    | Provider attesting this package.         |
| `attestation` | See below.                               |

If the `provider` is not found in the local key store when an `AttestUpdate`
action is encountered, an error **MUST** be raised. In most languages, this
means throwing an Exception.

If the `attestor` is not found in the local key store when an `AttestUpdate`
action is encountered, an error **MUST** be raised. In most languages, this
means throwing an Exception.

An attestation is a [NOP](https://en.wikipedia.org/wiki/NOP_(code)) for the
purposes of Gossamer's goals (key management and code-signing). However, other
protocols **MAY** wish to use attestations in order to provide third-party
oversight into the protocol.

Attestations **MUST** be one of the following:

| Attestation    | Meaning                                                              | Connotation |
|----------------|----------------------------------------------------------------------|-------------|
| `vote-against` | The attestor claims that this update should NOT be installed.        | Negative    |
| `reproduced`   | The attestor was able to reproduce this update from the source code. | Positive    |
| `spot-check`   | The attestor provided a spot check for this update.                  | Positive    |
| `code-review`  | The attestor performed a code review for this update.                | Positive    |
| `sec-audit`    | This update passed a security audit performed by the attestor.       | Positive    |

When implementing support for attestations in a protocol that builds atop
Gossamer, you **MAY** treat the above as an enum.

**Warning**: Attestations are immutable and **CANNOT** be revoked.

#### Negative Attestations

* If you want to advise users to not install an update, simply issue a `vote-against`
  attestation. This is useful for identifying that an old version of software SHOULD NOT
  be installed upon the discovery of a security flaw. However, unlike revocation,
  it will only take effect if users trust your `vote-against` attestations and configure
  them into their [attestation policies](../reference/Client/AttestPolicy.md).

#### Positive Attestations

* Reproducing a build package from the source code (`reproduced`) is important,
  but can be performed without human attention.
* A `spot-check` can be done in a hurry, and only certifies that nothing leaped
  out to the reviewer as "obviously malicious or broken".
* A code review (`code-review`) means that someone looked over the contents of
  the release and evaluated their contents carefully. It doesn't necessarily
  imply any sort of security guarantees; merely that this update was peer
  reviewed.
* A security audit (`sec-audit`) is the highest level of attestation that a
  third party can provide. It implies that not only was the release reviewed
  by an independent third party (like a `code-review`), but also that the third
  party in question claims to have discovered no security vulnerabilities in
  this particular version of the software.

## Invalid Messages in the Ledger

Any records in the ledger that do not contain a valid JSON message
**MUST** be skipped.

Any records in the ledger that contain a valid JSON message, but
do not contain a `verb` field conforming to one of the actions defined in
this specification, **MUST** be skipped.

The above two rules allow a single ledger instance to be used for
multiple purposes without interfering with the normal operation of the
Gossamer protocol.

### Validation

Any records that contain a JSON message and a `verb` field
conforming to an action, but are somehow invalid (as defined by the rules
of each particular `verb`; e.g. missing a mandatory field) but **NOT**
malicious (e.g. not violating a rule about who can author an `AppendKey`),
**MAY** be skipped, but the discrepancy **SHOULD** also be logged.

Alternatively, this can be a protocol error until the operator manually
skips the invalid message by fast-forwarding to the next ledger record.

### Security

Any records that obey the JSON message rules but violate the security
requirements of an action (e.g. trying to append an update for a different
provider, except if issued by the Super Provider) must constitute a protocol
error.

### Protocol Error Handling

The protocol will be stopped until a manual action is performed.

To recover, an operator **MUST** discard the malicious message in order and
fast-forward to the next ledger record. The operator **SHOULD** investigate
the cause of the breakage to ensure they are not the subject of a targeted
attack.

In the event of a network attack that led to invalid records being
replicated, operators **MAY** empty their keystore and replay the entire
protocol from the first record once they are sure the network is consistent
and trustworthy.

Tooling **SHOULD** be provided to allow an operator to specify the new
savepoint (summary hash, Merkle root, etc.) for the cryptographic ledger
powering the Gossamer protocol.

license: ISC

repository: paragonie/libgossamer

repo_name: paragonie/libgossamer


text:
# Design of a Content Delivery Network - CDN

These notes are supposed to help in designing a content delivery network, aka CDN,
from scratch. The solution needs to be scalable for millions of clients to use, with
billions of objects being used at the edges. It should also support features like
cache expiration, forced cache-expiration, custom time-to-live values (TTLs) and more.
It needs to support authenticated requests as well.

Before we being on the design of a CDN, let's see what is a CDN and how does a CDN
works to understand it better.

## What is a CDN

A Content-Delivery-Network is a proxy server, between the user's browser and the
actual server that is intended to speed-up serving resources by reducing the latency
between the browser and server. Thus, a CDN has servers across many countries (called
as edge locations) that serve browser clients from within their own geographical
fence. As the edge server is close to browser, the network latencies are reduced
drastically.

Sometimes CDN networks are also meant to take load-off the actual server, specially
for blob-storage kind of usages (images, thumbnails, binaries and all) - but this is
now known as blob-stores. CDNs may be deployed along with blob-stores to still reduce
the load and latencies between actual blob server and the end-user.

The source of truth in CDN world is called the origin server. This is the server from
where a CDN edge can always fetch the latest and greatest source of content, and is
always supposed to be available for a smaller number of requests. Remember, CDN is also
meant to take-off load of the origin servers.

A typical request flow with CDN edges can be visualized as:

```
                                        Actual Server
                                        Origin Server
                                              |
	   +----------------------------------+------------------------------------+
	   |                                  |                                    |

	Edge US 			 Edge Europe				Edge Asia

           |                                  |                                    |
     +-----+-+                        +---------------+                     +-------------+
     |       |                        |               |                     |             |

    CA    New-York                 England         Turkey                 India         Japan

```

There are two modes of working of a CDN server:

### 1. Push CDN

In a push CDN the origin server knows the CDN system and wants its files to be cached
eagerly - like Apple OS updates, Android OS updates, Adobe creative cloud binary
images - here we pay first and make sure that there is no bottleneck for the user. So
origin itself pushes the content ahead-of-time to the CDN system, which is turn propogates
it to multiple edge servers. This propogation usually takes a few minutes to replicate
around the world (also depending on the file size).

The files in this case are cached eternally, until the origin explitly does not update
them or delete them. The origin can also ask the CDN to refresh the file in this case,
which is nothing else but pushing the file again to all edge servers of the world.

### 2. Pull CDN

In a pull CDN, the content is lazily fetched. In this case the origin only pushes the
configuration to the CDN edge servers. Now when a request lands up on an edge server
the very first time, it goes ahead and fetches the response from the origin server,
caches it locally and serves to the browser. There is a TTL (via caching headers,
`max-age` header, `expires` header, CDN config or otherwise) associated with each file
(or CDN account depending on plan purchased). After this TTL has expired, the edge
server is mandated to fetch the file again from the origin server.

However, an edge server may revoke the file before the TTL if it deems that no new
request every came for the while to this edge server and it needs to free up space for
other requests. In such a case the edge server will make another request to origin
before the TTL actually expired.

This is cheaper due to the same edge server being reused for other requests and keeping
disk space low.

## Design Notes

Now that we understand what a CDN is, let's talk about the different components the
system will have, and the nuances of the same.

In most scenarios, a CDN system servers only `GET` requests and `POST`, `PUT` and other
VERB calls are handled by origin directly.

For the sake of this design we will assume that our Edge servers are deployed at `edge.
cdn.com` and that the origin server is available at `origin.my.com`. We will first
discuss the scenrio for a single origin server being served by a given edge server and
then extend it to multiple servers.

Let's assume the following value-object that will help us respond to requests on the
edge server. For simplicity sake, we will assume that we are using a `Servlet` based
system to code this CDN system.

We will first start with a `pull-CDN` and then extend it to the `push-CDN`.

```java
public class MyEdgeFacingServlet extends HttpServlet {

	/**
	 * This is the cache service that we use to check data in local cache at edge
	 * server. As CDN serves blobs the cache service is expected to be a scalable
	 * caching service, something on the lines, of a distributed cache.
	 */
	private CacheService cacheService;

	/**
	 * This is the service that we use to make the HTTP requests to origin server.
	 */
	private HttpService httpService;

	/**
	 * This service basically gets all the configuration related to the incoming
	 * query by identifying the origin host and other parameters.
	 */
	private CdnConfigService cdnConfigService;

	/**
	 *
	 * This method takes an incoming call and then responds to it by looking
	 * for something in the local cache. If present, it will return the same. If not
	 * present it will then query the origin server, get the response, and store in
	 * local cache, and serve to client.
	 *
	 */
	@Override
	public void doGET(HttpServletRequest request, HttpServletResponse response) {
		// find host for which we are proxying
		String incomingHost = RequestUtils.findIncomingHost(request);

		// there was nothing in cache - let's hit the origin server now
		CdnConfig config = this.cdnConfigService.getConfigForOrigin(incomingHost);

		// build a cache key that is unique to this request, including the URI path
		// and other request parameters
		final String cacheKey = RequestUtils.getCacheKey(request);

		// get the cached response
		CachedResponse cached = this.cacheService.getCachedResponseForKey(cacheKey);

		// was something in cache
		if(cached != null) {

			// copy this cached response to client
			ResponseUtils.copyResponse(response, cached);

			// we are done
			return;
		}

		// based on this config build the query for origin server
		HttpRequest originRequest = RequestUtils.proxyRequest(config, request);

		// fire this request
		HttpResponse originResponse = this.httpService.makeRequest(originRequest);

		// because CDN is meant for speed let's not delay the browser
		ResponseUtils.copyResponse(response, originResponse);

		// check if we have a valid response or not
		// depending on the configuration of origin settings or otherwise
		// say a 404 may be a valid response or not
		boolean canCacheResponse = CacheUtils.canCacheResponse(config, originResponse);

		if(!canCacheResponse) {
			// there is nothing to cache
			return;
		}

		// we need to cache this response
		this.cacheService.putInCache(cacheKey, originResponse);
	}

}
```

In the code above, we are trying to handle all `GET` calls that land up on the edge server
by checking in cache. If nothing is found, a similar `GET` call is made to origin server to
fetch the data from the origin server. It is first written to the client browser to have it
proceed ahead (as CDN is all about speed), and then the response is written to the distributed
cache.

### Scaling for multiple origins and clients

As in the code above, we use the following code line to find the origin server that we need
to hit to fetch the source of truth:

```java
String incomingHost = RequestUtils.findIncomingHost(request);
```

This is based on the host-name of the incoming request. CDN servers are transparent to the
end user and thus, always respond on the origin server's host name, say `cdn.my.com`.

We thus can code this method to use a mapping between different incoming host-names and the
associated customer/origin-host for them. In psuedo-code it could be as simple as:

```java
public class RequestUtils {

	private static final Map<String, String> map = new HashMap<>();

	/**
	 * Called when this instance is initialized
	 */
	public void init() {
		map.put("cdn.my.com", "origin.my.com");
		map.put("cdn.client1.com", "origin.client1.com");
		map.put("cdn.client2.com", "client2origin.com");
	}

	public String findIncomingHost(HttpServletRequest request) {
		return map.get(request.getServerName());
	}

}

```

This will ensure that we can fetch the configuration based on many different clients/customers.
The same code can be deployed on multiple edge-servers in a single data-center that is serving
as CDN. If the distributed cache works great, and we build guards towards concurrent requests
properly, then only one request will go to the origin server from the edge-server farm till the
response does not expire.

To scale we only need to scale the caching service (which is nothing but a local BLOB store).

### TTL and custom TTL

TTL, or time-to-live, is the time after which the entry in the edge-server is considered expired
and the edge-server is supposed to be making the request again to the origin server. In our code
above, we use `CacheUtils.canCacheResponse` to decide if the response should be cached or not.

Let's consider a typical simple implementation:

```java
public class CacheUtils {

	public boolean canCacheResponse(HttpResponse originResponse) {
		// let's check for expires header
		long expires = ResponseUtils.getExpiresValueInMillis(originResponse);
		if(expires > 0) {
			return true;
		}

		// let's check for max-age header
		long maxAge = ResponseUtils.getMaxAgeInMillis(originResponse);
		if(maxAge > 0) {
			return true;
		}

		return false;
	}

}
```

This method just checks for incoming response headers. But in a real-world scenario,  TTL for a
given request cache needs to be based on the following parameters:

* TTL as per the configuration of the CDN system globally
* TTL as per the configured client
* TTL as per the response headers of the request

Different priorities can be decided to the three to figure out whether a request should be cached
and if yes, for what time. This is the onus of the method `canCacheResponse` in the code above.

Let's update the method to return the `TTL` value as `milliseconds` than returning a `boolean`:

```java
public class CacheUtils {

	public long canCacheResponse(CdnConfig config, HttpResponse originResponse) {
		// let's check for expires header
		long expires = ResponseUtils.getExpiresValueInMillis(originResponse);
		if(expires > 0) {
			return expires;
		}

		// let's check for max-age header
		long maxAge = ResponseUtils.getMaxAgeInMillis(originResponse);
		if(maxAge > 0) {
			return maxAge;
		}

		// check for the value provided by client (origin-owner) configuration
		long ownerTTL = config.getOwnerTTL();
		if(ownerTTL > 0) {
			return ownerTTL;
		}

		// nothing is available - return the global TTL

		return this.cdnSystem.getGlobalTTL();
	}

}
```

Now we can modify our code to make use of this TTL value:

```java
long responseTTL = CacheUtils.canCacheResponse(config, originResponse);

if(responseTTL > 0) {
	// there is nothing to cache
	return;
}

// we need to cache this response
this.cacheService.putInCache(cacheKey, originResponse, responseTTL);
```

### Request Filters

Typical CDN systems also provide a way to set request filters to indicate what kind of queries
need to be cached or not. Say, all requests ending in `*.png` should always be cached irrespective
of the response headers, where as all requests to `*.meme` should never be cached. If you look at
the code in `CacheUtils.canCacheResponse` it becomes pretty simple to implement using the `CdnConfig`
object.

Consider the following `CdnConfig` object:

```java
public class CdnConfig {

	public String originHost;

	public final List<String> alwaysCache = new ArrayList();

	public final List<String> neverCache = new ArrayList();

	public boolean notCacheResponse(HttpResponse response) {
		String path = response.getRequestPath();

		for(String rule : neverCache) {
			if(StringUtils.wildcardMatch(rule, path)) {
				return true;
			}
		}

		return false;
	}

	public boolean alwaysCacheResponse(HttpResponse response) {
		String path = response.getRequestPath();

		for(String rule : alwaysCache) {
			if(StringUtils.wildcardMatch(rule, path)) {
				return true;
			}
		}

		return false;
	}

}
```

Now, code for `CacheUtils.canCacheResponse` will change to:

```java
public class CacheUtils {

	public long canCacheResponse(CdnConfig config, HttpResponse originResponse) {
		// check if this should not be cached per policy
		if(config.notCacheResponse(originResponse)) {
			return;
		}

		// check if this request is to be always cached per policy
		if(!config.alwaysCacheResponse(originResponse)) {
			// let's check for expires header
			long expires = ResponseUtils.getExpiresValueInMillis(originResponse);
			if(expires > 0) {
				return expires;
			}

			// let's check for max-age header
			long maxAge = ResponseUtils.getMaxAgeInMillis(originResponse);
			if(maxAge > 0) {
				return maxAge;
			}
		}

		// check for the value provided by client (origin-owner) configuration
		long ownerTTL = config.getOwnerTTL();
		if(ownerTTL > 0) {
			return ownerTTL;
		}

		// nothing is available - return the global TTL

		return this.cdnSystem.getGlobalTTL();
	}

}
```

### Cache expiration

Cache-expiration is the facility that the CDN system may allow to its clients to delete the
cache before the actual TTL flushes the data. This is useful when either the cache is corrupt,
or a new version of the asset is being pushed, or we just need to clean for another reason.

Consider the following `servlet` code at the edge location:

```java
public class MyEdgeFlushServlet extends HttpServlet {

	private CacheService cacheService;

	private CdnConfigService cdnConfigService;

	/**
	 *
	 * This method takes an incoming API call, authenticates it, and if genuine, clears
	 * the resources as requested by the client.
	 */
	@Override
	public void doGET(HttpServletRequest request, HttpServletResponse response) {
		String clientID = request.getParameter("clientID");

		// find client information
		CdnConfig config = this.cdnConfig.getConfigForClient(clientID);
		if(config == null) {
			// no such client is present
			response.sendError(HttpStatusCode.BAD_REQUEST, "No such client");
			return;
		}

		// make sure this is an authorized call
		boolean authenticatedRequest = SecurityUtils.isGenuineRequest(config, request);
		if(!authenticatedRequest) {
			response.sendError(HttpStatusCode.UNAUTHORIZED, "Not authenticated");
			return;
		}

		// find request or pattern to flush
		String resource = request.getParameter("requestPathToFlush");

		// clear it off the distributed cache
		this.cacheService.clearResource(config.originHost, resource);

		response.setStatus(HttpStatusCode.OK, "Resource cleared");
	}
```

Note that this `servlet` is deployed at the edge locations. A client is not supposed to call
all edge locations. Thus, a single admin end-point can be deployed that `fans-out` this call via
either direct-call to all edge-servers, or via a work `queue`. A `queue` based system is mostly
deployed to make sure that edge servers are not bombarded with requests from admin end-points
itself, and also to invert the `push` to `pull` which is simpler to implement.

### Authentication

Authentication in CDN usually applies to protected resources, say the paid-ones. For example,
`Disney` wants its paid users view the given movie `Moana` but not to every one. To improve
speed for all its subscribers they deploy a CDN, but the request has to be still authenticated.

Authentication for such request on a CDN is usually implemented in 2 ways:

#### Online Validation

In this methodology, the origin servers embed a validation code as a query-parameter to the request.
For example:

```
GET https://cdn.my.com/resource/at/path?validation=ABCDEFGHIJKLMNOPQRSTUVWXYZ
```

The validation code is generated by the origin server, `ABCDEFGHIJKLMNOPQRSTUVWXYZ` in this case,
and is per-request basis. It needs to change for all users, at every time. To make it more secure
the same token once generated should not be used in future.

The CDN system knows which parameter to read this validation code from, and is also aware of the
origin authentication end-point, where it can validate the token response from. Say the endpoint
is `auth.my.com`.

Thus, when CDN receives a request such as above, it will pass the parameter to my authentication
service, and see if it returns a `HTTP OK` response. If yes, it will serve the resource, else, shall
send the `HTTP FORBIDDEN` status to the client.

Thus, the code for our `MyEdgeFacingServle.doGET` changes to:

```java
	public void doGET(HttpServletRequest request, HttpServletResponse response) {
		// find host for which we are proxying
		String incomingHost = RequestUtils.findIncomingHost(request);

		// there was nothing in cache - let's hit the origin server now
		CdnConfig config = this.cdnConfigService.getConfigForOrigin(incomingHost);

		// check for authentication
		String valiationRequired = config.isValidationRequired(request);
		if(validationRequired) {
			String auth = request.getParameter("validation");

			if(AssertUtils.isEmpty(auth)) {
				response.sendError(HttpStatusCode.BAD_REQUEST, "Auth required");
				return;
			}

			AuthResponse authResponse = this.httpService.makeValidationRequest(config.getValidationEndPoint(), auth);

			// TODO: the below code is just a simple implementation
			// in essence, you need to check for all valid HTTP 2xx codes
			if(authResponse.getStatusCode() != 200) {
				response.sendError(HttpStatusCode.BAD_REQUEST, "Auth required");
				return;
			}
		}

		// continue with checking in cache and all
	}
```

#### Offline Validation

The above approach has a disadvantage that for every incoming request, the CDN system has to call
an authentication end-point. This makes the authentication end-point a bottleneck. If it fails, no
resource will be served by the CDN system. Also, this adds additional latency before the CDN system
can even serve the request.

To overcome this limitation, an offline-validation mechanism can be introduced in the above
authentication workflow. The origin host will generate a valid signature for the request using
known `client-id` and `client-secret` which is known between the origin server and the CDN server
only.

This is exactly the same workflow as maintaining stateless user sessions, [explained here]
(solutions/2016/stateless-user-session.md). The generated signature can thus be validated by the CDN
system just knowing the `client-secret` used to generate the key and can proceed ahead. For
additional security the concept of `nonce` can be applied here as well.

Another methodology being adopted in the wild is to encrypt the tokens using the public key of the
CDN system, which can only be decrypted by the private key that the CDN server has.

## Summary, TL;DR

Per design CDN is nothing but a simple reverse-proxy engine like `Apache http`, `nginx`,
`caddy` etc. The job is to check with the origin server and serve data to clients
downstream.

It is a simple web application that is serving its own clients, just that its disguised
in the name of another web application, with two main focus areas: speed and security.

license: MIT

repository: binhnguyennus/ps

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: binhnguyennus/ps


text:
### How did this project originate?

Daniel Lindsley gave a talk at PyCon 2013 about "How (Not) To Build An OSS
Community" which hit on many different points as to why most OSS communities
fail to grow or even begin. The overarching pattern with each point he made
was **communication**.

This wiki template acts as a starting point for doing just that. It also acts
as a checklist for the other various components that _should_ be in existence
to successfully grow a community around your project.

### Why are some of the pages partially filled out?

Because I haven't checked everything off the list for this project yet =)

repository: bruth/oss-community-wiki

repo_name: bruth/oss-community-wiki


text:
#### Gitlab

Whether you are using gitlab.com or hosting your own Gitlab instance
(e.g. at `gitlab.contoso.com`), create a new app by browsing to
one of the following URLs:

* `https://gitlab.contoso.com/-/profile/applications`
* `https://gitlab.com/-/profile/applications`

![Gitlab - New Application](./assets/docs/images/oauth_gitlab_new_app_1.png)

![Gitlab - New Application - Scopes](./assets/docs/images/oauth_gitlab_new_app_2.png)

![Gitlab - New Application - Review](./assets/docs/images/oauth_gitlab_new_app_2.png)

Next, create a configuration for the backend, e.g.:

```
app.contoso.com {
  route /auth* {
    authp {
      backends {
        gitlab_oauth2_backend {
          method oauth2
          realm gitlab
          provider gitlab
          domain_name gitlab.contoso.com
          client_id 522a2f714a1e978c52e80909e543e2a51
          client_secret d562a48c29a686b343978edbc8ac3d3
          scopes openid email profile
          user_group_filters barfoo
          user_group_filters ^a
        }
      }
      ui {
        links {
          "My Website" / icon "las la-star"
          "My Identity" "/auth/whoami" icon "las la-star"
        }
      }
      transform user {
        match origin gitlab
        action add role authp/user
      }
      transform user {
        match origin gitlab
        match email<EMAIL_ADDRESS>        action add role authp/admin
      }
    }
  }

  route {
    authorize {
      primary yes
      allow roles authp/admin authp/user
      validate bearer header
      set auth url /auth/oauth2/gitlab
      inject headers with claims
    }
    respond * "my app" 200
  }
}
```

By default, Gitlab groups are not included into the token, unless
the `user_group_filters` directive is being user in the configuration.

The following directives instruct the portal to add the groups having
`barfoo` in their name and the groups whose names start with the `a`.

```
          user_group_filters barfoo
          user_group_filters ^a
```

In this case, the groups making it to the JWT token are:

```
    "gitlab.contoso.com/barfoo",
    "gitlab.contoso.com/a-private-group/a-subgroup"
```

When a user browses to the app and clicks Gitlab link, the user is being redirected to
Gitlab instance.

![Gitlab - Initiate Login](./assets/docs/images/oauth_gitlab_init.png)

The user should click Authorize to continue.

![Gitlab - Authorize App](./assets/docs/images/oauth_gitlab_authorize_app_1.png)

Once logged in, the user may browse to "My Identity".

![Gitlab - Portal](./assets/docs/images/oauth_gitlab_portal.png)

The relevant Gitlab data became a part of the JWT token issued by the portal.

![Gitlab - User Identity](./assets/docs/images/oauth_gitlab_user_identity.png)

[:arrow_up: Back to Top](#table-of-contents)

repository: sivanamurugesan/caddy-auth-portal

branch_name: refs/heads/main

repo_name: sivanamurugesan/caddy-auth-portal


text:
# Integ_description

## Table of Contents
1. [General Info](#general-info)
2. [Installation](#installation)

### General Info
***
In this package you will find:
* A file "arm.urdf.xacro" which defines our RR robot and the onboard camera.
* A "target.urdf" file to describe the target model.

The camera is placed on the end effector and only tracks targets in the XY plane. The simulation is launched on Rviz. For a simulation on Gazebo, see the launch files in the integ_gazebo folder.

## Installation
***
Download the projet to your ROS workspace using the following commands
```
$ git clone https://github.com/Valentin-Molina/ecn-integ-project/src/integ_description.git
$ cd ../ros/src
$ catkin build integ_description
$ roslaunch integ_description arm.rviz
```

repository: Valentin-Molina/ecn-integ-project

gha_license_id: MIT

repo_name: Valentin-Molina/ecn-integ-project


text:
# bioinformatics-navarro

* Project done for [Bioinformatics course] (https://www.fer.unizg.hr/predmet/bio)
* [University of Zagreb - Faculty of Electrical Engineering and Computing](https://www.fer.unizg.hr/)

## About

Implementation of Navarro's algorithm (Improved approximate pattern matching on hypertext) described here
```https://www.sciencedirect.com/science/article/pii/S0304397599003333```

---
Comparing results with paper:
```https://academic.oup.com/bioinformatics/article/35/19/3599/5372677```

Implemented here:
```https://github.com/maickrau/GraphAligner/tree/PaperExperiments```

## Installing
---
Install boost, for Linux

---
```sudo apt install libboost-all-dev```

## Usage
---
For starting use compile-execute.sh scrypt which compiles and then executes program

---
```./compile-execute.sh```

## Note
---
Used from github (https://github.com/maickrau/GraphAligner/tree/PaperExperiments):

---
* AlignmentGraph.cpp
* AlignmentGraph.h
* BiGraphToDigraph.cpp
* BiGraphToDigraph.h
* CommonUtils.cpp
* CommonUtils.h
* fastloader.cpp
* fastloader.h
* ThreadReadAssertion.cpp
* ThreadReadAssertion.h


text:
---
title: Get Involved
---

You can contribute to libcoap in several ways:

Mailing Lists

      You are welcome to subscribe to the libcoap developers'
      mailing list and ask questions, discuss the latest
      features or report bugs.

View details »

Report Bugs

      If you have encountered bugs or want to request a new feature,
      you can open a new ticket at the libcoap bug tracking tool.

Bug Tracker »

Fix Bugs

      If you encounter bugs in the main tree please include a patch or
      a link to your fork with your bug tracker ticket. You can also
      start a discussion on the
      mailing list.

Write Documentation

Howtos, more API documentation

      platform-specific documentation

User Guides

    User guides and howtos are collected
    here.

API Reference

      {%- assign api = site.versions | first | prepend: "doc/reference/" | relative_url -%}
      A doxygen-generated API reference is available
      here

Cleanup Code

      As the library has evolved while no explicit coding style
      was published, the code style has become inconsistent and
      cluttered with whitespace here and there. You can help by
      cleaning it up!

Coding style

Develop Code

      New features are developed in topic branches and will be
      merged into the develop branch once they are tested and agreed
      to. Please make sure that your contribution adheres to the
      libcoap coding
      style and that you
      have read and understood the rules given in the <tt>CONTRIBUTE</tt>
      document of the library.

Write Tests

      libcoap comes with a set of unit tests to check the core
      functionality. As always, more tests are needed, including
      a port of the ETSI CoAP plugtest specs to the CUnit test
      framework used my libcoap.

        Moreover, libcoap is part of the Continuous Fuzzing
        for Open Source Software activity. More fuzzing tests are
        needed (see tests/oss-fuzz
        in the libcoap repository).

Branching Model

      libcoap follows a branching model presented in
      this post
      by Vincent Driessen. The main branches are the main
      where stable releases go. A release being in preparation
      lives in branch release, while the current development
      branch is develop.

Roadmap

      If you want to engage in developing libcoap features, please have a
      look at the Roadmap
      document for additional guidance on the bigger plan.

repository: obgm/libcoap-net

repo_name: obgm/libcoap-net


text:
---
layout: extension
name: archiver
title: Archive CKAN resources
author: CKAN
homepage: https://github.com/ckan/ckanext-archiver
github_user: ckan
github_repo: ckanext-archiver
category: Extension
featured: 1
permalink: /extension/archiver/
---

CKAN Archiver Extension
=======================

**Status:** Production

**CKAN Version:** 1.5.1+

Overview
--------

The CKAN Archiver Extension provides a set of Celery tasks for downloading and saving CKAN resources. It can be configured to run automatically, saving any new resources that are added to a CKAN instance (and saving any resources when their URL is changed). It can also be run manually from the command line in order to archive resources for specific datasets, or to archive all resources in a CKAN instance.

Installation
------------

Install the extension as usual, e.g. (from an activated virtualenv):

    $ pip install -e git+http://github.com/okfn/ckanext-archiver.git#egg=ckanext-archiver

Install the required libraries:

    $ pip install -r pip-requirements.txt

Configuration
-------------

1.  Enabling Archiver

    If you want the archiver to run automatically when a new CKAN resource is added, or the url of a resource is changed, then edit your CKAN config file (eg: development.ini) to enable the extension:

        ckan.plugins = archiver

    If there are other plugins activated, add this to the list (each plugin should be separated with a space).

    **Note:** You can still run the archiver manually (from the command line) on specific resources or on all resources in a CKAN instance without enabling the plugin. See section 'Using Archiver' for details.

2.  Other CKAN config options

    The following config variable should also be set in your CKAN config:

        ckan.site_url: URL to your CKAN instance

    This is the URL that the archive process (in Celery) will use to access the CKAN API to update it about the cached URLs. If your internal network names your CKAN server differently, then specify this internal name in config option ckan.site\_url\_internally.

    Optionally, the following config variables can also be set: :

        ckan.cache_url_root: URL that will be prepended to the file path and saved against the CKAN resource,
          providing a full URL to the archived file.

3.  Additional Archiver settings

    The following Archiver settings can be changed by creating a copy of `ckanext/archiver/default_settings.py` at `ckanext/archiver/settings.py`, and editing the variables:

        ARCHIVE_DIR: path to the directory that archived files will be saved
        to.
        MAX_CONTENT_LENGTH: the maximum size (in bytes) of files to archive.
        DATA_FORMATS: the data formats that are archived.
        USER_AGENT_STRING: the `User-Agent` header used when the archiver makes requests

    Alternatively, if you are running CKAN with this patch: <https://github.com/datagovuk/ckan/commit/83dcaf3d875d622ee0cd7f3c1f65ec27a970cd10> then you can instead add the settings to the CKAN config file as normal:

        ckanext-archiver.archive_dir
        ckanext-archiver.max_content_length
        ckanext-archiver.data_formats  (space separated)
        ckanext.archiver.user_agent_string

Using Archiver
--------------

First, make sure that Celery is running. For test/local use, you can do this by going to the CKAN root directory and typing:

    paster celeryd -c <path to CKAN config>

For production use, we recommend setting up Celery to run with supervisord. For more information see

> <http://docs.ckan.org/en/latest/maintaining/background-tasks.html>

The Archiver can be used in two ways:

1.  Automatically

    Install, enable and configure the plugin as described above. Any changes to resource URLs (either adding new or updating current URLs) in the CKAN instance will now call the archiver to try and download the resource.

2.  Manually

    From the ckanext-archiver directory run:

        paster archiver update [dataset] -c <path to CKAN config>

    Here `dataset` is an optional CKAN dataset name or ID. If given, all resources for that dataset will be archived.

    If omitted, all resources for all datasets will be archived.

    For a full list of manual commands run:

        paster archiver --help

Testing
-------

Tests should be run from the CKAN root directory (not the extension root).

    (pyenv)~/pyenv/src/ckan$ nosetests --ckan ../ckanext-archiver/tests/

branch_name: refs/heads/gh-pages

gha_language: HTML


text:
---
layout: default
---

# 1996-11-14 - Re: Black markets vs. cryptoanarchy

## Header Data

From: Jim Wise<EMAIL_ADDRESS>To: "Timothy C. May"<EMAIL_ADDRESS>Message Hash: b86d5db8a93a8b0482aaf3f94fa9d9e8f352c36c6e08744ccf331646cbe76069

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1996-11-14 03:06:51 UTC

Raw Date: Wed, 13 Nov 1996 19:06:51 -0800 (PST)

## Raw message

```
{% raw  %}From: Jim Wise<EMAIL_ADDRESS>Date: Wed, 13 Nov 1996 19:06:51 -0800 (PST)
To: "Timothy C. May"<EMAIL_ADDRESS>Subject: Re: Black markets vs. cryptoanarchy
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

On Wed, 13 Nov 1996, Timothy C. May wrote:

> At 10:56 PM -0500 11/12/96, Jim Wise wrote:

> >Which it does...  FWIW, I tend to agree with your general point, but I
> >moved from downtown Manhattan to Harlem recently, and was surprised to see
> >how many foodstuffs cost _more_ up here, as well as the obvious fact that
> >many are harder to get...  Junk food and cheap liquor are everywhere,
> >though...

> But you're conflating a separate issue: the cost of doing business in
> high-crime ghettoes. Both rich and poor alike find prices high and
> selection poor in high-crime ghettoes. Likewise, both rich and poor alike
> find prices low and selection good in low-crime, suburban locales.

I would hardly classify alphabet city as a `low-crime suburban locale'.
Much more of an issue is that the locals downtown are much closer to being
within walking distance of the higher-rent higher-income areas, so the
local bodegas must keep prices low to compete.  Up here, it's a lot
farther to an alternative, and a lot fewer people have cars, so you have a
lot fewer choices.  The result is that what choices there are can pretty
much stock what they please and charge what they please...

The issue here is much more one of the insularity of the ghetto than it's
crime rate...

--
				Jim Wise
				System Administrator
				GSAPP, Columbia University
<EMAIL_ADDRESS>				http://www.arch.columbia.edu/~jim
				* Finger for PGP public key *

{% endraw %}
```

## Thread

+ Return to [November 1996](/archive/1996/11)

+ Return to "[Adam Back <aba@dcs.ex.ac.uk>](/authors/adam_back_aba_at_dcs_ex_ac_uk_)"
+ Return to<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)](/authors/dlv_at_bwalk_dm_com_dr_dimitri_vulis_kotm_)"
+ Return to "[Jim Wise <jw250@columbia.edu>](/authors/jim_wise_jw250_at_columbia_edu_)"
+ Return to "["Mark M." <markm@voicenet.com>](/authors/mark_m__markm_at_voicenet_com_)"
+ Return to "[Mike Ingle <inglem@adnetsol.com>](/authors/mike_ingle_inglem_at_adnetsol_com_)"
+ Return to "[Open Net Postmaster <postmaster@opennet.net.au>](/authors/open_net_postmaster_postmaster_at_opennet_net_au_)"
+ Return to "[Ted Anderson <ota+@transarc.com>](/authors/ted_anderson_ota_at_transarc_com_)"
+ Return to "["Timothy C. May" <tcmay@got.net>](/authors/timothy_c_may_tcmay_at_got_net_)"
+ Return to "["Vladimir Z. Nuri" <vznuri@netcom.com>](/authors/vladimir_z_nuri_vznuri_at_netcom_com_)"

+ 1996-11-10 (Sat, 9 Nov 1996 23:47:47 -0800 (PST)) - [Black markets vs. cryptoanarchy](/archive/1996/11/599c6ee82220809e90a3eec319b2732e41128f57ede85c8733d3e167adb240e1) - _Mike Ingle<EMAIL_ADDRESS>  + 1996-11-10 (Sun, 10 Nov 1996 11:39:42 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/2a2b56595235a4c53ae25411503d561f2d97f3f7bde213b996d2fb01b4bb182b) - _"Vladimir Z. Nuri"<EMAIL_ADDRESS>    + 1996-11-11 (Mon, 11 Nov 1996 10:37:45 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/f9fea0da5df26b29f825cd1739119ee8dceeb7dcd51e7e36b05c6a3f33da2bfd) - _Adam Back<EMAIL_ADDRESS>    + 1996-11-12 (Tue, 12 Nov 1996 08:15:15 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/f0fece1c6f73e694e3c90e545f3892d4bfb278af160be6b7c084c6f254dc037a) - _Ted Anderson<EMAIL_ADDRESS>      + 1996-11-13 (Tue, 12 Nov 1996 18:56:06 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/74378ed086d7eed841cb4451b20c2b209f6d4169a14beddbba8dfd2c2ebe2ef3) - _"Vladimir Z. Nuri"<EMAIL_ADDRESS>    + 1996-11-12 (Tue, 12 Nov 1996 11:40:02 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/a0e2dbf40e9bc8cd3e69458e09c3702d1133f5b3ec7ae79f61fe9ca2664d8c9e) - _"Timothy C. May"<EMAIL_ADDRESS>      + 1996-11-13 (Tue, 12 Nov 1996 19:57:38 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/20a1dea7928808a627710df9dd3a55c54765ead13ad9aea22522f9251db5130c) - _Jim Wise<EMAIL_ADDRESS>        + 1996-11-13 (Wed, 13 Nov 1996 11:14:41 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/fabd40f21d53b21a9193531ac70d9a5cd304890d2f66b32ceba6e3bc0630e231) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
          + 1996-11-14 (Wed, 13 Nov 1996 20:29:18 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/7aa116624d226a4b4563dd4e73c0e54589de68edc18473069fd342a25db5f584) - _Jim Wise<EMAIL_ADDRESS>      + 1996-11-13 (Wed, 13 Nov 1996 08:12:33 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/3b8518b185afbd4c673155811f6f422858b6ccfbbcaadf77c403a78af45f1ef9) - _"Timothy C. May"<EMAIL_ADDRESS>        + 1996-11-14 (Wed, 13 Nov 1996 19:06:51 -0800 (PST)) - Re: Black markets vs. cryptoanarchy - _Jim Wise<EMAIL_ADDRESS>    + 1996-11-13 (Tue, 12 Nov 1996 16:26:56 -0800 (PST)) - [Returned mail: User Unknown](/archive/1996/11/d1a5b934b579b90430de2a025838063e445e7193b49be544cd33bd8f50fcfb59) - _Open Net Postmaster<EMAIL_ADDRESS>  + 1996-11-11 (Sun, 10 Nov 1996 18:45:49 -0800 (PST)) - [Re: Black markets vs. cryptoanarchy](/archive/1996/11/3a660da3e4d8a31bec02bc3e0f1cd400d4762ffb7e581a7a3dd476e1ebdbc69b) - _"Mark M."<EMAIL_ADDRESS>


text:
# Agoric JavaScript Programming Extensions

Agoric's platform lets you write secure smart contracts in JavaScript. The platform itself is mainly written in JavaScript. However, we've made several Agoric-specific additions to general JavaScript programming that you should know about and understand before programming on the platform. Some are *concepts*, others are *Agoric library additions*, and some are at the *syntax level*. All changes at the language level are in process to become official standards.

Extensions covered in this document are:
- **[`BigInt`](./bigint.md)**: JavaScript's `Number` primitive only represents
  numbers up to 253 - 1. `BigInt` is a newer built-in JavaScript
  object that represents arbitrarily large integers. Agoric uses `BigInts` for
  amount `values` and times.

- **[Vats](./vats.md)**: Objects and functions in the same JavaScript vat can
  communicate synchronously. Communication with objects outside the
  vat can only be done asynchronously.

- **[Secure EcmaScript (SES)](./ses/ses-guide.md)**: SES provides a secure platform for
  executing programs. With SES, you can run code you don't completely trust,
  without being vulnerable to bugs or bad intentions. It's a
  standards-track extension to the JavaScript standard.

- **[Realms and compartments](./ses/ses-guide.md)**: JavaScript code runs in the context of
  a *realm*, made up of the set of primordials (the standard
  library's objects and functions) and a global object. In a web
  browser, an iframe is a realm. In Node.js, a Node process is a
  realm. A *compartment* is a separate execution environment within a realm.

- **[`harden()`](./ses/ses-guide.md)**: A hardened object’s properties cannot be changed, so the only way to interact
with a hardened object is through its methods. `harden()`is similar to `Object.freeze()` but
more powerful. Objects provided as parameters that will be shared across vat boundaries must be hardened.

- **[Remote object communication using `E`](./eventual-send.md)**: `E` is a local "bridge" function that lets
you invoke methods on remote objects, whether in another vat, machine, or blockchain (for example).
It takes a local representative (a *proxy*) for a remote object as an argument and sends messages
to it using normal message-sending syntax. The local proxy forwards all messages to the remote
object to deal with. Sending a message to the remote object must be done by
using `E` (`E(remoteObj).myMethod()`).

- **[Notifiers](./notifiers.md):** The Agoric platform uses Notifiers to distribute state change
updates. Notifiers rely on promises to deliver a stream of messages as a publish-subscribe system
might, without requiring explicit management of lists of subscribers.

repository: Hayai-XxX/documentation

repo_name: Hayai-XxX/documentation


text:
# v@r

The v@r email or<EMAIL_ADDRESS>will serve as the bedrock of how we are looking to build our organisation. It serves as our communication, governance & transparency mechanism. This email will be the primary \(and even sole\) way one formally engages with the organisation.

Anybody who wishes to reach out to us for any guidance or sharing information are requested to make use of this email. All our volunteers receive a copy of the mail received on this email. You can then expect one or several of our volunteers to get back to you with the required guidance on the matter. Volunteers also ensure that<EMAIL_ADDRESS>is marked in cc whenever guidance is offered by them on any matter.

In summary

1. Whenever someone looks to engage with Rethink, ensure that you are writing to<EMAIL_ADDRESS>2. Whenever you receive a response to your mail from one of us, ensure that<EMAIL_ADDRESS>is marked in cc


text:
```r
library(fields)
```

```
## Loading required package: spam
```

```
## Loading required package: grid
```

```
## Spam version 1.0-1 (2014-09-09) is loaded.
## Type 'help( Spam)' or 'demo( spam)' for a short introduction
## and overview of this package.
## Help for individual functions is also obtained by adding the
## suffix '.spam' to the function name, e.g. 'help( chol.spam)'.
```

```
##
## Attaching package: 'spam'
```

```
## The following objects are masked from 'package:base':
##
##     backsolve, forwardsolve
```

```
## Loading required package: maps
```

```r
source("../../prior-component-analysis.R", chdir=TRUE)
source("../../covariance-functions.R")
```

```
##
## Attaching package: 'igraph'
```

```
## The following objects are masked from 'package:stats':
##
##     decompose, spectrum
```

```
## The following object is masked from 'package:base':
##
##     union
```

```r
source("/home/jim/Documents/feature_learning/ppca/ppca-simple.R")
source("/home/jim/R/Library/MultiImage.R")

data(ozone2)
```

Missing data: Replace missing values by column means

```r
X = ozone2$y
for (col in 1:ncol(X)) {
  ind = is.na(X[,col])
  X[ind,col] = mean(X[,col], na.rm=TRUE)
}
```

Plot a few samples

```r
locations = ozone2$lon.lat
locations = apply(locations, 2, function(col) (col-min(col))/(max(col)-min(col)))
multiImage(locations, X[1,], X[2,], X[3,], X[4,])
```

![plot of chunk plot-samples](figure/plot-samples-1.png)

Look at BIC for SPCA with and for PPCA. We can see that SPCA picks k=3, PPCA picks k=9 and SPCA always has a higher BIC. This is actually expected, since both SPCA and PPCA will have (approximately) the same model complexity penalty, but PPCA will have a greater model fit.

```r
source("../../prior-component-analysis.R", chdir=TRUE)
source("../../covariance-functions.R", chdir=TRUE)

source("/home/jim/Documents/feature_learning/ppca/ppca-simple.R")

ks = 1:12

bics.spca = numeric(length(ks))
bics.ppca = numeric(length(ks))
for (ki in seq_along(ks)) {
  out.spca = prca(X, ks[ki], locations, exp.MR.cov, exp.MR.cov.d,
                  beta0=log(c(0.1, 0.5)), maxit=15, max.dist=0.5,
                  trace=1, report_iter=15)
  bics.spca[ki] = out.spca$bic

  out.ppca = ppca(X, ks[ki])
  bics.ppca[ki] = out.ppca$bic
}
```

```
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -61100.3582 (increase=13360675607.4193)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -59710.0759 (increase=15174666855.6006)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -59410.5243 (increase=16022857802.8379)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -58005.6461 (increase=16929734687.5344)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57788.6356 (increase=19969798406.9073)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57493.1518 (increase=21130739587.9022)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57320.662 (increase=21812853409.9522)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57326.4191 (increase=23099546393.8104)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57289.5535 (increase=23449739358.3056)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57271.8731 (increase=24273622030.3798)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57064.5572 (increase=26019440806.3174)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -56894.9318 (increase=26840642879.3172)"
## [1] "Convergence criteria reached: 15 iterations"
```

```r
plot(ks, bics.spca, xlab='k', ylab='bic', type='b', main="BIC",
     ylim=range(bics.spca, bics.ppca))
points(ks, bics.ppca, col='green', type='b')
legend("top",
       legend=c("SPCA", "PPCA"),
       col=c("black", "green"),
       pch=1, lty=1)
```

![plot of chunk unnamed-chunk-3](figure/unnamed-chunk-3-1.png)

Try looking at crossvalidated log likelihoods instead, as a better measure of model fit. Better! SPCA always has a greater holdout data likelihood. Interesting that they peak at the same place (k=7).

```r
liks.spca = numeric(length(ks))
liks.ppca = numeric(length(ks))
for (ki in seq_along(ks)) {
  trainInd = sample(nrow(X), floor(nrow(X)*0.8))

  trainX = X[trainInd,]

  out.spca = prca(trainX, ks[ki], locations, exp.MR.cov, exp.MR.cov.d,
                  beta0=log(c(0.1, 0.5)), maxit=15, max.dist=0.5,
                  trace=1, report_iter=15)
  liks.spca[ki] = prca.log_likelihood(X[-trainInd,], out.spca$W, out.spca$sigSq)

  out.ppca = ppca(X, ks[ki])
  liks.ppca[ki] = prca.log_likelihood(X[-trainInd,], out.ppca$W, out.ppca$sigSq)
}
```

```
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -49444.2591 (increase=14407643695.1172)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -48048.8721 (increase=14897247998.2186)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -48446.8916 (increase=14385623029.4499)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -47677.5529 (increase=15746350649.1052)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -46852.9232 (increase=22081744807.9087)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -45957.5347 (increase=20537746311.7598)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -46313.1546 (increase=17044244131.4489)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -46252.331 (increase=24630266672.1623)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -46104.5255 (increase=25270113240.9913)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -46260.0727 (increase=24598662711.27)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -45665.0078 (increase=29609630399.0191)"
## [1] "Convergence criteria reached: 15 iterations"
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -46177.6119 (increase=33031837600.6249)"
## [1] "Convergence criteria reached: 15 iterations"
```

```r
plot(ks, liks.spca, xlab='k', ylab='bic', type='b',
     main="Heldout log likelihoods", ylim=range(liks.spca, liks.ppca))
points(ks, liks.ppca, col='green', type='b')
legend("top",
       legend=c("SPCA", "PPCA"),
       col=c("black", "green"),
       pch=1, lty=1)
```

![plot of chunk unnamed-chunk-4](figure/unnamed-chunk-4-1.png)

Rerun w/ k=7

```r
out.spca = prca(X, 7, locations, exp.MR.cov, exp.MR.cov.d,
                beta0=log(c(0.1, 0.5)), maxit=10, max.dist=0.5,
                trace=1, report_iter=5)
```

```
## [1] "Starting prca with 2 hyperparameters"
## [1] "Iteration 0: log likelihood = -57291.4579 (increase=21812853439.1562)"
## [1] "Iteration 5: log likelihood = -50006.0819 (increase=15.2569)"
## [1] "Convergence criteria reached: 10 iterations"
```

We have learnt quite a long length scale; a good sign!

```r
dist = matrix(seq(0, 1, length=100), ncol=1)
covar = exp.MR.cov(X=matrix(0), X2=dist, beta=out.spca$beta, max.dist=0.5)

plot(dist, covar, type='l', main='Learned covariance function')
```

![plot of chunk unnamed-chunk-6](figure/unnamed-chunk-6-1.png)

Another PrCA/PPCA comparison; cross-validate and project test-samples onto model. Compare distributions of distances to original points (generalisation error)

```r
#n        = nrow(X)
#err.wend = numeric(n)
#err.ppca = numeric(n)
#for (fold in 1:n) {
#  # PrCA
#  out.wend = prca(X[-fold,,drop=FALSE], 7, covar.fn, beta.init=c(1, log(0.3)),
#                  maxit=10)
#  rec.wend = predict(out.wend, X[fold,,drop=FALSE])
#  err.wend[fold] = sqrt(sum(rec.wend - X[fold,,drop=FALSE])^2)
#
#  # PPCA
#  out.ppca = prca(X[-fold,,drop=FALSE], 3, covar.fn.ppca, beta.init=c(),
#                  maxit=10, warnDiag=FALSE)
#  rec.ppca = predict(out.ppca, X[fold,,drop=FALSE])
#  err.ppca[fold] = sqrt(sum(rec.ppca - X[fold,,drop=FALSE])^2)
#}
#
## Histogram Colored (blue and red)
#hist(err.wend, 100, col=rgb(1,0,0,0.5), xlim=c(0, max(err.wend, err.ppca)),
#     ylim=c(0,45), xlab="Reconstruction error")
#hist(err.ppca, 100, col=rgb(0,0,1,0.5), add=T)
#box()
#
#boxplot(err.wend, err.ppca, names=c("PrCA", "PPCA"))
```

repository: seanigami/prior-component-analysis

repo_name: seanigami/prior-component-analysis


text:
---
name: Bug report
about: If you've found a bug that you think needs to be fixed then use this template. Be sure to fill out the form as shown and don't leave any sections off.
---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. step one

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable (delete if not), add screenshots to help explain your problem.

**Minecraft Enviorment**
 - Minecraft Version: [eg: 1.12.2]
 - XRay Mod Version: [eg: 1.5.0]
 - Mod Pack & Version if applicable
 - Forge Version if applicable

**Additional context**
Add any other context about the problem here.

repository: ScoPeZs/Auto-Clicker-Fabric

repo_name: ScoPeZs/Auto-Clicker-Fabric


text:
# Client use

So far, we have seen that Feathers with its services, events and hooks can also be used in the browser which is a very unique feature. By implementing custom services that talk to an API in the browser Feathers allows us to structure any client side application with any framework.

This is exactly what Feathers client side services do. In order to connect to a Feathers server, a client creates Services that use a REST or websocket connection to relay method calls and allow listening to events from the server. This means that we can use a client side Feathers application to transparently talk to a Feathers server the same way as if we would use it locally (like we did in all the previous examples)!

> __Note:__ The following examples show the use of the Feathers client through a `<script>` tag. For more information on using a module loader like Webpack or Browserify and loading individual modules see the [client API documentaiton](../../api/client.md).

## Real-time client

In the [real-time chapter](./real-time.md) we saw an example of how to directly use a websocket connection to make service calls an listen to events. We can also use a browser Feathers application and client services that use those conneciton. Let's update `public/client.js` to:

```js
// Create a websocket connecting to our Feathers server
const socket = io('http://localhost:3030');
// Create a Feathers application
const app = feathers();
// Configure Socket.io client services to use that socket
app.configure(feathers.socketio(socket));

app.service('messages').on('created', message => {
  console.log('Someone created a message', message);
});

async function createAndList() {
  await app.service('messages').create({
    text: 'Hello from Feathers browser client'
  });

  const messages = await app.service('messages').find();

  console.log('Messages', messages);
}

createAndList();
```

## REST client

We can also create services that communicate via REST using many different Ajax libraries like [jQuery](https://jquery.com) or [Axios](https://github.com/axios/axios). For this example we will be using [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) since it is already built into the browser.

> __Important:__ REST services do emit real-time events but only locally to themselves. REST does not support real-time updates from the server.

Since we a making a cross-domain request, we first have to enable [Cross-Origin Resource sharing (CORS)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) on the server by updating `app.js` to:

```js
const feathers = require('@feathersjs/feathers');
const express = require('@feathersjs/express');
const socketio = require('@feathersjs/socketio');
const memory = require('feathers-memory');

// This creates an app that is both, an Express and Feathers app
const app = express(feathers());

// This enables CORS
app.use(function(req, res, next) {
  res.header('Access-Control-Allow-Origin', '*');
  res.header('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept');
  next();
});

// Turn on JSON body parsing for REST services
app.use(express.json())
// Turn on URL-encoded body parsing for REST services
app.use(express.urlencoded({ extended: true }));
// Set up REST transport using Express
app.configure(express.rest());

// Configure the Socket.io transport
app.configure(socketio());

// On any real-time connection, add it to the `everybody` channel
app.on('connection', connection => app.channel('everybody').join(connection));

// Publish all events to the `everybody` channel
app.publish(() => app.channel('everybody'));

// Initialize the messages service
app.use('messages', memory({
  paginate: {
    default: 10,
    max: 25
  }
}));

// Set up an error handler that gives us nicer errors
app.use(express.errorHandler());

// Start the server on port 3030
const server = app.listen(3030);

server.on('listening', () => console.log('Feathers API started at localhost:3030'));
```

> __Note:__ This is just a basic middleware setting the headers. In production (and applications created by the Feathers generator) we will use the [cors](https://github.com/expressjs/cors) module.

Then we can update `public/client.js` to:

```js
// Create a Feathers application
const app = feathers();
// Initialize a REST connection
const rest = feathers.rest('http://localhost:3030');
// Configure the REST client by using `window.fetch`
app.configure(rest.fetch(window.fetch));

app.service('messages').on('created', message => {
  console.log('Created a new message locally', message);
});

async function createAndList() {
  await app.service('messages').create({
    text: 'Hello from Feathers browser client'
  });

  const messages = await app.service('messages').find();

  console.log('Messages', messages);
}

createAndList();
```

## What's next?

In this chapter we learned how to transparently connect to another Feathers server and use its services just the same as we are used to. In the [last chapter](./generator.md) let's briefly look at the Feathers generator (CLI) and the patterns it uses to structure an application before jumping into [building a full chat application](../chat/readme.md).

repository: aral/docs

repo_name: aral/docs


text:
# mysql-schema-updater

## What is it?

A tool to update MySQL schema in projects.

This tool can be used to bootstrap MySQL databases for your projects. It can be used inside a docker container or as a standalone binary. The application will read either os commands or environment variables.

The container can be pulled from [DockerHub | morfien101/mysql-schema-updater](https://hub.docker.com/r/morfien101/mysql-schema-updater/)

## How to use it

The below table will explain the environment variables and how they map to a command line:

Flag | Env var | Default | Description
---|---|---|---
sqlhost | SQL_HOST | localhost | Hostname or IP  of the SQL service you want to talk to.
sqlport | SQL_PORT | 3306 | Port number used while talking to the server.
sqlusername | SQL_USERNAME | root | Username to use when connecting to the SQL service.
sqlpassword | SQL_PASSWORD | root | Password to use when connecting to the SQL service.
sqldb | SQL_DB | test | Name of the database that should be used. If used in conjuntion with create-db it will make a DB with this name.
sqlversion | SQL_VERSION_TABLE | version | The name of the version table to use. If used with create-db it will create this table also.
scripts-path | SCRIPTS_PATH | /data | Where the sql files are kept.
create-db | CREATE_DB | false | Should the tool try to bootstrap the server for you also.
use-environment-variables | - | false | Use this flag if you want the application to read environment variables. Used primarly with docker containers on orchastrated platforms.
v | - | false | Shows the version of the application.
h | - | false | Shows the help menu.

Its its naked form the tool can be used with os arguments. See below for an example.

```bash
./mysql-schema-updater \
  --create-db \
  --scripts-path ./testsqlfiles \
  --sqldb go_is_awesome \
  --sqlhost <IP_ADDRESS> \
  --sqlpassword P4ssword \
  --sqlusername root
```

In a docker container you can use the following syntax:

```bash
# Whats in the environments file
cat fullEnvList
# Will produce:
# SQL_HOST=<IP_ADDRESS>
# SQL_PORT=3306
# SQL_USERNAME=root
# SQL_PASSWORD=P4ssword
# SQL_DB=go_is_awesome
# CREATE_DB=true

# Run the docker command in terminal mode to see the output.
docker run -it \
  --env-file $(pwd)/test_scripts/fullEnvList \
  -v $(pwd)/testsqlfiles:/data \
  morfien101/mysql-schema-updater:latest
```

The container used is started from __scratch__ so there is noting in there. You will not be able to use bash to see inside the container. Please also be aware of the mount to /data. This is the default location of where to look for your .sql files.

ALL FILES MUST END IN .sql TO BE PICKED UP.

All the sql statements will be printed to the terminal so that you can debug any crashes.
Important if you have sensitve data in there.
It is not currently supported to supress this printing.

## Why do we need it

This tool is primarly used in a situation where you want your updates to be automated. It will read files in a folder and process them. It will also check to see what versions are needed and only apply those files. You can mix this with something that will pull files from something like github and process the ones that are required.


text:
pelican-mg
==========

[![Build Status](https://travis-ci.org/Lucas-C/pelican-mg.svg?branch=master)](https://travis-ci.org/Lucas-C/pelican-mg)

A minimal theme for [Pelican](http://blog.getpelican.com/) that uses uikit.
The theme is suited for a single author blog. Feeds are provided via ATOM.

This theme was developped with accessibility in mind, but may not be perfectly WCAG-compliant.

Screenshots
--------------

Here is how the home page look like

![mg home page screenshot](https://raw.githubusercontent.com/lucachr/pelican-mg/master/home-page-screenshot.png)

This is the article page

![mg article page screenshot](https://raw.githubusercontent.com/lucachr/pelican-mg/master/article-screenshot.png)

The home page on a smartphone

![mg home page smartphone top screenshot](https://github.com/lucachr/pelican-mg/blob/master/home-page-smartphone-top.png)
![mg home page smartphone bottom screenshot](https://github.com/lucachr/pelican-mg/blob/master/home-page-smartphone-bottom.png)

This is the article layout on a smartphone screen

![mg article page smartphone top screenshot](https://raw.githubusercontent.com/lucachr/pelican-mg/master/article-page-smartphone-top.png)

Live Example
--------------
Check out [my blog](https://chezsoi.org/lucas).

Features
--------------

* [Open Graph](http://ogp.me) support.
* [Twitter Summary Card](https://dev.twitter.com/cards/types/summary) support.
* [Schema.org](http://schema.org) support.
* Search with [DuckDuckGo](https://duckduckgo.com/search_box).
* Responsive design.
* Comment with DISQUS or inno
* SCSS style sheets.
* Analytics with Google Analytics, PIWIK and StatCounter.
* Share buttons built with share urls.
* Custom footer notice.
* W3C-Validated HTML
* Home page images lazy-loading

Install
-------
This template uses the [representative_image](https://github.com/getpelican/pelican-plugins/tree/master/representative_image) and [image_process](https://github.com/getpelican/pelican-plugins/tree/master/image_process) plugins, so you will need to:

    ./run.sh install && ./run.sh dev_install

Settings
--------------

The following settings are required for a correct behaviour of this theme.

If you want to use the theme with search enabled (and probably you want it).

```python
    TAG_SAVE_AS = ''
    AUTHOR_SAVE_AS = ''
    DIRECT_TEMPLATES = ('index', 'categories', 'archives', 'past_readings')
```

If you want to use mg with search disabled.

```python
    TAG_SAVE_AS = ''
    AUTHOR_SAVE_AS = ''
    DIRECT_TEMPLATES = ('index', 'categories', 'archives')
    DISABLE_SEARCH = True
```

### Optional settings

**ALT_NAME**
An alternative name for your site. It appears in the header bar.

**DESCRIPTION**
A brief description of your site, for social networks and search engines.

**DISABLE_SEARCH**
Disable search, boolean.

**FAVICON**
The relative path of your favicon, this is needed for Disqus forum favicon.

**FAVICON_TYPE**
The MIME type of your favicon, this is needed for Disqus forum favicon.

**FOOTER**
A custom footer notice.

**META_IMAGE**
The absolute URL of a custom image for the `og:image` meta property, Twitter
summary card, and `image` meta property of Schema.org. This image is used in
every page of the blog. Articles and pages can override the default
**META_IMAGE** by setting the "image" metadata in the relative file.

**META_IMAGE_TYPE**
The MIME type for **META_IMAGE**, this is needed for `og:image:type`.

**SC_PROJECT**
The StatCounter project number.

**SC_SECURITY**
The StatCounter security code for the project.

**SHARE**
Enable share buttons, boolean.

**SOCIAL**
A list of tuples (icon, URL). The icons are from [Font Awesome]
(http://fortawesome.github.io/Font-Awesome/). The suffix "-square" is removed
in the footer icons of the small screen layout.
e.g.
```python
    SOCIAL = (('twitter', 'https://twitter.com/luca_chr'),
              ('google-plus-square', 'https://plus.google.com/117284397605208270870'),
              ('github', 'https://github.com/lucachr'),
              ('envelope', 'mailto:luca92web@gmail.com'),)
```

**SUPPORTS**
An optional list of tuples (img_alt, url, logo_url, title)

**READINGS**
An optional list of `dict`, with `img_url` & `description` fields, of books you read, from most recent to oldest.

**ISSO_BASE_URL**
An optional URL to an [isso](https://posativ.org/isso/) endpoint (serving `/js/embed.dev.js`).

**MG_DISABLE_SUMMARY**
If you do not manually provide a summary on your articles, set this to `True` so that that the articles summary is not included on the index / search result pages.

**MG_NO_EXCERPT**
Set this to `True`, if you do not want to display the last article in full on the index page.

**MG_FILTER_TAGS** and **MG_LANG_FILTER_TAGS**
Define those variables as list of tag names if you want to enable the tag filtering buttons.

**EXTRA_ATOM_FEED**
An extra URL providing an Atom feed of updates that you want inserted along the blog Atom feed.

Example Configuration
----------------------

This is the settings file for Dev's Bytes.

```python
    #!/usr/bin/env python
    # -*- coding: utf-8 -*- #
    from __future__ import unicode_literals

    AUTHOR = 'Luca Chiricozzi'
    SITENAME = "Dev's Bytes"
    ALT_NAME = "#! " + SITENAME
    SITESUBTITLE = "Random programming stuff"
    DESCRIPTION = "A blog about programming, *nix, and software development."
    SITEURL = 'http://www.devsbytes.com'
    FAVICON = 'favicon.ico'
    FAVICON_TYPE = 'image/vnd.microsoft.icon'

    META_IMAGE = SITEURL + "/static/img/og_logo.jpg"
    META_IMAGE_TYPE = "image/jpeg"

    PATH = 'content'

    TIMEZONE = 'Europe/Rome'

    DEFAULT_LANG = 'en'
    LOCALE = 'en_GB'

    THEME = "themes/mg"

    # Social widget
    SOCIAL = (('twitter', 'https://twitter.com/luca_chr'),
              ('google-plus-square', 'https://plus.google.com/+LucaChiricozzi'),
              ('github', 'https://github.com/lucachr'),
              ('envelope', 'mailto:luca92web@gmail.com'),)

    SHARE = True

    FOOTER = ("© 2015 Luca Chiricozzi. All rights reserved.<br>" +
              "Code snippets in the pages are released under " +
              "<a href=\"http://opensource.org/licenses/MIT\" target=\"_blank\">" +
              "The MIT License</a>, unless otherwise specified.")

    DEFAULT_PAGINATION = 10

    TAG_SAVE_AS = ''
    AUTHOR_SAVE_AS = ''
    PLUGINS = ['representative_image']
    DIRECT_TEMPLATES = ('index', 'categories', 'archives')

    RELATIVE_URLS = False

    FEED_ALL_ATOM = 'feeds/all.atom.xml'

    DELETE_OUTPUT_DIRECTORY = True

    TWITTER_USERNAME = 'luca_chr'
    # DISQUS_SITENAME = "devsbytes"
    # SC_PROJECT = '10224955'
    # SC_SECURITY = '1f2cc438'

```

License
---------

mg is released under [the MIT License](http://opensource.org/licenses/MIT).


text:
# Benchwrapper

A small gRPC wrapper around the pubsub client library. This allows the
benchmarking code to prod at pubsub without speaking Go.

## Running

```bash
cd pubsub/internal/benchwrapper
export PUBSUB_EMULATOR_HOST=localhost:8080
go run *.go --port=8081
```

repository: googleapis/google-cloud-go

gha_language: Go

repo_name: googleapis/google-cloud-go


text:
# bit-trader (WIP!)

## Disclaimer
kindly taken from (https://github.com/butor/blackbird/)

__USE THE SOFTWARE AT YOUR OWN RISK.

## About

bit-trader is a simple bitcoin and altcoin trading bot / bot-platform for the exchange [Kraken](http://kraken.com).
It isn't affilated or related to Payward, Inc. (the company behind kraken), kraken was just the first exchange to be implemented.

The whole project is currently very alpha-ish but I have the base "micro-trades"-strategy already running on my account and it already produces some money.

Since the whole thing is work-in-progress, currently there is no getting started but developers familiar with nodejs should be able to start it easely.

If you wan't to support the development of this bot, my Bitcoin Address is:
```
1JWbKdnVHYVW5vcajsTwJpLFUeBntpqsw1
```
also you can contribute anytime through a pull-request ;)

gha_language: JavaScript


text:
# CSIS Technology Policy Program Data Governance Table

## Quick-start Instructions

```shell
$ git clone https://github.com/CSIS-iLab/tech-policy-database.git
$ cd tech-policy-database
$ npm install
$ npm start
```

## Version

Note that this project uses React v16.9 and should **not** be updated until the [React Scroll Sync](https://github.com/okonet/react-scroll-sync) plugin has been updated to use new lifecycle hooks.

## Usage

### Local Development

This will give you file watching, browser synchronization, auto-rebuild, CSS injecting, etc.

```shell
$ npm start
```

### Build for Production

```shell
$ npm run build
```

### See More Commands

This will display all available commands, such as running eslint or imagemin independently.

```shell
$ npm run
```

### Branching

When modifying the code base, always make a new branch. Unless it's necessary to do otherwise, all branches should be created off of `master`.

Branches should use the following naming conventions:

| Branch type               | Name                                                      | Example                     |
| ------------------------- | --------------------------------------------------------- | --------------------------- |
| New Feature               | `feature/<short description of feature>`                  | `feature/header-navigation` |
| Bug Fixes                 | `bug/<short description of bug>`                          | `bug/mobile-navigation`     |
| Documentation             | `docs/<short description of documentation being updated>` | `docs/readme`               |
| Code clean-up/refactoring | `refactor/<short description>`                            | `refactor/apply-linting`    |
| Content Updates           | `content/<short description of content>`                  | `content/add-new-posts`     |

When ready to merge, submit a Pull Request into `master`. All code will be reviewed by the lead developer on the project before being merged into `master`.

### Commit Messages

Write clear and concise commit messages describing the changes you are making and why. If there are any issues associated with the commit, include the issue # in the commit title.

### CSS Styles

- This project uses the [BEM](http://getbem.com/introduction/) naming convention.

## Copyright / License

Copyright © 2019 CSIS iDeas Lab under the [MIT License](https://github.com/ixkaito/frasco/blob/master/LICENSE).

repository: CSIS-iLab/tech-policy-database

repo_name: CSIS-iLab/tech-policy-database


text:
# Fake.Commerce
```julia
Fake.Commerce.color() # => "lavender"

# Optional arguments max=3, fixed_amount=false
Fake.Commerce.department() # => "Grocery, Health & Beauty"
Fake.Commerce.department(5) # => "Grocery, Books, Health & Beauty"
Fake.Commerce.department(2, true) # => "Books & Tools"

Fake.Commerce.material() # => "Steel"

Fake.Commerce.product_name() # => "Practical Granite Shirt"

Fake.Commerce.price() # => 44.6

# Generate a random promotion code.
# Optional argument digits = 6 for number of random digits in suffix
Fake.Commerce.promotion_code() # => "AmazingDeal829102"
```

gha_language: Julia