text:
# Preface
The installation is orchestrated with Git, Pip, and Bash scripting. The process should require no manual intervention on your part, as long as you're running the latest stable LTS release of Ubuntu Server. Kibana requires some minimal configuration, but it is all point-and-click in the Kibana interface and only needs to be done once.

# Minimum Requirements
Each instance should be installed on a fresh 64-bit [Ubuntu Server](https://www.ubuntu.com/download/server) machine with **systemd support**, and at least the minimum required resources as listed in the [main README document Requirements section](../README.md#requirements).

The installation script is incompatible with Ubuntu versions prior to 15.04 due to a shift to SystemD by Canonical. Earlier versions of Ubuntu can use SystemD with a fair amount of work, but the installation script isn't supported on non-SystemD platforms with workarounds implemented.

While installing everything on one server is good for proof-of-concept or testing, additional Elasticsearch nodes will greatly increase performance and provide failover. Having additional Elasticsearch nodes will also allow you to retain more flow data, and tune overall performance to your needs.

# Overview
1. [Installation](#installation)
    1. [Clone the Git Repository](#clone-the-git-repository)
    2. [Run the Installation Script](#installation-script)
    3. [Build Elasticsearch Indexes](#build-elasticsearch-indexes)
        1. [Netflow Index](#netflow-index)
        2. [sFlow Index](#sflow-index)
    4. [Elasticsearch Tuning](#elasticsearch-tuning)
    5. [Firewall (Optional)](#firewall-optional)
    6. [Kibana Authentication (Optional)](#kibana-authentication-optional)
    7. [Reboot](#reboot)
    8. [Verify Services](#verify-services)
    9. [Disable Unused Collector Services](#disable-unused-collector-services)
2. [Configure Devices](#configure-devices)
3. [Kibana](#kibana)
    1. [Access Kibana](#access-kibana)
    2. [Configure Index Patterns](#configure-index-patterns)
        1. [Netflow Index Pattern](#netflow-index-pattern)
        2. [sFlow Index Pattern](#sflow-index-pattern)
    3. [Set Special Byte Fields](#set-special-byte-fields)
    4. [Import Kibana Visualizations and Dashboards](#import-kibana-visualizations-and-dashboards)
4. [Tuning](#tuning)
5. [Updates](#updates)
6. [Elasticsearch Clustering](#elasticsearch-clustering)

# Installation
Follow the steps below in order and the installation should take no more than about 20 minutes. Copy-paste commands for easier installation.

## Clone the Git Repository
If you don't have Git installed on your Ubuntu Server machine that's OK, just run the following:
```
sudo apt-get install git
```

Clone the Git repository:
```
git clone https://gitlab.com/thart/flowanalyzer.git
```

The download should only take a moment. Move into the repo directory:
```
cd flowanalyzer
```

## Installation Script
The ubuntu_install_elk5.sh script handles almost everything, just be sure to run it with sudo privileges:
```
sudo sh ./Install/ubuntu_install_elk5.sh
```

The ubuntu_install_elk5.sh script does the following:

- Shifts the Ubuntu Server to UTC time and configures an NTP client so timestamps are accurate
- Adds software repos for Elasticsearch and Kibana v5
- Updates software repos and keys
- Installs Elasticsearch v5 pre-reqs (Curl, OpenJDK 8, etc)
- Installs Elasticsearch and Kibana v5
- Creates the following collector services for Netflow v5/v9, IPFIX (aka Netflow v10), and sFlow:
  - netflow_v5
  - netflow_v9
  - ipfix
  - sflow
- Registers collector services, sets auto-start on boot, and auto-restart on failure (30 seconds)
- Creates an index pruning job (>30 days old) using Curator in /etc/cron.daily

## Build Elasticsearch Indexes
Index templates need to be created in Elasticsearch, so when data is collected the fields will be assigned the right data types and proper indexing settings.

### Netflow Index
The build_netflow_index_elk5.sh script creates the default index for storing Netflow and IPFIX flow data in Elasticsearch:
```
sh ./Install/build_netflow_index_elk5.sh
```

This should output the following message, indicating the template has been created:
```
{
    "acknowledged" : true
}
```

Additional output may also be included with details depending on the version of Elasticsearch.

### sFlow Index
The build_sflow_index_elk5.sh script creates the default index for storing sFlow data in Elasticsearch. A separate script and index are used because sFlow data can be very different from network flow data depending on what sFlow counter records are being exported.

Run the sFlow index script:
```
sh ./Install/build_sflow_index_elk5.sh
```

This should output the following message, indicating the template has been created:
```
{
    "acknowledged" : true
}
```

Additional output may also be included with details depending on the version of Elasticsearch.

## Elasticsearch Tuning
One of the biggest ways to destroy Elasticsearch performance is to not properly allocate the right sized heap. The creators of Elasticsearch recommend setting
Elasticsearch to use [50% of the available memory](https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html#_give_less_than_half_your_memory_to_lucene)
on a given server, up to the 32GB limit.

The configuration set in the installation script sets Elasticsearch memory to 2GB, assuming a server with at least 4GB of RAM. If you're using a 4GB server as a PoC or just a small installation you don't need to change anything.

If you have more RAM, Heap size is increased by setting the following in the **/etc/default/elasticsearch** configuration file:
```
ES_JAVA_OPTS="-Xms2g -Xmx2g"
```

**Note**: If you have a server with more RAM then you need to adjust this value and reboot the server (or restart Elasticsearch and then the collector services).

## Firewall (Optional)
These are examples of commands you may need to use if you're running a firewall on the Ubuntu Server installation:
```
ufw allow from xxx.xxx.xxx.xxx/xx to any port 80 proto tcp comment "Kibana interface"
ufw allow from xxx.xxx.xxx.xxx/xx to any port 9200 proto tcp comment "Elasticsearch CLI"
ufw allow from xxx.xxx.xxx.xxx/xx to any port 2055,9995,4739,6343 proto udp comment "Flow data in"
```

## Kibana Authentication (Optional)
By default Kibana comes with no authentication capabilities. This typically isn't an issue if flow data is not considered confidential, and if the Flow Analyzer server is running in an isolated VLAN or management network enclave.

If you need authentication for Kibana access there are two available options:

- Reverse Proxy with NGINX or Squid
- [Elastic's X-Pack commercial product](https://www.elastic.co/products/x-pack)

Putting a reverse proxy in front of Kibana blocks access to the web interface until the user has authenticated. Once successfully authenticated the user has access to all Searches, Visualizations, and Dashboards. The reverse proxy isn't integrated with Kibana, it simply sits in front of the interface. See the [Squid.md](Squid.md) file for steps to configure the reverse proxy.

Elastic's X-Pack commercial product is an integrated solution that works with Kibana, and provides more granular authentication, permissions, and roles. If you're looking for robust authentication, integration with Active Directory, and granular access permissions then X-Pack is the best solution available.

## Reboot
It's important to reboot so that we're sure the services were registered and start correctly:
```
sudo reboot
```

## Verify Services
Once the Ubuntu instance comes back up verify that the services have started:
```
systemctl status elasticsearch
systemctl status kibana
systemctl status netflow_v5
systemctl status netflow_v9
systemctl status ipfix
systemctl status sflow
```

## Disable Unused Collector Services
By default all the collectors are configured to auto-start on boot and run in the background. This is done for your convenience, but if you're not using a particular protocol then there's no reason to keep its collector running. For example, if your devices don't support sFlow you can disable the sFlow collector:
```
sudo systemctl stop sflow
sudo systemctl disable sflow
```

If you aren't using IPFIX:
```
sudo systemctl stop ipfix
sudo systemctl disable ipfix
```

Next time the server reboots those collector processes won't start and won't use any server resources.

# Configure Devices
Configure your devices to send Netflow, IPFIX, and sFlow data to the Flow Analyzer collector. Consult your vendor's documentation for configuring Netflow v5, Netflow v9, and IPFIX. Also see the [Flow Management blog](http://www.manitonetworks.com/flow-management/) at manitonetworks.com for instructions on configuring Cisco, Ubiquiti, Mikrotik, and other platforms.

* [Ubiquiti IPFIX](http://www.manitonetworks.com/flow-management/2016/7/1/ubiquiti-ipfix-configuration)
* [Mikrotik Netflow v5](http://www.manitonetworks.com/flow-management/2016/7/1/mikrotik-netflow-configuration)
* [Mikrotik Netflow v9](http://www.manitonetworks.com/flow-management/2016/10/10/mikrotik-netflow-v9-configuration)
* [Cisco Netflow v9](http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/netflow/configuration/15-mt/nf-15-mt-book/get-start-cfg-nflow.html#GUID-2B7E9519-66FE-4F43-B8D3-00CA38C1FA9A)

Use the following ports for each respective collector service:

Service     | Protocol  | Port |
--------    | --------  | -------- |
Netflow v5  | UDP       | 2055 |
Netflow v9  | UDP       | 9995 |
IPFIX       | UDP       | 4739 |
sFlow       | UDP       | 6343 |

These ports can be changed, see the [tuning documentation](../Tuning.md). Make sure your devices are configured to send flow data before moving on to [configuring Kibana](#kibana).

**Note**: Configuring devices and receiving flows **before** moving onto the next step will make adding the default index in Kibana much easier.

# Kibana
A few things have to be done first in Kibana before you get to see the Visualizations and Dashboards.
Ensure that you've already [configured your devices](#configure-devices) to send flows, so by the time you get to this point there is already some flow data in the index for Kibana to recognize.

## Access Kibana
There are a couple ways to access Kibana, depending on if you're using the [reverse proxy for authentication](#kibana-authentication-optional) or not.

### No Reverse Proxy (default):
Browse to Kibana at http://your_server_ip:5601

### Using a Reverse Proxy (optional, only if configured)
1. Browse to Kibana at http://your_server_ip
2. Log in with the default Squid credentials you created during the [Squid configuration](Squid.md).

## Configure Index Patterns
On first logging in to Kibana you'll notice an orange **Warning** message that states:
> No default index pattern. You must select or create one to continue.

The Netflow and sFlow index scripts already created the index templates and your devices should have been sending flows already which triggers index creation, but we need to point Kibana in the right direction.

### Netflow Index Pattern
Use the following steps to point Kibana to the Netflow index:
1. Click **Index Patterns**
2. Click the **Add New** button in the upper-left if not already placed at the **Configure an index pattern** page
3. Under **Index name or pattern** enter __flow\*__
4. Click outside the entry field and it should automatically parse your input, revealing more information below

**Note**: If you haven't already configured your devices to send flows to the collector go back and [perform that configuration](#configure-devices) because the following steps won't work.

5. Leave the automatically selected **Time** field under **Time-field name**.
6. Click the **Create** button.
7. You will be taken to the **flow\*** index pattern.

### sFlow Index Pattern
Use the following steps to point Kibana to the sFlow index:
1. Click **Index Patterns**
2. Click the **Add New** button in the upper-left if not already placed at the **Configure an index pattern** page
3. Under **Index name or pattern** enter __sflow\*__
4. Click outside the entry field and it should automatically parse your input, revealing more information below

**Note**: If you haven't already configured your devices to send flows to the collector go back and [perform that configuration](#configure-devices) because the following steps won't work.

5. Leave the automatically selected **Time** field under **Time-field name**.
6. Click the **Create** button.
7. You will be taken to the **sflow\*** index pattern.

## Set Special Byte Fields
Most flow protocols report packet and data sizes in Byte units, which is fine for computers, but isn't very helpful when trying to present a human-readable dashboard. By setting the field formatting in Kibana we can automatically have number fields with raw octets displayed in KB, MB, GB, TB, etc.

### Netflow
Perform the following steps below on the following Netflow index fields:
- Bytes In
- Bytes Out

### sFlow
Perform the following steps below on the following sFlow index fields:
- Bytes In
- Bytes Out
- Bytes Read
- Bytes Written
- Current Allocation
- Memory Buffers
- Memory Cached
- Memory Free
- Memory Shared
- Memory Total
- Memory Used
- Total Available
- Total Capacity

### Steps to Set Byte Field Formatting
1. Click **Management** on the left navigation bar
2. Click **Index Patterns** at the top of the window
4. Select the index to modify fields in
5. Sort field names by clicking the **name** heading
6. Click the **pencil icon** in the far-right **controls** column to edit the field
7. Under the **Format** drop-down choose **Bytes**
8. Click **Update Field**

## Import Kibana Visualizations and Dashboards
1. Download the export.json from the Kibana directory.
2. In Kibana perform the following steps:
    1. Click **Management** on the left navigation bar
    2. Click **Saved Objects** at the top of the window
    3. Click the **Import** button
    4. Browse to the downloaded JSON file

# Tuning
There are additional features that you can utilize, but they have to be enabled by you. This includes:

 - [Reverse DNS Lookups](../Tuning.md#lookups) and Content Tagging
 - MAC Address Prefix Tagging (in development)

See [the tuning documentation](../Tuning.md) for how to enable these features, and recommendations for baseline settings.

# Updates
To get the latest updates do the following:

Change to the flowanalyzer directory and fetch the latest stable code via Git from the Master branch:
```
cd /your/directory/flowanalyzer
git pull
```

Any changes to files will be shown. If nothing has changed disregard the next steps that restart the collector services.

Restart the collector services you are actively using on your network (all are listed here for documentation purposes):
```
systemctl restart netflow_v5
systemctl restart netflow_v9
systemctl restart ipfix
systemctl restart sflow
```

# Elasticsearch Clustering
Elasticsearch works best in a cluster, and as your Elasticsearch cluster grows you'll get better performance and more storage. The default installation creates one instance of Elasticsearch, which is fine for testing or small organizations, but you'll get the best performance from two or three (or more!) instances of Elasticsearch. As you sample more traffic from more devices you can grow the cluster over time.

Fortunately almost everything you need is included in the default installation script, you just need to tweak a few options and Elasticsearch does the rest automatically.

See the [Elasticsearch Cluster file for steps to configure a cluster](Cluster.md).

# ---
**Copyright (c) 2017, Manito Networks, LLC**
**All rights reserved.**

repository: cpajr/flowanalyzer

repo_name: cpajr/flowanalyzer


text:
Minifiers

Several minifiers are available. Minification can take several seconds so it is recommended to [set up a cache directory](Introduction.md#speed-up-minification-with-a-cache).

### Google Closure Compiler service

This is the best choice for most users. The Closure Compiler service is [hosted by Google](https://developers.google.com/closure/compiler/docs/terms_ui?csw=1) and is accessible via HTTP. The default configuration gives the best results.

```php
$configurator = new s9e\TextFormatter\Configurator;
$configurator->enableJavaScript();
$configurator->javascript->setMinifier('ClosureCompilerService');
```

### Google Closure Compiler application

Alternatively, the [Google Closure Compiler application](https://developers.google.com/closure/compiler/docs/gettingstarted_app) can be used. This requires PHP to be able to use `exec()` to execute the relevant executables. Like the Google Closure Compiler service, configuration is automatic.

```php
$configurator = new s9e\TextFormatter\Configurator;
$configurator->enableJavaScript();

// Using the Java application
$configurator->javascript->setMinifier(
	'ClosureCompilerApplication',
	'java -jar /usr/local/bin/compiler.jar'
);

// You can use npx or a native executable
$configurator->javascript->setMinifier(
	'ClosureCompilerApplication',
	'npx google-closure-compiler'
);
```

### MatthiasMullie\\Minify

[Minify](https://www.minifier.org/) is a JavaScript minifier written in PHP. Its minification is not as extensive as Google's Closure Compiler but it is fast and does not use any external service. In order to use this minifier you must have Minify already installed.

```php
$configurator = new s9e\TextFormatter\Configurator;
$configurator->enableJavaScript();
$configurator->javascript->setMinifier('MatthiasMullieMinify');
```

## Meta-minifiers

### Noop

As the name implies, the no-op minifier will preserve the original source as-is. This is the default setting.

### FirstAvailable

The FirstAvailable strategy allows multiple minifiers to be set. They are executed in order and the result of the first successful minification is returned.

In the following example, we set up the ClosureCompilerService minifier to handle minification. If it fails, the MatthiasMullieMinify minifier will be executed. If it fails too, the Noop (no-op) minifier is executed as a fail-safe and the original source is returned.

```php
$configurator = new s9e\TextFormatter\Configurator;
$configurator->enableJavaScript();

$minifier = $configurator->javascript->setMinifier('FirstAvailable');
$minifier->add('ClosureCompilerService');
$minifier->add('MatthiasMullieMinify');
$minifier->add('Noop');
```

license: MIT

branch_name: refs/heads/master

extension: md

gha_language: PHP

gha_license_id: MIT

license_type: permissive


text:
# Integratej2objc

J2Objc and Gradle will generate your files and put them in a source directory, but they won't update your xcode project to actually reflect the new truth. This gem provides the integratej2objc executable that uses the xcodeproj gem to update your project.  Can be included as a last task after gradle does it's thing.

## Installation
Currently the gem is not deployed to rubygems.  The best way I've found to use it locally is with bundler

Gemfile entry:
```
gem 'integratej2objc', git:'https://github.com/developertown/integratej2objc.git'
```

`bundle; bundle exec integratej2objc help;`

## Usage

```
Usage:
  integratej2objc integrate_source -g, --group=GROUP -s, --source-root=SOURCE_ROOT -t, --target=TARGET -x, --xcodeproj=XCODEPROJ

Options:
  -x, --xcodeproj=XCODEPROJ      # relative or absolute path to yourproject.xcodeproj, including yourproject.xcodeproj
  -s, --source-root=SOURCE_ROOT  # the relatitve path from yourproject.xcodeproj to the generated sources directory.
  -g, --group=GROUP              # This is the path to the group you are targeting from the xcode project, in Xcode For example in Xcode Project Navigator you see:
           MyProject.xcodeproj
            └ MyProject
                └ generated
           would have a group argument: -g "MyProject/generated"
  -t, --target=TARGET            # the name of the target that generated sources should be added to in your xcode project

For use with any source directory and Xcode project. Removes GROUP and descendant files from XCODEPROJ and then adds all directories and files from SOURCE_ROOT, recursively, to the GROUP and TARGET
```

Worked example given a project structure:

* my_project/
   * shared/
      * generated_objc/
   * ios/
      * my_proj.xcodeproj/

and a group for your generated source files in the my_proj.xcodeproj called "generated" and a target called MyProj

from my_project/

```
integratej2objc integrate_source  -x ios/my_proj.xcodeproj -s ../shared/generated_objc -g generated -t MyProj
```

this will remove the existing `generated` group from my_proj.xcodeproj and all of it's children files. It will then add it back with a path relative to the project file and add all descendant .h and .m files.  these files will then be added to the target so that they will be compiled.

## Contributing

1. Fork it ( https://github.com/[my-github-username]/integratej2objc/fork )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request

gha_language: Ruby


text:
# NAME
uniq - report or omit repeated lines.

# SYNOPSIS

    uniq [OPTION]...

# DESCRIPTION
Print the unique lines in the standard input.

# OPTIONS

    -c
        Prefix lines by the number of occurrences.
    -d
        Only print duplicate lines.
    -u
        Only print unique lines.

# NOTES
The input must be sorted.

# SEE ALSO
- `sort(1)`

repository: mukaiizawa/Paren

gha_language: OpenEdge ABL

gha_license_id: NOASSERTION

repo_name: mukaiizawa/Paren


text:
# Kafka Listeners

tl;dr : You need to set advertised.listeners (or KAFKA_ADVERTISED_LISTENERS if you're using Docker images) to the external address (host/IP) so that clients can correctly connect to it. Otherwise they'll try to connect to the internal host address--and if that's not reachable then problems ensue

![image](../../media/Technologies-Kafka-Kafka-Listeners-image1.jpg)

In this post I'll talk about *why* this is necessary, and then show *how* to do it, based on a couple of scenarios - Docker, and AWS.

## Is anyone listening?

Kafka is a distributed system. Data is read from & written to the *Leader* for a given partition, which could be on any of the brokers in a cluster. When a client (producer/consumer) starts, it will request metadata about which broker is the leader for a partition - and it can do this from *any* broker. The metadata returned will include the endpoints available for the Leader broker for that partition, and the client will then use those endpoints to connect to the broker to read/write data as required.

It's these endpoints that cause people trouble. On a *single machine, running 'bare metal'* (no VMs, no Docker), everything might be the hostname (or just *localhost*) and it's easy. But once you move into more complex networking setups, and multiple nodes, you have to pay more attention to it.

Let's assume you have more than one network. This could be things like:

- Docker internal network(s) plus host machine
- Brokers in the cloud (eg. AWS EC2), and on-premises machines locally (or even in another cloud)

You need to tell Kafka how the brokers can reach each other, but also make sure that external clients (producers/consumers) can reach the broker they need to.

The key thing is that when you run a client, the broker you pass to it is just where it's going to go and get the metadata about brokers in the cluster from. The actual host & IP that it will connect to for reading/writing data is based on the data that the broker passes back in that initial connection - even if it's just a single node and the broker returned is the same as the one connected to.

For configuring this correctly, you need to understand that Kafka brokers can have multiple *listeners*. A listener is a combination of

1. Host/IP
2. Port
3. Protocol

Let's check out some config. Often the protocol is used for the listener name too, but here let's make it nice and clear by using abstract names for the listeners:

```bash
KAFKA_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://localhost:9092
KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_BOB:PLAINTEXT,LISTENER_FRED:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_BOB
```

*I'm using the Docker config names - the equivalents if you're configuring server.properties directly (e.g. on AWS etc) are shown indented in the following list*

- KAFKA_LISTENERS is a comma-separated list of listeners, and the host/ip and port to which Kafka binds to on which to listen. For more complex networking this might be an IP address associated with a given network interface on a machine. The default is <IP_ADDRESS>, which means listening on all interfaces.

- listeners

- KAFKA_ADVERTISED_LISTENERS is a comma-separated list of listeners with their the host/ip and port. This is the metadata that's passed back to clients.

- advertised.listeners

- KAFKA_LISTENER_SECURITY_PROTOCOL_MAP defines key/value pairs for the security protocol to use, per listener name.

- listener.security.protocol.map

*Kafka brokers communicate between themselves*, usually on the internal network (e.g. Docker network, AWS VPC, etc). To define which listener to use, specify KAFKA_INTER_BROKER_LISTENER_NAME (inter.broker.listener.name). The host/IP used must be accessible from the broker machine to others.

Kafka *clients* may well not be local to the broker's network, and this is where the additional listeners come in.

Each listener will, when connected to, report back the address on which it can be reached. *The address on which you reach a broker depends on the network used*. If you're connecting to the broker from an internal network it's going to be a different host/IP than when connecting externally.

When connecting to a broker, the listener that will be returned to the client will be the listener to which you connected (based on the port).

kafkacat is a useful tool for exploring this. Using -L you can see the metadata for the listener to which you connected. Based on the same listener config as above (LISTENER_BOB / LISTENER_FRED), check out the respective entries for **broker 0 at**: -

- Connecting on port 9092 (which we map as LISTENER_FRED), the broker's address is given back as localhost

```bash
$ kafkacat -b kafka0:9092 -L
Metadata for all topics (from broker -1: kafka0:9092/bootstrap):
1 brokers:
broker 0 at localhost:9092

- Connecting on port 29092 (which we map as LISTENER_BOB), the brokers address is given back as kafka0:

$ kafkacat -b kafka0:29092 -L
Metadata for all topics (from broker 0: kafka0:29092/0):
1 brokers:
broker 0 at kafka0:29092
```

You can also use tcpdump to examine the traffic from a client connecting to the broker, and spot the hostname that's returned from the broker.

## Why can I connect to the broker, but the client still fails?

*tl;dr* Even if you can make the initial connection to the broker, the address returned in the metadata may still be for a hostname that is not accessible from your client.

Let's walk this through step by step.

1. We've got a broker on AWS. We want to send a message to it from our laptop. We know the external hostname for the EC2 instance (ec2-54-191-84-122.us-west-2.compute.amazonaws.com). We've created the necessary entry in the security group to open the broker's port to our inbound traffic. We do smart things like checking that our local machine can connect to the port on the AWS instance:

```bash
$ nc -vz ec2-54-191-84-122.us-west-2.compute.amazonaws.com 9092
found 0 associations
found 1 connections:
1: flags=82<CONNECTED,PREFERRED>
outif utun5
src <IP_ADDRESS> port 53352
dst <IP_ADDRESS> port 9092
rank info not available
TCP aux info available

Connection to ec2-54-191-84-122.us-west-2.compute.amazonaws.com port 9092 [tcp/XmlIpcRegSvc] succeeded!
```

Things are looking good! We run:

`echo "test"|kafka-console-producer --broker-list ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092 --topic test`

Now... what happens next?

2. Our laptop resolves ec2-54-191-84-122.us-west-2.compute.amazonaws.com successfully (to the IP address <IP_ADDRESS>), and connects to the AWS machine on port 9092

3. The broker receives the inbound connection on port 9092. *It returns the metadata to the client, with the hostname ip-172-31-18-160.us-west-2.compute.internal* because this is the host name of the broker and the default value for listeners.

4. The client the tries to send data to the broker using the metadata it was given. Since ip-172-31-18-160.us-west-2.compute.internal is not resolvable from the internet, it fails.

```bash
$ echo "test"|kafka-console-producer --broker-list ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092 --topic test
>>[2018-07-30 15:08:41,932] ERROR Error when sending message to topic test with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0: 1547 ms has passed since batch creation plus linger time
```

5. Puzzled, we try the same thing from the broker machine itself:

```bash
$ echo "foo"|kafka-console-producer --broker-list ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092 --topic test

$ kafka-console-consumer --bootstrap-server ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092 --topic test --from-beginning
foo
```

It works fine! That's because we are connecting to port 9092, which is configured as the *internal* listener, and thus reports back its hostname as ip-172-31-18-160.us-west-2.compute.internal which *is* resolvable from the broker machine (since it's its own hostname!)

6. We can make life even easier by using [kafkacat](https://docs.confluent.io/current/app-development/kafkacat-usage.html). Using the -L flag we can see the metadata returned by the broker:

```bash
$ kafkacat -b ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092 -L
Metadata for all topics (from broker -1: ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092/bootstrap):
1 brokers:
broker 0 at ip-172-31-18-160.us-west-2.compute.internal:9092
```

Clear as day, the *internal* hostname is returned. This also makes this seemingly-confusing error make a lot more sense - connecting to one hostname, getting a lookup error on another:

```bash
$ kafkacat -b ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092 -C -t test
% ERROR: Local: Host resolution failure: ip-172-31-18-160.us-west-2.compute.internal:9092/0: Failed to resolve 'ip-172-31-18-160.us-west-2.compute.internal:9092': nodename nor servname provided, or not known
```

Here we're using kafkacat in producer mode (-C) from our local machine to try and read from the topic. As before, because we're getting the *internal* listener hostname back from the broker in the metadata, the client cannot resolve that hostname to read/write from.

I saw a StackOverflow answer suggesting to just update my hosts file...isn't that easier?

This is nothing more than a hack to workaround a mis-configuration, instead of actually fixing it.

If the broker is reporting back a hostname to which the client cannot connect, then hardcoding the hostname/IP combo into the local /etc/hosts may seem a nice fix. But this is a very brittle and manual solution. What happens when the IP changes, when you move hosts and forget to take the little hack with you, when other people want to do the same?

Much better is to understand and actually fix the advertised.listeners setting for your network.

## HOWTO: Connecting to Kafka on Docker

![image](../../media/Technologies-Kafka-Kafka-Listeners-image1.jpg)

Run within Docker, you will need to configure two listeners for Kafka:

1. Communication *within the Docker network*. This could be inter-broker communication (i.e. between brokers), and between other components running in Docker such as Kafka Connect, or third-party clients or producers.

For these comms, we need to use *the hostname of the Docker container(s)*. Each Docker container on the same Docker network will use the hostname of the Kafka broker container to reach it

2. Non-Docker network traffic. This could be clients running local on the Docker host machine, for example. The assumption is that they will connect on localhost, to a port exposed from the Docker container.

Here's the docker-compose snippet:

```yaml
kafka0:
    image: "confluentinc/cp-enterprise-kafka:5.0.0-rc3"
    ports:
    - '9092:9092'
    depends_on:
    - zookeeper
    environment:
    KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://localhost:9092
    KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_BOB:PLAINTEXT,LISTENER_FRED:PLAINTEXT
    [...]
```

3. Clients *within* the Docker network connect using listener "BOB", with port 29092 and hostname kafka0. In doing so, they get back the hostname kafka0 to which to connect. Each docker container will resolve kafka0 using Docker's internal network, and be able to reach the broker.

4. Clients *external* to the Docker network connect using listener "FRED", with port 9092 and hostname localhost. Port 9092 is exposed by the Docker container and so available to connect to. When clients connect, they are given the hostname localhost for the broker's metadata, and so connect to this when reading/writing data.

5. The above configuration would *not* handle the scenario in which a client external to Docker *and* external to the host machine wants to connect. This is because neither kafka0 (the internal Docker hostname) *or* localhost (the loopback address for the Docker host machine) would be resolvable.

## HOWTO: Connecting to Kafka on AWS/IaaS

*I'm naming AWS because it's what the majority of people use, but this applies to any IaaS/Cloud solution.*

Exactly the same concepts apply here as with Docker. The main difference is that whilst with Docker the external connections may well be just on localhost (as above), with Cloud-hosted Kafka (such as on AWS) the external connection will be from a machine not local to to the broker and which will need to be able to connect to the broker.

A further complication is that whilst Docker networks are heavily segregated from the host's, on IaaS often the *external* hostname is resolvable *internally*, making it hit and miss when you may actually encounter these problems.

There are two approaches, depending on whether the external address through which you're going to connect to the broker is also resolvable locally to all of the brokers on the network (e.g VPC).

### Option 1 - external address IS resolvable locally

![image](../../media/Technologies-Kafka-Kafka-Listeners-image2.jpg)

You can get by with one listener here. The existing listener, called PLAINTEXT, just needs overriding to set the advertised hostname (i.e. the one that is passed to inbound clients)

`advertised.listeners=PLAINTEXT://ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092`

Now connections both internally and externally will use ec2-54-191-84-122.us-west-2.compute.amazonaws.com for connecting. Because ec2-54-191-84-122.us-west-2.compute.amazonaws.com can be resolved both locally and externally, things work fine.

### Option 2 - external address is NOT resolvable locally

You will need to configure two listeners for Kafka:

1. Communication *within the AWS network (VPC)*. This could be inter-broker communication (i.e. between brokers), and between other components running in the VPC such as Kafka Connect, or third-party clients or producers.

For these comms, we need to use *the internal IP of the EC2 machine* (or hostname, if DNS is configured).

2. External AWS traffic. This could be testing connectivity from a laptop, or simply from machines not hosted in Amazon. In both cases, the external IP of the instance needs to be used (or hostname, if DNS is configured).

![image](../../media/Technologies-Kafka-Kafka-Listeners-image3.jpg)

Here's an example configuration:

```bash
listeners=INTERNAL://<IP_ADDRESS>:19092,EXTERNAL://<IP_ADDRESS>:9092
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
advertised.listeners=INTERNAL://ip-172-31-18-160.us-west-2.compute.internal:19092,EXTERNAL://ec2-54-191-84-122.us-west-2.compute.amazonaws.com:9092
inter.broker.listener.name=INTERNAL
```

## Exploring listeners with Docker

Take a look at <https://github.com/rmoff/kafka-listeners>. This includes a docker-compose to bring up a Zookeeper instance along with Kafka broker configured with several listeners.

- Listener BOB (port 29092) for internal traffic on the Docker network

```bash
$ docker run -t --network kafka-listeners_default confluentinc/cp-kafkacat

kafkacat -b kafka0:29092 -L

Metadata for all topics (from broker 0: kafka0:29092/0):
1 brokers:
broker 0 at kafka0:29092

- Listener FRED (port 9092) for traffic from the Docker-host machine (localhost)

$ docker run -t --network kafka-listeners_default confluentinc/cp-kafkacat

kafkacat -b kafka0:9092 -L

Metadata for all topics (from broker -1: kafka0:9092/bootstrap):
1 brokers:
broker 0 at localhost:9092

- Listener ALICE (port 29094) for traffic from outside, reaching the Docker host on the DNS name never-gonna-give-you-up

$ docker run -t --network kafka-listeners_default confluentinc/cp-kafkacat

kafkacat -b kafka0:29094 -L
Metadata for all topics (from broker -1: kafka0:29094/bootstrap):
1 brokers:
broker 0 at never-gonna-give-you-up:29094
```

## References

<https://rmoff.net/2018/08/02/kafka-listeners-explained>

gha_language: JavaScript


text:
# Visual Studio Code with IoT Hub Hands on Lab

This documentation for this lab has been moved to Microsoft Learn.

[Develop secure IoT Solutions for Azure Sphere with IoT Hub](https://docs.microsoft.com/en-us/learn/modules/develop-secure-iot-solutions-azure-sphere-iot-hub/)


text:
# Introduction to node_hello

Hello World app in Node Dockerize!

## Before you begin
In order to run this hello app in you machine you need to have the following software installed:
* Install [Docker](https://docs.docker.com/install/#supported-platforms)
* Install [NodeJS](https://nodejs.org/en/download)

## Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Create a local image of the hello app
```sh
docker build -t hello .
```

Run the newly create hello app docker image
```sh
docker run -p 3000:3000 -d hello
```

Test the [Hello World](http://localhost:3000) app
```sh
curl http://localhost:3000
```


text:
#Lower&Upper Bound
> The C++ standard library contains the following functions that are based on binary search and work in logarithmic time.

- lower_bound: returns a pointer to the first array element whose value is at least $x$.

- upper_bound: returns a pointer to the first array element whose value is larger than $x$.

- equal_range: returns both above pointers

> The functions assume that the array is sorted.

> If there is no such element, the pointer points to the element after the last array element.

> For example, the following code finds out whether an array contains an element with vlaue $x$.

```cpp
auto k = lower_bound(array, array + n, x) - array;
if (k < n && array[k] == x) {
    // x found at index k
}
```

> Then, the following code counts th enumber o f elements whose value is $x$

```cpp
auto a = lower_bound(array, array + n, x);
auto b = upper_bound(array, array + n, x);
cout << b - a << '\n';
```

Using `equal_range`, the code becomes shorter:

```cpp
auto r = equal_range(array, array + n, x);
cout << r.second - r.first << '\n';
```

## Finding the smallest solution
> An important use for binary search is to find the position where the value of a function change.

> Suppose that we wish to find the smallest vlaue $k$ that is a valid solution for a problem.

> We are given a function `ok(x)` that returns `true` if $x$ is valid solution and false otherwise.

> In addition, we know that `ok(x)` is `false` when $x < k$ and `true` when $x \geq k$.

> Now, the value of $k$ can be found using binary search.

```cpp
int x = - 1;
for (int b = z; b >= 1; b /= 2) {
    while (!ok(x + b)) x += b;
}
int k = x + 1;
```

> The search finds the largest value of $x$ for which `ok(x)` is `false`.

> Thus, the next value `k = x + 1` is the smallest possible value for which `ok(k)` is `true`.

> The initial jump length $z$ has to be lar enough, for exmaple some vlaue for which we know beforehand that `ok(z)` is `true`.

> The algorithm calls the function `ok` $\Omicron(lgz)$ times, so the total time complexity depends on the function `ok`.

> For example, if the function works in works in $\Omicron(n)$ time, the total time complexity is $Omicron(nlgz)$.

## Finding the maximum value
> Binary search can also be used to find the maximum value for a function that is first increasing and then decreasing. Our task is to find a poisition $k$ such that

- $f(x) < f(x + 1 )$ when $x < k $, and
- $f(x) > f(x + 1)$ when $ x \geq k$

> The idea is to use binary search for finding the largest value of $x$ for which $f(x) < f(x + 1)$.

> This implies that $ k = x + 1$ because $f(x+1) > f(x + 2).

```cpp
int x = -1;
for (int b = z; b >= 1; b /= 2) {
    while (f(x + b) < f(x + b + 1)) x += b;
}
int k = x + 1;
```
> ==Note that unlike in the ordinary binary search, here it is not allowed that consecutive values of the function are equal.==

> In this case it would not be possible to know how to continue the search.

repository: jchrys/icpc

repo_name: jchrys/icpc


text:
# NoughtPad <!-- omit in toc -->

This is a basic blog website made using Django and Bootstrap. It is my first ever project in Django, hence the Nought(`[0]`) in NoughtPad.

## Table of Contents <!-- omit in toc -->
- [Features](#features)
- [Screenshots](#screenshots)
- [Technologies](#technologies)
- [Setup Instructions](#setup-instructions)
- [Project Status](#project-status)
- [Contributions](#contributions)
- [Sources](#sources)

## Features

- Sign-in/Sign-up
- Adding, Editing, Deleting Notes
  - Using RichText in Note body
  - Uploading header images in Notes
- Liking anyone's Notes
- Author page
  - Profile card with profile picture, bio and external links
  - All Notes posted by the author
- Edit profile/user settings

## Screenshots

![Home Page](docs/images/screenshots/home.png)
![Author Page](docs/images/screenshots/author.png)

## Technologies

This project is built with:

* [Django 3.2.4](https://www.djangoproject.com/)
* [Bootstrap 4.4.1](https://getbootstrap.com/docs/4.4/getting-started/introduction/)

## Setup Instructions

Clone the repository:
```sh
$ git clone https://github.com/codelixir/nought-pad.git
$ cd nought-pad
```
Now if you wish, you can create ans activate a virtual environment (recommended).

You also need a Secret Key for Django. A good way to do it is storing `SECRET_KEY` as an environment variable.
```sh
$ export SECRET_KEY='your_secret_key'
```
and then change line 24 of [`settings.py`](noughtpad/settings.py) to:
```py
SECRET_KEY = os.environ['SECRET_KEY']
```
Another way of doing it is to put it in line 1 of [`keys.py`](noughtpad/keys.py), although this is not the best practice.

Then install the dependencies:
```sh
$ pip3 install -r requirements.txt
```
Once `pip` has finished downloading, you can run the server:
```sh
$ python3 manage.py runserver
```
and navigate to `http://<IP_ADDRESS>:8000/`

## Project Status

I will not be working on this project for now, but there are a number of improvements that can be made. For example:

- [ ] Making the navbar sticky
- [ ] Improving responsiveness
- [ ] Adding dark mode
- [ ] Making a separate profile page
- [ ] Option to upload in-text images in notes
- [ ] Adding comments to notes
- [ ] Adding categories to notes

## Contributions

Want to contribute? Awesome!

To fix a bug or add/enhance a feature (out of those listed above, or anything else you find interesting), follow these steps:

- Fork the repo
- Create a new branch (`git checkout -b improve-feature`)
- Make the appropriate changes in the files
- Add changes to reflect the changes made
- Commit your changes (`git commit -am 'Improve feature'`)
- Push to the branch (`git push origin improve-feature`)
- Create a Pull Request

If you want to add a feature not listed under [project status](#project-status), please open an issue [here](https://github.com/codelixir/nought-pad/issues/new) before working on it.

## Sources

This project is inspired by a Django tutorials playlist by [Codemy](https://codemy.com/).

---
[![forthebadge](https://forthebadge.com/images/badges/built-with-love.svg)](https://forthebadge.com)

repository: codelixir/nought-pad

branch_name: refs/heads/main

repo_name: codelixir/nought-pad


text:
[Hanlon](https://github.com/csc/Hanlon) Vagrant[ified]
==============
Please note that the provisioning shell "script" is being ported to chef-solo, puppet and ansible.

Getting Started
=====

Install Vagrant + VirtualBox
---
Requires [Vagrant](http://www.vagrantup.com/) and [VirtualBox](https://www.virtualbox.org/wiki/Downloads) - pick installers appropriate for your OS.

Set up Hanlon Server
---
Clone this repo:

```
git clone https://github.com/ap1kenobi/hanlon-vagrant ~/hanlon
cd ~/hanlon
vagrant up
```
Once it's finished provisioning (hopefully without errors), ssh to the VM:

```
vagrant ssh

```

and fire up Hanlon:

```
cd /opt/hanlon/script
./run-tri.sh
```

Notes
---
Only Hanlon Server and a couple of pre-requisites such as ipxe, tftpd and dhcpd, are deployed and configured.  See ```bootstrap.sh``` for details.

What still remains is automation of image and model creation - see manual steps below...

Set up provisioning
---
Open a second ssh connection:

```
vagrant ssh
cd /opt/hanlon
```

Create iPXE configuration:

```
./cli/hanlon config ipxe > /tftpboot/hanlon.ipxe
```

Get Hanlon Microkernel and add it to the image repo:

```
wget https://github.com/csc/Hanlon-Microkernel/releases/download/v1.0/hnl_mk_prod-image.1.0.iso
./cli/hanlon image add -t mk -p hnl_mk_prod-image.1.0.iso

```

Add CentOS ISO to the image repo:

```
wget http://reflector.westga.edu/repos/CentOS/6.5/isos/x86_64/CentOS-6.5-x86_64-minimal.iso
./cli/hanlon image add -t os -p CentOS-6.5-x86_64-minimal.iso -n CentOS65 -v 6.5

```

Create model:

```
./cli/hanlon model add -t centos_6 -l centos65-model -i K3v2suiIkSYNEdx1CTKqs

```
Create the policy:

```
./cli/hanlon policy add --template=linux_deploy -l centos65-policy -m 2J5Vb2ztkks9pz1iLrCxks -t memsize_1GiB --enabled true
```

Credits
===

```bootstrap.sh``` and the manual steps are based on Joe's work outlined in his wiki page: [here](https://github.com/jcpowermac/hanlon-wiki/blob/master/detailed-hanlon-install.md)

repository: toepi/hanlon-vagrant

repo_name: toepi/hanlon-vagrant


text:
# Notification engine

**Plugin is currently in Beta status. Proceed with caution.**

Adds support for sending notifications across a variety of different channels, including mail, SMS and Slack.

Notifications are managed in the back-end area by navigating to *Settings > Notification rules*.

## Notification workflow

When a notification fires, it uses the following workflow:

1. Plugin registers associated actions, conditions and events using `registerNotificationRules`
1. A notification class is bound to a system event using `Notifier::bindEvent`
1. A system event is fired `Event::fire`
1. The parameters of the event are captured, along with any global context parameters
1. A command is pushed on the queue to process the notification `Queue::push`
1. The command finds all notification rules using the notification class and triggers them
1. The notification conditions are checked and only proceed if met
1. The notification actions are triggered

Here is an example of a plugin registering notification rules. The `groups` definition will create containers that are used to better organise events. The `presets` definition specifies notification rules defined by the system.

```php
public function registerNotificationRules()
{
    return [
        'events' => [
            \Winter\User\NotifyRules\UserActivatedEvent::class,
        ],
        'actions' => [
            \Winter\User\NotifyRules\SaveToDatabaseAction::class,
        ],
        'conditions' => [
            \Winter\User\NotifyRules\UserAttributeCondition::class
        ],
        'groups' => [
            'user' => [
                'label' => 'User',
                'icon' => 'icon-user'
            ],
        ],
        'presets' => '$/winter/user/config/notify_presets.yaml',
    ];
}
```

Here is an example of triggering a notification. The system event `winter.user.activate` is bound to the `UserActivatedEvent` class.

```php
// Bind to a system event
\Winter\Notify\Classes\Notifier::bindEvents([
    'winter.user.activate' => \Winter\User\NotifyRules\UserActivatedEvent::class
]);

// Fire the system event
Event::fire('winter.user.activate', [$this]);
```

Here is an example of registering context parameters, which are available globally to all notifications.

```php
\Winter\Notify\Classes\Notifier::instance()->registerCallback(function($manager) {
    $manager->registerGlobalParams([
        'user' => Auth::getUser()
    ]);
});
```

Here is an example of an event preset:

```yaml
# ===================================
#  Event Presets
# ===================================

welcome_email:
    name: Send welcome email to user
    event: Winter\User\NotifyRules\UserRegisteredEvent
    items:
        - action: Winter\Notify\NotifyRules\SendMailTemplateAction
          mail_template: winter.user::mail.welcome
          send_to_mode: user
    conditions:
        - condition: Winter\Notify\NotifyRules\ExecutionContextCondition
          subcondition: environment
          operator: is
          value: dev
          condition_text: Application environment <span class="operator">is</span> dev
```

## Creating Event classes

An event class is responsible for preparing the parameters passed to the conditions and actions. The static method `makeParamsFromEvent` will take the arguments provided by the system event and convert them in to parameters.

```php
class UserActivatedEvent extends \Winter\Notify\Classes\EventBase
{
    /**
     * @var array Local conditions supported by this event.
        */
    public $conditions = [
        \Winter\User\NotifyRules\UserAttributeCondition::class
    ];

    /**
     * Returns information about this event, including name and description.
     */
    public function eventDetails()
    {
        return [
            'name'        => 'Activated',
            'description' => 'A user is activated',
            'group'       => 'user'
        ];
    }

    /**
     * Defines the usable parameters provided by this class.
     */
    public function defineParams()
    {
        return [
            'name' => [
                'title' => 'Name',
                'label' => 'Name of the user',
            ],
            // ...
        ];
    }

    public static function makeParamsFromEvent(array $args, $eventName = null)
    {
        return [
            'user' => array_get($args, 0)
        ];
    }
}
```

## Creating Action classes

Action classes define the final step in a notification and subsequently perform the notification itself. Some examples might be sending and email or writing to the database.

```php
class SendMailTemplateAction extends \Winter\Notify\Classes\ActionBase
{
    /**
     * Returns information about this event, including name and description.
     */
    public function actionDetails()
    {
        return [
            'name'        => 'Compose a mail message',
            'description' => 'Send a message to a recipient',
            'icon'        => 'icon-envelope'
        ];
    }

    /**
     * Field configuration for the action.
     */
    public function defineFormFields()
    {
        return 'fields.yaml';
    }

    public function getText()
    {
        $template = $this->host->template_name;

        return 'Send a message using '.$template;
    }

    /**
     * Triggers this action.
     * @param array $params
        * @return void
        */
    public function triggerAction($params)
    {
        $email =<EMAIL_ADDRESS>        $template = $this->host->template_name;

        Mail::sendTo($email, $template, $params);
    }
}
```

A form fields definition file is used to provide form fields when the action is established. These values are accessed from condition using the host model via the `$this->host` property.

```yaml
# ===================================
#  Field Definitions
# ===================================

fields:

    template_name:
        label: Template name
        type: text
```

An action may choose to provide no form fields by simply returning false from the `defineFormFields` method.

```php
public function defineFormFields()
{
    return false;
}
```

## Creating Condition classes

A condition class should specify how it should appear in the user interface, providing a name, title and summary text. It also must declare an `isTrue` method for evaluating whether the condition is true or not.

```php
class MyCondition extends \Winter\Notify\Classes\ConditionBase
{
    /**
     * Return either ConditionBase::TYPE_ANY or ConditionBase::TYPE_LOCAL
     */
    public function getConditionType()
    {
        // If the condition should appear for all events
        return ConditionBase::TYPE_ANY;

        // If the condition should appear only for some events
        return ConditionBase::TYPE_LOCAL;
    }

    /**
     * Field configuration for the condition.
     */
    public function defineFormFields()
    {
        return 'fields.yaml';
    }

    public function getName()
    {
        return 'My condition is checked';
    }

    public function getTitle()
    {
        return 'My condition';
    }

    public function getText()
    {
        $value = $this->host->mycondition;

        return 'My condition <span class="operator">is</span> '.$value;
    }

    /**
     * Checks whether the condition is TRUE for specified parameters
     * @param array $params
        * @return bool
        */
    public function isTrue(&$params)
    {
        return true;
    }
}
```

A form fields definition file is used to provide form fields when the condition is established. These values are accessed from condition using the host model via the `$this->host` property.

```yaml
# ===================================
#  Field Definitions
# ===================================

fields:

    mycondition:
        label: My condition
        type: dropdown
        options:
            true: True
            false: False
```

## Model attribute condition classes

Model attribute conditions are designed specially for applying conditions to sets of model attributes.

```php
class UserAttributeCondition extends \Winter\Notify\Classes\ModelAttributesConditionBase
{
    protected $modelClass = \Winter\User\Models\User::class;

    public function getGroupingTitle()
    {
        return 'User attribute';
    }

    public function getTitle()
    {
        return 'User attribute';
    }

    /**
     * Checks whether the condition is TRUE for specified parameters
     * @param array $params Specifies a list of parameters as an associative array.
        * @return bool
        */
    public function isTrue(&$params)
    {
        $hostObj = $this->host;

        $attribute = $hostObj->subcondition;

        if (!$user = array_get($params, 'user')) {
            throw new ApplicationException('Error evaluating the user attribute condition: the user object is not found in the condition parameters.');
        }

        return parent::evalIsTrue($user);
    }
}
```

An attributes definition file is used to specify which attributes should be included in the condition.

```yaml
# ===================================
#  Condition Attribute Definitions
# ===================================

attributes:

    name:
        label: Name

    email:
        label: Email address

    country:
        label: Country
        type: relation
        relation:
            model: Winter\Location\Models\Country
            label: Name
            nameFrom: name
            keyFrom: id
```

## Save to database action

There is a dedicated table in the database for storing events and their parameters. This table is accessed using the `Winter\Notify\Models\Notification` model and can be referenced as a relation from your own models. In this example the `MyProject` model contains its own notification channel called `notifications`.

```php
class MyProject extends Model
{
    // ...

    public $morphMany = [
        'my_notifications' => [
            \Winter\Notify\Models\Notification::class,
            'name' => 'notifiable'
        ]
    ];
}
```

This channel should be registered with the `Winter\Notify\NotifyRules\SaveDatabaseAction` so it appears as a related object when selecting the action.

```php
SaveDatabaseAction::extend(function ($action) {
    $action->addTableDefinition([
        'label'    => 'Project activity',
        'class'    => MyProject::class,
        'relation' => 'my_notifications',
        'param'    => 'project'
    ]);
});
```

The **label** is shown as the related object, the **class** references the model class, the **relation** refers to the relation name. The **param** defines the parameter name, passed to the triggering event.

So essentially if you pass a `project` to the event parameters, or if `project` is a global parameter, a notification model is created with the parameters stored in the `data` attribute. Equivalent to the following code:

```php
$myproject->my_notifications()->create([
    // ...
    'data' => $params
]);
```

## Dynamically adding conditions to events

Events can be extended to include new local conditions. Simply add the condition class to the event `$conditions` array property.

```php
UserActivatedEvent::extend(function($event) {
    $event->conditions[] = \Winter\UserPlus\NotifyRules\UserLocationAttributeCondition::class;
});
```

repository: wintercms/wn-notify-plugin

repo_name: wintercms/wn-notify-plugin


text:
# ADR 012: State Accessors

## Changelog

* 2019 Sep 04: Initial draft

## Context

Cosmos SDK modules currently use the `KVStore` interface and `Codec` to access their respective state. While
this provides a large degree of freedom to module developers, it is hard to modularize and the UX is
mediocre.

First, each time a module tries to access the state, it has to marshal the value and set or get the
value and finally unmarshal. Usually this is done by declaring `Keeper.GetXXX` and `Keeper.SetXXX` functions,
which are repetitive and hard to maintain.

Second, this makes it harder to align with the object capability theorem: the right to access the
state is defined as a `StoreKey`, which gives full access on the entire Merkle tree, so a module cannot
send the access right to a specific key-value pair (or a set of key-value pairs) to another module safely.

Finally, because the getter/setter functions are defined as methods of a module's `Keeper`, the reviewers
have to consider the whole Merkle tree space when they reviewing a function accessing any part of the state.
There is no static way to know which part of the state that the function is accessing (and which is not).

## Decision

We will define a type named `Value`:

```go
type Value struct {
  m   Mapping
  key []byte
}
```

The `Value` works as a reference for a key-value pair in the state, where `Value.m` defines the key-value
space it will access and `Value.key` defines the exact key for the reference.

We will define a type named `Mapping`:

```go
type Mapping struct {
  storeKey sdk.StoreKey
  cdc      *codec.LegacyAmino
  prefix   []byte
}
```

The `Mapping` works as a reference for a key-value space in the state, where `Mapping.storeKey` defines
the IAVL (sub-)tree and `Mapping.prefix` defines the optional subspace prefix.

We will define the following core methods for the `Value` type:

```go
// Get and unmarshal stored data, noop if not exists, panic if cannot unmarshal
func (Value) Get(ctx Context, ptr interface{}) {}

// Get and unmarshal stored data, return error if not exists or cannot unmarshal
func (Value) GetSafe(ctx Context, ptr interface{}) {}

// Get stored data as raw byte slice
func (Value) GetRaw(ctx Context) []byte {}

// Marshal and set a raw value
func (Value) Set(ctx Context, o interface{}) {}

// Check if a raw value exists
func (Value) Exists(ctx Context) bool {}

// Delete a raw value value
func (Value) Delete(ctx Context) {}
```

We will define the following core methods for the `Mapping` type:

```go
// Constructs key-value pair reference corresponding to the key argument in the Mapping space
func (Mapping) Value(key []byte) Value {}

// Get and unmarshal stored data, noop if not exists, panic if cannot unmarshal
func (Mapping) Get(ctx Context, key []byte, ptr interface{}) {}

// Get and unmarshal stored data, return error if not exists or cannot unmarshal
func (Mapping) GetSafe(ctx Context, key []byte, ptr interface{})

// Get stored data as raw byte slice
func (Mapping) GetRaw(ctx Context, key []byte) []byte {}

// Marshal and set a raw value
func (Mapping) Set(ctx Context, key []byte, o interface{}) {}

// Check if a raw value exists
func (Mapping) Has(ctx Context, key []byte) bool {}

// Delete a raw value value
func (Mapping) Delete(ctx Context, key []byte) {}
```

Each method of the `Mapping` type that is passed the arguments `ctx`, `key`, and `args...` will proxy
the call to `Mapping.Value(key)` with arguments `ctx` and `args...`.

In addition, we will define and provide a common set of types derived from the `Value` type:

```go
type Boolean struct { Value }
type Enum struct { Value }
type Integer struct { Value; enc IntEncoding }
type String struct { Value }
// ...
```

Where the encoding schemes can be different, `o` arguments in core methods are typed, and `ptr` arguments
in core methods are replaced by explicit return types.

Finally, we will define a family of types derived from the `Mapping` type:

```go
type Indexer struct {
  m   Mapping
  enc IntEncoding
}
```

Where the `key` argument in core method is typed.

Some of the properties of the accessor types are:

* State access happens only when a function which takes a `Context` as an argument is invoked
* Accessor type structs give rights to access the state only that the struct is referring, no other
* Marshalling/Unmarshalling happens implicitly within the core methods

## Status

Proposed

## Consequences

### Positive

* Serialization will be done automatically
* Shorter code size, less boilerplate, better UX
* References to the state can be transferred safely
* Explicit scope of accessing

### Negative

* Serialization format will be hidden
* Different architecture from the current, but the use of accessor types can be opt-in
* Type-specific types (e.g. `Boolean` and `Integer`) have to be defined manually

### Neutral

## References

* [#4554](https://github.com/cosmos/cosmos-sdk/issues/4554)

repository: cosmos/cosmos-sdk

gha_language: Go

repo_name: cosmos/cosmos-sdk


text:
`build01.nix-community.org`

We provide an `x86_64-linux` build machine as a public remote builder for the nix community.

#### Access

If you want access read the security guide lines on [aarch64-build-box](https://github.com/nix-community/aarch64-build-box). Than add your username to [`builder/users.nix`](https://github.com/nix-community/infra/blob/master/modules/nixos/community-builder/users.nix). Don't keep any important data in your home! We will regularly delete `/home` without further notice.

#### Using your NixOS home-manager configuration on the hosts

If you happen to have your NixOS & home-manager configurations intertwined but you'd like your familiar environment on our infrastructure you can evaluate `pkgs.writeShellScript "hm-activate" config.systemd.services.home-manager-<yourusername>.serviceConfig.ExecStart` from your NixOS configuration, and send this derivation to be realized remotely: (in case you aren't a Nix trusted user)

```console
# somehow get the .drv of the above expression into $path
$ nix copy --to ssh://build01.nix-community.org --derivation $path
$ ssh build01.nix-community.org
$ nix-store -r $path
$ $path
```

_(My [implementation](https://github.com/ckiee/nixfiles/blob/aac57f56e417e31f00fd495d8a30fb399ecbc19b/deploy/hm-only.nix#L10) of [this](https://github.com/ckiee/nixfiles/blob/aac57f56e417e31f00fd495d8a30fb399ecbc19b/bin/c#L92-L95) ~ckie)_

repository: nix-community/infra

gha_language: Nix

repo_name: nix-community/infra


text:
Functions are Objects

Consequently: (PREVIEW of IMPLICATIONS)

1) Multiple Creation forms

* function name() {}
```
function plus(x,y) { return x+y;}
```

* name = function() {}
```
var plus = function(x,y) { return x+y;}
```

* name = new Function()
```
var plus = new Function('x','y','return x+y;');
```

2) name of a function is a variable holding a reference

3) functions have no intrinsic name

4) functions can be shared (e.g. two objects sharing one method)

5) functions can have properties

6) Functions can have methods

7) Functions can be passed as arguments to other functions

8) Functions are subject to operators

---

3 invocations of functions:

* definition
* call/use
* mention/reference

1) Defining function creates object, returns reference to it
[Diagram function obj]

3) Mention returns a reference to that fn

CALL OPERATOR
2) Call is operator: takes a function reference + args,
builds scope for that call, outputs return value associated with that scope

---

Consider this:
`f(f)`

Hint: first consider this: `f(obj)`

Write a function f for which f(f) does something sensible
(i.e. what sort of function might reasonably receive itself as an argument)

(Ans: a function which does anything reasonable to an object!  "Function, whatever you do, do it to yourself")

```
function f(obj) {  // f --> paintObjBlue
  obj.color = 'blue';
}

function f(obj) {  // f --> wrapObj
  return {inner:obj};
}

function f(obj) {  // f --> getKeysOf
  return Object.keys(obj);
}

function f(obj) { // f --> isFunction
  return (typeof obj === 'function');
}
```

---

Deferred eval

```
function someFn() { blahblahblah;}
someFn; //OK
typeof someFn; //OK
var alias = someFn; //ok
```

---

SCOPES contain variables:

* all local vars declared with _var_, and
* all normal parameters,
* and 2 special parameters:
  * _this_
  * _arguments_

```
function plus(x,y) { return x+y; } //[FIG 1a]
var sum = plus(1,2); //[FIG 1b]
sum = plus(3,4); //[FIG 1c]
```

---

LEXICAL CONTEXT (CLOSURE)

When fn is called, an new environment/scope is created.

When fn is created, retains secret link to the scope which defined it, called 'closure'

'Closure'="Where Function was born"

[FIG2]
Add detail to Fig1: Dotted arrow from _plus_ Function back to global context.
Arrow called: "lexical context":
 scope/context surrounding the function object.
It's where function goes to look for any variables not declared locally.

(Etymology: variables which are not declared locally are "open",
JS must "close" them to use them; closure is where to look to close them)

Closure arrow is dotted b/c not a normal reference:
*  Can't follow it / get its value
*  Refers to a scope, not an object

Represents scope nesting, how to find variables

[FIG3]
```
var x=0;
function plusX(y) {
    return x+y;
} //[FIG3a]
x = plusX(1); //[FIG3b]
x = plusX(5); //[FIG3c]
```

---

```
var x = 1;
function outerA() {
  var x=2;
  return x;
} //[FIG4a]
outerA() //2 [FIG4b]
```

```
var x = 1;
function outerA() {
  var x = 2;
  return outerB();
}
function outerB() {
  return x;
}  //[FIG5a]
outerB() //1 [FIG5b]
outerA() //1 [FIG5c]
```
---

NESTED EXAMPLE

```
function outer() {
  function inner() {}
} //[Fig6a]
// So far, only outer is built
outer(); //[Fig6b]
// Now: both built, only outer has scope
```

How to call inner()?  Can't from console; need outer to do it:

```
function outer() {
  function inner() {}
  return inner();
}  //[Fig7a]
outer(); // [Fig7b]
```

---

NESTED with CLOSURE

```
var x = 1;
function outer() {
  var x = 2;
  function inner() {
    return x;
  }
  return inner();
}  // [Fig8a]
var y = outer() //2 [Fig8b]
```

---
<!--  Fork in road; remember this spot...

CLOSURE prevents SCOPE COLLAPSE
allows PERSISTENT ("static") SCOPE

As before, with one tiny change:

```
var x = 1;
function outer() {
  var x = 2;
  function inner() {
    return x;
  }
  return inner; //<<- change from inner() to inner
}  // [Fig9a]
var y=outer() //inner [Fig9b]
y() //2
```

NOTICE: inner still needs the "page" where the variable for its _x_ is written.
That scope created by outer(), which serves as the closure for inner,
if forced to persist even once outer() returns!

The scope is now a "secret space": it's not directly accessible, nor can its variables be addressed directly from outside, but they can still be referenced and set _indirectly_.

-->

---

Functions are Objects (Reprise)

Consequently:

#1) Multiple Creation forms
* function name() {}
* name = function() {}
* name = new Function()

#2) name of a function is a variable holding a reference

#3) functions have no intrinsic name
	a) can be aliased: multiple references to same function-obj
	b) can be temporarily anonymous

#8) Functions are subject to operators
Call operator:
()

---
THIS

#4) functions can be shared (e.g. two objects sharing one method)
*  _this_ depends on how fn is called:
```
function myName() {return this.name; }
var a = {name:'a', myName:myName};
var b = {name:'b', myName:myName};
a.myName(); //'a'
b.myName(); //'b'
```

Reinterpretation of _this_:
_this_ is the object holding the reference we followed to get here

---

#5) functions can have properties
* Function.length  (automatic)
Custom:

* plus.sideEffects = false;
* console.log.sideEffects = true;
* perpetualMotion.status = 'needs debugging'

---

#6) Functions can have methods
* Function.apply
* Function.call
```
function paintThis(newcolor) {
   this.color = newcolor;
}
var car {color:'blue', paint:paintThis}
car.paintThis('red');
car.color //

// another object may borrow it:
var fence = {color:'white'};
paintThis.call(fence,'red');
// OR
car.paintThis.call(fence,'red');
// either way:
fence.color //red

//equivalent to:
fence.paint = paintThis;
fence.paint('red');
delete fence.paint;
```

Call method means: 'function, lend yourself to obj as if it were your owner'
OR: call function 'through this object'
vs normal call operator: call function through whatever obj used to name it

---

GLOBAL OBJECT

1) It's an environment:
a) Contains variables declared w. _var_
b) Lexical context for global functions
c) Execution context: place for JS robot to stand when interpreting code, meaning of 'this'

In other execution contexts (i.e. in methods), 'this' is owner object, the object with a reference property we followed to get to the function.
By that interpretation...

[DRAW global var reference to function]
... in a global function, 'this' means Global Object Itself!

2) It's an object!
a) contains properties: every variable is also a property of global object;
therefore can delete global variables
b) contains methods: every global function is a method of global object, so 'this' points to it
c) reference properties and variables may refer to it.

It's the only object in JS which is both an Object and a Scope!

In browser:
this === window
window === this.window
window.window.window.window === this

In Node:
this === global
etc.

---

7) Functions can be passed as arguments to other functions
Callbacks

function callFunct(fn) {
  return fn();
}
function callFunctWithArg(fn,arg) {
  return fn(arg);
}

Array.map
Array.forEach
Array.sort

```
var result = {};
function addPair(key) {
    result[key] = key;
}
var keys = ['one','two','three','four'];
keys.forEach(addPair);

result={};
keys.forEach(function (key) {
              result[key]=key;
            });

function addPairToObj(key,obj) {
    obj[key] = key;
}
var obj = {};
keys.forEach(function (key) {
              addPairToObj(key,obj);
            });
```

---

SORTING ARRAY

Array.sort(compareFn)...

function compareNums(x,y) {
  return x-y;
}
var arr = [5,8,2,4,9,0,1];
arr.sort(compareNums);
arr //

---

<!--
Nested functions:
```

function capitalizeWord(word) {
	 return word.charAt(0).toUpperCase() + word.slice(1).toLowerCase();
}
function normalizeTitleWord(word) {
  if (word==='a' || word==='the') {
     return word.toLowerCase;
  return capitalizeWord(word);
}

function normalizeName(name) {
  var words = name.split(' ');
  if (words.length > 0)
     words[0]=capitalizeWord(words[0]);
  for (var i=1; i < words.length; i++) {
         words[i]=normalizeTitleWord(words[i]);
-->

---

Factory pattern

```
function makeThing(val) {
	 var thing = {};
	 thing.value = val;
	 thing.personalMethod = function() {};

	 thing.sharedMethod = makeThing.sharedMethod;
	 return thing;
}
makeThing.sharedMethod = function() {
		   return this.value;
	}
var thing1 = makeThing(1);
var thing2 = makeThing(2);
thing1.sharedMethod(); //1
thing2.sharedMethod(); //2
//Problems:
//
```

repository: portlandcodeschool/js-night

repo_name: portlandcodeschool/js-night


text:
# Pier 11
Pier 11 is my experimental Vagrant machine for building and utilizing development Docker containers, particularly when running macOS as a host machine.

## Vision
The three primary inconveniences I am trying to avoid are:

1. [Experiencing a dramatic I/O slowdown of mounted Docker volumes on macOS](https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/276)
2. Needing to rebuild my containers after each code change
3. Deciding whether I need to be SSH’d into a VM before running a command

Born were Pier 11 as well as the [`pier` and `moor` command line tools](https://github.com/lightster/pier-cli).

## Requirements
Before installing Pier 11, you will need to download and install:

 - [Vagrant](https://www.vagrantup.com/)
 - [VirtualBox](https://www.virtualbox.org/)

If you have [Brew](https://brew.sh/), you can install Vagrant and Virtualbox via Brew:

```bash
brew cask install vagrant
brew cask install virtualbox
```

## Installation

```bash
git clone git@github.com:lightster/pier-11.git

cd pier-11
make install
```

Additionally, if you want to be able to use `moor cd` to change directories between projects, make sure to add the following to your `~/.bashrc`:

```bash
source /path/to/pier-11/codebase/lightster/pier-cli/bin/bash_functions.sh
```

## The backstory
My motivation is to create an easy to setup environment for running development Docker containers without experiencing
[a dramatic I/O slowdown of mounted volumes](https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/276) that exists in Docker when running on macOS.

I also consider it unacceptable as a developer to need to rebuild a container between making a code change and running tests.  I want to be able to test changes quickly and not wait for a build to finish.

As I began to brainstorm ways to do achieve this, I also recognized I did not want the overhead of deciding if I needed to SSH into a VM to run a command or to run it on the host machine.  I have maintained a Vagrant development environment at work for a few years, and I often have asked and have been asked:

> I need to run `git clone` on my Mac? I should run `docker-compose` on the VM? What about the `make` command—should I run that on the VM or my Mac?

I decided that to avoid the dramatic I/O slowdown of mounted volumes on macOS without having to rebuild my containers constantly, I would need to run my containers on a VM.  In order to also avoid inconvenience #3, I decided I would build relatively simple tools that I can run on my Mac that act as a proxy to run commands on the VM.


text:
# Live USB Installer

This folder contains scripts and resources to facilitate the creation of a Live USB installer.

It uses the [Ubuntu Live CD/USB
process](https://help.ubuntu.com/community/LiveCDCustomizationFromScratch)
([casper](http://manpages.ubuntu.com/manpages/wily/man7/casper.7.html)) with some modifications to
create an installer that runs under a GUI and copies a pre-installed set of packages + files to a
target system.

## Benefits over the previous installer

* No need for USB workarounds on fresh systems
  ([AmericanRedCross/posm#116](https://github.com/AmericanRedCross/posm/issues/116))
* Installs quickly
* Doesn't require an internet connection
* Distributed as an ISO (for Legacy BIOS support + VMs)
* Consistently installs a known set of dependencies / versions, simplifying validation
* Theoretically more CI-friendly (depends if LXD is installable in CI environments)
* Image can be created in a controlled environment with an HTTP cache / fast internet

## Downsides over the previous installer

* Less control over disk partitioning (creates single `/` partition)
* Larger initial download (full ISO)
* Not directly suited for cloud hosts (although it uses the same bootstrapping process)

## Why LXD?

If you read the
[LiveCDCustomizationFromScratch](https://help.ubuntu.com/community/LiveCDCustomizationFromScratch)
wiki, you'll see that `chroot`s are used extensively. `initctl` is diverted and
[`policy-rc.d`](https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt) is used to
prevent services from starting during installation. For producing a base system installer (i.e.
Ubuntu Desktop), this is desirable.

However, POSM needs to access a running PostgreSQL instance in order to initialize OSM (etc.). It's
possible to do this in a `chroot`, but with a high likelihood of port and device conflicts. Using
something that uses `cgroups` (e.g. Docker or LXC) makes it feasible.

[LXD](https://www.ubuntu.com/cloud/lxd) is an Ubuntu (etc.) wrapper around LXC that makes it behave
like a VM host, even allowing containers to run without privileges (so the full gamut of users
present in a container map to the user who invoked it). Guests are persistent, so commands can be
executed as though they were targeting a `chroot`ed environment while being able to access services
that were started (after removing the `initctl` diversion).

## ISO Creation

To create a Live Installer ISO, run:

```bash
make
```

This will produce `live-cd.iso`.

The git repo and branch used to bootstrap the installer can be provided as environment variables,
allowing for builds containing experimental features:

```bash
GIT_REPO=https://github.com/mojodna/posm-build GIT_BRANCH=integration make
```

To clean both the file artifacts and to remove the container used for installer creation
(appropriately named after adjectified animals, e.g. `noble gopher` or `promoted mastodon`), run:

```bash
make clean
```

LXD snapshots are created during each phase of the bootstrap process, allowing one to rollback to a
specific stage and re-run it if it failed. To do so, remove each of the marker files (e.g. `admin`,
`base`), delete newer snapshots (if using ZFS; `lxc delete $(cat container)/admin`), and restore the
parent of the stage you wish to re-run (`lxc info $(cat container)` provides a sequential list of
snapshots and therefore stages): `lxc restore $(cat container) <stage>`

This current snapshot recovery process is awkward, but may prove helpful under some circumstances.
In practice, I usually run `make clean` and start from scratch.

If you'd like to investigate the container after bootstrapping a specific phase, you can do this:

```bash
# start the container if it had been stopped, i.e. after postinstall
lxc start $(cat container)

# invoke a shell within the container
lxc exec $(cat container) bash
```

gha_language: Shell


text:
# Introduction to CM-Well #

[What is CM-Well?](#hdr1)

[CM-Well as a Platform for Data Modeling](#hdr3)

[How Do I Access CM-Well?](#hdr4)

[Why Should I Use CM-Well?](#hdr5)

[A CM-Well Use Case: Open PermID](#hdr6)

[Getting Started with CM-Well](#hdr7)

## What is CM-Well? ##
CM-Well is a writable [Linked Data](https://en.wikipedia.org/wiki/Linked_data) repository, which can be used to model data from various datasets.

CM-Well (Content Matrix Well) adheres to Open Data principles, meaning that its data is in a standard, machine-readable format, and is open to access by anyone in Thomson Reuters. (Currently access from outside TR is allowed only for TR products.)

CM-Well's data is represented as a graph database, which means that it contains both entities and relationships between pairs of entities. For instance, it  contains facts such as "CompanyA is a subsidiary of CompanyB", or "PersonX works at CompanyA". Conceptually, a graph database is a structure composed of nodes representing entities, and directed arrows representing relationships among them. See the image below for an example.

## CM-Well as a Platform for Data Modeling ##
CM-Well can be used to build your own linked data repository. In addition to API calls for reading CM-Well information, there are calls for creating new entries and connections.

For example, you could create a data repository describing countries, their capitals, continents and currencies. For each of these entities, you would create a corresponding entity in CM-Well, with appropriate relationships between them (e.g. "Paris" - "is capital of" - "France").

## How Do I Access CM-Well? ##

Entities and their related information can be accessed by a URI (Uniform Resource Identifier). A URI looks like an internet link, but it points to a data structure rather than a typical web page.

Here is an example of a CM-Well URI:
[http://cm-well-host.com/permid.org/1-5035948006](http://cm-well-host.com/permid.org/1-5035948006)

The "5035948006" identifier at the end is a company's TR PermID. If you access this URI in a browser within Thomson Reuters you will see this page:

You can also access CM-Well programatically, by making REST API calls.
Go to Getting Started with CM-Well to learn more.

## Why Should I Use CM-Well? ##
CM-Well has many advantages and convenient features, including:

* Easy-to-use Linked Data sharing as a service.
* Direct access to entity info via URI.
* Does not require installation.
* Does not require downloading large files and subsequent processing and database insertion.
* Contains up-to-the-minute current info, as well as all historical versions of entities.
* Supports retrieval, querying and filtering of content at different levels of granularity - from a single entity, to a stream of updates, to the entire repository.
* Enables navigation among related entities via the links returned within an entity's info.
* Allows you to create your own data repository.
* As opposed to standard triple-stores that focus only on reading relatively static data, CM-Well is specifically designed to handle data that is constantly updated. This includes features such as convenient upload, handling historical versions, subscribing to real-time updates, and more.

## A CM-Well Use Case: Open PermID ##
The [Thomson Reuters Open PermID](https://permid.org/) platform is a beta-release service that allows anyone to look up companies, organizations, instruments and quotes. The matched entity is returned with a TR "PermID", which is a unique, permanent identifier for the entity. This is important for applications that must work with verified financial entity IDs, rather than name strings that can be ambiguous or even incorrect.

Open PermId users can view entity information in a browser, as shown below.

Programs can access the same information via the Open PermID API.

CM-Well is the underlying information provider for the Open PermID platform.

## Getting Started with CM-Well ##

Here are some more topics to help you get started with CM-Well:

[Technical Aspects of CM-Well](Intro.TechnicalAspectsOfCM-Well.md)
[CM-Well Data Paradigms](Intro.CM-WellDataParadigms.md)
[CM-Well Input and Output Formats](API.InputAndOutputFormats.md)
[Overview of the CM-Well API](Intro.OverviewOfTheCM-WellAPI.md)
[CM-Well Environments](CM-WellEnvironments.md)
[Basic Queries](DevGuide.BasicQueries.md)

repository: PJLeyden/CM-Well

repo_name: PJLeyden/CM-Well


text:
Contribution Guide
===================

Welcome! Here you can find guidance on the best way to contribute to this repo.

Bugs Reports / Feature Requests
---------------------------------

We are happy to receive feature request or bug reports on the GitHub [issue tracker].

A bug report is most useful if it gives detailed, *reproducible* instructions. Additionally, it should include:

* the version.
* the exact arguments used.
* the exact config used.
* the output received.
* the output you expected instead.
* screenshots if available.

This will allow us to help you more quickly. A template is included via `.github/ISSUE_TEMPLATE/bug.md` as well.

Pull Requests
-------------

We appreciate PRs but encourage you to follow these guidelines for the reviewer:

* Provide context to the reviewer in the "Description" of the PR.
* Add/update tests for the code modified.
* PRs should be kept small and cohesive. If you're making bigger change, consider chunking up the PR into small pieces.

Github actions are used to run tests on the latest ubuntu:20.04 image.

Commits
-------

Please follow the usual guidelines for git commits: keep commits atomic, self-contained, and add a brief but clear
commit message. This [guide](https://chris.beams.io/posts/git-commit/) by Chris Beams is a good resource if you'd like
to learn more.

However, don't fret over this too much. You can also just accumulate commits without much thought for this rule. All
commits in a PR can be squashed into a single commit upon merging. It is still appreciated if the commit message
doesn't need to be rewritten.

Releases
--------

* Use tags for each Github release.
* Avoid checking in large files into the repo. Instead, attach files as artifacts to Github releases.

gha_language: Python


text:
# Title: Source Code Control Provider Sample
**Abstract:** Provides a Source Code Control Provider.

* Technologies: Visual Studio 2015 SDK
* Topics: MSBuild, VSX
* Last Updated: 05/14/2015

**Description**

This sample demonstrates how to register a source control provider with Visual
Studio that can be selected as active source control provider.

  * Implement a source control provider package
  * Adds an Options page visible only when the provider is active
  * Adds a tool window visible only when the provider is active
  * Menu items are only exposed when the provider is active
  * Adds a source control commands toolbar. The buttons are disabled when when the provider is inactive
  * Add source control command shortcuts to right-click menu in Solution Explorer
  * Enable persisting and reading solution properties
  * Enable persisting and reading user options in the .suo file
  * Implement basic QueryEdit / QuerySave service functionality
  * Demonstrate how to implement IVsTrackProjectDocumentsEvents2 functions
  * Provide basic source control functionality, such as add to source control, check in, check out, and work offline

![image](C%23/Resources/Example.SccProvider.png)

**Requirements**

[ Visual Studio 2015 ](http://www.microsoft.com/visualstudio/en-us/try/default.mspx#download)

[ Visual Studio 2015 SDK ](https://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx)

** Run the sample

  1. To run the sample, hitF5** or choose theDebug > Start Debugging** menu command. A new experimental instance of Visual Studio will launch.
  2. Once loaded, navigate toTools > Options > Source Control > Plug-In Selection** and set _Managed Source Control Sample Provider_ as the active source control provider.
  3. Once the active source control provider has been selected, several changes will be automatically applied to Visual Studio:
  4. The _Source control provider toolwindow_ automatically launches. The toolwindow can now be launched at any time by pressingView >_Source Control Provider Toolwindow_
  5. The _Source control provider toolwindow_ contains a toolbar with a single button. Pressing the button causes the the text and text field to change colors.
  6. A new page appears in the Options dialog. You can now navigate to Tools > Options > Source Control > _SccProviderOptions_
  7. The Sample Source Control Toolbar provides AddToSourceControl, Checkin, Checkout, and Offline commands. The toolbar can be accessed fromView > Toolbars > Sample Source Control Toolbar**
  8. Toolbar commands have also been added to the right-click menu of the Solution Explorer
  9. The visibility of menu commands, tool window, and Options page is controlled by the active state of the provider. They are not visible when the provider is not active.

**Details**

The provider demonstrates basic source control functionality:

  1. The sample does not implement a real source control store. To track which files in a project are controlled, the sample will create a projectname.storage text file on the hard drive that contains the list of controlled files in that project.
  2. To simplify things, if a project is added to the solution after the solution was added to source control, that project will not be automatically controlled. The user must explicitly use the _AddToSourceControl_ command to add it to source control. (Exceptions are solution folders, which are controlled automatically after closing and reopening the solution)
  3. Controlled solutions may be opened from disk.
  4. Controlled files can be checked out and checked in, but these actions only toggle the read-write attribute of the files. Source control status of files will be inferred from the file attributes on local disk.

The _SccProvider_ class derives from theIVsPersistSolutionProps** interface
to persist properties in the solution file. Uses:SolutionIsControlled**
indicates the controlled status of the solution.SolutionBindings** shows
the solution location in the source control database.

The _Use Scc Offline_ command is visible in the right-click menus of the
Solution Explorer. _Use Scc Offline_ acts like a toggle that allows projects
to be taken online or offline. The _Sample Source Control Provider_ options
key in the .suo file stores the offline status of a project. Other source
control commands are not affected by the offline status of a project.

The _SccProviderService_ class implements basic QueryEdit / QuerySave
functionality. _OnEdit_ prompts the user to check out the file, edit in
memory, or cancel the edit. _OnSave_ prompts the user to check out the file,
skip the save, save as a new file, or cancel the operation. _OnSave_ code
triggers only if files have been edited in memory or if the user forces the
file to be saved when it is already checked in.

_SccProviderService_ implements theIVsTrackProjectDocumentsEvents2**
interface to intercept project-change events in the integrated development
environment (IDE). Uses:OnAfterRenameFiles** renames files in source
control whenever they are renamed in the project.OnQueryRemoveFiles** warns
the user when they are deleting a file that is under source control.

**Project Files**

* **AssemblyInfo.cs**

This file contains assembly custom attributes.

* **SccProvider.cs*

This file contains the Package implementation. It also handles the enabling
and execution of source control commands.

* **CommandId.cs**

This is a list of GUIDs specific to this sample, especially the package GUID
and the commands group GUID. It also includes GUIDs of other elements used by
the sample.

* **DataStreamFromComStream.cs**

Implements a Stream object that wraps an IStream COM object. Facilitates the
read/write of solution user options from an IStream by using a BinaryFormatter
object that serializes data to and from Stream objects

* **DlgQueryEditCheckedInFile.cs**

A Form that is displayed when a checked in file is edited. The user can choose
to check out the file, continue editing the file in memory, or cancel the edit

* **DlgQuerySaveCheckedInFile.cs**

A Form that is displayed when a checked in file is saved. The user can choose
to checkout the file, discard the in-memory changes and skip saving it, save
the file with a different name, or cancel the operation.

* **ProvideSolutionProps.cs**

Contains the implementation of a custom registration attribute that declares
the key used by the package to persist solution properties. When encountering
a solution containing this key, the IDE will know which package it has to call
to read that block of solution properties.

* **ProvideSourceControlProvider.cs**

This file contains the implementation of a custom registration attribute that
registers a source control provider. It is used to make the source control
provider visisble on theTools > Options > SourceControl > Plug-Ins page

* **ProvideToolsOptionsPageVisibility.cs**

This file contains the implementation of a custom registration attribute that
defines the visibility of a tool window. It is used to make the tool window
implemented by the provider visible only when the provider is active (that is,
when the provider context UI has been asserted)

* **SccProviderOptions.cs**

This class derives from MsVsShell.DialogPage and provides the Options page. It
is responsible for handling Option page events, such as activation, apply, and
close. It hosts the SccProviderOptionsControl.

* **SccProviderOptionsControl.cs**

This class is a UserControl that will be hosted on the Options page. It has a
label to demonstrate display of controls in the page.

* **SccProviderService.cs**

Implementation of Sample Source Control Provider Service.

* **SccProviderStorage.cs**

Contains the implementation of a pseudo-source control storage. The class
creates a *.storage text file for each controlled project in the solution and
stores in it the list of the controlled files. The checked in and checked out
status of controlled files is inferred from the their attributes on disk.

* **SccProviderToolWindow.cs**

This class derives from ToolWindowPane, which provides the IVsWindowPane
implementation. It is responsible for defining the window frame properties
such as caption and bitmap. It hosts the SccProviderToolWindowControl.

* **SccProviderToolWindowControl.cs**

This class is a UserControl that will be hosted in the tool window. It has a
label to demonstrate display of controls in the page.

* **PkgCmd.vsct**

This file describes the menu structure and commands for this sample.

* **Resources.resx**

Resource strings (localizable).

**Functional Tests**

  * Verify the sample builds in all configurations
  * Verify that the sample was registered. The About box should list the product as installed
  * Verify that the provider is accessible in Tools, Options, SourceControl and can be made the active source control provider
  * Verify that the menu commands are visible only when the provider is active, after it was displayed once
  * Verify that the Options page is visible only when the provider is active
  * Verify that Sample Source Control Toolbar can be displayed
  * Verify prompts are dispayed when saving or editing a file that is currently checked in
  * Verify source control buttons are displayed in the right-click menu of the Solution Explorer

**Related topics**

  * [ SCC Provider Integration ](https://msdn.microsoft.com/en-us/library/bb166434(v=vs.140).aspx)
  * [ Visual Studio SDK Documentation ](https://msdn.microsoft.com/en-us/library/bb166441(v=vs.140).aspx)


text:
experimental-functors
=====================

An experiment at expressing functors in F# the .NET type system.

This solution attempts to implement the data structures detailed in Chris Okasaki's *Optimal Purely Functional Priority Queues*
by using static fields of generic types and generic interfaces to encode ML functors.

repository: jack-pappas/experimental-functors

repo_name: jack-pappas/experimental-functors


text:
# This folder contains the EAGLE schematics for the various hardware boards.

## munia
This contains schematics and board layout for the 3-in-1 device with support for the Nintendo Gamecube, Nintendo 64 and Super Nintendo/SFC consoles.
The enclosure also makes use of the

## munia-ngc
This is the simplified Gamecube-only version of the device

## munia-n64
This is the simplified N64-only version of the device

## munia-snes
This is the simplified SNES-only version of the device

## musia
This is the Sony PS2 version

repository: zzattack/munia

gha_language: C

repo_name: zzattack/munia


text:
**Quarry Stop Checker** utilizes the codelock from QuarryLocks. When a quarry is turned off it checks nearby players to ensure
that someone that is authorized is near by and if not turns the quarry back on.

The plugin can also be configured to announce in chat that someone tried to turn off a quarry but was thwarted and/or to slap the player(s) involved.

## Configuration

```json
{
  "EnableChatOnCaught": false,
  "EnableSlapOnCaught": false,
  "EnableServerLog": true
}
```

repository: OldGuyWoody/QuarryStopChecker

repo_name: OldGuyWoody/QuarryStopChecker