text:
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [Pre-Requisites](#pre-requisites)
  - [Test your operator bundle](#test-your-operator-bundle)
  - [Push the operator to quay.io](#push-the-operator-to-quayio)
  - [Test your operator standalone on the Openshift console](#test-your-operator-standalone-on-the-openshift-console)
    - [1. Create OperatorSource](#1-create-operatorsource)
    - [2. Create a Namespace for your operator](#2-create-a-namespace-for-your-operator)
    - [3. Install Operator](#3-install-operator)
- [Resolving services dependency](#resolving-services-dependency)
  - [How to share secret or configmap between services](#how-to-share-secret-or-configmap-between-services)
    - [Background](#background)
    - [As an information provider](#as-an-information-provider)
    - [As a information requester/adopter](#as-a-information-requesteradopter)
    - [One more thing](#one-more-thing)
- [Common Service Onboarding](#common-service-onboarding)
  - [1. Clone the git repository of operand deployment lifecycle manager](#1-clone-the-git-repository-of-operand-deployment-lifecycle-manager)
  - [2. Edit default value of custom resource](#2-edit-default-value-of-custom-resource)
    - [Edit the OperandRegistry custom resource](#edit-the-operandregistry-custom-resource)
    - [Edit the OperandConfig custom resource](#edit-the-operandconfig-custom-resource)
    - [Edit a OperandRequest custom resource](#edit-a-operandrequest-custom-resource)
    - [Edit alm-example in the operand-deployment-lifecycle-manager CSV](#edit-alm-example-in-the-operand-deployment-lifecycle-manager-csv)
  - [3.Make a pull request to merge the changes](#3make-a-pull-request-to-merge-the-changes)
- [End to end test](#end-to-end-test)
  - [1. Create an OperatorSource in the Openshift cluster](#1-create-an-operatorsource-in-the-openshift-cluster)
  - [2. Create a Namespace `ibm-common-services`](#2-create-a-namespace-ibm-common-services)
  - [3. Install operand deployment lifecycle manager](#3-install-operand-deployment-lifecycle-manager)
  - [4. Check the installed operators](#4-check-the-installed-operators)
  - [5. Edit the OperandConfig custom resource and the OperandRegistry custom resource](#5-edit-the-operandconfig-custom-resource-and-the-operandregistry-custom-resource)
  - [6. Create a OperandRequest](#6-create-a-operandrequest)
  - [7. Check the installed operators and their custom resource](#7-check-the-installed-operators-and-their-custom-resource)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

# Pre-Requisites

## Test your operator bundle

- [operator-courier](https://github.com/operator-framework/operator-courier)
- [scorecard](https://github.com/operator-framework/operator-sdk/blob/master/doc/test-framework/scorecard.md)

**Note:** Follow this [How to push to Quay.io](https://github.com/operator-framework/community-operators/blob/master/docs/testing-operators.md#push-to-quayio) to push your operator bundle (including CSV, CRD and Package.yaml) to the Quay.io.

**Note:** The CSV generated by operator sdk is not a complete CSV. The following fields need to be added manually. You can get more informations from [generating-a-csv](https://github.com/operator-framework/operator-sdk/blob/master/doc/user/olm-catalog/generating-a-csv.md) and [csv-annotations](https://github.com/operator-framework/operator-sdk/blob/master/doc/user/olm-catalog/csv-annotations.md).

- `description`, `displayName`, `resources`, `specDescriptors`, `statusDescriptors`, `actionDescriptors` fields in each spec.customresourcedefinitions.owned element will not be filled by the CSV generator.

- `displayName` and `description` are necessary for the CSV.

- `spec.replaces`, `spec.links`, `spec.selector`, `spec.icon` and `spec.maturity` fields can't be generated as well, but they are optional and will not be overwritten by the operator-sdk generate command.

## Push the operator to quay.io

All common services OLMs and operator images should be published in public org in Quay.io: [OpenCloudio](https://quay.io/organization/opencloudio)

more information see the [push olm to quay.io](https://github.com/operator-framework/community-operators/blob/master/docs/testing-operators.md#push-to-quayio).

## Test your operator standalone on the Openshift console

### 1. Create OperatorSource

Open OCP console, click the `Plus` button on the top right and paste the following content, then click `Create`.

```yaml
apiVersion: operators.coreos.com/v1
kind: OperatorSource
metadata:
  name: opencloud-operators
  namespace: openshift-marketplace
spec:
  authorizationToken: {}
  displayName: IBMCS Operators
  endpoint: https://quay.io/cnr
  publisher: IBM
  registryNamespace: opencloudio
  type: appregistry
```

### 2. Create a Namespace for your operator

Open the `OperatorHub` page in OCP console left menu, then `Create Project`.

### 3. Install Operator

Open `OperatorHub`, search your operator and install it.

# Resolving services dependency

## How to share secret or configmap between services

This topic is for the services which need to share their `secret` or `configmap` to other services.

### Background

In Q1, all the common services will be installed into the same namespace.

The namespace is something like: `ibm-common-services`.

In the Q1 release, there is no dependency management from ODLM, all the required operators need to be added into `OperandRequest` instance.

In the Q2 release, operators will use `OperandRequest` to manage their own dependencies.

### As an information provider

Take MongoDB as an example:

MongoDB provides two secrets and a configmap to all the service accounts in the `ibm-common-services`.

The `deploy/role.yaml`:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  creationTimestamp: null
  name: ibm-mongodb-operator-info
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  resourceNames:
  - icp-mongodb-client-cert
  - icp-mongodb-admin
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  resourceNames:
  - icp-mongodb
  verbs:
  - get
```

The `deploy/role_binding.yaml`:

```yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ibm-mongodb-operator-info
subjects:
- kind: Group
  name: system:serviceaccounts:ibm-common-services
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: ibm-mongodb-operator-info
  apiGroup: rbac.authorization.k8s.io
```

The above role/rolebinding means: IAM and Metering can get and mount the secret/configmap in their deployment/daemonset/statefulset.

### As a information requester/adopter

I need to know the information provider's secret/configmap name.

Take Metering as an example:

Metering needs to know the MongoDB secret and configmap:

- secret: icp-mongodb-client-cert, icp-mongodb-admin
- configmap: icp-mongodb

Metering needs to mount these configmap/secret to metering deployment/daemonset.

### One more thing

As we can see from above, the operators should appoint the service account name and the secret/configmap name to share info.

In the future releases, we will use Bindings and Requests to elegantly handle the information between operators.

# Common Service Onboarding

**Note:** Before onboard your operater to the operand deployment lifecycle manager, make sure your operater has finished the [pre-requisites](#pre-requisites), technically, can be deployed manually by OLM.

## 1. Clone the git repository of operand deployment lifecycle manager

```bash
git clone git@github.com:IBM/operand-deployment-lifecycle-manager.git
```

## 2. Edit default value of custom resource

### Edit the OperandRegistry custom resource

```bash
cd operand-deployment-lifecycle-manager
vi deploy/crds/operator.ibm.com_v1alpha1_operandregistry_cr.yaml
```

Append the operator package information under the `operators` field.

```yaml
apiVersion: operator.ibm.com/v1alpha1
kind: OperandRegistry
metadata:
name: common-service
spec:
operators:
...
- name: jenkins
    namespace: jenkins-operator
    channel: alpha
    packageName: jenkins-operator
    sourceName: community-operators
    sourceNamespace: openshift-marketplace
    description: The jenkins service depends on ...
...
```

- `name` is the name of the operator, which should be the same as the services name in the `OperandConfig` and `OperandRequest`.
- `namespace` is the namespace the operator will be deployed in. For the common services, they need to be installed in the `ibm-common-services` namespace.
- `channel` is the name of a tracked channel.
- `packageName` is the name of the package in `CatalogSource` will be deployed.
- `sourceName` is the name of the `CatalogSource`.
- `sourceNamespace` is the namespaces of the `CatalogSource`.
- `description` is used to add a detailed description for service including clarifying the dependency.

### Edit the OperandConfig custom resource

```bash
cd operand-deployment-lifecycle-manager
vi deploy/crds/operator.ibm.com_v1alpha1_operandconfig_cr.yaml
```

Append the operator custom resource information under the `services` field.

```yaml
apiVersion: operator.ibm.com/v1alpha1
kind: OperandConfig
metadata:
  name: common-service
spec:
  services:
  ...
  - name: jenkins
    spec:
      jenkins:
        service:
          port: 8081
  ...
```

Take the jenkins operator as an example.
- The `name` field defines the name of the operator.
- The `spec` field defines a map. Its key `jenkins` is the kind name of the custom resource. Its value `service.port: 8081` will be merged to the `spec` field of custom resource `jenkins`.

In this example:
The configuration for custom resource `Jenkins` is:

```yaml
      jenkins:
        service:
          port: 8081
```

The configuration will be merged in the `spec` of the relevant `alm-example` in the `cluster service version` and generate custom resource `Jenkins`

```yaml
apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  ...
  name: example
  namespace: jenkins-operator
spec:
  ...
  service:
    port: 8081
    type: ClusterIP
```

**Note:** When onboarding your operator, you need to put all the default configuration into the `alm-example`of your CSV and leave the value of map in `spec` to `{}`.
For example:

```yaml
apiVersion: operator.ibm.com/v1alpha1
kind: OperandConfig
metadata:
  name: common-service
spec:
  services:
  ...
  - name: jenkins
    spec:
      jenkins: {}
  ...
```

### Edit a OperandRequest custom resource

```bash
cd operand-deployment-lifecycle-manager
vi deploy/crds/operator.ibm.com_v1alpha1_operandrequest_cr.yaml
```

Append the operator information under the `services` field.

```yaml
apiVersion: operator.ibm.com/v1alpha1
kind: OperandRequest
metadata:
  name: common-service
spec:
  services:
  ...
  requests:
  - operands:
      - name: jenkins
    registry: common-service
    registryNamespace: ibm-common-services
  ...
```

- `services` is a list defines the set for each service.
- `name` is the service name, which should be the same as the services name in the `OperandConfig` and operator name in the `OperandRegistry`.
- `channel` is an optional setting, it can overwrite the `channel` defined in the `OperandRegistry`.
- `state` defines if the service should be present or absent.
- `description` is the description of the service.

### Edit alm-example in the operand-deployment-lifecycle-manager CSV

Add the changes for operandrequest operandregistry and operandconfig in the [alm-example](https://github.com/IBM/operand-deployment-lifecycle-manager/blob/master/deploy/olm-catalog/operand-deployment-lifecycle-manager/1.2.0/operand-deployment-lifecycle-manager.v1.2.0.clusterserviceversion.yaml#L5)

## 3.Make a pull request to merge the changes

# End to end test

**Note:** before running the e2e test, users have to push your own CSV package to Quay.io.
more information see [Push the operator to quay.io](#push-the-operator-to-quayio).

## 1. Create an OperatorSource in the Openshift cluster

Open OCP console, click the `Plus` button on the top right and paste the following content, then click `Create`.

```yaml
apiVersion: operators.coreos.com/v1
kind: OperatorSource
metadata:
  name: opencloud-operators
  namespace: openshift-marketplace
spec:
  authorizationToken: {}
  displayName: IBMCS Operators
  endpoint: https://quay.io/cnr
  publisher: IBM
  registryNamespace: opencloudio
  type: appregistry
```

## 2. Create a Namespace `ibm-common-services`

Open `OperatorHub` page in OCP console left menu, then `Create Project`, e.g., create a project named `ibm-common-services`.

## 3. Install operand deployment lifecycle manager

Open `OperatorHub` and search `operand-deployment-lifecycle-manager` to find the operator, and install it.

## 4. Check the installed operators

Open `Installed Operators` page to check the installed operators.

## 5. Edit the OperandConfig custom resource and the OperandRegistry custom resource

- [Editing OperandConfig](#edit-the-operandconfig-custom-resource)
- [Editing OperandRegistry](#edit-the-operandregistry-custom-resource)

## 6. Create a OperandRequest

```bash
vi deploy/crds/operator.ibm.com_v1alpha1_operandrequest_cr.yaml
oc apply -f deploy/crds/operator.ibm.com_v1alpha1_operandrequest_cr.yaml -n ibm-common-services
```

- [Editing OperandRequest](#edit-a-operandrequest-custom-resource)

## 7. Check the installed operators and their custom resource


text:
# catalyst-lazy-properties-mixin

[![Travis](https://img.shields.io/travis/catalyst/catalyst-lazy-properties-mixin/master.svg?style=flat-square)](https://travis-ci.org/catalyst/catalyst-lazy-properties-mixin)
[![David](https://img.shields.io/david/catalyst/catalyst-lazy-properties-mixin.svg?style=flat-square)](https://david-dm.org/catalyst/catalyst-lazy-properties-mixin)
[![David](https://img.shields.io/david/dev/catalyst/catalyst-lazy-properties-mixin.svg?style=flat-square)](https://david-dm.org/catalyst/catalyst-lazy-properties-mixin?type=dev)
[![npm (scoped)](https://img.shields.io/npm/v/@catalyst-elements/catalyst-lazy-properties-mixin.svg?style=flat-square)](https://www.npmjs.com/package/@catalyst-elements/catalyst-lazy-properties-mixin)
[![Bower not supported](https://img.shields.io/badge/bower-not_supported-red.svg?style=flat-square)]()
[![Polymer 2 not supported](https://img.shields.io/badge/Polymer_2-not_supported-red.svg?style=flat-square)]()
[![Polymer 3 support pending](https://img.shields.io/badge/Polymer_3-support_pending-yellow.svg?style=flat-square)]()

[API documentation ↗](https://catalyst.github.io/CatalystElementsBundle/#/classes/catalystLazyPropertiesMixin)

`<catalyst-lazy-properties-mixin>` is a mixin that provides an element with toggle behavior.

## Installation

Install with npm:

```sh
npm install --save @catalyst-elements/catalyst-lazy-properties-mixin
```

Install with yarn:

```sh
yarn add @catalyst-elements/catalyst-lazy-properties-mixin
```

Please note that this package is not compatible with Bower.

## Usage

### As a Module (Recommend)

```js
import catalystLazyPropertiesMixin from<EMAIL_ADDRESS>
class MyLazyPropertiesElement extends catalystLazyPropertiesMixin(HTMLElement) {
  constructor() {
    super();
    // ...
  }

  // ...
}
```

### As a Script

```html
<script<EMAIL_ADDRESS><script>
  var SuperClass = window.CatalystElements.catalystLazyPropertiesMixin(HTMLElement);

  function MyLazyPropertiesElement() {
    SuperClass.call(this);
    // ...
  };
  MyLazyPropertiesElement.prototype = Object.create(SuperClass.prototype);
  MyLazyPropertiesElement.prototype.constructor = MyLazyPropertiesElement;

  // ...
</script>
```

Please note that this script has been transpiled to es5 and thus use of `custom-elements-es5-adapter.js` or an equivalent library is required. See [es5 support](https://github.com/catalyst/CatalystElements/wiki/Browser-Compatibility#es5-support) on the Catalyst Elements wiki for details.

## Contributions

Contributions are most welcome.

Please read our [contribution guidelines](./CONTRIBUTING.md).

repository: catalyst/catalyst-lazy-properties-mixin

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: catalyst/catalyst-lazy-properties-mixin


text:
# Lesson Plan - Closures

## Overview

In this lesson, we will expand on our understanding of scope with the discussion of closures as a programming tool to solve unusual problems.

## Assumptions

This lesson assumes the students have a strong understanding of defining and using functions, as well as a cursory knowledge of lexical scope.

By this point, students should be able to:

- Write JavaScript in an HTML page using the `<script>` tag
- Use Chrome DevTools to view the output of the HTML page
- Write and use functions
- Wite and use object literals
- Understand that variables created in higher scope are available in lower scopes
- Understand that variables created inside of functions are not accessible outside of them
- Manipulate HTML elements with the DOM

## Objectives

- Complete activities 1-7 in the activities folder

- For each student activity, (3, 5, and 7) refer the student to the README in the unsolved folder for directions to complete that activity.

## Instructor Notes

Today's class is focused on demonstrating closures. Reassure students that closures can be confusing at first and that it may take them some time to fully wrap their heads around the concept.

Feel free to approach this lesson as a gentle exploration into the wild side of programming. If time allows, take the opportunity to explore edge-cases, and answer students' what-ifs.

## Learning Objectives

- To understand the closure shared-scope paradigm

- Implement closures to re-use code

- Use JS methods with "private" state

- Demonstrate the use of closures to make a function modular

---

## Class Instruction

### 1. Instructor Do: Closure Intro (5-10 min)

- Open the file `01-intro.html` in the `01-intro` folder, and ask the students to compare and contrast the two code blocks, then open the browser and show them the output.

- While the first block is cleaner, the second block uses closures and allows us to do some interesting things because it returns a function of its own.

- The goal is to pique the students' interest; sometimes we have to write more code to write less.

### 2. Instructor Do: Closer look at Closures (10 min)

- In the `02-easy-closures` folder, open the file `02-easy-closures.html` in VSCode. Demonstrate to students how our closure approach can be implemented to re-use the last names for the "family" we're creating.

- This is helpful if we want to initialize some functionality in one place, but maybe execute the rest of the logic later. In this case, we can set up a family unit with the last name and then get around to adding members later.

- If there's time, it may be helpful to demonstrate how you would write this without the closure, highlighting how often we need to pass in the same strings which may be prone to misspellings, even though this is a pretty contrived example.

### 3. Students Do: Closures and Cats (15 min)

- Proceed with the first activity, have students open `03-cats-and-closure/unsolved/03-cats-and-closure.html` and ask students to fill in the inner function that creates an object with a property defined from the outer function.

- This should be similar to the previous example, except instead of returning just a string we're returning an object.

### 4. Instructor Do: Review Closures and Cats (5 min)

- Demonstrate the solved solution from `03-cats-and-closure-solved.html` in the `solved` folder. Again, this should have a similar signature to the `02` activity.

- Summarize this activity by pointing out that the last two activities are simply functions returning functions.

**Note**: While it's tempting to call closures function factories or generators, we don't want to sully the student's GoogleFu and have them type in JS Generators or factories when trying to use closures since those concepts open up a whole can of worms. For this reason, the lesson avoids using the terms consistently.

Focus on the concept of shared scope.

### 5. Instructor Do: "Private" Variables (15 min)

- Open the file `04-private-variables/04-private-variables.html` in VSCode and walk through the closure function with the class.

- Attempt to generate discussion on what the console would show in the example where we try to access variables in the outer function vs the inner functions.

- This should tie back to the discussion of lexical scope and not being able to access variables outside their functions.

- If any students have had any exposure to OOP or objects in other languages, they might recognize this as a parallel to private variables, but we should avoid a discussion about OOP and classes as this won't be very helpful for beginners and we want to stay on the topic at hand.

**Important**: Over-emphasize that when we say "private variables" in the realm of closures, we **DO NOT MEAN SECURE**.
Students should never put sensitive information in code that will be used in the browser.

- If you have time remaining you can demonstrate that these variables aren't private (and that we're using closures) by calling `console.log(mySecrets)` in the DevTools console and inspecting the `tellMeYourSecrets` method as shown here:

![images/not-so-private-console-dir.png](images/not-so-private-console-dir.png)

### 6. Student Do: Private Treasure (20 min)

- Direct the students to `05-private-treasure/unsolved/05-private-treasure.html` and ask the students to create a closure function using private variables.

- This is going to challenge some students as it surreptitiously requires Object Oriented Programming skills they may not have learned yet, so be sure to check in with students periodically to drop hints or you can partially code-behind them if they get stuck.

### 7. Instructor Do: Private Variables Review (10 min)

- Open the solved solution from `05-private-treasure-solved.html` in the `solved` folder and demonstrate how we can use closure functions to build these "what-we-will-eventually-call-object-oriented-programming" objects.

- The key takeaway should be that you can enforce controlled get/set patterns by abstracting away from the variables themselves.

### 8. Instructor Do: Semi-Practical Example (15 min)

- Now we'll get into some more practical examples of closure convenience working with actual HTML elements

- Underline to the students that closures aren't the solution to every problem. Some developers strongly recommend against them unless necessary, but they're a good tool to have for certain situations, and they're great interview questions.

- This functionality could likely be achieved more easily with a different solution, but being familiar with closures gives us more flexibility when trying to solve the problem.

- Open `06-callbacks-with-closure.html` in the browser and demonstrate how we can click the bordered buttons and update the root element, then inspect the `<script>` to show how we use a closure to mass-produce event handlers for a given affected element.

- We can then apply the **same** functionality if we wanted to change the border of ANY element using the same method, so here's some code to do this to the whole `document.body`.

```javascript
var borderizeWindow = makeBorderizer(document.body);

document.getElementById("btn-1").onclick = borderizeWindow;
document.getElementById("btn-2").onclick = borderizeWindow;
document.getElementById("btn-3").onclick = borderizeWindow;
document.getElementById("btn-4").onclick = borderizeWindow;
```

- Note that unlike `addEventListenter`, setting the click event with `.onclick` will OVERWRITE the click handler for the buttons, but don't let students fixate on this minor difference since this isn't a review of DOM API.

### 9. Student Do: DOM Discovery (10 min)

- Direct the students to open `07-dom-discovery/unsolved/07-dom-discovery.html` in the browser and ask the students to implement a closure function that outputs information about a hovered element into another element.

- Ensure that the students are only looking at the `README.md` and modifying the `js/discovery.js`. There's a lot in this activity and we don't want them to get lost in all the directories. The browser's dev tools should be open the whole time as well.

- This is either going to be a challenge or very simple. Encourage the students to refer to previous activities, and look things up early and often when they don't know. There's a lot to dig through in event objects!

### 10. Instructor Do: DOM Discovery Review (15 min)

- Open the solved solution from `js/discover.js` in the `solved` folder and walk through the few lines it should take to accomplish this seemingly harrowing task.

- With the given target output, we create a function that takes this `outputBox` as a parameter and returns a ready-made event handler.

- They may struggle with how to attach an event handler to the whole page, but we can remind them of a similar example in activity `06-callbacks-with-closure`, where we optionally attached an event listener to the body tag.

- The value proposition of using closures here is that this JS file could be made into a module and another developer could add this feature to their code by just plug in a different target and using the generated event handler.

## Conclusion

- By the end of this lesson, students should have familiarity in recognizing a function with closure, and a general understanding of the "signature" involved with a closure function.

- It's okay if students can't precisely pin down WHEN they might need to use closures; it's more important that they can identify them when they come up, and know how to implement them when they do need one.

- Encourage students to spend some time brainstorming problems that might be solved with closures, or even how to avoid them in these activities. Exposure and practice are key so the more they use them, the better they'll understand this hairy concept.

## Additional Resources for Closures

- [MDN Closures](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Closures)
- [W3Schools Closures](https://www.w3schools.com/js/js_function_closures.asp)

license: MIT

repository: actuallydan/trilogy-assessment-closures

branch_name: refs/heads/main

repo_name: actuallydan/trilogy-assessment-closures


text:
# Full Stack MERN APP

## Introduction

Just a full stack app with MERN.

## Instructions

In order to run the project just type `npm run dev` which will both run back and front end at the same time.

For adding the data to the system, just use `npm run data:import`, to delete data use `npm run data:delete`.

repository: nijatmursali/mern-ecommerce

repo_name: nijatmursali/mern-ecommerce


text:
# Vision

This script analyses the Nmap XML scanning results, parses each CPE context and correlates to search CVE on NIST. You can use that to find public vulnerabilities in services.

```
	..::: VISION v0.3 :::...
        Nmap\'s XML result parser and NVD's CPE correlation to search CVE

	Example:
		python Vision-cpe.py result_scan.xml 3 txt
	argv 1 = Nmap scanner results in XML
	argv 2 = Limit CVEs per CPE to get
	argv 3 = Type of output (xml or txt)

To install modules:
$ sudo python3 -m pip install -r requirements.txt
```

## Example of results:
```
$ sudo nmap -sS -sV -O -P0 02:42:0A:00:00:03 -oX result.xml
$ python3 Vision-cpe.py result.xml 3 txt

::::: Vision v0.3 - nmap NVD's cpe correlation with CVE - Coded by CoolerVoid

Find CPE : vsftpd vsftpd 2.3.4
Host: 02:42:0A:00:00:03
Port: 21
vsftpd vsftpd 2.3.4

	URL: https://nvd.nist.gov/vuln/detail/CVE-2011-2523
	Description: vsftpd 2.3.4 downloaded between 20110630 and 20110703 contains a backdoor which opens a shell on port 6200/tcp.

	Risk cvss-3: 9.8 CRITICAL

	Risk cvss-2: 10.0 HIGH

	URL: https://nvd.nist.gov/vuln/detail/CVE-2011-0762
	Description: The vsf_filename_passes_filter function in ls.c in vsftpd before 2.3.3 allows remote authenticated users to cause a denial of service (CPU consumption and process slot exhaustion) via crafted glob expressions in STAT commands in multiple FTP sessions, a different vulnerability than CVE-2010-2632.

	Risk cvss-2: 4.0 MEDIUM

Find CPE : openbsd openssh 4.7p1
Host: 02:42:0A:00:00:03
Port: 22
openbsd openssh 4.7p1

Find CPE : apache http_server 2.2.8
Host: 02:42:0A:00:00:03
Port: 80
apache http_server 2.2.8

Find CPE : proftpd proftpd 1.3.1
Host: 02:42:0A:00:00:03
Port: 2121
proftpd proftpd 1.3.1

	URL: https://nvd.nist.gov/vuln/detail/CVE-2009-0543
	Description: ProFTPD Server 1.3.1, with NLS support enabled, allows remote attackers to bypass SQL injection protection mechanisms via invalid, encoded multibyte characters, which are not properly handled in (1) mod_sql_mysql and (2) mod_sql_postgres.

	Risk cvss-2: 6.8 MEDIUM

	URL: https://nvd.nist.gov/vuln/detail/CVE-2009-0542
	Description: SQL injection vulnerability in ProFTPD Server 1.3.1 through 1.3.2rc2 allows remote attackers to execute arbitrary SQL commands via a "%" (percent) character in the username, which introduces a "'" (single quote) character during variable substitution by mod_sql.

	Risk cvss-2: 7.5 HIGH

	URL: https://nvd.nist.gov/vuln/detail/CVE-2008-4242
	Description: ProFTPD 1.3.1 interprets long commands from an FTP client as multiple commands, which allows remote attackers to conduct cross-site request forgery (CSRF) attacks and execute arbitrary FTP commands via a long ftp:// URI that leverages an existing session from the FTP client implementation in a web browser.

	Risk cvss-2: 6.8 MEDIUM

	URL: https://nvd.nist.gov/vuln/detail/CVE-2006-6563
	Description: Stack-based buffer overflow in the pr_ctrls_recv_request function in ctrls.c in the mod_ctrls module in ProFTPD before 1.3.1rc1 allows local users to execute arbitrary code via a large reqarglen length value.

	Risk cvss-2: 6.6 MEDIUM

Find CPE : mysql mysql 5.0.51a
Host: 02:42:0A:00:00:03
Port: 3306
mysql mysql 5.0.51a

	URL: https://nvd.nist.gov/vuln/detail/CVE-2009-4484
	Description: Multiple stack-based buffer overflows in the CertDecoder::GetName function in src/asn.cpp in TaoCrypt in yaSSL before 1.9.9, as used in mysqld in MySQL 5.0.x before 5.0.90, MySQL 5.1.x before 5.1.43, MySQL 5.5.x through 5.5.0-m2, and other products, allow remote attackers to execute arbitrary code or cause a denial of service (memory corruption and daemon crash) by establishing an SSL connection and sending an X.509 client certificate with a crafted name field, as demonstrated by mysql_overflow1.py and the vd_mysql5 module in VulnDisco Pack Professional 8.11. NOTE: this was originally reported for MySQL 5.0.51a.

	Risk cvss-2: 7.5 HIGH

	URL: https://nvd.nist.gov/vuln/detail/CVE-2008-4097
	Description: MySQL 5.0.51a allows local users to bypass certain privilege checks by calling CREATE TABLE on a MyISAM table with modified (1) DATA DIRECTORY or (2) INDEX DIRECTORY arguments that are associated with symlinks within pathnames for subdirectories of the MySQL home data directory, which are followed when tables are created in the future. NOTE: this vulnerability exists because of an incomplete fix for CVE-2008-2079.

	Risk cvss-2: 4.6 MEDIUM

	URL: https://nvd.nist.gov/vuln/detail/CVE-2007-6303
	Description: MySQL 5.0.x before 5.0.51a, 5.1.x before 5.1.23, and 6.0.x before 6.0.4 does not update the DEFINER value of a view when the view is altered, which allows remote authenticated users to gain privileges via a sequence of statements including a CREATE SQL SECURITY DEFINER VIEW statement and an ALTER VIEW statement.

	Risk cvss-2: 3.5 LOW

	URL: https://nvd.nist.gov/vuln/detail/CVE-2007-6304
	Description: The federated engine in MySQL 5.0.x before 5.0.51a, 5.1.x before 5.1.23, and 6.0.x before 6.0.4, when performing a certain SHOW TABLE STATUS query, allows remote MySQL servers to cause a denial of service (federated handler crash and daemon crash) via a response that lacks the minimum required number of columns.

	Risk cvss-2: 5.0 MEDIUM

Find CPE : postgresql postgresql 8.3
Host: 02:42:0A:00:00:03
Port: 5432
postgresql postgresql 8.3

	URL: https://nvd.nist.gov/vuln/detail/CVE-2013-1903
	Description: PostgreSQL, possibly 9.2.x before 9.2.4, 9.1.x before 9.1.9, 9.0.x before 9.0.13, 8.4.x before 8.4.17, and 8.3.x before 8.3.23 incorrectly provides the superuser password to scripts related to "graphical installers for Linux and Mac OS X," which has unspecified impact and attack vectors.

	Risk cvss-2: 10.0 HIGH

	URL: https://nvd.nist.gov/vuln/detail/CVE-2013-1902
	Description: PostgreSQL, 9.2.x before 9.2.4, 9.1.x before 9.1.9, 9.0.x before 9.0.13, 8.4.x before 8.4.17, and 8.3.x before 8.3.23 generates insecure temporary files with predictable filenames, which has unspecified impact and attack vectors related to "graphical installers for Linux and Mac OS X."

	Risk cvss-2: 10.0 HIGH

	URL: https://nvd.nist.gov/vuln/detail/CVE-2013-0255
	Description: PostgreSQL 9.2.x before 9.2.3, 9.1.x before 9.1.8, 9.0.x before 9.0.12, 8.4.x before 8.4.16, and 8.3.x before 8.3.23 does not properly declare the enum_recv function in backend/utils/adt/enum.c, which causes it to be invoked with incorrect arguments and allows remote authenticated users to cause a denial of service (server crash) or read sensitive process memory via a crafted SQL command, which triggers an array index error and an out-of-bounds read.

	Risk cvss-2: 6.8 MEDIUM

	URL: https://nvd.nist.gov/vuln/detail/CVE-2012-3489
	Description: The xml_parse function in the libxml2 support in the core server component in PostgreSQL 8.3 before 8.3.20, 8.4 before 8.4.13, 9.0 before 9.0.9, and 9.1 before 9.1.5 allows remote authenticated users to determine the existence of arbitrary files or URLs, and possibly obtain file or URL content that triggers a parsing error, via an XML value that refers to (1) a DTD or (2) an entity, related to an XML External Entity (aka XXE) issue.

	Risk cvss-2: 4.0 MEDIUM

Find CPE : apache coyote_http_connector 1.1
Host: 02:42:0A:00:00:03
Port: 8180
apache coyote_http_connector 1.1

Start parser

```

## Common questions:

## How to write XML output on Nmap ?
https://nmap.org/book/output-formats-xml-output.html

## What is a CPE  ?

https://nmap.org/book/output-formats-cpe.html

https://nvd.nist.gov/products/cpe

## What is a CVE ?

https://cve.mitre.org/

## This is a true vulnerability scanner ?

Nop, this script is util to audit banners of services, this tool don't test inputs... Full vulnerability scanner its complex, look that following http://www.openvas.org/

Author: CoolerVoid - just another computer programmer...

Contact: coolerlair at gmail dot com

Ter Set  5 02:00:09 2017

repository: CoolerVoid/Vision

repo_name: CoolerVoid/Vision


text:
# Setup GCP Project Security

### Why do this
 - Learn how to secure your GCP project resources (Virtual machine instances, Cloud storage buckets,...)
 - Learn how to secure your analysis data via assigned access permissions and/or data encryption

### What is this
 - Google Cloud provides uses IAM (Identity and Access Management) objects to allow you to configure security
 - GCP IAM objects includes members, roles, permissions, keys and other security objects
 - GCP allows you to enable the use of selected cloud services in your account in one of three ways:
    - using enabled **services** (such as 'create VM using Google Compute Engine service' by clicking on GCP
    console or using `gcloud create...`)
    - using enabled **APIs** (such as 'create VM instance' using GCP Cloud SDK) --or--
    - using enabled services which include API access

NOTE: Most GCP services are disabled by default.  You must first explicitly enable a service in your account, then you can create service instances of that service (such as VM instances in Google Compute Engine)

<img src="https://github.com/lynnlangit/gcp-for-bioinformatics/raw/master/images/roles.png" width="460" align="right">

### Key considerations
 - Only enable services/APIs that you are using in your GCP Project
 - Consider auditing service usage by using Cloud Monitoring logs - [link](https://cloud.google.com/solutions/exporting-stackdriver-logging-for-security-and-access-analytics)

### How to do this
 - Associate GCP IAM User accounts to GCP Roles (BigQuery User role shown to the right)
 - Best Practice: Use GCP-created roles.  Service-level roles include individual service permissions
 - You can create custom IAM (or security) roles if needed

### How to verify you've done it
 - Test permissions assigned to roles/users
 - TIP: You can use the GCP IAM & Admin -> `Policy Troubleshooter` tool to verify that an account has been assigned the appropriate permissions.  This tools shows the (policy) bindings assigned to one or more (service account) resources.

### 📺 Click below to see Lynn's 'Use GCP IAM security roles' 6 min video
[![GCP IAM roles - Google Cloud Platform for Bioinformatics](http://img.youtube.com/vi/4oSA9bdwlPs/0.jpg)](http://www.youtube.com/watch?v=4oSA9bdwlPs "GCP IAM roles - Google Cloud Platform for Bioinformatics")

### Other Things to Know
 - Data is encrypted by default in Cloud Storage using a Google-provided encryption key.  It is possible for you to supply your own encryption key when your requirements necessitate this.
 - You can use GCP service object labels to help you to identify services instances that have different security requirements.  They are implemented as key:value tag pairs, i.e. `"lab":"langit-lab"` etc...

### How to learn More
 - 📘 IAM Best practices - [link](https://cloud.google.com/iam/docs/using-iam-securely)
 - 📘 GCP IAM Roles Explained covers when to use which type of role -[link](https://medium.com/google-cloud/gcp-iam-roles-explained-af84955346e7)
 - 📘 GCP Security Best Practice from The Broad - [link](https://dsp-security.broadinstitute.org/cloud-security/google-cloud-platform)
 - 📘 HIPPA compliant GCP services - [link](https://cloud.google.com/security/compliance/hipaa/)
 - 📘 GCP Security Whitepapers - [link](https://services.google.com/fh/files/misc/security_whitepapers_march2018.pdf)
 - 📺 Watch [understanding Terra security roles and other topics](https://www.youtube.com/watch?v=SRVrzXHkZKU) - 27 minute video from the Broad Institute, security discussion starts at 10:00 in the video. Terra.bio runs on GCP.
 - 📘 IAM release updates, adding more granular control via service improvements - [link](https://cloud.google.com/iam/docs/release-notes)

repository: lynnlangit/gcp-for-bioinformatics

gha_language: Jupyter Notebook

repo_name: lynnlangit/gcp-for-bioinformatics


text:
# Advanced Steps
Contribute to open source. When deciding of whether or not to contribute to a project consider the following questions.

- Does the open source project interest you?
- Is the architecture documented?
- Are there any contributing instructions?
- Is there a tool you particularly like but feel it could be better if it had a specific feature?
- Does the open source project have continuous integration?
- As a Syncroness employee, is the contribution valuable to the company?

If you answered yes to many of these questions, consider contributing to the open source project.

If you are having trouble finding projects you would like to contribute to, consider one of the following.
- [Servo](https://servo.org/): <https://github.com/servo/servo>
- iced: <https://github.com/hecrj/iced>
- [probe.rs](https://probe.rs/): <https://github.com/probe-rs/probe-rs>
- [Seed](https://seed-rs.org/): <https://github.com/seed-rs/seed>
- tarpaulin: <https://github.com/xd009642/tarpaulin>
- [tauri](https://tauri.studio/en/) - <https://github.com/tauri-apps/tauri>

repository: seanybaggins/rust-at-syncroness

repo_name: seanybaggins/rust-at-syncroness


text:
# Docker image basic R image

## Purpose

Many R replication packages have dependencies, which sometimes include the specific version of R (in particular before/after release of v4).
This Docker image is meant to isolate and stabilize that environment, and should be portable across
multiple operating system, as long as [Docker](https://docker.com) is available.

## Build

### Adjust the needed packages

See the [setup.R](setup.R) file, and update accordingly.

> WARNING: not all packages might build, depending on whether the R base image has the relevant libraries. You might want to change R base image, or switch to another image from [rocker](https://hub.docker.com/u/rocker).

### Setup info

Set the `TAG` and `IMAGEID` accordingly.

```
TAG=v$(date +%F)
MYIMG=aer-9999-8888
MYHUBID=aeadataeditor
```
### Build the image

```
docker build  . -t $MYIMG:$TAG
```
or if using the newer build system
```
DOCKER_BUILDKIT=1 docker build . -t $MYIMG:$TAG
```

## Publish the image

The resulting docker image can be uploaded to [Docker Hub](https://hub.docker.com/), if desired.

```
docker push $MYHUBID/${MYIMG}:$TAG
```

## Using the image

If using a pre-built image on [Docker Hub](https://hub.docker.com/repository/docker/larsvilhuber/):

```
docker run -it --rm $MYHUBID/${MYIMG}:$TAG
```

If using the image you just created:

```
docker run -it --rm $MYHUBID/${MYIMG}:$TAG
```

Somewhat more sophisticated, if you are in a project directory (for instance, the replication package you just downloaded), you can access it directly within the image as follows:

```
docker run -it --rm -v $(pwd)/subdir:/code -w /code $MYHUBID/${MYIMG}:$TAG
```

You can now start to run code.

## NOTE

This entire process could be automated, using [Travis-CI](https://docs.travis-ci.com/user/docker/#pushing-a-docker-image-to-a-registry) or [Github Actions](https://github.com/marketplace/actions/build-and-push-docker-images). Not done yet.

repository: alaskievic/docker-r-starter

repo_name: alaskievic/docker-r-starter


text:
# Grover
(aka, code for [Defending Against Neural Fake News](https://arxiv.org/abs/1905.12616))

Grover is a model for Neural Fake News -- both generation and detection. However, it probably can also be used for other generation tasks.

Visit our project page at [rowanzellers.com/grover](https://rowanzellers.com/grover), [the AI2 online demo](https://grover.allenai.org), or read the full paper at [arxiv.org/abs/1905.12616](https://arxiv.org/abs/1905.12616).

![teaser](https://i.imgur.com/VAGFpBe.png "teaser")

## What's in this repo?

We are releasing the following:
* Code for the Grover generator (in [lm/](lm/)). This involves training the model as a language model across fields.
* Code for the Grover discriminator in [discrimination/](discrimination/). Without much changing, you can run Grover as a discriminator to detect Neural Fake News.
* Code for generating from a Grover model, in [sample/](sample/).
* Code for making your own RealNews dataset in [realnews/](realnews/).
* Model checkpoints freely available online for the Grover-Base and Grover-Large models. For Grover-Mega or the RealNews dataset, please [submit this form](https://docs.google.com/forms/d/1LMAUeUtHNPXO9koyAIlDpvyKsLSYlrBj3rYhC30a7Ak) and we will get back to you as soon as possible.

Scroll down 👇 for some easy-to-use instructions for setting up Grover to generate news articles.

## Setting up your environment

*NOTE*: If you just care about making your own RealNews dataset, you will need to set up your environment separately just for that, using an AWS machine (see [realnews/](realnews/).)

There are a few ways you can run Grover:
* **Generation mode (inference)**. This requires a GPU because I wasn't able to get top-p sampling, or caching of transformer hidden states, to work on a TPU.
* **LM Validation mode (perplexity)**. This could be run on a GPU or a TPU, but I've only tested this with TPU inference.
* **LM Training mode**. This requires a large TPU pod.
* **Discrimination mode (training)**. This requires a TPU pod.
* **Discrimination mode (inference)**. This could be run on a GPU or a TPU, but I've only tested this with TPU inference.

I used Python3.6 for everything. Usually I set it up using the following commands:
```
curl -o ~/miniconda.sh -O  https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh  && \
     chmod +x ~/miniconda.sh && \
     ~/miniconda.sh -b -p ~/conda && \
     rm ~/miniconda.sh && \
     ~/conda/bin/conda install -y python=3.6
```
Then `pip install -r requirements-gpu.txt` if you're installing on a GPU, or `pip install requirements-tpu.txt` for TPU.

Misc notes/tips:
* If you have a lot of projects on your machine, you might want to use an anaconda environment to handle them all. Use `conda create -n grover python=3.6` to create an environment named `grover`. To enter the environment use `source activate grover`. To leave use `source deactivate`.
* I'm using tensorflow `1.13.1` which requires Cuda `10.0`. You'll need to install that from the nvidia website. I usually install it into `/usr/local/cuda-10.0/`, so you will need to run `export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64` so tensorflow knows where to find it.
* I always have my pythonpath as the root directory. While in the `grover` directory, run `export PYTHONPATH=$(pwd)` to set it.

## Quickstart: setting up Grover for generation!

1. Set up your environment. Here's the easy way, assuming anaconda is installed: `conda create -y -n grover python=3.6 && source activate grover && pip install -r requirements-gpu.txt`
2. Download the model using `python download_model.py base`
3. Now generate: `PYTHONPATH=$(pwd) python sample/contextual_generate.py -model_config_fn lm/configs/base.json -model_ckpt models/base/model.ckpt -metadata_fn sample/april2019_set_mini.jsonl -out_fn april2019_set_mini_out.jsonl`

Congrats! You can view the generations, conditioned on the domain/headline/date/authors, in `april2019_set_mini_out.jsonl`.

### Bibtex

```
@inproceedings{zellers2019grover,
    title={Defending Against Neural Fake News},
    author={Zellers, Rowan and Holtzman, Ari and Rashkin, Hannah and Bisk, Yonatan and Farhadi, Ali and Roesner, Franziska and Choi, Yejin},
    journal={arXiv preprint arXiv:1905.12616},
    year={2019}
}
```

repository: comex/grover

repo_name: comex/grover


text:
# 05to12 Iquery STECData

## Dataset

| Name | Value |
| :--- | :---- |
| Catalog | [Link](https://catalog.data.gov/dataset/05to12-iquery-stecdata-b3da9) |
| Metadata | [Link](https://data.illinois.gov/api/views/f55m-q7ae) |
| Data: JSON | [100 Rows](https://data.illinois.gov/api/views/f55m-q7ae/rows.json?max_rows=100) |
| Data: CSV | [100 Rows](https://data.illinois.gov/api/views/f55m-q7ae/rows.csv?max_rows=100) |
| Host | data.illinois.gov |
| Id | f55m-q7ae |
| Name | 05to12 Iquery STECData |
| Created | 2013-11-05T21:12:19Z |
| Publication Date | 2013-11-06T15:01:42Z |

## Description

The reported number of illnesses caused by strains of the Escherichia coli bacteria that produce shiga toxin. Shiga toxin-producing E. coli (STEC) strains are a very small subset of the numerous Escherichia coli bacteria strains. These strains produce the powerful Shiga toxin as a byproduct of their infection. The most common strain that does this is E. coli O157:H7, but there are several other closely related strains that also produce Shiga toxin. The major symptom of illness is diarrhea, which may be bloody. Severe cases of STEC may result in hemolytic uremic syndrome (HUS) which, in turn, may lead to acute renal dysfunction necessitating the use of dialysis. While anyone may contract illness, children under age 5 and the elderly seem to be most prone to serious complications.

## Columns

```ls
| Included | Schema Type    | Field Name    | Name          | Data Type | Render Type |
| ======== | ============== | ============= | ============= | ========= | =========== |
| Yes      | series tag     | disease       | Disease       | text      | text        |
| Yes      | series tag     | yeartext      | YearText      | text      | text        |
| Yes      | time           | year          | Year          | number    | number      |
| Yes      | series tag     | stateorcounty | StateOrCounty | text      | text        |
| No       |                | il            | IL            | text      | text        |
| Yes      | series tag     | race          | Race          | text      | text        |
| Yes      | series tag     | ethnicity     | Ethnicity     | text      | text        |
| Yes      | series tag     | gender        | Gender        | text      | text        |
| Yes      | series tag     | agegrouptext  | AgeGroupText  | text      | text        |
| Yes      | series tag     | agegroup      | AgeGroup      | text      | text        |
| Yes      | numeric metric | count         | Count         | number    | number      |
```

## Time Field

```ls
Value = year
Format & Zone = yyyy
```

## Series Fields

```ls
Excluded Fields = il
```

## Data Commands

```ls
series e:f55m-q7ae d:2005-01-01T00:00:00.000Z t:yeartext=Year t:agegroup="25-29 Years" t:gender=Male t:stateorcounty=S t:disease=STEC t:ethnicity=UnkwnNotClassified t:race=White t:agegrouptext=CD5yr m:count=1

series e:f55m-q7ae d:2005-01-01T00:00:00.000Z t:yeartext=Year t:agegroup="40-44 Years" t:gender=Male t:stateorcounty=S t:disease=STEC t:ethnicity=UnkwnNotClassified t:race=White t:agegrouptext=CD5yr m:count=1

series e:f55m-q7ae d:2005-01-01T00:00:00.000Z t:yeartext=Year t:agegroup="65 + Years" t:gender=Male t:stateorcounty=S t:disease=STEC t:ethnicity=UnkwnNotClassified t:race=White t:agegrouptext=CD5yr m:count=2
```

## Meta Commands

```ls
metric m:count p:integer l:Count t:dataTypeName=number

entity e:f55m-q7ae l:"05to12 Iquery STECData" t:url=https://data.illinois.gov/api/views/f55m-q7ae

property e:f55m-q7ae t:meta.view v:id=f55m-q7ae v:averageRating=0 v:name="05to12 Iquery STECData"

property e:f55m-q7ae t:meta.view.owner v:id=r2ci-32dw v:screenName=rlucht v:displayName=rlucht

property e:f55m-q7ae t:meta.view.tableauthor v:id=r2ci-32dw v:screenName=rlucht v:roleName=publisher v:displayName=rlucht
```

## Top Records

```ls
| disease | yeartext | year | stateorcounty | il | race                 | ethnicity          | gender | agegrouptext | agegroup    | count |
| ======= | ======== | ==== | ============= | == | ==================== | ================== | ====== | ============ | =========== | ===== |
| STEC    | Year     | 2005 | S             | IL | White                | UnkwnNotClassified | Male   | CD5yr        | 25-29 Years | 1     |
| STEC    | Year     | 2005 | S             | IL | White                | UnkwnNotClassified | Male   | CD5yr        | 40-44 Years | 1     |
| STEC    | Year     | 2005 | S             | IL | White                | UnkwnNotClassified | Male   | CD5yr        | 65 + Years  | 2     |
| STEC    | Year     | 2006 | S             | IL | AmericanIndianAlaskn | NotHispNotLatino   | Male   | CD5yr        | 55-59 Years | 1     |
| STEC    | Year     | 2006 | S             | IL | Asian                | NotHispNotLatino   | Female | CD5yr        | 65 + Years  | 1     |
| STEC    | Year     | 2006 | S             | IL | Asian                | NotHispNotLatino   | Male   | CD5yr        | 5-9 Years   | 1     |
| STEC    | Year     | 2006 | S             | IL | Asian                | UnkwnNotClassified | Male   | CD5yr        | 0-4 Years   | 1     |
| STEC    | Year     | 2006 | S             | IL | BlackAfricanAmer     | NotHispNotLatino   | Female | CD5yr        | 0-4 Years   | 1     |
| STEC    | Year     | 2006 | S             | IL | BlackAfricanAmer     | NotHispNotLatino   | Female | CD5yr        | 40-44 Years | 1     |
| STEC    | Year     | 2006 | S             | IL | BlackAfricanAmer     | NotHispNotLatino   | Female | CD5yr        | 60-64 Years | 1     |
```

repository: SpellarBot/open-data-catalog

repo_name: SpellarBot/open-data-catalog


text:
## Clash Royale Themed Infinite Runner
This is our graphics project for DMET 502 Course at the GUC.

## Gameplay Video
[![Video](https://img.youtube.com/vi/CSetTcO1r_o/0.jpg)](https://youtu.be/CSetTcO1r_o)

###### You need CMAKE and OSX to be able to run this code.

repository: melzareix/cr-infinite-runner

repo_name: melzareix/cr-infinite-runner


text:
# Safety Results Over Time
![Example](https://github.com/RhoInc/safety-results-over-time/wiki/img/default.PNG)

## Overview
Safety Results Over Time is a JavaScript library built using Webcharts ([1](https://github.com/RhoInc/Webcharts), [2](https://github.com/RhoInc/webcharts-wrapper-boilerplate)) that allows users to view summaries of clinical outcomes such as lab results and vital signs over time. A typical chart created with safety results over time looks like the chart pictured above.
Users can specify grouping variables and overlay violin plots:

![Example](https://github.com/RhoInc/safety-results-over-time/wiki/img/customized.PNG)

Users can view any outcome in the data and stratify that outcome by sex or treatment, for example; the full functionality is described [here](https://github.com/RhoInc/safety-results-over-time/wiki/Technical-Documentation).
The library expects an [ADaM-esque data structure](https://www.cdisc.org/system/files/members/standard/foundational/adam/analysis_data_model_v2.1.pdf) by default but can be customized to use any dataset that is one row per participant per timepoint per measure.
Full details about chart configuration are [here](https://github.com/RhoInc/safety-results-over-time/wiki/Configuration).

Users can interact with the display in real time to:
* See the chart compare lab and vital sign values at different time points
* See the number and percentage of participants displayed in the current view (updates with each user interaction)
* Change the measure of interest, and see an updated chart
* Hide visits that do not include data
* View box plots within the chart
* Remove box plots from the chart
* View violin plots in the chart
* Remove violin plots in the chart
* Filter the shift plot for selected criteria, and see an updated chart (optional)

## Typical Usage
The code to initialize the chart looks like this:

```javascript

    d3.csv(
        'https://rawgit.com/RhoInc/viz-library/master/data/safetyData/ADBDS.csv',
        function(data) {
            safetyResultsOverTime('body', {}).init(data);
        }
    );

```

The chart can be configured to facilitate non-standard data formats and to alter the chart itself. Overwrite the defaults with a custom settings object like so:

```javascript
    const settings = {
        time_settings: {
            value_col: 'AVISIT',
            type: 'ordinal',
            label: 'Visit',
            rotate_tick_labels: true,
            vertical_space: 100
        },
        measure_col: 'PARAM',
        value_col: 'AVAL',
        normal_col_low: 'ANRLO',
        normal_col_high: 'ANRHI',
        groups: [
            {value_col: 'TRT01P', label: 'Treatment Group'},
            {value_col: 'SEX', label: 'Sex'},
            {value_col: 'RACE', label: 'Race'}],
        ],
        filters: [
            {value_col: 'TRT01P', label: 'Treatment Group'},
            {value_col: 'SEX', label: 'Sex'},
            {value_col: 'RACE', label: 'Race'}
        ],
        violins: true
    };

    d3.csv(
        'ADBDS.csv',
        function(data) {
            safetyResultsOverTime('body', settings).init(data);
        }
    );
```

Click [here](https://rhoinc.github.io/safety-results-over-time/test-page/) to open an interactive example of the Safety Results Over Time.

## Links
- [Interactive Example](https://rhoinc.github.io/safety-results-over-time/test-page/)
- [Configuration](https://github.com/RhoInc/safety-results-over-time/wiki/Configuration)
- [API](https://github.com/RhoInc/safety-results-over-time/wiki/API)
- [Technical Documentation](https://github.com/RhoInc/safety-results-over-time/wiki/Technical-Documentation)
- [Data Guidelines](https://github.com/RhoInc/safety-results-over-time/wiki/Data-Guidelines)

repository: RhoInc/safety-results-over-time

gha_language: JavaScript

gha_license_id: MIT

repo_name: RhoInc/safety-results-over-time


text:
# slack-parrot
dont let your memes be dreams

a clone of mimic

### Setup and Installation
* Make a new bot user in your Slack workspace and get yourself a Slack API token (see https://api.slack.com/bot-users)
* Clone the repo
* `pip install -r requirements.txt` in your preferred environment
* Add a file called `slackbot_settings.py
    * Add the setting `API_TOKEN = 'your_token_here'`
* Run the `parrot.py` in the background of your cloud instance with nohup or something

license: Unlicense

repository: SzySteve/slack-parrot

gha_language: Python

gha_license_id: Unlicense

repo_name: SzySteve/slack-parrot


text:
# Docker Datastack

[![Travis-CI Build Status](https://travis-ci.org/mikebirdgeneau/Docker-Datastack.svg?branch=master)](https://travis-ci.org/mikebirdgeneau/Docker-Datastack)

This project contains a general purpose set of Docker containers including:

1. Jupyter (with Python 3 & R kernels)
2. Rstudio
3. Shiny Server
4. Jupyterlab (Pre-Release)

We start with a custom set of packages for each based on things I find myself commonly using.
The purpose is a consistent (clean) starting point for data analysis and/or application development.

## Getting started

To launch this project, clone this repository then run:
`./launch` (launch.sh) from the project directory.

Alternately use `docker-compose up -d` then open a web-browser and visit: [http://localhost/](http://localhost).

Note that you may have issues if you have services running on ports 80, 8787, 8888, 8989, or 3838 as these will conflict with these services. In this case, you may need to edit `docker-compose.yml` before starting these containers.

repository: fredrijo/Docker-Datastack

repo_name: fredrijo/Docker-Datastack


text:
---
title: "Contextual Explanation Network, with Maruan"
hosts: ["Matt Gardner","Waleed Ammar"]
guests: ["Maruan Al-Shedivat"]
number: "021"
tags: []
description: "https://arxiv.org/abs/1705.10301

Maruan, Avinava Dubey and Eric Xing essentially put the post-hoc decision boundary explanations from the \"Why Should I Trust You?\" paper as a core component of a predictive model. Maruan comes on to tell us about it.

https://www.semanticscholar.org/paper/Why-Should-I-Trust-You-Explaining-the-Predictions-Ribeiro-Singh/5636dca44384240ce9aff2b10b78458cd3c2f450"
type: episode
---

<iframe width="100%" height="166" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?&url=https%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F328210590&show_artwork=true&show_comments=false"></iframe>

<turn speaker="Matt Gardner" timestamp="00:00">

Hello and welcome to the NLP highlights podcast where we talk about interesting recent work in
natural language processing.

</turn>

<turn speaker="Waleed Ammar" timestamp="00:06">

This is Matt Gardner and Waleed Ammar, we are research scientists at the Allen Institute for
Artificial Intelligence.

</turn>

<turn speaker="Waleed Ammar" timestamp="00:12">

Today our guest is my Maruan Al-Shedivat, Maruan is a PhD student in the machine learning department
at Carnegie Mellon University advised by Eric P. Xing. His research interests are in learning theory
information theory, representation learning, including various latent variable models. He is also
interested in applications in healthcare and natural language. It's a great pleasure to have you
with us today. Maruan

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="00:36">

Thank you. Waleed. It's a great for you to have me.

</turn>

<turn speaker="Waleed Ammar" timestamp="00:41">

All right. So we're going to talk about the paper that you put on archive recently. The paper is
called: Contextual Explanation Networks. And the main motivation or professional explanation
networks is that low interpretability of standard neural network values, which makes it hard to use
them in mission critical problems. Could you describe an example problem where contextual
explanation network can help?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="01:10">

Right. So the original motivation was exactly as you said. That, we'd like to have models that are
powerful, but at the same time interpretable. And a motivating example would be, for example, a
healthcare application where a medical doctor uses a model but not to make like predictions in the
sense where, for example, we tried to categorize images into multiple categories and then that's
just the end product, but rather than to get, you know, a sense of the problem and then try to make
a decision based on whatever predictions that were obtained in the first place. So for example, so
we give this example in the paper it's a small motivation to consider you have the patients and then
you have different data for that patient. So you'd like to make a prediction whether a patient has
some sort of condition, for example, heart arrhythmia.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="02:12">

And in this case you can have multiple variables that could contribute to the risk of having heart
arrhythmia. For example, diabetes, smoking, blood pressure and medical history, previous part
attacks. So whenever you ask a doctor, a doctor would tell you, for example you know, you have such
and such risk because such and such, you know variables are going to have certain values but you,
but the doctor takes into account, you know, or the context and probably knows the patient from
multiple visits. So we would like to build a model that would sort of, you know, would give you an
interpretable prediction such as a doctor's explanation, but at the same time the accurate enough to
match the actual risk.

</turn>

<turn speaker="Waleed Ammar" timestamp="03:07">

Right. So the inputs here are going to be some text that represents what we know about this patient
and also a set of categorical attributes.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="03:17">

Right, so the vision is that like, if we design a model of this class for healthcare, the input
would be text or it can be also images. And right now healthcare databases are growing and they're
becoming a pretty vast in terms of the types of data that you can collect about a patient. But also
you can have some sort of very specific crisp attributes that you'd like to interpret your decision
in terms of.

</turn>

<turn speaker="Waleed Ammar" timestamp="03:44">

All right, so what exactly do you mean by an explanation here in this paper? And how can we
interpret this explanation?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="03:51">

Right? so in this paper we define an explanation as a probabilistic model, as a conditional
probabilistic model that conditions on the attributes and gives you a probability of a target or an
outcome. And in a sense, it's a pretty general definition because let's say you have a neural
network multi-layer neural net that takes in the attributes and then output your probability of a
target. And it's still satisfies the explanation definition. But probably it's not good enough for
us because we would like to have the explanation to be interpretable. So that's why we restrict the
class of these models to be, let's say linear or some simple class of models. But how we overcome
this problem of the restrictiveness of this class we say that the explanations are contextual. So
this probabilistic model is valid in a certain context. And this context can be specified by let's
say, a deep neural network or something else.

</turn>

<turn speaker="Matt Gardner" timestamp="04:56">

Do you think you lose anything by making this restricted class of models?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="05:02">

That's true. That's a great question. So of course you lose something. For example your targets,
let's say your, targets, let's say not just scalars, but some sort of vector output. And then there
is some sort of relationships between these outputs, right? If you try to predict this using a model
that completely factorizes over the dimensions of the targets then you will not be able to capture
any dependencies between these dimensions. So for example, you're trying to predict three different
conditions that are somehow interrelated or something like that. If your model completely factorizes
over these dimensions, you will not be able to capture some correlations between those.

</turn>

<turn speaker="Waleed Ammar" timestamp="05:54">

So I'm missing a little bit of information here. How does making the explanation condition the
context make it interpretable? Rather, what do you mean by context for this composition?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="06:10">

Right. So for the context we we say that any data representation such as images or text or raw text
that is processed by whatever model that we're enough to learn from this from this data modality
will represent the context. So we take in, let's say it texts, we process it by an LSTM or maybe a
multi-layer recurrent bidirectional model. And then whatever the output vector would be, that would
be, that would represent a context for this particular instance. And once we condition on that we
say, well now we would like to construct a conditional model on this. That would have, would be from
the class of linear models from the class of linear models whose features are some specific
attributes such that we can relate weights to every single attribute. And then in this case, the
weight will have a very specific meaning.

</turn>

<turn speaker="Waleed Ammar" timestamp="07:11">

I see.

</turn>

<turn speaker="Matt Gardner" timestamp="07:12">

So this sounds pretty similar to some work that was by Sameer Singh and some co-authors that I guess
was a post-hoc explanation. I guess that works says these models are really complex and it's really
hard to give some kind of intuitive explanation for what's going on. But for any particular
instance, I can find that some linear approximation to the classification decision that was made on
this particular data point by doing some fancy sampling and then show a linear approximation for the
decisions surface at that point. And so I'm pretty sure you mentioned this in the paper, but can
you, can you tell us, like what are the trade offs involved in doing this post-hoc thing versus
putting it in the model? Like what do you think?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="07:56">

Right, so initially we thought that it's a great idea to do this post-hoc interpretation. But the
problem is that when you construct a model post-hoc you don't actually use that model in the first
place to make a prediction. So it means that this post-hoc interpretation while it approximates the
decision boundary. In certain cases, if you have selected this representation in terms of which
you'd like to make you know, your linear interpretation to construct this decision boundary. So,
everything depends upon how you selected that.  And this will not affect the performance, the model. So you
don't have a way to sort of judge in a very you know, principal manner, how well, you know, your
interpretation matches what actually the model is doing internally. In our case, we combine these
two paradigms together and then learn to interpret and to predict jointly. And as we show in the
experiments, in this case, if there's something wrong with the representation in terms of which
you're trying to interpret the model the performance will deteriorate while with the line framework.
It's not the case.

</turn>

<turn speaker="Matt Gardner" timestamp="09:34">

Interesting.

</turn>

<turn speaker="Waleed Ammar" timestamp="09:37">

So this is a lot of teasing for our audience. Could you explain to them what the context of
explanation is, what you actually do, how do they work?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="09:47">

Right, right. I probably should have started from first explaining how it works and then, go into
details. But so let's let's look at an example. So what happens? So consider a different application
or let's consider the same application. So we have some texts and then we apply to the context for a
medical note or just a paragraph an LSTM network. We pre-process it, we get a hidden vector. Now we
apply to this hidden vector a linear layer and then apply a softmax to get a probability
distribution so why we do that is because we store somewhere a dictionary of explanations. So a
dictionary of this linear models that we would like to apply to make the final prediction. And so
what we do, we use the softt attention mechanism to retrieve a model from this dictionary and then
work parameters of that model from the dictionary. And then we applied this model to the attributes
that we'd like to make prediction.

</turn>

<turn speaker="Matt Gardner" timestamp="11:02">

So, so does that mean that you have this dictionary of weights associated with linear your models
and you're actually doing a weighted sum over all of these givens, like some attention over each,
over all of the explanations in the dictionary? Is that, is that right?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="11:18">

That's, correct. That's correct. So in practice, what happens is that the dictionary becomes pretty
sparse, and also the attention vector becomes extremely sparse. So technically in practice what
happens is that we select one element from the dictionary all the time.

</turn>

<turn speaker="Matt Gardner" timestamp="11:34">

Okay.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="11:35">

And even if the dictionary is sort of over parameterized we observe and practice that's a lot of
elements will just die out by themselves.

</turn>

<turn speaker="Matt Gardner" timestamp="11:42">

Do you enforce that sparsity particularly, or is it just?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="11:44">

That's right. We do enforce it. We opened an aligned organization on the entire dictionary.

</turn>

<turn speaker="Matt Gardner" timestamp="11:50">

Okay.

</turn>

<turn speaker="Waleed Ammar" timestamp="11:51">

So, I think this refers to the constraint of the deterministic map is that what you're mentioning.
So, how does this compare to the deterministic encoding and the Mixture of Experts? Like two
different modeling this direction?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="12:05">

Right. So dictionary the sort of the constraint deterministic map where we use the dictionary is not
the only one. So the, the easiest one would be to amid this dictionary and try to predict parameters
of these explanations directly from the context. So that's the deterministic encoding where you just
take in the context and then you try to predict parameters of an explanation. So Mixture of Experts
is also a related model. So instead of trying to combine elements from the dictionary through
attention, what you do, you build every single model in the dictionary and then combine the
predictions together. So you're look at the predictions and then you say the final prediction is
going to be a weighted sum of the predictions of every single model and that the weights are going
to be the probabilities of these every single model in the dictionary according to the, according to
the context. So you out output this probability conditional on the context. And then wait with these
probabilities the models in the dictionary, in this case, we call them experts. So it's the
classical mixture of experts models. So maybe one, difference is that mixture of experts usually has
context. And in all these models, usually context and attributes are the same representation, but in
this case we would separate them into two different conditions.

</turn>

<turn speaker="Waleed Ammar" timestamp="13:39">

I see. So the paper compares the explanations generated by the contextual explanation network and
the post-hoc explanations could you shed some light on the results of this comparison?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="13:54">

Right. So there are two results. So one of them is theoretical results and the, and the other one
that I've already mentioned is more like empirical results. So the theoretical results says that
under certain conditions if we're trying to interpret our contextual explanation network with a
post-hoc methods, or we're trying basically to locally approximate the decision boundary of our
contextual explanation network, we will recover the explanation that the network used in the first
place.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="14:32">

So this means basically that the explanation's generated by the networks are consistent with the
post-hoc interpretations. In this particular case. So it also tells us that if post-hoc whatever
post-hoc method that you use or local boundary approximation method that we use so these
explanations, if they are interpretable, it means that the explanation is generated by contextual
explanation networks are also interpretable because they will basically match. So the other is also
empirical where we're trying to inject noise into the attributes in terms of which the explanations
are constructed. And the idea here is that so if you add noise to the attributes, you've sort of
decorrelate your targets and the attributes. So whatever your model conditional model that you're
trying to build, basically we're trying to predict from noise some outcome.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="15:44">

And this in the worst case when that, when the noise level is pretty high this should lead to
basically random prediction. And that's what we observe with contextual information networks. So
their performance deteriorates. The more noise you add the, the worst performances. Because they
basically, regardless of which explanation you construct, you can do better than random. But then
we're trying to also, we tried to approximate decision boundaries of let's say a network, in this
case an LSTM or a convolutional network trained on the context on me. And then when we tried to
explain it in terms of these noisy attributes in post-hoc manner. So because we were trying to in
post-hoc manner, in post-hoc approximation, because we're fitting those local linear models to the
actual predictions, they still match locally the departments of the original model. But the
interpretations are providing in terms of pretty much noise. Does that make sense?

</turn>

<turn speaker="Waleed Ammar" timestamp="17:00">

It does, yes.

</turn>

<turn speaker="Matt Gardner" timestamp="17:01">

So I have a high level question. What, I've seen a bunch of work, I guess people are really
interested these days and how to make these models interpretable. Do you think that showing like
linear feature weights is the ends like, is this good enough? Should we be looking for something
more? Like what do you think about the future of work on interpretability?

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="17:29">

I don't know, in my opinion. Both methods should be used sort of together. Because visualizing
feature weights allows you to somewhat debug the model that you trained on your data and just see
what are the internals, how it functions. It doesn't give you a specific, it doesn't give you an
explanation of why the model makes this or that prediction. So but it gives you, it gives you a lot
of other information that probably could be used for improving, you know training procedures. Of,
these models. So there's neural networks.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="18:11">

In my opinion, like an explanation is by definition, like explanation, for example, I explained to
you a concept in math rights. I, tell you, I tell you a model that you can further take and then use
for other math exercises and it pretty much, you know, solving correctly. So I think an explanation
is by itself a model. And so it's a little bit different perspective than a neural network. So I'm
not sure how to combine these two together in in a sense that you can still use both, but I think a
way, an interesting way an interesting extension would be to try to combine where you sort of can
visualize, but at the same time, you can relate your visualizations to explanations in terms of some
sort of crisp features.

</turn>

<turn speaker="Matt Gardner" timestamp="19:01">

Interesting. Thanks.

</turn>

<turn speaker="Waleed Ammar" timestamp="19:03">

Thank you very much for for spending the time to talk to us. And yeah, I hope to see you soon.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="19:09">

All right. Thanks so much for having me.

</turn>

<turn speaker="Waleed Ammar" timestamp="19:12">

Bye. Bye.

</turn>

<turn speaker="Maruan Al-Shedivat" timestamp="19:12">

Bye

</turn>

repository: allenai/nlphighlights

gha_license_id: NOASSERTION

repo_name: allenai/nlphighlights


text:
```
                                 __   _ __      ____
                                / /  (_) /____ / __/______  ___  ___
                               / /__/ / __/ -_)\ \/ __/ _ \/ _ \/ -_)
                              /____/_/\__/\__/___/\__/\___/ .__/\__/
                                                         /_/
                               Copyright 2015-2022 / EnjoyDigital

                           A small footprint and configurable Logic Analyzer
                                    core powered by Migen & LiteX
```

[![](https://github.com/enjoy-digital/litescope/workflows/ci/badge.svg)](https://github.com/enjoy-digital/litescope/actions) ![License](https://img.shields.io/badge/License-BSD%202--Clause-orange.svg)

[> Intro
--------
LiteScope provides a small footprint and configurable embedded logic analyzer that you
can use in your FPGA and aims to provide a free, portable and flexible
alternative to vendor's solutions!

LiteScope is part of LiteX libraries whose aims are to lower entry level of
complex FPGA cores by providing simple, elegant and efficient implementations
of components used in today's SoC such as Ethernet, SATA, PCIe, SDRAM Controller...

Using Migen to describe the HDL allows the core to be highly and easily configurable.

LiteScope can be used as LiteX library or can be integrated with your standard
design flow by generating the verilog rtl that you will use as a standard core.

[> Features
-----------
- IO peek and poke with LiteScopeIO.
- Logic analyser with LiteScopeAnalyzer:
  - Subsampling.
  - Data storage in Block RAM.
  - Configurable triggers.
- Bridges:
  - UART <--> Wishbone (provided by LiteX)
  - Ethernet <--> Wishbone ("Etherbone") (provided by LiteEth)
  - PCIe <--> Wishbone (provided by LitePCIe)
- Exports formats: .vcd, .sr(sigrok), .csv, .py, etc...

[> Proven
---------
LiteScope has already been used to investigate issues on several commercial or
open-source designs.

[> Possible improvements
------------------------
- add standardized interfaces (AXI, Avalon-ST)
- add protocols analyzers
- add signals injection/generation
- add storage in DRAM
- add storage in HDD with LiteSATA core
- ... See below Support and consulting :)

If you want to support these features, please contact us at florent [AT]
enjoy-digital.fr.

[> Getting started
------------------
1. Install Python 3.6+ and FPGA vendor's development tools.
2. Install LiteX and the cores by following the LiteX's wiki [installation guide](https://github.com/enjoy-digital/litex/wiki/Installation).
3. You can find examples of integration of the core with LiteX in LiteX-Boards and in the examples directory.

[> Tests
--------
Unit tests are available in ./test/.
To run all the unit tests:
```sh
$ ./setup.py test
```

Tests can also be run individually:
```sh
$ python3 -m unittest test.test_name
```

[> License
----------
LiteScope is released under the very permissive two-clause BSD license. Under
the terms of this license, you are authorized to use LiteScope for closed-source
proprietary designs.
Even though we do not require you to do so, those things are awesome, so please
do them if possible:
 - tell us that you are using LiteScope
 - cite LiteScope in publications related to research it has helped
 - send us feedback and suggestions for improvements
 - send us bug reports when something goes wrong
 - send us the modifications and improvements you have done to LiteScope.

[> Support and consulting
-------------------------
We love open-source hardware and like sharing our designs with others.

LiteScope is developed and maintained by EnjoyDigital.

If you would like to know more about LiteScope or if you are already a happy
user and would like to extend it for your needs, EnjoyDigital can provide standard
commercial support as well as consulting services.

So feel free to contact us, we'd love to work with you! (and eventually shorten
the list of the possible improvements :)

[> Contact
----------
E-mail: florent [AT] enjoy-digital.fr

repository: enjoy-digital/litescope

repo_name: enjoy-digital/litescope


text:
# Implementation Version 1

## Overall Idea of implementation

According to the problem statement, We need to identify (detect) the logo based on an input image.

### About the dataset

We are given a [dataset](./training_data/) of 12 images. These images belong to the following classes : `Bank of America`, `Capital One`, `Citigroup`, `JPMorganChase`, `WellsFargo` and `Others`. Additionally we are given few sample images to do the model validation. These sample images consists of `headers` and `logos(from different classes)`

## Implementation:

The most obvious method of implementation is by using [template matching](https://www.wiley.com/en-us/Template+Matching+Techniques+in+Computer+Vision%3A+Theory+and+Practice-p-9780470517062). To find the implementation please refer [here](https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html). But unfortunately, this naive technique have some implementation issues as far as logo detection is concerned.  The major issue is that if the test image is scaled, then the template will not be able to detect the exact position of the logo itself. This would make the model unstable under the scale variance.

The other technique that we can use is [matching descriptors](https://docs.opencv.org/2.4/doc/tutorials/features2d/feature_homography/feature_homography.html). In this implementation we can extract the descriptor vectors of the image which are invariant to scaling and intensity. This would make our model robust to scale and intensity variation.

For this version I have chosen BLOB DETECTOR which uses corner detection technique (Harris corner detector) for the descriptor and key-points extraction. The blob detectors detect the image corners and represent as blobs which plays an important role in tracking and object detection.  Below are the steps involved in the implementation :

1. Some hyper parameters are necessary to define for a single implementation. Here, initial sigma value for gaussian function used for filtering in laplacian of gaussian, number of step in the space scale and the threshold values for maximum suppression are those hyper parameters. Each combination of these parameters would result in different outcome and can be tuned according to user preferences.

2. The input image provided is raw and need to preprocess. The image is first converted to grayscale and then converted its data type to double followed by normalization.

3. After preprocessing, the blob detection algorithm is applied as follows:

   1. Scaling the filter operation (inefficient):

      ![Image result for scale invariant](https://docs.opencv.org/3.0-beta/_images/sift_dog.jpg)

   2. Convolve the image with filter of laplacian of gaussian with given sigma value. For this process MATLAB fspecial function is used (with keeping an odd filter as a constraint via sigma value) to generate filter.

   3. Compare each pixel in the convolved image with a set of neighbourhood and selection of the best is performed.
      Repeat the above 2 step to create space scale (with hyperparameter of steps) with different values of sigma obtained by factor multiplication of K and thereby generating different size of filter to convolve an image with. As, we are only concerned with the maximum response of a pixel to LOG filter, non- maximum suppression is performed in 2D slices.
      Maximum value across the scale space is then taken as non-maximum suppression in 3D.
      Now we consider only those pixel which passes survival threshold. The threshold value is the same value form hyperparameter. Then the characteristic radius and the center is calculated with the following equation:  r = σ. √2

   4. Scaling the image operation (efficient):

      Given an image and a sigma value, the laplacian filter is created to convolve with the image.
      The given image is then resized with factor of inverse of k .

      ![Related image](http://campar.in.tum.de/twiki/pub/Chair/KlinkerCMU/FidoWarpPyramid.JPG)

      The similar operation of the LoG convolution is applied to the resized image. The image is then rescaled again to normal dimensions.
      Similar steps starting from 2D non maximum suppression is applied to this procedure as well.

### Results:

Here's how the filter looks like:

![filter](./output/Process/filter.gif)

Below are the images snaps taken on different level of filter. This makes the processes scale invariant. One of the sample image of `Bank of America` is taken here:

![sample](./output/sample/sample.gif)

Here is one of the output of the BLOB DETECTORS:

![final_blob](./output/final_blob.jpg)

## Bag of Descriptors:

> **NOTE**:  The above implementation is done on MATLAB. Please refer to the repository for codes.

Now that we have extracted the features that invariant to rotation as well as as scaling, we can collect all these descriptors `Bag of Descriptors`and then compare the testing image via brute force or RANSAC.

The further implementation is done in version 2 of the project.

## Running the project

To run the project the you will need MATLAB 2017b.

```
Steps to run the code:
	1. Place the images in the data folder.
	2. Replace the variable name in the main.m
	3. execute main.m
```

# References :

1. https://www.wiley.com/en-us/Template+Matching+Techniques+in+Computer+Vision%3A+Theory+and+Practice-p-9780470517062
2. https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html
3. https://docs.opencv.org/2.4/doc/tutorials/features2d/feature_homography/feature_homography.html
4. David G Lowe. Distinctive image features from scale-invariant keypoints.
5. Sample Harris detector code


text:
# behna Libarary

## CDN
Just copy and paste this link to your head section in link tag

```
https://behna.herokuapp.com/behna.css
```

## Demo
For Demonstration

https://behna.herokuapp.com

### License

BehnaCss is [MIT licensed](./LICENSE).

license: ISC,MIT

repository: raitalharehman/behnaCssDemo

detected_licenses: ISC MIT

repo_name: raitalharehman/behnaCssDemo


text:
# docker-traefik-nextcloud-nginx
docker compose files for traefik, nextcloud and nginx

#### Docker versions in test environment

- Docker version 20.10.6, build 370c289
- docker-compose version 1.25.5, build 8a1c60f6

### Getting started

**ENV**

Adjust the the .env files in nextcloud, traefik and  nginx

in line 26 in traefik/volume enter your auth string (replace USER:AUTH). You can generate the authentification string with this commands:
```
apt-get install apache2-utils
echo $(htpasswd -nb user password) | sed -e s/\\$/\\$\\$/g
```
change the emailadress in traefik/volume/traefik.yaml file (line 20)

**Next Steps**
1. ```docker network create proxy```
2. ```bash startall.sh```

### Nextcloud
After the nextcloud url has been called, nextcloud must be configured like this


text:
## Unrestricted Multi-Shift-And Algorithm (UMSA)

An unrestricted implementation of the Multi-Shift-And algorithm for multiple exact pattern matching. Patterns can be longer than the computer word size.

The example program (main.cpp) is coded to only take DNA alphabet (A, C, G and T) patterns, but the UMSA class itself can handle any ASCII patterns and any text.

Example program usage: `./umsa DNA_SEQUENCE DNA_PATTERN1 [DNA_PATTERN2 ...]`


text:
## Quick start: landmark detection

[![Paper](http://img.shields.io/badge/paper-arXiv.1812.01584-B3181B.svg)](https://arxiv.org/abs/1812.01584)

### Install DELF library

To be able to use this code, please follow
[these instructions](INSTALL_INSTRUCTIONS.md) to properly install the DELF
library.

### Download Oxford buildings dataset

To illustrate detector usage, please download the Oxford buildings dataset, by
following the instructions
[here](EXTRACTION_MATCHING.md#download-oxford-buildings-dataset). Then, create
the file `list_images_detector.txt` as follows:

```bash
# From tensorflow/models/research/delf/delf/python/examples/
echo data/oxford5k_images/all_souls_000002.jpg >> list_images_detector.txt
echo data/oxford5k_images/all_souls_000035.jpg >> list_images_detector.txt
```

### Download detector model

Also, you will need to download the pre-trained detector model:

```bash
# From tensorflow/models/research/delf/delf/python/examples/
mkdir parameters && cd parameters
wget http://storage.googleapis.com/delf/d2r_frcnn_20190411.tar.gz
tar -xvzf d2r_frcnn_20190411.tar.gz
```

**Note**: this is the Faster-RCNN based model. We also release a MobileNet-SSD
model, see the [README](README.md#pre-trained-models) for download link. The
instructions should work seamlessly for both models.

### Detecting landmarks

Now that you have everything in place, running this command should detect boxes
for the images `all_souls_000002.jpg` and `all_souls_000035.jpg`, with a
threshold of 0.8, and produce visualizations.

```bash
# From tensorflow/models/research/delf/delf/python/examples/
python3 extract_boxes.py \
  --detector_path parameters/d2r_frcnn_20190411 \
  --detector_thresh 0.8 \
  --list_images_path list_images_detector.txt \
  --output_dir data/oxford5k_boxes \
  --output_viz_dir data/oxford5k_boxes_viz
```

Two images are generated in the `data/oxford5k_boxes_viz` directory, they should
look similar to these ones:

![DetectionExample1](delf/python/examples/detection_example_1.jpg)
![DetectionExample2](delf/python/examples/detection_example_2.jpg)

### Troubleshooting

#### `matplotlib`

`matplotlib` may complain with a message such as `no display name and no
$DISPLAY environment variable`. To fix this, one option is add the line
`backend : Agg` to the file `.config/matplotlib/matplotlibrc`. On this problem,
see the discussion
[here](https://stackoverflow.com/questions/37604289/tkinter-tclerror-no-display-name-and-no-display-environment-variable).


text:
---
title: "One Platform"
sub_link: "one-platform"
meta_title: "Single Platform - Polyaxon As A Single Platform For Machine Learning and Deep Learning Life Cycle"
meta_description: "Polyaxon is a single platform that does everything from accessing data to managing machine learning models. This provides more results, happier developers and data scientists, improved cycle times, and a great return on investment."
visibility: public
status: published
tags:
  - reference
---

## One platform

Polyaxon is a single platform that does everything from accessing data to managing machine learning models. This provides more results, happier developers and data scientists, improved cycle times, and a great return on investment.

Other solutions currently in the market offer a kit plane you have to assemble yourself, or lock-in solution with a rigid approach, Polyaxon is based on an open-source end-to-end opinionated platform with flexible and extensible interfaces.

## One access interface

Users on your teams have to access to one and only one platform to perform most of the operations needed to develop and manage ML models.
No time is wasted switching between platforms with different access interfaces.

Teams can set one high-level authorization for managing access to the different aspects of the platform.
This means that you set permissions once and everyone in your organization has the right access to every component.

One interface can have a major impact on distributing knowledge within and between teams, which will result in improving all your employees' productivity.

## One project interface

Polyaxon provides one interface for managing the lifecycle of your projects, with all needed components, steps, and plugins,
this gives Polyaxon all information needed at every step to expose the relevant context to users.
This allows natural, fast, and intuitive access to the needed information in an easy and optimal way.
Finally, the single project interface reduces information dissipation due to the nonexistent context switching, which results in more productivity.

## One on-boarding process

New team members only need to learn about a single, simple, and intuitive interface and tools. Our tools and documentation are integrated, non-dependent on other unestablished projects, and open-source.
This means that not only you don't have to write your own documentation, but you can contribute to make on-boarding your users, as well as anyone in the community a breeze.

## One deployment

Running Polyaxon means that you only need to install, maintain, scale, backup, network, and secure one deployment.
One deployment results in saving time from your developers, reduces risk, increases reliability, and lowers the bill.

## One upgrade

Features and plugins developed and maintained by Polyaxon are guaranteed to work properly after every upgrade.
Maintaining several different ad-hoc applications and components increase the complexity of your deployments,
and makes upgrades a complex process prone to errors and incompatibilities.

## One centralized database

Managing the state of an application is hard, so imagine what it means to manage several stateful applications with different scalability, migrations, and backup processes.
In addition to the non-necessary complexity, several applications plane will result in redundant, and eventually inconsistent, data.

Polyaxon comes with one database, PostgreSQL, providing you with one consistent source of truth about the state of your deployment, so you don't have to manually extract data or create a warehouse.

## One Vendor

You only have to deal with Polyaxon, while ensuring that you have access to an open-source ecosystem, instead of multiple vendors with different licensing and risk pointing to each-other.

If you decide to invest in an internal solution, you can base it on a tool you are already using, and you only need to learn one code base, which is open-source.

repository: polyaxon/polyaxon

repo_name: polyaxon/polyaxon


text:
# ReactYouTubeSearch

React App using YouTube API to search for the top 5 vidoes. Built through Stephen Grider's Udemy React course

```
> npm install
> npm start
```

repository: CarrieHappoldt/ReactYouTubeSearch

repo_name: CarrieHappoldt/ReactYouTubeSearch


text:
# [Kingdom Hearts Melody of Memory](index.md) Database

Most of the game logic is able to be manipulated using a SQLite database located in...

| Platform | Location                          |
|----------|-----------------------------------|
| Switch   | `StreamingAssets\SQLite\Table.db` |
| PC       | `StreamingAssets\Data\data.dat`   |

The database is encrypted using SQLCipher 3 with default values.

## Documented Tables

| Table Name         | Notes |
|--------------------|-------|
| AbilityTable       | Lists IDs of all [abilities](abilities.md)
| DoorTable          | Defines criteria for doors to unlock other worlds |
| GameSceneFlowTable | Defines menu layout, eg. Title>Tutorial>Main Screen, comments describe what each flow does, translation required however |
| MusicTable         | Defines music track properties, like difficulty values, item drops and drop rates |
| PartyTable         | Defines which characters are in each team, and if they're selectable by the player or not. |
| TripWorldTable     | Defines worlds, eg. which tracks should be included (up to 8), world icon, clearing rewards, and more |

## Noteworthy Details

* 3 regions are defined in MusicTable, "Japan", "World", and "Asia", with boolean values on each song defining if they are "included" in that region.
* This may be used for different translations of music like Hikari/Simple and Clean

## Example Edits

Using a SQLite DB editor, such as "DB Browser for SQLite" execute these SQL queries:

### Disabling first start tutorial

Title Screen goes directly to Main Menu:

```sql
UPDATE GameSceneFlowTable
SET NextGameSceneID = 10000034
WHERE GameSceneID=10000005 AND NextGameSceneID=10000077;
```

### Disabling promotion screen after playing a few songs

The trigger still happens automatically after finishing a few songs but drops you at the music list instead of the eShop promo screen:

```sql
UPDATE GameSceneFlowTable
SET NextGameSceneID = 10000000
WHERE GameSceneID=10000003 AND NextGameSceneID=11000000;
```

### Changing Team Classic to contain Sora, Roxas, and Ventus

Can be done with other characters, however you will need to assign them DeformedChara and Ability values in CharacterTable.

```sql
UPDATE PartyTable
SET Character1 = 110000000
WHERE PartyID=120000000;
UPDATE PartyTable
SET Character2 = 110000006
WHERE PartyID=120000000;
UPDATE PartyTable
SET Character3 = 110000009
WHERE PartyID=120000000;
```

### Unlock all worlds on the world map by default

```sql
UPDATE TripWorldTable
SET StartFlag = 1;
```

repository: OpenKH/OpenKh

gha_language: C#

repo_name: OpenKH/OpenKh


text:
#Class: mutual_trust

Provides a defined type named "mutual_trust::ssh" which enables mutual
trust between users on different systems.

All uses of mutual_trust::ssh with the same tag will result in trust to every other usage of mutual_trust::ssh with that same tag!

*For this module to work, you **must** enable storeconfigs on the puppetmaster server.* For instance, you can put this in `/etc/puppet/puppet.conf`:

    [master]
        storeconfigs = true

##Parameters

* **tag**

Optional (defaults to *$name*).

The tag used to collect resources. All nodes declared with the same tag will trust each other.

* **user**

Optional (defaults to *root*).

The user for whom to collect public ssh keys.

* **homedir**

Optional (defaults to a guess such as */root* or */home/$user*)

The home directory for the user. This is only used to guess the *sshdir* parameter.

* **sshdir**

Optional (defaults to *$homedir/.ssh*)

The ssh directory containing the authorized_keys file.

##Examples

    include mutual_trust
    node /foo/ {
        mutual_trust::ssh {"web":}
    }
    node /bar/ {
        mutual_trust::ssh {"web":}
        mutual_trust::ssh {"db":}
    }
    node /baz/ {
        mutual_trust::ssh {"db":
            user => oracle
        }
    }

As a result, root@foo and root@bar will be able to login to each other.

root@bar can also login to oracle@baz and vice versa.

oracle@baz cannot login directly to root@foo, but can do so anyway
while logged into root@bar.

##Authors

Maarten Thibaut<EMAIL_ADDRESS>
##Copyright

Copyright 2012 Maarten Thibaut. Distributed under the Apache License,
Version 2.0.

repository: mthibaut/puppet-mutual-trust

repo_name: mthibaut/puppet-mutual-trust