text:
# Liferay GSearch
The Google like search for Liferay 7 CE and Liferay DXP.

# Table of contents

1. [Background](#Background)
1. [Features](#Features)
1. [Screenshots](#Screenshots)
1. [Requirements](#Requirements)
1. [Project Modules](#Modules)
1. [Installation](#Installation)
1. [Enabling Audience Targeting support](#Audience_Targeting)
1. [Embedding Search Field into a Theme](#Search_Field)
1. [Sample Configurations](#Configurations)
1. [Troubleshooting](#Troubleshooting)
1. [Important Notes](#Important)
1. [FAQ](#FAQ)
1. [Project Roadmap](#Roadmap)
1. [Credits](#Credits)
1. [Disclaimer](#Disclaimer)
1. [Changelog](#Changelog)

# 1 Background

This is the Google like search project for Liferay CE & DXP. The code is originally created for the blog series:

 * [Part 1: Creating a google like search](https://web.liferay.com/web/petteri.karttunen/blog/-/blogs/creating-a-google-like-search)
 * [Part 2: Filter by structure and document type](https://web.liferay.com/web/petteri.karttunen/blog/-/blogs/creating-a-google-like-search-part-ii-filter-by-structure-and-document-type)
 * [Part 3: Autocompletion and suggestions](https://web.liferay.com/web/petteri.karttunen/blog/-/blogs/creating-a-google-like-search-part-iii-autocompletion-suggestions)
 * [Part 4: Query tuning and Lucene syntax](https://web.liferay.com/web/petteri.karttunen/blog/-/blogs/creating-a-google-like-search-part-iv-query-tuning-and-lucene-syntax)
 * [Part 5: Finale](https://web.liferay.com/web/petteri.karttunen/blog/-/blogs/creating-a-google-like-search-part-v-finale)

This project served many purposes for me. I wanted to experiment with SOY & Metal.JS (which are a really great combination), practise writing OSGI compliant code but most importantly: I wanted to have a highly configurable, alternative search portlet for Liferay having support for Boolean operators and Lucene syntax and many other features currently missing in the standard search portlet, like autocompletion. I wanted to have a search portlet with better control over hits relevancy.

# 2 Features

* Google like appearance and UX
* Completely ajaxed interface (no page transitions)
* 3 selectable search result layouts (image card layout available for files / image)
* Sortable search results (not available in default Liferay search)
* Bookmarkable searches with short urls which can easily be collected to Google Analytics
* Autocompletion using aggregate query suggestions (phrase and completion suggesters)
* Automatic alternate search
* Support for Boolean operators and Lucene syntax
* Completely configurable:
    * Asset types to search for
    * Facets to retrieve
    * Sort fields
    * Fields to search for and their boosting
    * Suggesters
* Audience targeting support: configurable boosting for content matching user's user segments.
* Ability to include non-Liferay resources in the search results
* Speed ; It's fast

# 3 Screenshots

## Basic Usage
![Basic Usage](https://github.com/peerkar/liferay-gsearch/blob/master/gsearch-doc/screenshots/basic-usage.gif)

## Image Layout
![Image Layout](https://github.com/peerkar/liferay-gsearch/blob/master/gsearch-doc/screenshots/image-layout.gif)

## Configuration
![Configuration ](https://github.com/peerkar/liferay-gsearch/blob/master/gsearch-doc/screenshots/configuration.gif)

## Non-Liferay Assets in the Search Results
![Non Liferay Assets](https://github.com/peerkar/liferay-gsearch/blob/master/gsearch-doc/screenshots/non-liferay-asset.gif)

# 4 Requirements

## Mandatory

* At least Liferay __DXP SP5__ / fixpack version 28 OR Liferay __CE GA5__

## Optional

Some of the features require the custom Elasticsearch adapter. If you plan to use that, please also install a standalone Elasticsearch server to be able to configure custom type mappings. Installing a standalone server is recommended anyways.

# 5 Project Modules

## gsearch-core-api
API module for the backend logic.

## gsearch-core-impl
Implementation module for the backend logic.

## gsearch-query-api
This module contains contains the extension for portal search API's StringQuery type, the QueryStringQuery query type.

## gsearch-web
UI module written using SOY & Metal.js.

## gsearch-mini-web
Miniportlet for adding a search field into a theme. Please see the gsearch-test-theme for an example.

## gsearch-test-theme
A test theme embedding miniportlet into it.

## gsearch-audience-targeting
Module enabling Audience Targeting support.

## gsearch-elasticsearch-adapter
A custom Elasticsearch adapter implementing Elasticsearch QueryStringQuery translator into Liferay portal search API. It has also index setting customizations: custom analyzers for keyword suggester and ascii folding filter for the title, description and content fields to better support non-ascii characters (for us non English speaking people).

Please see the adapter in its' [own repo](https://github.com/peerkar/gsearch-elasticsearch-adapter).

# 6 Installation
If you find the instructions insufficient or need more information, please leave a ticket and I'll do my best.

__If you are updating the modules__:

* Please remember to remove the older versions from osgi/modules and clean up osgi/state folder first.
* Please check the configuration file for changes. Preferably put the sample configuration file in the [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest/) to the osgi/configs folder.

## Step 1

### Option 1 (The Easy Way)

Download following jars from [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest) and deploy:

* fi.soveltia.liferay.gsearch.core-api-VERSION.jar
* fi.soveltia.liferay.gsearch.core-impl-VERSION.jar
* fi.soveltia.liferay.gsearch.query-VERSION.jar
* fi.soveltia.liferay.gsearch.web-VERSION.jar

After installing the easiest way to check that modules have been properly installed, is to use Gogo shell. For example:

```
> telnet localhost 11311
> lb -s GSearch"

```

Check that status of all the modules is active.

### Option 2 (The Advanced Way)
If you want to build everything by yourself, you need to have:

1. Liferay workspace
2. Clone this repository in the modules directory
3. Do a Gradle refresh (in the context menu of Eclipse) for the modules directory
4. Build and deploy to you Liferay (DXP) installation

If you need to build the Elasticsearch adapter please see [this repository](https://github.com/peerkar/gsearch-elasticsearch-adapter)

## Step 2 - Install the Custom Elasticsearch Adapter

This is not mandatory but if you want to have all the features and configurability there, then use this one and use a standalone Elasticsearch server.

First, uninstall the standard Elasticsearch adapter from control panel apps management or, preferably, from Gogo shell:

```
> telnet localhost 11311
> lb -s com.liferay.portal.search.elasticsearch (to get the bundle_id)
> uninstall bundle_id
```
After that, deploy the custom adapter (com.liferay.portal.search.elasticsearch-VERSION-GSEARCH-PATCHED.jar) which you downloaded earlier from the [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest/). Please see again from Gogo shell that it's deployed properly.

Please note that by default the standard search adapter reinstalls every time, you reboot the portal. It's not fatal if both adapters start simultaneously but search just won't work before you uninstall either one.

## Step 3 - Configuration

After succesfully deploying the modules portlet has to be configured. Otherwise it doesn't work.

Portlet configuration can be found in Control Panel -> Configuration -> System Settings -> Other -> Gsearch Configuration.

The easy and fast way to get this to work is to download the default configuration file **fi.soveltia.liferay.gsearch.core.configuration.GSearchConfiguration.config** from [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest/) and copy it to Liferay\_home\_folder/osgi/configs/

There's then just one more thing to do:

1. Create a page and put there an Asset Publisher portlet to show any contents that are not bound to any layout (page). By default this pages' friendlyUrl should be "/viewasset". Typically, you would configure this page to be hidden from navigation menu.
2. In the portlet configuration, in Control Panel -> Configuration -> System Settings -> Other -> Gsearch Configuration, point "Asset Publisher page friendly URL" to the friendly of of the page you just created.

You can find the sample configurations in the end of this documentation.

## Step 4 - Update Suggester mapping

If you are not using the custom Elasticsearch adapter, you can skip this one. Otherwise create querySuggestion mapping from command line using CURL (or with Kibana). If the command succeeds, you should get an "acknowledged" output from the script.

Please change the index name in the sample (liferay-20116) to correspond to your company id.

Please also note that this mapping change might fail if you have already suggestions in the index. The best way is to do this right after reindexing.

```
curl -XPUT 'localhost:9200/liferay-20116/_mapping/querySuggestion?pretty' -H 'Content-Type: application/json' -d'
{
	"dynamic_templates": [
	{
		"template_keywordSearch": {
			"mapping": {
				"type": "string",
				"fields": {
					"ngram": {
						"type": "string",
					        "search_analyzer": "standard",
						"analyzer": "gsearch_shingle_analyzer"
					},
					"suggest" : {
						"type" : "completion",
						"analyzer" : "simple",
						"search_analyzer" : "simple"
					}
				}
			},
			"match": "keywordSearch_*",
			"match_mapping_type": "string"
		}
	}
	],
	"properties": {
		"companyId": {
			"index": "not_analyzed",
			"store": "yes",
			"type": "string"
		},
		"groupId": {
			"index": "not_analyzed",
			"store": "yes",
			"type": "string"
		},
		"priority": {
			"index": "not_analyzed",
			"type": "float"
		},
		"uid": {
			"index": "not_analyzed",
			"store": "yes",
			"type": "string"
		}
	}
}'
```

## Step 5 - Reindex

If you were transitioning from embedded Elasticsearch server to standalone server, please reindex search indexes from Control panel -> Server Administration.

To be sure that index type mappings have been refreshed please aldo restart the portal and Elasticsearch server and you are ready to go.

# 7 Enabling Audience Targeting Support

Download the following jar from [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest) and deploy:

* fi.soveltia.liferay.gsearch.ct-VERSION.jar

After the module has been installed, please enable that in the portlet configuration ontrol Panel -> Configuration -> System Settings -> Other -> Gsearch Configuration.

With this feature you can boost relevancy for the contents falling into current user's user segments. You can adjust the boost factor in the configuration. Create test segments and contents having those segments and play with the boost to see, how it affects hits relevancy.

# 8 Embedding Search Field into a Theme
Download the following jar from [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest) and deploy:

* fi.soveltia.liferay.gsearch.mini.web-VERSION.jar

Please see the gsearch-test-theme templates folder for an example of how to embed that into a theme.
There's also a theme binary in the [latest folder](https://github.com/peerkar/liferay-gsearch/tree/master/latest) if you want to test it.

# 9 Sample Configurations

Please see the portlet configuration in Control Panel -> Configuration -> System Settings -> Other -> Gsearch Configuration.

Documentation about the configuration values is sparse at the moment but I'll try to improve it with time. If you want to know what all those fields are about, the best source of information is Elasticsearch documentation and code of this application.

If you are having problems with configuration (portlet stops working) it's usually because of malformed JSON. Please see Tomcat log first, if you're having problems.

### Suggester Sample Configuration

If you don't want to use the custom Elasticsearch adapter (loosing much of the suggester functionalities), please use the settings below:

```
[
	{
		"suggesterName": "phrase",
		"suggesterType": "phrase",
		"fieldPrefix": "keywordSearch_",
		"fieldSuffix": "",
		"isLocalized": true,
		"numberOfSuggestions":  5,
		"confidence": "0.1f",
		"gramSize": "2",
		"maxErrors": "2.0f",
		"realWordErrorLikelihood": "0.95f"
	}
]
```

If you want to use the custom Elasticsearch adapter, please use this one:

```
[
	{
		"suggesterName": "phrase",
		"suggesterType": "phrase",
		"fieldPrefix": "keywordSearch_",
		"fieldSuffix": ".ngram",
		"isLocalized": true,
		"numberOfSuggestions":  5,
		"confidence": "0.1f",
		"gramSize": "2",
		"maxErrors": "2.0f",
		"realWordErrorLikelihood": "0.95f"
	},
	{
		"suggesterName": "completion",
		"suggesterType": "completion",
		"fieldPrefix": "keywordSearch_",
		"fieldSuffix": ".suggest",
		"isLocalized": true,
		"numberOfSuggestions":  5
	}
]
```

### Search types Sample Configuration

This configuration defines the asset types to search for.

```
[
	{
		"key": "web-content",
		"entryClassName": "com.liferay.journal.model.JournalArticle"
	},
	{
		"key": "file",
		"entryClassName": "com.liferay.document.library.kernel.model.DLFileEntry"
	},
	{
		"key": "discussion",
		"entryClassName": "com.liferay.message.boards.kernel.model.MBMessage"
	},
	{
		"key": "blog",
		"entryClassName": "com.liferay.blogs.kernel.model.BlogsEntry"
	},
	{
		"key": "wikipage",
		"entryClassName": "com.liferay.wiki.model.WikiPage"
	},
	{
		"key": "non-liferay-type-example",
		"entryClassName": "non-liferay-type"
	}
]
```

### Facets Sample configuration

This configuration defines the available facets in the secondary filter menu. You can add there any indexed field.

```
[
	{
		"fieldName": "entryClassName",
		"icon": "icons/icon-ddm-structure.png"
	},
	{
		"fieldName": "ddmStructureKey",
		"icon": "icons/icon-ddm-structure.png"
	},
	{
		"fieldName": "fileEntryTypeId",
		"icon": "icons/icon-file-entry-type.png"
	},
	{
		"fieldName": "extension",
		"icon": "icons/icon-file-extension.png",
		"aggregations": [
			{
				"key": "Image",
				"values": "png,jpg,gif"
			},
			{
				"key": "MS Word",
				"values": "doc,docx"
			},
			{
				"key": "MS Excel",
				"values": "xls,xlsx"
			},
			{
				"key": "MS Powerpoint",
				"values": "ppt,pptx"
			},
			{
				"key": "Video",
				"values": "mp4,avi"
			},
			{
				"key": "Audio",
				"values": "mp3"
			}
		]
	},
	{
		"fieldName": "userName",
		"icon": "icons/icon-user.png"
	},
	{
		"fieldName": "assetCategoryTitles",
		"icon": "icons/icon-category.png"
	},
	{
		"fieldName": "assetTagNames",
		"icon": "icons/icon-tag.png"
	}
]
```

### Sortfields Sample Configuration

Sort fields. You can add there any indexed field. Translations need to be added to gsearch-core-api module localization file. Please see [Sort.java](https://github.com/liferay/liferay-portal/blob/master/portal-kernel/src/com/liferay/portal/kernel/search/Sort.java) for the available values for "fieldType".

```
[
	{
		"key": "score",
		"fieldType":  0,
		"default": true,
		"localized": false
	},
	{
		"key": "title",
		"fieldName": "title",
		"fieldPrefix": "localized_",
		"fieldSuffix": "_sortable",
		"fieldType":  3,
		"localized": true
	},
	{
		"key": "modified",
		"fieldName": "modified",
		"fieldSuffix": "_sortable",
		"fieldType":  6,
		"localized": false
	}
]
```

### Query Sample Configuration

This defines the main query. You can have there just a single query or construct it of many queries. The supported types ("queryType") at the moment are query_string, match and wildcard. Please see the gsearch-core-impl module QueryBuilders code for more information.

```
[
	{
		"queryType": "query_string",
		"occur": "must",
		"operator": "and",
		"fuzziness": "",
		"boost": 1,
		"fields": [
			{
				"fieldName": "title",
				"localized": true,
				"boost": 2,
				"boostForLocalizedVersion": 3
			},
			{
				"fieldName": "description",
				"localized": true,
				"boost": 1,
				"boostForLocalizedVersion": 1.5
			},
			{
				"fieldName": "content",
				"localized": true,
				"boost": 1,
				"boostForLocalizedVersion": 1.5
			}
		]
	},
	{
		"queryType": "wildcard",
		"occur": "should",
		"fieldName": "userName",
		"boost": "0.5",
		"keywordSplitter":  " ",
		"valuePrefix":  "*",
		"valueSuffix":  "*"
	}
]
```

# 10 Troubleshooting

## Querysuggester Not Working

This might happen at least of two reasons:

 1. You are not using the custom Elasticsearch adapter but are using the query suggester configuration for the custom adapter. Please check the configuration samples below in this document.
 2. querySuggestion type mapping has not been refreshed, probably because there was an existing one already. Please reindex and immediately after reindexing, run the CURL command from Step 5.

In any case, if you are having problems, please check both Liferay logs and Elasticsearch logs. Elasticsearch log would be by default ELASTICSEARCH_SERVER_PATH/logs/LiferayElasticsearchCluster.log

# 11 Important Notes

## Permissions
Search result permissions rely currently on three fields in the indexed document: roleId, groupRoleId and userName(current owner). Thes role fields contain content specific the roles that have access to the document. When you create a content these fields contain correctly any inherited role information. However, when you update role permissions to, for example, grant web content view access to a contents on a site, these fields won't update in the index.

This is how Liferay works at least currently. This issue will be revisited later but it's important to know about it.

# 12 FAQ

## This Portlet Doesn't Return the Same Results as the Standard Liferay Search Portlet?!

That's right. By default this portlet targets the search only to title, description and content fields (with localization support) and not for example username, tags, categories etc. which I thought, are generally better suitable for secondary facet filtering. For example, if I want to find documents where my name is mentioned in the content, I don't want to get all the documents where I'm a document author (username field). In the portlet configuration however, you can configure the target fields and their boost factors without any restrictions to your likings.

## Does This Work on CE?

It's not tested but basically there is one thing preventing this to work on CE: new version of Soy bridge. You can however remove this barrier fairly easy:

* Downgrade the Soy bridge dependency in the web module's build.gradle, rebuild  and see if it works. Other way around, you can also try to upgrade the bridge to the portal instance.

If you need help with creating a CE compatible version, please create a ticket.

## Do I Have to Use the Custom Search Adapter?
No you don't but then you loose the possibility to fine tune search field configuration and their boosts as well as a much better autocompletion / keyword suggester functionality.

## How Does the Suggester Work?
The suggester works by storing succesfull search keywords/phrases and offering them as autocompletion options. That's the way Google works, too.

Suggestions are bound to the portal instance (companyId) and to the UI language.

Obviously, after installing there are no stored queries, so you have to teach the suggester to make it offer opions. Have also a look at the configuration options.

Remember that search phrases are not persisted but are index time only. If you reindex, the options have to be rebuilt again.

## How Can I Make Non-Liferay Assets Findable in Index
To get this portlet to find "external" assets in the Liferay index you have to index those assets to that portal company's index, you want them to be findable at. It's possible to spread the search across multiple indexes but that would require some more customization of the search adapter.

There's a sample non Liferay result item builder (gsearch-core-impl) and here's how you can test it. Add a sample document into your index:

```
curl -XPOST 'localhost:9200/liferay-20116/LiferayDocumentType/?pretty' -H 'Content-Type: application/json' -d'
{
   "title": "Non Liferay Asset In Elasticsearch Index Hey",
   "description": "This is definitely not a Liferay Asset. Check it out by yourself.",
   "content":  "Still trying this out...",
   "modified":  "20171028095357",
   "entryClassName": "non-liferay-type",
   "treePath": "http://www.google.fi",
   "companyId": "20116",
   "stagingGroup": "false",
   "status": "0",
   "roleId": "20123",
   "groupRoleId": "20143-20131",
   "entryClassPK": "0"
}
'
```

Things to know about the fields:

* title, content, description: the fields which are targeted for search. There can also be localized versions of those.
* modified: modification date. See syntax from other Liferay documents
* entryClassName: this should match the type you configured in GSearch configuration
* treePath: this is just a random field existing in the type mapping we use for the link. That can be any other of the mapping, as long as you implement the getLink() method of the result item builder.
* companyId: your portal instance's id
* stagingGroup: this just tell's that this is a "live" content
* status: 0 = published
* roleId: roles allowed to view this content
* groupRoleId: group roles allowed to view the content
* entryClassPK: any long value. Doesn't really matter here.

## How Do I Connect a Search Field in a Theme to This Portlet?
Please see the gsearch-test-theme for an example how to embed a Miniportlet (with autocompletion) into a theme.

If you want to create your own form/field, just create a search form and make it to redirect (GET) to the page having the search portlet. You can find the parameter list in the source code but only required parameter is "q", having the keywords.

## Can I Use This With an Embedded Elasticsearch Server
If you don't want to use the custom Elasticsearch adapter, you can run this also with an embedded server. However not only the custom adapter brings a lot of extra value to this search portlet, but it makes debugging and tweaking easy or in some cases, even possible. Please remember that using embedded server is not supported option in a production environment anyways.

Installation instructions are [here](https://dev.liferay.com/discover/deployment/-/knowledge_base/7-0/installing-elasticsearch).

ElasticHQ is an excellent lightweight Elasticsearch plugin for managing and monitoring indexes and Elastic cluster. Installation instructions [here](http://www.elastichq.org/support_plugin.html). After installation point your browser (by default) to http://localhost:9200/_plugin/hq/

## There's Audience Targeting Integration Now. How Could I Add Something Like GeoDistance There?

See 'fi.soveltia.liferay.gsearch.core.impl.query.QueryBuilderImpl'. That's where the Audience Targeting condition gets added and the place where any other logic like that should be added. If you want to improve relevancy by a field, please remember, that it has to be in the query, not in the filter.

# 13 Project Roadmap
Upcoming:

 * Integration tests

# 14 Credits
Thanks to Tony Tawk for the Arabic translation!

# 15 Disclaimer
This portlet hasn't been thoroughly tested and is provided as is. You can freely develop it further to serve your own purposes. If you have good ideas and would like me to implement those, please leave ticket or ping me. Also many thanks in advance for any bug findings.

# 16 Changelog

## 2017-12-21

* Validated CE support.
* Added Minisearch portlet for embedding a searchfield into a theme.
* Added a test theme for Minisearch portlet.
* Changed searchfield configuration into a more flexible query configuration. Now the main query can be constructed of multiple queries.
* Bugs fixed:
 * Custom ES adapter adding fields twice into QueryStringQuery
 * Query suggester splitting suggestions into a comma separated list

## 2017-12-16

* Removed Audience Targeting requirement and put the functionality in its' own module.

## 2017-12-4

* Keywords suggester / autocomplete refactoring:
	* UI component changed from Metal.js to Devbridge Autocomplete
	* Suggester configuration changed to a JSON string
	* Switched to aggregate suggester by default now
	* Added configuration for the completion type suggest field
	* Added custom analyzers and filters for the query suggesters in the index-settings.json (see custom Elasticsearch Adapter project)
	* As index field mapping for title, description and content doesn't use asciifolding filter and doesn't recognize accent characters, modified analyzers for these fields to use asciifolding filter in liferay-type-mappings.json (see custom Elasticsearch Adapter project)

repository: gbovyn/liferay-gsearch

repo_name: gbovyn/liferay-gsearch


text:
> We are given a color picture consisting of an $m \times n$ array $A[1..m, 1..n]$ of pixels, where each pixel specifies a triple of red, green, and blue (RGB) intensities. Suppose that we wish to compress this picture slightly. Specifically, we wish to remove one pixel from each of the $m$ rows, so that the whole picture becomes one pixel narrower. To avoid disturbing visual effects, however, we require that the pixels removed in two adjacent rows be in the same or adjacent columns; the pixels removed form a "seam" from the top row to the bottom row where successive pixels in the seam are adjacent vertically or diagonally.
>
> **a.** Show that the number of such possible seams grows at least exponentially in $m$, assuming that $n > 1$.
>
> **b.** Suppose now that along with each pixel $A[i, j]$, we have calculated a real-valued disruption measure $d[i, j]$, indicating how disruptive it would be to remove pixel $A[i, j]$. Intuitively, the lower a pixel's disruption measure, the more similar the pixel is to its neighbors. Suppose further that we define the disruption measure of a seam to be the sum of the disruption measures of its pixels.
>
> Give an algorithm to find a seam with the lowest disruption measure. How efficient is your algorithm?

**a.** Let us set up a recurrence for the number of valid seams as a function of $m$. Suppose we are in the process of carving out a seam row by row, starting from the first row. Let the last pixel carved out be $A[i, j]$. How many choices do we have for the pixel in row $i + 1$ such that the pixel continues the seam? If the last pixel $A[i, j]$ were on the column boundary ($i = 1$ or $i = n$), then there would be two choices for the next pixel. For example, when $i = 1$, the two choices for the next pixel are $A[i + 1, j]$ and $A[i + 1, j + 1]$. Otherwise, there would be three choices for the next pixel: $A[i + 1, j - 1], A[i + 1, j], A[i + 1, j + 1]$. Thus, for a general pixel $A[i, j]$, there are at least two possible choices for a pixel $p$ in the next row such that $p$ continues a seam ending in $A[i, j]$. Let $T(i)$ denote the number of possible seams from row $1$ to row $i$. Then, for $i = 1$, we have $T(i) = n$, and for $i > 1$,

$$T(i) \ge 2T(i - 1).$$

It is easy to guess that $T(i) \ge n2^{n - 1}$, which we verify by direct substitution. For $i = 1$, we have $T(i) = n \ge n \cdot 2^0$. For $i > 1$, we have

$$
\begin{aligned}
T(i) & \ge 2T(i - 1) \\\\
     & \ge 2 \cdot n 2^{i - 1} \\\\
     & =   n 2^{i - 1}.
\end{aligned}
$$

Thus, the total number $T(m)$ of seams is at least $n2^{m - 1}$. We conclude that the number of seams grows at least exponentially in $m$.

**b.** As proved in the previous part, it is infeasible to systematically check every seam, since the number of possible seams grows exponentially.

The structure of the problem allows us to build the solution row by row. Consider a pixel $A[i, j]$. We ask the question: 'If $i$ were the first row of the picture, what is the minimum disruptive measure of seams that start with the pixel $A[i, j]$?'

Let $S^\*$ be a seam of minimum disruptive measure among all seams that start with pixel $A[i, j]$. Let $A[i + 1, p]$, where $p \in \\{j - 1, j, j + 1\\}$, be the pixel of $S^\*$ in the next row. Let $S'$ be the sub-seam of $S^\*$ that starts with $A[i + 1, p]$. We claim that $S'$ has the minimum disruptive measure among seams that start with $A[i + 1, p]$. Why? Suppose there exists another seam $S''$ that starts with $A[i + 1, p]$ and has disruptive measure less than that of $S'$. By using $S''$ as the sub-seam instead of $S'$, we can obtain another seam that starts with $A[i, j]$ and has a disruptive measure which is less than that of $S^\*$. Thus, we obtain a contradiction to our assumption that $S^\*$ is a seam of minimum disruptive measure.

Let $disr[i, j]$ be the value of the minimum disruptive measure among all seams that start with pixel $A[i, j]$. For row $m$, the seam with the minimum disruptive measure consists of just one point. We can now state a recurrence for $disr[i, j]$ as follows. In the base case, $disr[m, j] = d[m, j]$ for $j = 1, 2, \ldots, n$. In the recursive case, for $j = 1, 2, \ldots, n$,

$$disr[i, j] = d[i, j] + \min_{k \in K}{dist[i + i, j + k]},$$

where the set $K$ of index offsets is

$$
K =
\begin{cases}
{0, 1}     & \text{if $j = 1$}, \\\\
{-1, 0, 1} & \text{if $1 < j < m$}, \\\\
{-1, 0}    & \text{if $j = n$}.
\end{cases}
$$

Since every seam has to start with a pixel of the first row, we simply find the minimum $disr[1, j]$ for pixels in the first row to obtain the minimum disruptive measure.

```cpp
COMPRESS-IMAGE(d)
    m = d.rows
    n = d.columns
    let disr[1..m, 1..n] and next[1..m, 1..n] be new tables
    for j = 1 to n
        disr[m, j] = d[m, j]
    for i = m - 1 downto 1
        for j = 1 to n
            low = max(-1, 1 - j)
            high = min(1, n - j)
            disr[i, j] = ∞
            for k = low to high
                if disr[i + 1, j + k] < disr[i, j]
                    disr[i, j] = disr[i + 1, j + k]
                    next[i, j] = j + k
            disr[i, j] = disr[i, j] + d[i, j]
    val = ∞
    start = 1
    for j = 1 to n
        if disr[1, j] < val
            val = disr[1, j]
            start = j
    print "The minimum value of the disruptive measure is" val
    for i = 1 to m
        print "cup point at" (i, start)
        start = next[i, start]
```

The procedure $\text{COMPRESS-IMAGE}$ is simply an implementation of this recurrence in a bottom-up fashion.

We first carry out the initialization of the base cases, which are the cases when row $i = m$. The minimum disruptive measure for the base cases is simply $d[m, j]$.

The next **for** loop runs down from $m - 1$ to $1$. Thus, $disr[i + 1, j]$ is already available before computing $dist[i, j]$ for pixels of row $i$.

The assignments to $low$ and $high$ allow the index offset $k$ to range over the correct set $K$ from above. We set $low$ to $0$ when $j = 1$ and to $-1$ when $j > 1$, and we set $high$ to $0$ when $j = n$ and to $1$ when $j < n$. The innermost **for** loop sets $dist[i, j]$ to the minimum value of $disr[i + 1, j + k]$ for all $k \in K$, and the line that follows this loop adds in $d[i, j]$.

We use the $next$ table to reconstruct the actual seam. For a given pixel, it records which pixel was used as the next pixel. Specifically, for a pixel $A[i, j]$, if $next[i, j] = p$, where $p \in {j - 1, j, j + 1}$, then the next pixel of the seam is $A[i + 1, p]$.

The last line of the **for** loop adds the disruptive measure of the current pixel to the disruptive measure of the seam.

The next **for** loop finds the minimum disruptive measure of pixels in the first row. We print the minimum disruptive measure as the answer.

The rest of the code reconstructs the actual seam, using the information stored in the $next$ array.

Noting that the innermost **for** loop runs over at most three values of $k$, we see that the running time of $\text{COMPRESS-IMAGE}$ is $O(mn)$. The space requirement is also $O(mn)$. We can improve upon the space requirement by observing that row $i$ of the $disr$ table depends on only row $i + 1$. Therefore, we can store just two rows at any time. Thus, we can improve the space requirement of $\text{COMPRESS-IMAGE}$ to $O(n)$.

license: MIT

branch_name: refs/heads/master

extension: md

gha_license_id: MIT

license_type: permissive


text:
# BigIntJs

This recipe shows how to print, create, and use values of the `BigIntJs` type in either the node.js or web browser console.

## Expected Behavior:

- Shows the difference between using `show` and `toString`
- Shows how to create values of this type
- Shows a few operations one can use

If viewing the web version, open the browser's console with dev tools first, then reload/refresh the page.

## Dependencies used

[big-integer](https://www.npmjs.com/package/big-integer)

repository: Ebmtranceboy/purescript-cookbook

repo_name: Ebmtranceboy/purescript-cookbook


text:
---
layout: default
---

# 1995-07-14 - Re: Root Causes

## Header Data

From<EMAIL_ADDRESS>(Lucky Green)

To<EMAIL_ADDRESS>Message Hash: be307884c07bc63f3e0f0133b42ca9d9ac14111bca86ba31cfa7fcb24e20f335

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1995-07-14 02:27:17 UTC

Raw Date: Thu, 13 Jul 95 19:27:17 PDT

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Lucky Green)
Date: Thu, 13 Jul 95 19:27:17 PDT
To<EMAIL_ADDRESS>Subject: Re: Root Causes
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

-----BEGIN PGP SIGNED MESSAGE-----

In article<EMAIL_ADDRESS><EMAIL_ADDRESS>(David
K. Merriman) wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>
>While I respect the ideas and opinions submitted by the majority of the
>members of this list, I wonder if perhaps we're failing to deal with the
>_root_ problem of such things as the CDA, Clipper, DTA, etc.
>
>Specifically, I wonder if it wouldn't be a better approach to *prevent* such
>measures from ever being proposed in the first place.

Short of a 50 kilo ton bomb on Washington, I don't see any way that could
be accomplished.
<No, I don't advocate nuking DC.>

>Is there any precedence or possibility of either filing civil or criminal
>charges against a Government official for their _official_ actions?
>Something that will not only make for some Serious Press, but hit them from
>an unexpected angle?

You can't sue the government without its prior consent. Government
officials are also usually immune from being sued over their official
actions.

>It would seem that things such as the CDA, etc, are patent violations of the
>Bill of Rights. As such, wouldn't the Congressrodent(s) proposing such
>measures be violating our civil rights, and thus be criminally liable?
>Aren't Congressrodents supposed to take an Oath of Office that involves
>upholding the Constitution?

The oath is not ment to be kept. It's sole purpose is to provide a photo
op for the incomming congresscritters.

- --
- -- Lucky Green<EMAIL_ADDRESS>   PGP encrypted mail preferred.
- ---
[This message has been signed by an auto-signing service.  A valid signature
means only that it has been received at the address corresponding to the
signature and forwarded.]

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Comment: Gratis auto-signing service

iQBFAwUBMAXVPioZzwIn1bdtAQFp5gF/WnEoNO15G11gXi9G/BmtFzu/toHZPBmj
ldONnU+mbB5c9LIGeJH3usQZLdT/D4Sw
=NpN9
-----END PGP SIGNATURE-----

{% endraw %}
```

## Thread

+ Return to [July 1995](/archive/1995/07)

+ Return to<EMAIL_ADDRESS>(Lucky Green)](/authors/shamrock_at_netcom_com_lucky_green_)"

+ 1995-07-14 (Thu, 13 Jul 95 19:27:17 PDT) - Re: Root Causes -<EMAIL_ADDRESS>(Lucky Green)_


text:
hls-downloader
==============

Download all files as is from HLS (HTTP Live Streaming) VoD (Video on Demand) playlist m3u8 from <https://developer.apple.com/streaming/examples/>

![Python application](https://github.com/denex/hls-downloader/workflows/Python%20application/badge.svg)

Python 2 suport
---------------

Branch python2 should work in Python 2

Requirements:

* m3u8
* python-slugify
* requests

Usage:

```sh
python main.py 'http://some.m3u8' /some/dir/for/files/
```

Known Issues:

* No encryption/decryption supported yet

repository: lr-zhao/hls-downloader

repo_name: lr-zhao/hls-downloader


text:
---
layout:     project
title:      Steganography
code:       SRC_Steganography.zip
doc:        DOC_Steganography.pdf
demo:
best:       true
---

It’s a text based menu-driven application written in C and it can hide a text file in a Bitmap image and extract it back from the image. After researching and reading some papers and articles, our group planned to implement a method called “[Adaptive Data Hiding in Edge Areas of Images With Spatial LSB Domain Systems](http://dx.doi.org/10.1109/TIFS.2008.926097)”. Our program can hide up to 15 bits in each pixel of the image. I personally really enjoyed writing this project and I learned a lot during working on it. This was the project of my second programming course at the University and was marked as the best project both by the instructor and the students.


text:
Elasticsearch Aggregation Path Hierarchy Plugin
=========================================

This plugin adds the possibility to create hierarchical aggregations.
Each term is split on a provided separator (default "/") then aggregated by level.
For a complete example see https://github.com/elastic/elasticsearch/issues/8896

Two different aggregations are available:
 - `path_hierarchy` for hierarchical aggregations on `keywords` field or `scripts`
 - `date_hierarchy` for hierachical aggregations on `date` fields. It is more optimized to use this aggregation for date instead of a script.

This is a multi bucket aggregation.

Installation
------------

`bin/plugin --install path_hierarchy --url "https://github.com/opendatasoft/elasticsearch-aggregation-pathhierarchy/releases/download/v<IP_ADDRESS>/pathhierarchy-aggregation-<IP_ADDRESS>.zip"`

Build
-----

Development Environment Setup
------------

Build the plugin using gradle:

`./gradlew build`

OR

`./gradlew assemble` (to avoid tests)

Then the following command will start a dockerized ES and will install the previously built plugin:

`
docker-compose up
`

> NOTE: In `docker-compose.yml` you can uncomment the debug env and attach a REMOTE JVM on `*:5005` to debug the plugin.

Path hierarchy aggregation
--------------------------

### Parameters

 - `field` or `script` : field to aggregate on
 - `separator` : separator for path hierarchy (default to "/")
 - `order` : order parameter to define how to sort result. Allowed parameters are `_key`, `_count` or sub aggregation name. Default to {"_count": "desc}.
 - `size`: size parameter to define how many buckets should be returned. Default to 10.
 - `shard_size`: how many buckets returned by each shards. Set to size if smaller, default to size if the search request needs to go to a single shard, and (size * 1.5 + 10) otherwise (more information here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_shard_size_3).
 - `min_depth`: Set minimum depth level. Default to 0.
 - `max_depth`: Set maximum depth level. `-1` means no limit. Default to 3.
 - `depth`: Retrieve values for specified depth. Shortcut, instead of setting `min_depth` and `max_depth` parameters to the same value.
 - `keep_blank_path`: Keep blank path as bucket. if this option is set to false, chained separator will be ignored. Default to false.
 - `min_doc_count`: Return buckets containing at least `min_doc_count` document. Default to 0

Examples
-------

#### String field

```
# Add data:
PUT filesystem
{
  "mappings": {
    "properties": {
      "path": {
        "type": "keyword"
      }
    }
  }
}
POST filesystem/_bulk?refresh
{"index":{}}
{"path":"/My documents/Spreadsheets/Budget_2013.xls","views":10}
{"index":{}}
{"path":"/My documents/Spreadsheets/Budget_2014.xls","views":7}
{"index":{}}
{"path":"/My documents/Test.txt","views":1}

# Path hierarchy request :
GET /filesystem/_search?size=0
{
  "aggs": {
    "tree": {
      "path_hierarchy": {
        "field": "path",
        "separator": "/"
      },
      "aggs": {
        "total_views": {
          "sum": {
            "field": "views"
          }
        }
      }
    }
  }
}

Result :
{"aggregations": {
   "tree": {
     "sum_other_doc_count": 0,
     "buckets": [
       {
         "key": "My documents",
         "doc_count": 3,
         "total_views": {
           "value": 18
         },
         "tree": {
           "buckets": [
             {
               "key": "Spreadsheets",
               "doc_count": 2,
               "total_views": {
                 "value": 17
               },
               "tree": {
                 "buckets": [
                   {
                     "key": "Budget_2013.xls",
                     "doc_count": 1,
                     "total_views": {
                       "value": 10
                     }
                   },
                   {
                     "key": "Budget_2014.xls",
                     "doc_count": 1,
                     "total_views": {
                       "value": 7
                     }
                   }
                 ]
               }
             },
             {
               "key": "Test.txt",
               "doc_count": 1,
               "total_views": {
                 "value": 1
               }
             }
           ]
         }
       }
     ]
   }
}
```

#### Script

```
PUT calendar
{
  "mappings": {
    "properties": {
      "date": {
        "type": "date"
      }
    }
  }
}

POST calendar/_bulk?refresh
{"index":{}}
{"date": "2012-01-10T02:47:28"}
{"index":{}}
{"date": "2012-01-05T01:43:35"}
{"index":{}}
{"date": "2012-05-01T12:24:19"}

GET /calendar/_search?size=0
{
  "aggs": {
    "tree": {
      "path_hierarchy": {
        "script": "doc['date'].value.toOffsetDateTime().format(DateTimeFormatter.ofPattern('yyyy/MM/dd'))",
        "order": {
          "_key": "asc"
        }
      }
    }
  }
}

Result :

{"aggregations": {
    "tree": {
      "buckets": [
        {
          "key": "2012",
          "doc_count": 3,
          "tree": {
            "buckets": [
              {
                "key": "01",
                "doc_count": 2,
                "tree": {
                  "buckets": [
                    {
                      "key": "05",
                      "doc_count": 1
                    },
                    {
                      "key": "10",
                      "doc_count": 1
                    }
                  ]
                }
              },
              {
                "key": "05",
                "doc_count": 1,
                "tree": {
                  "buckets": [
                    {
                      "key": "01",
                      "doc_count": 1
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

```

Date hierarchy
--------------

### Parameters

 - `field` : field to aggregate on. This parameter is mandatory
 - `interval`: date interval used to create the hierarchy. Allowed values are: `years`, `months`, `days`, `hours`, `minutes`, `seconds` Default to `years`.
 - `order` : order parameter to define how to sort result. Allowed parameters are `_key`, `_count` or sub aggregation name. Default to {"_count": "desc}.
 - `size`: size parameter to define how many buckets should be returned. Default to 10.
 - `shard_size`: how many buckets returned by each shards. Set to size if smaller, default to size if the search request needs to go to a single shard, and (size * 1.5 + 10) otherwise (more information here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_shard_size_3).
 - `min_doc_count`: Return buckets containing at least `min_doc_count` document. Default to 0

Example
-------

```

PUT calendar
{
  "mappings": {
    "properties": {
      "date": {
        "type": "date"
      }
    }
  }
}

PUT /calendar/_doc/1
{
  "date": "2012-01-10T02:47:28"
}
PUT /calendar/_doc/2
{
  "date": "2012-01-05T01:43:35"
}
PUT /calendar/_doc/3
{
  "date": "2012-05-01T12:24:19"
}

GET /calendar/_search?size=0
{
  "aggs": {
    "tree": {
      "date_hierarchy": {
        "field": "date",
        "interval": "days",
        "order": {
          "_key": "asc"
        }
      }
    }
  }
}

```

Installation
------------

Built with Java 17.

The first 3 digits of plugin version is Elasticsearch versioning. The last digit is used for plugin versioning under an elasticsearch version.

To install it, launch this command in Elasticsearch directory replacing the url with a release suiting your case (please check available releases [here](https://github.com/opendatasoft/elasticsearch-aggregation-pathhierarchy/releases)):
`./bin/elasticsearch-plugin install https://github.com/opendatasoft/elasticsearch-aggregation-pathhierarchy/releases/download/v<IP_ADDRESS>/pathhierarchy-aggregation-<IP_ADDRESS>.zip`

License
-------

This software is under The MIT License (MIT).

gha_language: Java


text:
# ROOT running with noVNC Display Container

Based on https://github.com/theasp/docker-novnc

## Image Contents

* [Xvfb](http://www.x.org/releases/X11R7.6/doc/man/man1/Xvfb.1.xhtml) - X11 in a virtual framebuffer
* [x11vnc](http://www.karlrunge.com/x11vnc/) - A VNC server that scrapes the above X11 server
* [noNVC](https://kanaka.github.io/noVNC/) - A HTML5 canvas vnc viewer
* [Fluxbox](http://www.fluxbox.org/) - a small window manager
* [socat](http://www.dest-unreach.org/socat/) - for use with other containers
* [xterm](http://invisible-island.net/xterm/) - to demo that it works
* [supervisord](http://supervisord.org) - to keep it all running

## Usage

### Variables

You can specify the following variables:
* `DISPLAY_WIDTH=<width>` (1024)
* `DISPLAY_HEIGHT=<height>` (768)
* `RUN_XTERM={yes|no}` (yes)
* `RUN_FLUXBOX={yes|no}` (yes)

### Stand-alone Demo
Run:
```bash
$ docker run --rm -it -p 8080:8080 root-docker-novnc
```
Open a browser and see the `xterm` demo at `http://<server>:8080/vnc.html`

# Thanks
___
This is based on the alpine container by @psharkey: https://github.com/psharkey/docker/tree/master/novnc
Based on [wine-x11-novnc-docker](https://github.com/solarkennedy/wine-x11-novnc-docker) and [octave-x11-novnc-docker](https://hub.docker.com/r/epflsti/octave-x11-novnc-docker/).


text:
# Attacking SQuAD Models

## Files and Usage
This folder contains the following files:
+ `squad.py` Creates the triggers using the optimization procedure described in the paper.
+ `universal_adversarial_sentences.txt` contains the adversarial sentences used in the paper. You can copy and paste these into the variable `adv_token_idxs` in `squad.py` to test out different attacks.

## Models

We have pre-trained models available for use (see `squad.py` for how to use a model archive):
+ `bidaf` as described in the paper using GloVe word embeddings. The model archive is available here https://s3-us-west-2.amazonaws.com/allennlp/models/bidaf-glove-2019.05.09.tar.gz

repository: Eric-Wallace/universal-triggers

gha_language: Python

repo_name: Eric-Wallace/universal-triggers


text:
# This folder contains the examples code of [Tiny-Prolog-in-OCaml-OneFile](https://github.com/Naereen/Tiny-Prolog-in-OCaml-OneFile)
> A tiny implementation of a small subset of the Prolog language, in OCaml. With small and fun examples.
>
> WARNING: this project only has an **educational purpose**, for a real-world use of Prolog, please refer to [GNU Prolog (gprolog)](XXX).

## List of examples
### Basic examples
- [`pair.pl`](pair.pl) defines a predicate `pair` on integer, using `o` as 0 and `s(X)` for `X+1`.
- [`impair.pl`](impair.pl) defines a predicate `pair` on integer, using `o` as 0 and `s(X)` for `X+1`.
- [`lapin.pl`](lapin.pl) is a fun small example on a family of bunnies.
- [`tomandjerry.pl`](tomandjerry.pl) is another fun small example on Tom and Jerry.

### Other examples
- [`famille.pl`](famille.pl) and [`genealogie.pl`](genealogie.pl) are two implementations of a same examples, the first one is as detailed as possible, and the other one is as concise as possible.
- [`nat.pl`](nat.pl) implements basic arithmetic operations on integers.
- [`domino.pl`](domino.pl) is the solution of a harder problem.

## How to run them?
- First, [build `prolog`](../prolog/)
- Then:

```bash
cd exemples
../prolog/prolog ./pair.pl "pair(o)."  # load a theory and ask a question
```

---

### :scroll: License ? [![GitHub license](https://img.shields.io/github/license/Naereen/Tiny-Prolog-in-OCaml-OneFile.svg)](https://github.com/Naereen/Tiny-Prolog-in-OCaml-OneFile/blob/master/LICENSE)
This (small) repository is published under the terms of the [MIT license](http://lbesson.mit-license.org/) (file [LICENSE](LICENSE)).
© [Lilian Besson](https://GitHub.com/Naereen), 2018.

[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Naereen/Tiny-Prolog-in-OCaml-OneFile/graphs/commit-activity)
[![Ask Me Anything !](https://img.shields.io/badge/Ask%20me-anything-1abc9c.svg)](https://GitHub.com/Naereen/Tiny-Prolog-in-OCaml-OneFile)
[![Analytics](https://ga-beacon.appspot.com/UA-38514290-17/github.com/Naereen/Tiny-Prolog-in-OCaml-OneFile/README.md?pixel)](https://GitHub.com/Naereen/Tiny-Prolog-in-OCaml-OneFile/)

[![made-with-OCaml](https://img.shields.io/badge/Made%20with-OCaml-1f425f.svg)](https://ocaml.org/)
[![made-for-teaching](https://img.shields.io/badge/Made%20for-Teaching-6800ff.svg)](https://perso.crans.org/besson/teach/)

[![ForTheBadge built-with-science](http://ForTheBadge.com/images/badges/built-with-science.svg)](https://GitHub.com/Naereen/)
[![ForTheBadge uses-badges](http://ForTheBadge.com/images/badges/uses-badges.svg)](http://ForTheBadge.com)
[![ForTheBadge uses-git](http://ForTheBadge.com/images/badges/uses-git.svg)](https://GitHub.com/)

repository: Naereen/Tiny-Prolog-in-OCaml-OneFile

repo_name: Naereen/Tiny-Prolog-in-OCaml-OneFile


text:
# The Complete Guide to SAS Event Stream Processing 6.2 with Kubernetes

The complete guide to SAS Event Stream Processing with Kubernetes provides all the necessary information and guidelines with commands to set up your own Kubernetes cluster and its peripherals. It helps the user to start the SAS ESP Operator, SAS ESP Studio, SAS ESP Streamviewer and SAS Event Stream Manager in the user-defined namespace.

## Table of Contents
* [Introduction to Container Architecture](architecture/ContainerArchitecture.md)
* [Kubernetes Architecture](architecture/KubernetesArchitecture.md)
* [Kubernetes Cluster Environment Setup](cluster_setup/KubernetesClusterEnvironmentSetup.md)
* [End-to-End ESP Kubernetes (ESP-Kube) Architecture](architecture/EndToEndESPKubeArchitecture.md)
* [ESP Operator](operator/ESPOperator.md)
    * [Demo Examples to use ESP Operator](operator/DemoExamples.md)
* [Single-User Clients](single_user_clients/SingleUserClients.md)
    * [Demo Examples for cloud-native deployments using web-clients](single_user_clients/DemoExamplesClients.md)

## Overview

SAS Event Stream Processing 6.2 with Kubernetes is a framework for controlled and automatic deployment, management, and scaling of SAS Event Stream Processing Servers running in Docker containers across the Kubernetes cluster in a cloud environment. It provides high guarantees on availability, reliability and fault tolerance. The framework leverages features of Kubernetes for resource allocation, storage orchestration, self-monitoring and healing, and secret and configuration management.

The framework comprises of SAS Event Stream Processing Operator which allows starting, stopping, updating, deleting, and monitoring of the ESP Servers docker containers running in the Kubernetes cluster.

SAS Web Clients (SAS Event Stream Processing Studio, SAS Event Stream Processing Streamviewer and SAS Event Stream Processing Manager) also runs in the docker containers in the same cluster and allows seamless access to all the ESP Servers.

This paper provides a thorough deep dive into the SAS Event Stream Processing 6.2 Kubernetes Architecture and guidelines for users to set up their own ESP Operator framework with web clients in a Kubernetes cluster.

### Installation
Follow the table of contents in order for step-by-step instructions to install, configure and use SAS ESP 6.2 Kubernetes Framework.

## Contact Information
Your comments and suggestions are valuable and most welcome. Contact the author at:

Divya Gupta<EMAIL_ADDRESS>
Solutions Architect, IoT Global Division

## Contributing

We welcome your contributions! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to submit contributions to this project.

## License

This project is licensed under the [Apache 2.0 License](LICENSE).

## Additional Resources

* R&D Projects on [ESP Kubernetes](https://github.com/sassoftware/esp-kubernetes)
* [SAS Event Stream Processing](https://www.sas.com/en_us/software/event-stream-processing.html)
* Documentation for [SAS Event Stream Processing Kubernetes](https://go.documentation.sas.com/?cdcId=espcdc&cdcVersion=6.2&docsetId=espex&docsetTarget=n1nbfkss07x269n1in6yfnpju3i9.htm&locale=fr)
* Documentation for [SAS Event Stream Processing Studio](https://go.documentation.sas.com/?cdcId=espcdc&cdcVersion=6.2&docsetId=espstudio&docsetTarget=titlepage.htm&locale=en)
* Documentation for [SAS Event Stream Manager](https://go.documentation.sas.com/?cdcId=espcdc&cdcVersion=6.2&docsetId=esmuse&docsetTarget=titlepage.htm&locale=en)
* Documentation for [SAS Event Stream Processing Streamviewer](https://go.documentation.sas.com/?cdcId=espcdc&cdcVersion=6.2&docsetId=espvisualize&docsetTarget=titlepage.htm&locale=en)
* [SAS Support Communities](https://communities.sas.com/)


text:
# Contributing to Baleen

Thank you, for wanting to contribute and improve the world of data validation.

## How Can I Contribute?

If you have a change you would like to make, open a pull request and one of the maintainers will review it.

### Have an idea of a neat feature

Create a new issue with the label of "enhancement"

### Find a bug

Create an issue with the label bug.  Please include details on how to recreate the issue and the impact of the issue.

repository: ShopRunner/baleen

gha_language: Kotlin

repo_name: ShopRunner/baleen


text:
# @treblefm/eslint-config

Shared ESLint config for Treble.fm

## Quick Install

(Copy/paste into your terminal)
```sh
npm install --save-dev \
  @treblefm/eslint-config \
  eslint@4 \
<EMAIL_ADDRESS>\
  eslint-import-resolver-typescript@1 \
  eslint-plugin-import@2 \
  eslint-plugin-node@5
```

## Dependencies

This config requires the following peer dependencies:
-<EMAIL_ADDRESS>(obviously)
-<EMAIL_ADDRESS>1
-<EMAIL_ADDRESS>1
-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>
Additionally, these optional peer dependencies will automatically be used if installed:
-<EMAIL_ADDRESS>2
-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>3
-<EMAIL_ADDRESS>4
-<EMAIL_ADDRESS>4
-<EMAIL_ADDRESS>4

1: Required even if you aren't using React Native or TypeScript
2: See additional GraphQL notes below

3: Required by the React Native config

4: Required by the TypeScript config

-------------------

## Usage

By default, this config assumes a Node.js project with ES2017 syntax (though unsupported features will be reported as errors by `eslint-plugin-node`):
```js
{
  "extends": "@treblefm" // or @treblefm/eslint-config/node
}
```

Browser (ES6):
```js
{
  "extends": "@treblefm/eslint-config/browser"
}
```

React Native (includes JSX support):
```js
{
  "extends": "@treblefm/eslint-config/react-native"
}
```

-------------------

JSX support can be added to any of the above configs:
```js
{
  "extends": [
    "@treblefm",
    "@treblefm/eslint-config/jsx"
  ]
}
```

Similarly, TypeScript support can also be added (includes JSX support):
```js
{
  "extends": [
    "@treblefm",
    "@treblefm/eslint-config/typescript"
  ]
}
```

-------------------

## Note about GraphQL

While GraphQL is supported, the rules are disabled by default and some configuration is required ([click here for more info](https://github.com/apollographql/eslint-plugin-graphql)).

[`.graphqlconfig`](https://github.com/graphcool/graphql-config)
```js
{
  "schemaPath": "./path/to/schema.json"
}
```

`.eslintrc`
```js
{
  "extends": "@treblefm",
  "rules": {
    "graphql/template-strings": [2, {
      "env": "apollo",
      // "validators": [/* see GraphQL's `specifiedRules` */]
    }],
    "graphql/named-operations": [2, {
      "env": "apollo"
    }],
    "graphql/required-fields": [2, {
      "env": "apollo",
      "requiredFields": ["id"]
    }],
    "graphql/capitalized-type-name": [2, {
      "env": "apollo"
    }]
  }
}
```

repository: TrebleFM/eslint-config

repo_name: TrebleFM/eslint-config


text:
# Team Profile Generator
![](https://img.shields.io/badge/HTML5-E34F26?style=for-the-badge&logo=html5&logoColor=white)
![](https://img.shields.io/badge/Tailwind_CSS-38B2AC?style=for-the-badge&logo=tailwind-css&logoColor=white)
![](https://img.shields.io/badge/Node.js-43853D?style=for-the-badge&logo=node.js&logoColor=white)
![](https://img.shields.io/badge/JavaScript-323330?style=for-the-badge&logo=javascript&logoColor=F7DF1E)
## Description
An app that allows you to create an HTML page based on user input in the integrated terminal. The team consists of a mandatory manager, as well as any number of engineers and interns.
## Table of Contents
- [Installation](#installation)
- [Usage](#usage)
- [License](#license)
- [Contributions](#contributions)
- [Test](#test)
- [Questions](#questions)

## Installation
Open the integrated terminal on the index.js file

![Integrated Terminal](./images/openIntegrated.png)

Type ```npm i``` in the integrated terminal(if all the files do not install, attempt it again)

![npm I](./images/npmI.png)

## Usage
Run the application in the integrated terminal by typing "node index.js".

![node](./images/nodeIndex.png)

Fill out the manager and once prompted with a choice to add another member; either choose an Engineer, Intern or finish your team by selecting None.

Example inputing data:

![Instuction GIF](https://user-images.githubusercontent.com/85041715/129671323-e03265cc-3eb2-4e28-bb0d-273b990722ba.gif)

Example of the page created with the application:

![ss](./images/pageSS.png)

Link to deployed Example:
https://aabazary.github.io/team-profile-generator/
## License
This project is covered under MIT License
## Contributions
Feel free to fork and use at will!
## Tests
Type `npm test` in the integrated terminal for a list of tests

![npm Test](./images/npmTest.png)

![npm TestGIF](https://user-images.githubusercontent.com/85041715/129673367-7d623aa2-9595-4d24-ab18-7b65103d5389.gif)

## Questions
Contact the creator of Team Profile Generator at<EMAIL_ADDRESS>Github link: https://github.com/aabazary

repository: aabazary/team-profile-generator

branch_name: refs/heads/main

repo_name: aabazary/team-profile-generator


text:
---
id: 310
title: Broken Apple App Store Discovery
date: 2014-12-20T18:53:13+00:00
author: dmarges
layout: post
guid: http://www.donmarges.io/?p=310
permalink: /broken-apple-app-store-discovery/
categories:
  - Mobile Apps
tags:
  - apple
  - apple app store
  - apple app store discovery
---
So this is something I’ve been ranting about for a while and I’m glad a lot of other people are too.

Many times I have been looking for an App that I heard about from a friend or saw online. I would go to the App Store and search for the app I wanted to download only to not see it in the search results. Let me restate that:

I COULDN’T FIND AN APP EVEN THOUGH I TYPED THE EXACT NAME OF THE APP!!!

This is incredibly frustrating as an App Developer to have your work virtually unsearchable in the store!

In this article, it looks like the author had similar problems:

How Broken is Discovery on the App Store? This Broken.

I really hope Apple cleans this up as this just further stacks the odds against us developers!


text:
---
title: Creating a Plugin Project
redirect_from:
  - /basics/getting_started/creating_an_action.html
---

> **NOTE** For new projects, it is highly recommended to use [Gradle](/tutorials/build_system.md).

This section explains how you can create a new plugin project from scratch using the New Project wizard.  Optionally, you can import an existing project or import a project from external models. You can also add a new plugin module to a current *IntelliJ Platform* project.
For more information, refer to the [IntelliJ IDEA Web Help](https://www.jetbrains.com/idea/help/new-project-wizard.html).

### To Create an IntelliJ Platform Plugin Project:

*  On the main menu, choose **File \| New \| Project**. The *New Project*
   wizard starts.

   ![New Project Wizard](img/new_project_wizard.png)

*  Set *IntelliJ Platform Plugin* project type.
*  Click **Next**.
*  Set the desired project name.
*  Click **Finish** to generate project structure files.
*  Go to **File \| Project Structure** to customize project settings if required.

### To Create an IntelliJ Platform Plugin Module
*  Select **File \| New \| Module** and choose the *IntelliJ Platform Plugin* module type

   ![IntelliJ Platform Plugin Module](img/intellij_platform_plugin_module.png)

*  Enter your desired plugin name.

*  Go to **File \| Project Structure** and select the newly created *IntelliJ Platform SDK* as the default SDK for the plugin module:

   ![Set Plugin Module SDK](img/set_plugin_module_sdk.png)

### Adding Code to the Project
Before running the new project, add some code to provide simple functionality.
See the [Creating Actions](/tutorials/action_system/working_with_custom_actions.md) tutorial for step-by-step instructions for adding a menu action.


text:
# burger-project
Aplicação onde o usuario consiga dinamicamente montar um hamburguer. Projeto criado no curso
[React - The Complete Guide (incl Hooks, React Router, Redux)](https://www.udemy.com/react-the-complete-guide-incl-redux/)

## Getting Started

### Prerequisites

You'll need to have a basic environment with Reactjs and Create React app

**Installing dependencies**
```
$ npm install
```

## Built With
- ReactJs
- WebPack
- Redux

repository: LorenaKauane/burger-project

repo_name: LorenaKauane/burger-project


text:
This is the current, previous and future development milestones and contains the features backlog.

0.1
===
* initial README.md describing goals
* command-line tool
* basic options all supported
* create repo on GitHub
* publish as a gem on RubyGems

0.2
===
* switch to mixlib-cli
* --dryrun The option `--dryrun` will print the commands to run, but not actually execute them (the default behavior)
* --delete The option `--delete` will delete each cookbook, role, data bag, environment and node described in the yml file. All nodes from the system are deleted with `knife node bulk_delete`.
* --rebuild The option `--rebuild` will remove all currently managed infrastructure for this chef repository and rebuild it from scratch.

0.3
===
* renamed MILESTONES.md to CHANGELOG.md
* fixed version number
* updated YAML schema and examples because Ruby 1.8 does not order hashes.
* validate that the recipes and roles listed in the nodes are loaded

0.4
===
* support versions for cookbooks
* support site vendor for cookbooks

0.5
===
* support JSON and YAML

0.6
===
* add support for cookbook options

0.7
=============
* add support for environments
* rescue from parser errors
* update cookbook download syntax
* multiple nodes with same runlists syntax
* add support for encrypted data bags
* wildcard support for data bag items

0.7.1 *CURRENT*
===============
* fixed run list parsing
* updated examples for Chef 0.10
* fixed validation on existing cookbooks
* added examples directory

BACKLOG
=======
Next
----
* make spiceweasel a library rather than an executable
* switch to using lib/ directory
* convert to a knife plugin
 * knife batch create from file infrastructure.yml
 * knife batch delete from file infrastructure.json
 * knife batch rebuild from file infrastructure.yml
Future
------
* --chef-client The option `--chef-client` will make a `knife ssh` call to each box and run `chef-client` on each.
* --chef-client validation that nodes are added
* -e/--execute execute the commands
 * catching return codes and retrying (with retry count?)
* make the JSON calls directly with Chef APIs
* execution-phase validation
 * check metadata.rb of cookbooks for their dependencies
 * validate within role files rather than the names of files (assumption that they are the same)
 * validate cookbooks referenced in roles
 * validate not trying to use environments with 0.9.x
 * validate within environment files rather than the names of files (assumption that they are the same)
 * validate cookbooks referenced in environments
 * validate recipes from cookbooks in run_lists
* wildcards for environments and roles
* on provider delete take count of vendor-specific, delete if match (ec2 server delete and node delete)
* knife winrm bootstrap FQDN [RUN LIST...] (options)
* use GNU parallel with knife?
* extract existing infrastructure
 * knife batch extract

repository: Atalanta/spiceweasel

repo_name: Atalanta/spiceweasel


text:
### Authentication

Authentication is an **essential** part of most applications. There are many different approaches and strategies to handle authentication. The approach taken for any project depends on its particular application requirements. This chapter presents several approaches to authentication that can be adapted to a variety of different requirements.

Let's flesh out our requirements. For this use case, clients will start by authenticating with a username and password. Once authenticated, the server will issue a JWT that can be sent as a [bearer token](https://tools.ietf.org/html/rfc6750) in an authorization header on subsequent requests to prove authentication. We'll also create a protected route that is accessible only to requests that contain a valid JWT.

We'll start with the first requirement: authenticating a user. We'll then extend that by issuing a JWT. Finally, we'll create a protected route that checks for a valid JWT on the request.

#### Creating an authentication module

We'll start by generating an `AuthModule` and in it, an `AuthService` and an `AuthController`. We'll use the `AuthService` to implement the authentication logic, and the `AuthController` to expose the authentication endpoints.

```bash
$ nest g module auth
$ nest g controller auth
$ nest g service auth
```

As we implement the `AuthService`, we'll find it useful to encapsulate user operations in a `UsersService`, so let's generate that module and service now:

```bash
$ nest g module users
$ nest g service users
```

Replace the default contents of these generated files as shown below. For our sample app, the `UsersService` simply maintains a hard-coded in-memory list of users, and a find method to retrieve one by username. In a real app, this is where you'd build your user model and persistence layer, using your library of choice (e.g., TypeORM, Sequelize, Mongoose, etc.).

```typescript
@@filename(users/users.service)
import { Injectable } from '@nestjs/common';

// This should be a real class/interface representing a user entity
export type User = any;

@Injectable()
export class UsersService {
  private readonly users = [
    {
      userId: 1,
      username: 'john',
      password: 'changeme',
    },
    {
      userId: 2,
      username: 'maria',
      password: 'guess',
    },
  ];

  async findOne(username: string): Promise<User | undefined> {
    return this.users.find(user => user.username === username);
  }
}
@@switch
import { Injectable } from '@nestjs/common';

@Injectable()
export class UsersService {
  constructor() {
    this.users = [
      {
        userId: 1,
        username: 'john',
        password: 'changeme',
      },
      {
        userId: 2,
        username: 'maria',
        password: 'guess',
      },
    ];
  }

  async findOne(username) {
    return this.users.find(user => user.username === username);
  }
}
```

In the `UsersModule`, the only change needed is to add the `UsersService` to the exports array of the `@Module` decorator so that it is visible outside this module (we'll soon use it in our `AuthService`).

```typescript
@@filename(users/users.module)
import { Module } from '@nestjs/common';
import { UsersService } from './users.service';

@Module({
  providers: [UsersService],
  exports: [UsersService],
})
export class UsersModule {}
@@switch
import { Module } from '@nestjs/common';
import { UsersService } from './users.service';

@Module({
  providers: [UsersService],
  exports: [UsersService],
})
export class UsersModule {}
```

#### Implementing the "Sign in" endpoint

Our `AuthService` has the job of retrieving a user and verifying the password. We create a `signIn()` method for this purpose. In the code below, we use a convenient ES6 spread operator to strip the password property from the user object before returning it. This is a common practice when returning user objects, as you don't want to expose sensitive fields like passwords or other security keys.

```typescript
@@filename(auth/auth.service)
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { UsersService } from '../users/users.service';

@Injectable()
export class AuthService {
  constructor(private usersService: UsersService) {}

  async signIn(username: string, pass: string): Promise<any> {
    const user = await this.usersService.findOne(username);
    if (user?.password !== pass) {
      throw new UnauthorizedException();
    }
    const { password, ...result } = user;
    // TODO: Generate a JWT and return it here
    // instead of the user object
    return result;
  }
}
@@switch
import { Injectable, Dependencies, UnauthorizedException } from '@nestjs/common';
import { UsersService } from '../users/users.service';

@Injectable()
@Dependencies(UsersService)
export class AuthService {
  constructor(usersService) {
    this.usersService = usersService;
  }

  async signIn(username: string, pass: string) {
    const user = await this.usersService.findOne(username);
    if (user?.password !== pass) {
      throw new UnauthorizedException();
    }
    const { password, ...result } = user;
    // TODO: Generate a JWT and return it here
    // instead of the user object
    return result;
  }
}
```

> Warning **Warning** Of course in a real application, you wouldn't store a password in plain text. You'd instead use a library like [bcrypt](https://github.com/kelektiv/node.bcrypt.js#readme), with a salted one-way hash algorithm. With that approach, you'd only store hashed passwords, and then compare the stored password to a hashed version of the **incoming** password, thus never storing or exposing user passwords in plain text. To keep our sample app simple, we violate that absolute mandate and use plain text. **Don't do this in your real app!**

Now, we update our `AuthModule` to import the `UsersModule`.

```typescript
@@filename(auth/auth.module)
import { Module } from '@nestjs/common';
import { AuthService } from './auth.service';
import { AuthController } from './auth.controller';
import { UsersModule } from '../users/users.module';

@Module({
  imports: [UsersModule],
  providers: [AuthService],
  controllers: [AuthController],
})
export class AuthModule {}
@@switch
import { Module } from '@nestjs/common';
import { AuthService } from './auth.service';
import { AuthController } from './auth.controller';
import { UsersModule } from '../users/users.module';

@Module({
  imports: [UsersModule],
  providers: [AuthService],
  controllers: [AuthController],
})
export class AuthModule {}
```

With this in place, let's open up the `AuthController` and add a `signIn()` method to it. This method will be called by the client to authenticate a user. It will receive the username and password in the request body, and will return a JWT token if the user is authenticated.

```typescript
@@filename(auth/auth.controller)
import { Body, Controller, Post, HttpCode, HttpStatus } from '@nestjs/common';
import { AuthService } from './auth.service';

@Controller('auth')
export class AuthController {
  constructor(private authService: AuthService) {}

  @HttpCode(HttpStatus.OK)
  @Post('login')
  signIn(@Body() signInDto: Record<string, any>) {
    return this.authService.signIn(signInDto.username, signInDto.password);
  }
}
```

> info **Hint** Ideally, instead of using the `Record<string, any>` type, we should use a DTO class to define the shape of the request body. See the [validation](/techniques/validation) chapter for more information.

<app-banner-courses-auth></app-banner-courses-auth>

#### JWT token

We're ready to move on to the JWT portion of our auth system. Let's review and refine our requirements:

- Allow users to authenticate with username/password, returning a JWT for use in subsequent calls to protected API endpoints. We're well on our way to meeting this requirement. To complete it, we'll need to write the code that issues a JWT.
- Create API routes which are protected based on the presence of a valid JWT as a bearer token

We'll need to install one additional package to support our JWT requirements:

```bash
$ npm install --save @nestjs/jwt
```

> info **Hint** The `@nestjs/jwt` package (see more [here](https://github.com/nestjs/jwt)) is a utility package that helps with JWT manipulation. This includes generating and verifying JWT tokens.

To keep our services cleanly modularized, we'll handle generating the JWT in the `authService`. Open the `auth.service.ts` file in the `auth` folder, inject the `JwtService`, and update the `signIn` method to generate a JWT token as shown below:

```typescript
@@filename(auth/auth.service)
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { UsersService } from '../users/users.service';
import { JwtService } from '@nestjs/jwt';

@Injectable()
export class AuthService {
  constructor(
    private usersService: UsersService,
    private jwtService: JwtService
  ) {}

  async signIn(username, pass) {
    const user = await this.usersService.findOne(username);
    if (user?.password !== pass) {
      throw new UnauthorizedException();
    }
    const payload = { sub: user.userId, username: user.username };
    return {
      access_token: await this.jwtService.signAsync(payload),
    };
  }
}
@@switch
import { Injectable, Dependencies, UnauthorizedException } from '@nestjs/common';
import { UsersService } from '../users/users.service';
import { JwtService } from '@nestjs/jwt';

@Dependencies(UsersService, JwtService)
@Injectable()
export class AuthService {
  constructor(usersService, jwtService) {
    this.usersService = usersService;
    this.jwtService = jwtService;
  }

  async signIn(username, pass) {
    const user = await this.usersService.findOne(username);
    if (user?.password !== pass) {
      throw new UnauthorizedException();
    }
    const payload = { username: user.username, sub: user.userId };
    return {
      access_token: await this.jwtService.signAsync(payload),
    };
  }
}
```

We're using the `@nestjs/jwt` library, which supplies a `signAsync()` function to generate our JWT from a subset of the `user` object properties, which we then return as a simple object with a single `access_token` property. Note: we choose a property name of `sub` to hold our `userId` value to be consistent with JWT standards. Don't forget to inject the JwtService provider into the `AuthService`.

We now need to update the `AuthModule` to import the new dependencies and configure the `JwtModule`.

First, create `constants.ts` in the `auth` folder, and add the following code:

```typescript
@@filename(auth/constants)
export const jwtConstants = {
  secret: 'DO NOT USE THIS VALUE. INSTEAD, CREATE A COMPLEX SECRET AND KEEP IT SAFE OUTSIDE OF THE SOURCE CODE.',
};
@@switch
export const jwtConstants = {
  secret: 'DO NOT USE THIS VALUE. INSTEAD, CREATE A COMPLEX SECRET AND KEEP IT SAFE OUTSIDE OF THE SOURCE CODE.',
};
```

We'll use this to share our key between the JWT signing and verifying steps.

> Warning **Warning** **Do not expose this key publicly**. We have done so here to make it clear what the code is doing, but in a production system **you must protect this key** using appropriate measures such as a secrets vault, environment variable, or configuration service.

Now, open `auth.module.ts` in the `auth` folder and update it to look like this:

```typescript
@@filename(auth/auth.module)
import { Module } from '@nestjs/common';
import { AuthService } from './auth.service';
import { UsersModule } from '../users/users.module';
import { JwtModule } from '@nestjs/jwt';
import { AuthController } from './auth.controller';
import { jwtConstants } from './constants';

@Module({
  imports: [
    UsersModule,
    JwtModule.register({
      global: true,
      secret: jwtConstants.secret,
      signOptions: { expiresIn: '60s' },
    }),
  ],
  providers: [AuthService],
  controllers: [AuthController],
  exports: [AuthService],
})
export class AuthModule {}
@@switch
import { Module } from '@nestjs/common';
import { AuthService } from './auth.service';
import { UsersModule } from '../users/users.module';
import { JwtModule } from '@nestjs/jwt';
import { AuthController } from './auth.controller';
import { jwtConstants } from './constants';

@Module({
  imports: [
    UsersModule,
    JwtModule.register({
      global: true,
      secret: jwtConstants.secret,
      signOptions: { expiresIn: '60s' },
    }),
  ],
  providers: [AuthService],
  controllers: [AuthController],
  exports: [AuthService],
})
export class AuthModule {}
```

> hint **Hint** We're registering the `JwtModule` as global to make things easier for us. This means that we don't need to import the `JwtModule` anywhere else in our application.

We configure the `JwtModule` using `register()`, passing in a configuration object. See [here](https://github.com/nestjs/jwt/blob/master/README.md) for more on the Nest `JwtModule` and [here](https://github.com/auth0/node-jsonwebtoken#usage) for more details on the available configuration options.

Let's go ahead and test our routes using cURL again. You can test with any of the `user` objects hard-coded in the `UsersService`.

```bash
$ # POST to /auth/login
$ curl -X POST http://localhost:3000/auth/login -d '{"username": "john", "password": "changeme"}' -H "Content-Type: application/json"
{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."}
$ # Note: above JWT truncated
```

#### Implementing the authentication guard

We can now address our final requirement: protecting endpoints by requiring a valid JWT be present on the request. We'll do this by creating an `AuthGuard` that we can use to protect our routes.

```typescript
@@filename(auth/auth.guard)
import {
  CanActivate,
  ExecutionContext,
  Injectable,
  UnauthorizedException,
} from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import { jwtConstants } from './constants';
import { Request } from 'express';

@Injectable()
export class AuthGuard implements CanActivate {
  constructor(private jwtService: JwtService) {}

  async canActivate(context: ExecutionContext): Promise<boolean> {
    const request = context.switchToHttp().getRequest();
    const token = this.extractTokenFromHeader(request);
    if (!token) {
      throw new UnauthorizedException();
    }
    try {
      const payload = await this.jwtService.verifyAsync(
        token,
        {
          secret: jwtConstants.secret
        }
      );
      // 💡 We're assigning the payload to the request object here
      // so that we can access it in our route handlers
      request['user'] = payload;
    } catch {
      throw new UnauthorizedException();
    }
    return true;
  }

  private extractTokenFromHeader(request: Request): string | undefined {
    const [type, token] = request.headers.authorization?.split(' ') ?? [];
    return type === 'Bearer' ? token : undefined;
  }
}
```

We can now implement our protected route and register our `AuthGuard` to protect it.

Open the `auth.controller.ts` file and update it as shown below:

```typescript
@@filename(auth.controller)
import {
  Body,
  Controller,
  Get,
  HttpCode,
  HttpStatus,
  Post,
  Request,
  UseGuards
} from '@nestjs/common';
import { AuthGuard } from './auth.guard';
import { AuthService } from './auth.service';

@Controller('auth')
export class AuthController {
  constructor(private authService: AuthService) {}

  @HttpCode(HttpStatus.OK)
  @Post('login')
  signIn(@Body() signInDto: Record<string, any>) {
    return this.authService.signIn(signInDto.username, signInDto.password);
  }

  @UseGuards(AuthGuard)
  @Get('profile')
  getProfile(@Request() req) {
    return req.user;
  }
}
```

We're applying the `AuthGuard` that we just created to the `GET /profile` route so that it will be protected.

Ensure the app is running, and test the routes using `cURL`.

```bash
$ # GET /profile
$ curl http://localhost:3000/auth/profile
{"statusCode":401,"message":"Unauthorized"}

$ # POST /auth/login
$ curl -X POST http://localhost:3000/auth/login -d '{"username": "john", "password": "changeme"}' -H "Content-Type: application/json"
{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2Vybm..."}

$ # GET /profile using access_token returned from previous step as bearer code
$ curl http://localhost:3000/auth/profile -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2Vybm..."
{"sub":1,"username":"john","iat":...,"exp":...}
```

Note that in the `AuthModule`, we configured the JWT to have an expiration of `60 seconds`. This is too short an expiration, and dealing with the details of token expiration and refresh is beyond the scope of this article. However, we chose that to demonstrate an important quality of JWTs. If you wait 60 seconds after authenticating before attempting a `GET /auth/profile` request, you'll receive a `401 Unauthorized` response. This is because `@nestjs/jwt` automatically checks the JWT for its expiration time, saving you the trouble of doing so in your application.

We've now completed our JWT authentication implementation. JavaScript clients (such as Angular/React/Vue), and other JavaScript apps, can now authenticate and communicate securely with our API Server.

#### Enable authentication globally

If the vast majority of your endpoints should be protected by default, you can register the authentication guard as a [global guard](/guards#binding-guards) and instead of using `@UseGuards()` decorator on top of each controller, you could simply flag which routes should be public.

First, register the `AuthGuard` as a global guard using the following construction (in any module, for example, in the `AuthModule`):

```typescript
providers: [
  {
    provide: APP_GUARD,
    useClass: AuthGuard,
  },
],
```

With this in place, Nest will automatically bind `AuthGuard` to all endpoints.

Now we must provide a mechanism for declaring routes as public. For this, we can create a custom decorator using the `SetMetadata` decorator factory function.

```typescript
import { SetMetadata } from '@nestjs/common';

export const IS_PUBLIC_KEY = 'isPublic';
export const Public = () => SetMetadata(IS_PUBLIC_KEY, true);
```

In the file above, we exported two constants. One being our metadata key named `IS_PUBLIC_KEY`, and the other being our new decorator itself that we’re going to call `Public` (you can alternatively name it `SkipAuth` or `AllowAnon`, whatever fits your project).

Now that we have a custom `@Public()` decorator, we can use it to decorate any method, as follows:

```typescript
@Public()
@Get()
findAll() {
  return [];
}
```

Lastly, we need the `AuthGuard` to return `true` when the `"isPublic"` metadata is found. For this, we'll use the `Reflector` class (read more [here](/guards#putting-it-all-together)).

```typescript
@Injectable()
export class AuthGuard implements CanActivate {
  constructor(private jwtService: JwtService, private reflector: Reflector) {}

  async canActivate(context: ExecutionContext): Promise<boolean> {
    const isPublic = this.reflector.getAllAndOverride<boolean>(IS_PUBLIC_KEY, [
      context.getHandler(),
      context.getClass(),
    ]);
    if (isPublic) {
      // 💡 See this condition
      return true;
    }

    const request = context.switchToHttp().getRequest();
    const token = this.extractTokenFromHeader(request);
    if (!token) {
      throw new UnauthorizedException();
    }
    try {
      const payload = await this.jwtService.verifyAsync(token, {
        secret: jwtConstants.secret,
      });
      // 💡 We're assigning the payload to the request object here
      // so that we can access it in our route handlers
      request['user'] = payload;
    } catch {
      throw new UnauthorizedException();
    }
    return true;
  }

  private extractTokenFromHeader(request: Request): string | undefined {
    const [type, token] = request.headers.authorization?.split(' ') ?? [];
    return type === 'Bearer' ? token : undefined;
  }
}
```

#### Passport integration

[Passport](https://github.com/jaredhanson/passport) is the most popular node.js authentication library, well-known by the community and successfully used in many production applications. It's straightforward to integrate this library with a **Nest** application using the `@nestjs/passport` module.

To learn how you can integrate Passport with NestJS, check out this [chapter](/recipes/passport).

#### Example

You can find a complete version of the code in this chapter [here](https://github.com/nestjs/nest/tree/master/sample/19-auth-jwt).

gha_language: TypeScript


text:
A generated wrapper library for the closed-source Agilysys OnDemand system used by RIT.

Generated by [ondemand_wrapper_gen](https://github.com/RubbaBoy/ondemand_wrapper_gen)

### Usage

In your pubspec.yml, include the following:

```yaml
dependencies:
  ondemand:
    git: https://github.com/RubbaBoy/ondemand
```

### Examples

The least amount of code needed to start making requests:

```dart
import 'package:ondemand/ondemand.dart';

// ...

// The object to invoke requests with
var onDemand = await Initialization.create();

// The site-wide config
var config = initialization.config;
```

The following is a relatively barebones example of making an account inquiry to view balances (Source: [example/inquiry_example.dart](/example/inquiry_example.dart))

```dart
import 'package:ondemand/account_inquiry.dart' as _account_inquiry;
import 'package:ondemand/get_kitchens.dart' as _get_kitchens;
import 'package:ondemand/ondemand.dart';

// The UID to make the inquiry to
const UID = 'my-uid';

Future<void> main(List<String> args) async {
  var onDemand = await Initialization.create();
  // The context ID, which should be the same no matter what
  var contextId = onDemand.config.contextID;

  // Fetch the kitchens available in the store
  var siteRequest = await onDemand.getKitchens(_get_kitchens.Request());

  // Just get the first kitchen, as a terminal ID is required
  var firstKitchen = siteRequest.kitchens.first;

  // Get the terminal ID
  var terminalId = firstKitchen.displayOptions.onDemandTerminalId;

  print('Using terminal: ${firstKitchen.name}');
  print('Terminal ID: $terminalId\n');

  // Check the account balance
  var tenderResponse = await makeInquiry(onDemand, contextId, terminalId, UID);
  for (var inq in tenderResponse) {
    print('Tender: ${TenderId.getTender(inq.tenderId).name}:');
    print('  Remaining: ${inq.amount.remaining} ${inq.amount.currency}');

    print('\n  Individual accounts:');
    for (var account in inq.accounts) {
      print('  ${account.name}: ${account.balance} ${account.currency}');
    }
  }
}
```

Output of this for my own account:

```
Using terminal: The Commons
Terminal ID: 1706

Tender: Dining Dollars:
  Remaining: 1424.87 USD
    Individual accounts:
     Dining Dollars (Meal Plan): 142487 USD
     Dining Dollars (Voluntary): 0 USD
```

repository: RubbaBoy/ondemand

repo_name: RubbaBoy/ondemand


text:
---
# Display name
title: Tomas Ahumada

# Username (this should match the folder name)
authors:
- ahumada

# Is this the primary user of the site?
superuser: true

# Role/position
role: GWs/GRBs

# Organizations/Affiliations
organizations:
- name: University of Maryland
  url: ""

# Short bio (displayed in user profile at end of posts)
bio: My research interests include white dwarf binaries and other gravitational-wave sources identified by the Zwicky Transient Facility.

# Enter email to display Gravatar (if Gravatar enabled in Config)
email<EMAIL_ADDRESS>
# Organizational groups that you belong to (for People widget)
#   Set this to `[]` or comment out if you are not using People widget.
user_groups:
- "Grad Students"
---

Tomás Ahumada is a third year graduate student at the University of Maryland. Originally from Chile, Tomás now works with Dr. Leo Singer (NASA) searching for the optical counterpart to gravitational waves and short gamma ray bursts using the Zwicky Transient Facility (ZTF). Tomás uses KPED to photometrically monitor the evolution of ZTF transients that are spatially and temporarily consistent with LIGO/Virgo or Fermi triggers.

repository: mcoughlin/kped-website

repo_name: mcoughlin/kped-website


text:
# API Review 2016-01-26

This API review was also recorded and is available on [Google+](https://plus.google.com/events/c3qat6vd0ic623s0t53ccv6fb9k).

## Overview

In this API review we took a look at the differences in reflection between .NET Core and the .NET Framework. Specifically, we compared `TypeInfo` and `Type`.

## Notes

Status: **In Progress** |
[Video](https://plus.google.com/events/c3qat6vd0ic623s0t53ccv6fb9k) |
[API Difference](NetFx-vs-NetCore.md)

This is a continuation of the [previous meeting](https://github.com/dotnet/apireviews/tree/master/2016-01-19-reflection). We reviewed the [API differences between .NET Framework and .NET Core](NetFx-vs-NetCore.md).

1. Equality is odd
    * Can we introduce `==` and `!=` later on?
    * it might be better to introduce `==` and `!=` now and have it call `Equals`
    * However, can't do it for `TypeInfo` because it requires new APIs in .NET Framework
    * For now, we decided to not add any operators because we can't get consistency anyways -- even in .NET Framework `Equals` and `==` don't do the same on the `*Info` classes

2. For overload resolution tie breakers Roslyn needs a way to differentiate
   between a substituted generic and actual type, e.g:

    ```C#
        class Foo<T>
        {
            void Foo(T v);
            void Foo(int v);
        }
    ```

    for `Foo<int>`, overload resolution should pick the second method, instead of failing. The currently use `MetadataToken` to compare the original definitions. Given this API throws in certain cases now (e.g. CoreRT), it would be better if we could introduce a new APIs. However, as mentioned earlier this a separate issue. Filed [#5744](https://github.com/dotnet/corefx/issues/5744).

3. `ObfuscationAttribute` / `ObfuscationAssemblyAttribute`: Do not expose. Not meaningful to the runtime or the framework and usage is quite low.

4. Don't expose module resolve event (should be `AssemblyLoadContext`, if ever)

5. Expose `ICustomAttributeProvider`, but implement explicitly

6. Do not expose `IReflect`, used for `IDispatch`

7. We will expose the overloads that take `Binder` as object and make those extension methods in S.R.TE, which is a compat contract. We should add 'real' overloads that allow people to accomplish the same scenarios but take no Binder the next time we can add APIs to .NET Framework.

8. Expose `TargetException`

repository: dotnet/apireviews

repo_name: dotnet/apireviews


text:
# tokio-native-tls
[![github actions](https://github.com/tokio-rs/tls/workflows/CI/badge.svg)](https://github.com/tokio-rs/tls/actions)
[![crates](https://img.shields.io/crates/v/tokio-native-tls.svg)](https://crates.io/crates/tokio-native-tls)
[![license](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/tokio-rs/tls/blob/master/tokio-native-tls/LICENSE)
[![docs.rs](https://docs.rs/tokio-native-tls/badge.svg)](https://docs.rs/tokio-native-tls/)

An implementation of TLS/SSL streams for Tokio built on top of the
[`native-tls`](https://crates.io/crates/native-tls) crate.

## License

This project is licensed under the [MIT license](./LICENSE).

### Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.

repository: tokio-rs/tls

gha_language: Rust

repo_name: tokio-rs/tls


text:
![logo](contrib/assets/protector_rabbit.png)

# Protector

[![Build Status](https://travis-ci.org/trivago/Protector.svg?branch=master)](https://travis-ci.org/trivago/Protector)
[![Coverage Status](https://coveralls.io/repos/github/trivago/Protector/badge.svg?branch=master)](https://coveralls.io/github/trivago/Protector?branch=master)

A circuit breaker for Time Series Databases like InfluxDB

## Introduction

This project aims to create a protection mechanism for time series databases.
It prevents malicious queries that could overload the browser or the database itself.
**Currently only InfluxDB 0.8 is supported**. But other databases could be added as well.

Goals:

* Prevent malicious queries to Time Series Databases (e.g. InfluxDB) that could bring it down by running out of memory.
* Prevent the Browser from freezing because of too many incoming datapoints.
* Provide helpful error messages for rejected queries.

Protector is supposed to be **running between your dashboard solution and the database**.
For example, you should point Grafana to Protector instead of InfluxDB to filter the queries:

![logo](contrib/assets/diagram_protector.jpeg)

For more information on why we built this tool, please read our announcement post on the trivgo techblog.

## How it works

Protector checks each query against a list of rules before it even reaches the database.
Filtered queries will not be executed at all and an error message will be returned instead.
Rules can be specified in your `config.yaml`.

You can show a description of all available rules with `protector --show_rules`.
Here's the current list of rules:

#### Prevent negative group by statements (`negative_groupby_statement`) ####
Negative group by statements lead to undefined behavior.
They can even bring down the server. That's why they are forbidden.

#### Prevent drop queries (`prevent_drop`) ####
Drop queries mean data loss. This is a risky operation that should be restricted to admin users

#### Prevent delete queries (`prevent_delete`) ####
Deleting data can be a very expensive operation. This should not be done in InfluxDB 0.8 and before.
See: https://influxdb.com/docs/v0.8/api/query_language.html#deleting-data-or-dropping-series

#### Prevent series names that end with a dot (`series_endswith_dot`) ####
Such series usually indicate that the query is unfinished and
was executed by accident. To avoid the error, just remove the dot
or add another word (e.g. 'my.graphite.series.' -> 'my.graphite.series')

#### Prevent too many datapoints per query (`too_many_datapoints`) ####
Such queries can bring down the time series database
or overload the client with too much data transferred over the wire.

#### Prevent queries for short series names (`short_series_name`) ####
The shorter the regex for the series name, the more series names get potentially matched.
This is a huge performance hit for InfluxDB.

#### Prevent querying for very old data (`query_old_data`) ####
Such queries can bring down the time series database
because it needs to open and parse very old shards from disk

## Whitelisting

You can whitelist series names in the config. Queries for series names matching one of the patterns are always executed without any previous checking. This is useful if you want to allow all queries for [Grafana annotations](http://docs.grafana.org/reference/annotations/) for example. Look at `config.yaml` for some sample patterns.

## Usage

You can either run Protector as a Docker container or as a stand-alone Python application.

But first, please create a `config.yaml` which includes all your settings.
We've added a sample config file that should get you started.
Please make sure to adjust the `backend_host` and `backend_port` to point to your database

### Option 1: Running as a stand-alone application

You need to have Python 2.7 installed on your server

```Python
pip install protector
protector -c config.yaml
```

### Option 2: Running as a Docker container

Start the container in the same directory where your `config.yaml` is located:

```
docker run -it -v `pwd`:config trivago/protector
```

### Wiring up

After you've started Protector, point all your user-facing endpoints (e.g. Grafana) to Protector instead of InfluxDB.
That should do the trick.

## Commandline options

You can overwrite the following settings from the command-line:

```
usage: protector [-h] [--host HOST] [--port PORT]
                   [--backend_host BACKEND_HOST] [--backend_port BACKEND_PORT]
                   [-c CONFIGFILE] [-v] [--show_rules] [-f] [--version]
                   [{start,stop,status,restart}]

Protector - A circruit breaker for Time Series Databases

positional arguments:
  {start,stop,status,restart}
                        One of the following options:
                        start: Start the daemon (default)
                        stop: Stop the daemon
                        status: Show current status
                        restart: Restart the daemon

optional arguments:
  -h, --help            show this help message and exit
  --host HOST           Hostname to bind to (default: localhost)
  --port PORT           Port to bind to (default: 8888)
  --backend_host BACKEND_HOST
                        Hostname of Time Series Database (default: localhost)
  --backend_port BACKEND_PORT
                        Port of Time Series Database (default: 8086)
  -c CONFIGFILE, --configfile CONFIGFILE
                        Configfile path (default: None)
  -v, --verbose         Set verbosity level. Increase verbosity by adding a v:
                        -v -vv -vvv (default: 0)
  --show_rules          Show a list of available rules and quit
  -f, --foreground      Run in foreground. Don't daemonize on start.
  --version             Show version
```

## Performance

Protector can easily check hundreds of queries per second.
We included a load test that you can run yourself. Look into `docker-compose-integration.yml` for more info.

## Credits

Logo created by [Bradley Ashburn](https://thenounproject.com/bashburn/). Thanks for this!

## License

BSD-3

repository: hateif/Protector

repo_name: hateif/Protector