text:
**Project Overview**

Bank Of New York wants to expand its branches and for that it has certain hypothesis and statements it wants to verify. Using the inferential statistics method you just learned, help the bank.

**Dataset**

It has data from 9578 customers with the 15 features.

**Learnings from the project**

After completing this project, I have a better understanding of how to use inferential statistics. In this project, I have  applied the following concepts

    Confidence Interval
    Central Limit Theorem
    Hypothesis Testing
    Chi Square Test


text:
class: title

# Course Conclusion

![end](images/end.jpg)

---

## Questions & Next Steps

A bunch of useful links:

* Docker homepage - http://www.docker.com/
* Docker Hub - https://hub.docker.com
* Docker blog - http://blog.docker.com/
* Docker documentation - http://docs.docker.com/
* Docker code on GitHub - https://github.com/docker/docker
* Docker mailing list - [https://groups.google.com/forum/#!forum/docker-user
* Docker on IRC: irc.freenode.net and channels `#docker` and `#docker-dev`
* Docker on Twitter - http://twitter.com/docker
* Get Docker help on Stack Overflow - http://stackoverflow.com/search?q=docker
* Play With Docker Hands-On Labs - http://training.play-with-docker.com/

These slides are at: http://container.training/

---

class: title

Thank You!

.small[http://container.training/]

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
four-means
==========

Download images from my Tumblr feed, find their four dominant colors, make a web page

Incidentally this [article about Color Detection from Lyst](http://developers.lyst.com/data/images/2014/02/22/color-detection/) appeared on my radar ([lobste.rs](https://lobste.rs)) the same day I open-sourced this.

The gh-pages branch lives in a sub-folder to the master branch. I'm working on making these scripts more generally usable.

license: MIT

repository: jiko/four-means

repo_name: jiko/four-means


text:
# React FilePond

React FilePond is a handy wrapper component for [FilePond](https://github.com/pqina/filepond), a JavaScript library that can upload anything you throw at it, optimizes images for faster uploads, and offers a great, accessible, silky smooth user experience.

[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/pqina/react-filepond/blob/master/LICENSE)
[![npm version](https://badge.fury.io/js/react-filepond.svg)](https://www.npmjs.com/package/react-filepond)
![npm](https://img.shields.io/npm/dt/react-filepond)

---

[](https://www.buymeacoffee.com/rikschennink/)

[Buy me a Coffee](https://www.buymeacoffee.com/rikschennink/) / [Use FilePond with Pintura](https://pqina.nl/pintura/?ref=github-filepond) / [Dev updates on Twitter](https://twitter.com/rikschennink/)

---

### Core Features

*   Accepts **directories**, **files**, blobs, local URLs, **remote URLs** and Data URIs.
*   **Drop files**, select on filesystem, **copy and paste files**, or add files using the API.
*   **Async uploading** with AJAX, or encode files as base64 data and send along form post.
*   **Accessible**, tested with AT software like VoiceOver and JAWS, **navigable by Keyboard**.
*   **Image optimization**, automatic image resizing, **cropping**, and **fixes EXIF orientation**.
*   **Responsive**, automatically scales to available space, is functional on both **mobile and desktop devices**.

[Learn more about FilePond](https://pqina.nl/filepond/)

---

### Also need Image Editing?

**Pintura the modern JavaScript Image Editor** is what you're looking for. Pintura supports setting **crop aspect ratios**, **resizing**, **rotating**, **cropping**, and **flipping** images. Above all, it integrates beautifully with FilePond.

[Learn more about Pintura](https://pqina.nl/pintura/?ref=github-react-filepond)

---

## Installation

```bash
npm install react-filepond filepond --save
```

Hooks:

```jsx
import React, { useState } from 'react'
import ReactDOM from 'react-dom'

// Import React FilePond
import { FilePond, registerPlugin } from 'react-filepond'

// Import FilePond styles
import 'filepond/dist/filepond.min.css'

// Import the Image EXIF Orientation and Image Preview plugins
// Note: These need to be installed separately
// `npm i filepond-plugin-image-preview filepond-plugin-image-exif-orientation --save`
import FilePondPluginImageExifOrientation from 'filepond-plugin-image-exif-orientation'
import FilePondPluginImagePreview from 'filepond-plugin-image-preview'
import 'filepond-plugin-image-preview/dist/filepond-plugin-image-preview.css'

// Register the plugins
registerPlugin(FilePondPluginImageExifOrientation, FilePondPluginImagePreview)

// Our app
function App() {
  const [files, setFiles] = useState([])
  return (
    <div className="App">
      <FilePond
        files={files}
        onupdatefiles={setFiles}
        allowMultiple={true}
        maxFiles={3}
        server="/api"
        name="files" /* sets the file input name, it's filepond by default */
        labelIdle='Drag & Drop your files or <span class="filepond--label-action">Browse</span>'
      />
    </div>
  )
}
```

Component:

```jsx
import React, { useState } from 'react'
import ReactDOM from 'react-dom'

// Import React FilePond
import { FilePond, registerPlugin } from "react-filepond";

// Import FilePond styles
import "filepond/dist/filepond.min.css";

// Import the Image EXIF Orientation and Image Preview plugins
// Note: These need to be installed separately
import FilePondPluginImageExifOrientation from "filepond-plugin-image-exif-orientation";
import FilePondPluginImagePreview from "filepond-plugin-image-preview";
import "filepond-plugin-image-preview/dist/filepond-plugin-image-preview.css";

// Register the plugins
registerPlugin(FilePondPluginImageExifOrientation, FilePondPluginImagePreview);

// Our app
class App extends Component {
  constructor(props) {
    super(props);

    this.state = {
      // Set initial files, type 'local' means this is a file
      // that has already been uploaded to the server (see docs)
      files: [
        {
          source: "index.html",
          options: {
            type: "local"
          }
        }
      ]
    };
  }

  handleInit() {
    console.log("FilePond instance has initialised", this.pond);
  }

  render() {
    return (
      <div className="App">
        <FilePond
          ref={ref => (this.pond = ref)}
          files={this.state.files}
          allowMultiple={true}
          allowReorder={true}
          maxFiles={3}
          server="/api"
          name="files" /* sets the file input name, it's filepond by default */
          oninit={() => this.handleInit()}
          onupdatefiles={fileItems => {
            // Set currently active file objects to this.state
            this.setState({
              files: fileItems.map(fileItem => fileItem.file)
            });
          }}
        />
      </div>
    );
  }
}

```

[Read the docs for more information](https://pqina.nl/filepond/docs/patterns/frameworks/react/)

[Live Demo on Code Sandbox](https://codesandbox.io/s/react-filepond-live-demo-iw9ri)

repository: pqina/react-filepond

gha_language: JavaScript

gha_license_id: MIT

repo_name: pqina/react-filepond


text:
# auto-broadcast
Simple auto message broadcasting plugin for (Paper) Spigot
(I just made this in a couple of hours for a server and thought it might be useful for some people here)

### Tested on MC 1.13.2 (might work on other versions)

### Features
- **Simple, easy-to-use configuration**
- Reload config command
- **Coloured broadcasts**
- Random order
- Multi-line supported
- **Sounds**

### Usage
Currently, the plugin is not on Spigot. You can download the latest version of
the plugin from the [Releases page](https://github.com/jellz/auto-broadcast/releases).

Just install it on your server like any other plugin. You can edit the config after the plugin generates it.

This is the default config:
```yaml
broadcasts-enabled: true
broadcast-interval: 3600 # ticks between each broadcast (default: 3 minutes)
broadcasts:
  test:
    sound: BLOCK_NOTE_BLOCK_PLING # see https://hub.spigotmc.org/javadocs/bukkit/org/bukkit/Sound.html for a list of sounds, leave blank for no sound
    messages:
      - '&6Test broadcast!'
  nosound:
    messages:
      - ' '
      - '&bThis broadcast takes up 3 lines!'
      - ' '
```

Reload the config with `/reloadbroadcasts`.

### Ideas
- PlaceholderAPI support
- Action bar / Titles
- Permission checks

repository: jellz/auto-broadcast

repo_name: jellz/auto-broadcast


text:
# Jsoniter Scala [![build status](https://travis-ci.org/plokhotnyuk/jsoniter-scala.svg?branch=master)](https://travis-ci.org/plokhotnyuk/jsoniter-scala) [![code coverage](https://codecov.io/gh/plokhotnyuk/jsoniter-scala/branch/master/graph/badge.svg)](https://codecov.io/gh/plokhotnyuk/jsoniter-scala) [![license](http://img.shields.io/:license-Apache%202-green.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)

Macros that generates `com.jsoniter.spi.Decoder` and `com.jsoniter.spi.Encoder` interfaces in compile time,
based on compile time reflection.

Main goal is to provide a static code generation for Scala case classes, standard types and collections
to get maximum performance of JSON parsing & serialization.

## Features and limitations
- JSON parsing from `Array[Byte]` or `java.io.InputStream` using `com.jsoniter.JsonIterator`
- JSON serialization to `Array[Byte]` or `java.io.OutputStream` using `com.jsoniter.JsonStream`
- Support of UTF-8 encoding (including parsing of hex encoded JSON field names, keys & values)
- Case classes should be defined as a top-level class or directly inside of another class or object
- Enums, `BigInt`, `BigDecimal`, `Option` & Scala collections as class fields are supported
- Fields with null values, default values, empty options & empty collections are not serialized to provide sparse output
- Fields can be annotated as transient or just not defined in constructor to avoid parsing and serializing
- Other fields are required (no special annotation required)
- Need to materialize for all case classes used in nested structures
- Key names can be overridden by field annotation
- Configurable indenting of output
- TODO: configurable by field annotations custom encoder(s)/decoder(s) for fields and their type parameter(s)
- TODO: efficient specialized immutable list & set for primitive types
- TODO: throw exception instead of replacing by placeholder character(s) during parsing invalid UTF-8 bytes

## How to use

Build and publish locally (release to Sonatype repo is coming)

```sh
sbt clean +publishLocal
```

Add the library to your dependencies list

```sbt
libraryDependencies += "com.github.plokhotnyuk.jsoniter-scala" %% "macros" % "0.1-SNAPSHOT"
```

Generate some serializers for your case classes

```scala
import com.github.plokhotnyuk.jsoniter_scala.Codec

case class User(name: String)

val userCodec = Codec.materialize[User]
```

That's it! You have generated an instance of `com.github.plokhotnyuk.jsoniter_scala.Codec` which implements both
`com.jsoniter.spi.Decoder` and `com.jsoniter.spi.Encoder` interfaces for your `User`.

Now you can use it:

```scala
userCodec.read("""{"name":"John"}""".getBytes("UTF-8"))
userCodec.write(User(name = "John"))
```

To see generated code add the following line to your sbt build file

```sbt
scalaOptions += "-Xmacro-settings:print-codecs"
```

For more features & examples, please, check out
[CodecSpec](https://github.com/plokhotnyuk/jsoniter-scala/blob/master/macros/src/test/scala/com/github/plokhotnyuk/jsoniter_scala/CodecSpec.scala)

## How to develop

Feel free to ask questions by opening issues (Gitter chat is coming), or contribute by creating pull requests (code or/and tests are highly appreciated)

Run tests and check coverage

```sh
sbt clean +coverage +test +coverageReport
```

Run benchmarks

```sh
sbt -no-colors clean 'benchmark/jmh:run -prof gc .*CodecBenchmark.*' >results.txt
```

Currently [./results.txt](https://github.com/plokhotnyuk/jsoniter-scala/blob/master/results.txt) contains results for the following environment:

Intel(R) Core(TM) i7-2760QM CPU @ 2.40GHz (max 3.50GHz), RAM 16Gb DDR3-1600, Ubuntu 15.04, Linux 4.4.0-38-generic, Oracle JDK build 1.8.0_112-b15 64-bit

## Acknowledges

[Jsoniter Java](https://github.com/json-iterator/java)

[Kryo Macros](https://github.com/evolution-gaming/kryo-macros)

[AVSystem Commons Library for Scala](https://github.com/AVSystem/scala-commons)

repository: kostjas/jsoniter-scala

repo_name: kostjas/jsoniter-scala


text:
# How to add new natural language support into Sumy

Let's say [someone wants](https://github.com/miso-belica/sumy/issues/62) to summarize documents in the Russian language by the Sumy. The first thing you will need is a [tokenizer](docs/index.md#Tokenizer). So we can try if there is some support in Sumy.

```python
from sumy.nlp.tokenizers import Tokenizer
tokenizer = Tokenizer("ru")

# https://ru.m.wikipedia.org/wiki/Тунберг,_Грета
sentences = tokenizer.to_sentences("Гре́та Тинтин Элеонора Э́рнман Ту́нберг (швед. Greta Tintin Eleonora Ernman Thunberg; род. 3 января 2003[1][2], Стокгольм[1]) — шведская школьница, экологическая активистка. В 15 лет начала протестовать возле шведского парламента с плакатом «Школьная забастовка за климат», призывая к незамедлительным действиям по борьбе с изменением климата в соответствии с Парижским соглашением. Её действия нашли отклик по всему миру, породив массовые мероприятия, известные как «школьные забастовки за климат» или «пятницы ради будущего»")

for sentence in sentences:
    print(tokenizer.to_words(sentence))
```

So we are good here. But you know, what if the tokenizer is missing? Then you need to implement one by yourself. Or to find some library for your language and wrap it into the API for Sumy. It should be easy because Sumy expects an object with two methods. The simplest naive tokenizer would look like this.

```python
from typing import List

class Tokenizer:
    @staticmethod
    def to_sentences(text: str) -> List[str]:
        return [s.strip() for s in text.split(".")]

    @staticmethod
    def to_words(sentence: str) -> List[str]:
        return [w.strip() for w in sentence.split(" ")]
```

Another language-specific thing is **Stemmer**. For Sumy, the stemmer is any callable object which accepts word (string) and returns word (string). But the word may be somehow changed. The role of the stemmer is to normalize words into the same form. For example, you have words: _teacher_, _teaching_, _teach_ and you want to return the root of the word _teach_ for all these because they have the same meaning. But of course, you want to return _sleep_ for _sleeping_. Some languages like the Japanese language does not care and can simply return the original word. But for another like Slovak/Czech/English language, it's quite important to normalize all the different forms of the word. The simplest stemmer looks like the one below:

```python
def null_stemmer(word):
    """I am the same as from sumy.nlp.stemmers import null_stemmer :)"""
    return word
```

```python
# seems NLTK covers our back again :)
from sumy.nlp.stemmers import Stemmer
stemmer = Stemmer("ru")
stem = stemmer("Элеонора")  # элеонор
```

The last piece is a list of stop-words. Sumy has some stop-words in, but you can download any free list from the internet. This piece is also optional because the summarizers can work without it but it's **highly recommended** to provide one because it may increase the quality of the summaries dramatically.

And that's all. You will these parts together with the other parts from the README and send a pull request with your code :)

repository: miso-belica/sumy

branch_name: refs/heads/main

gha_language: Python

repo_name: miso-belica/sumy


text:
---
templateKey: virtual-booth-page
title: 'Virtual Booth'
intro: >-
  Hello! Sponsor ads and logos can link out to any sponsor-related page or back to the sponsor’s virtual sponsor booth page.

  Two samples of virtual sponsor booth layouts are shown below.
columns:
  leftColumn:
    title: TOP-TIER SPONSOR PAGE
    content: >-
      All the bells and whistles: promotional videos and images; scrolling ticker messaging; links to documents, URLs, and social media sites, featured upcoming event  sessions   (hand selected by the sponsor), interactive chat, lead retrieval, meeting scheduling, and more!
    image: /img/YOCO-GoldSponsor-Page-Mock-960.jpg
  rightColumn:
    title: MIDDLE-TIER SPONSOR PAGE
    content: >-
      Perhaps a static image header, brief company intro, fewer documents/links, etc.
    image: /img/YOCO-SilverSponsor-Page-Mock-960.jpg
endText: >-
  As the client, you set the parameters. Your sponsorship features drive the layout … and if you’re unsure exactly what you want to offer, our creative and logistic pros are here to help walk you through the process!

  With our flexible system, the sky’s the limit! Let us help make your sponsors happy!
---

repository: gcutrini/virtualevent-poc

repo_name: gcutrini/virtualevent-poc


text:
# composite transaction

When performing a composite transaction that involves 2 (or more) unit transactions,
it is crucial to guarantee its _atomicity_.

Suppose a composite transaction involving 2 unit transactions, `tx1` and `tx2`.
For atomicity,
only one of the following 2 cases can be the result of the transaction:

- both `tx1` and `tx2` succeed
- neither succeeds (rollbacked)

## protocol

Such atomicity requirement can be defined as dsl4sc specification as follows:

```
protocol tx1; (tx1_err; rollback + tx2; (tx2_err; rollback)?);;
```

This protocol specifies that

- the first event is `tx1`, from which the remaining part is divided into the following 2 cases
- [case1] `tx1_err; rollback`
  `tx1_err` is emitted, indicating faiure of `tx1`, which is immediately followed by `rollback`
- [case2] `tx2; (tx2_err; rollback)?`
  `tx1` succeeds, and `tx2` is emitted subsequently.
  If it succeeds as well, the composition transaction terminates successfully.
  Otherwise, `tx2_err` and `rollback` are emitted in the same way as `tx1` fails.


text:
# MVC-Express example using Sequelize

To run the app, create the DB, use [the Sequelize CLI](https://github.com/sequelize/cli) to
run the migrations and the seeds, then `npm run dev`.

```bash
mysql -uroot

create database mvc;
exit

sequelize db:migrate
sequelize db:seed:all

npm run dev
```

repository: pajtai/mvc-express

repo_name: pajtai/mvc-express


text:
---
title: Trained Models & Pipelines
teaser: Downloadable trained pipelines and weights for spaCy
menu:
  - ['Quickstart', 'quickstart']
  - ['Conventions', 'conventions']
  - ['Pipeline Design', 'design']
---

<!-- TODO: include interactive demo -->

### Quickstart {hidden="true"}

> #### 📖 Installation and usage
>
> For more details on how to use trained pipelines with spaCy, see the
> [usage guide](/usage/models).

import QuickstartModels from 'widgets/quickstart-models.js'

<QuickstartModels id="quickstart" />

## Package naming conventions {#conventions}

In general, spaCy expects all pipeline packages to follow the naming convention
of `[lang]\_[name]`. For spaCy's pipelines, we also chose to divide the name
into three components:

1. **Type:** Capabilities (e.g. `core` for general-purpose pipeline with
   tagging, parsing, lemmatization and named entity recognition, or `dep` for
   only tagging, parsing and lemmatization).
2. **Genre:** Type of text the pipeline is trained on, e.g. `web` or `news`.
3. **Size:** Package size indicator, `sm`, `md`, `lg` or `trf` (`sm`: no word
   vectors, `md`: reduced word vector table with 20k unique vectors for ~500k
   words, `lg`: large word vector table with ~500k entries, `trf`: transformer
   pipeline without static word vectors)

For example, [`en_core_web_sm`](/models/en#en_core_web_sm) is a small English
pipeline trained on written web text (blogs, news, comments), that includes
vocabulary, syntax and entities.

### Package versioning {#model-versioning}

Additionally, the pipeline package versioning reflects both the compatibility
with spaCy, as well as the model version. A package version `a.b.c` translates
to:

- `a`: **spaCy major version**. For example, `2` for spaCy v2.x.
- `b`: **spaCy minor version**. For example, `3` for spaCy v2.3.x.
- `c`: **Model version**. Different model config: e.g. from being trained on
  different data, with different parameters, for different numbers of
  iterations, with different vectors, etc.

For a detailed compatibility overview, see the
[`compatibility.json`](https://github.com/explosion/spacy-models/tree/master/compatibility.json).
This is also the source of spaCy's internal compatibility check, performed when
you run the [`download`](/api/cli#download) command.

## Trained pipeline design {#design}

The spaCy v3 trained pipelines are designed to be efficient and configurable.
For example, multiple components can share a common "token-to-vector" model and
it's easy to swap out or disable the lemmatizer. The pipelines are designed to
be efficient in terms of speed and size and work well when the pipeline is run
in full.

When modifying a trained pipeline, it's important to understand how the
components **depend on** each other. Unlike spaCy v2, where the `tagger`,
`parser` and `ner` components were all independent, some v3 components depend on
earlier components in the pipeline. As a result, disabling or reordering
components can affect the annotation quality or lead to warnings and errors.

Main changes from spaCy v2 models:

- The [`Tok2Vec`](/api/tok2vec) component may be a separate, shared component. A
  component like a tagger or parser can
  [listen](/api/architectures#Tok2VecListener) to an earlier `tok2vec` or
  `transformer` rather than having its own separate tok2vec layer.
- Rule-based exceptions move from individual components to the
  `attribute_ruler`. Lemma and POS exceptions move from the tokenizer exceptions
  to the attribute ruler and the tag map and morph rules move from the tagger to
  the attribute ruler.
- The lemmatizer tables and processing move from the vocab and tagger to a
  separate `lemmatizer` component.

### CNN/CPU pipeline design {#design-cnn}

![Components and their dependencies in the CNN pipelines](../images/pipeline-design.svg)

In the `sm`/`md`/`lg` models:

- The `tagger`, `morphologizer` and `parser` components listen to the `tok2vec`
  component.
- The `attribute_ruler` maps `token.tag` to `token.pos` if there is no
  `morphologizer`. The `attribute_ruler` additionally makes sure whitespace is
  tagged consistently and copies `token.pos` to `token.tag` if there is no
  tagger. For English, the attribute ruler can improve its mapping from
  `token.tag` to `token.pos` if dependency parses from a `parser` are present,
  but the parser is not required.
- The `lemmatizer` component for many languages (Catalan, Dutch, English,
  French, Greek, Italian Macedonian, Norwegian, Polish and Spanish) requires
  `token.pos` annotation from either `tagger`+`attribute_ruler` or
  `morphologizer`.
- The `ner` component is independent with its own internal tok2vec layer.

### Transformer pipeline design {#design-trf}

In the transformer (`trf`) models, the `tagger`, `parser` and `ner` (if present)
all listen to the `transformer` component. The `attribute_ruler` and
`lemmatizer` have the same configuration as in the CNN models.

### Modifying the default pipeline {#design-modify}

For faster processing, you may only want to run a subset of the components in a
trained pipeline. The `disable` and `exclude` arguments to
[`spacy.load`](/api/top-level#spacy.load) let you control which components are
loaded and run. Disabled components are loaded in the background so it's
possible to reenable them in the same pipeline in the future with
[`nlp.enable_pipe`](/api/language/#enable_pipe). To skip loading a component
completely, use `exclude` instead of `disable`.

#### Disable part-of-speech tagging and lemmatization

To disable part-of-speech tagging and lemmatization, disable the `tagger`,
`morphologizer`, `attribute_ruler` and `lemmatizer` components.

```python
# Note: English doesn't include a morphologizer
nlp = spacy.load("en_core_web_sm", disable=["tagger", "attribute_ruler", "lemmatizer"])
nlp = spacy.load("en_core_web_trf", disable=["tagger", "attribute_ruler", "lemmatizer"])
```

<Infobox variant="warning" title="Rule-based and POS-lookup lemmatizers require
Token.pos">

The lemmatizer depends on `tagger`+`attribute_ruler` or `morphologizer` for
Catalan, Dutch, English, French, Greek, Italian, Macedonian, Norwegian, Polish
and Spanish. If you disable any of these components, you'll see lemmatizer
warnings unless the lemmatizer is also disabled.

</Infobox>

#### Use senter rather than parser for fast sentence segmentation

If you need fast sentence segmentation without dependency parses, disable the
`parser` use the `senter` component instead:

```python
nlp = spacy.load("en_core_web_sm")
nlp.disable_pipe("parser")
nlp.enable_pipe("senter")
```

The `senter` component is ~10× faster than the parser and more accurate
than the rule-based `sentencizer`.

#### Switch from rule-based to lookup lemmatization

For the Dutch, English, French, Greek, Macedonian, Norwegian and Spanish
pipelines, you can switch from the default rule-based lemmatizer to a lookup
lemmatizer:

```python
# Requirements: pip install spacy-lookups-data
nlp = spacy.load("en_core_web_sm")
nlp.remove_pipe("lemmatizer")
nlp.add_pipe("lemmatizer", config={"mode": "lookup"}).initialize()
```

#### Disable everything except NER

For the non-transformer models, the `ner` component is independent, so you can
disable everything else:

```python
nlp = spacy.load("en_core_web_sm", disable=["tok2vec", "tagger", "parser", "attribute_ruler", "lemmatizer"])
```

In the transformer models, `ner` listens to the `transformer` component, so you
can disable all components related tagging, parsing, and lemmatization.

```python
nlp = spacy.load("en_core_web_trf", disable=["tagger", "parser", "attribute_ruler", "lemmatizer"])
```

#### Move NER to the end of the pipeline

<Infobox title="For v3.0.x models only" variant="warning">

As of v3.1, the NER component is at the end of the pipeline by default.

</Infobox>

For access to `POS` and `LEMMA` features in an `entity_ruler`, move `ner` to the
end of the pipeline after `attribute_ruler` and `lemmatizer`:

```python
# load without NER
nlp = spacy.load("en_core_web_sm", exclude=["ner"])

# source NER from the same pipeline package as the last component
nlp.add_pipe("ner", source=spacy.load("en_core_web_sm"))

# insert the entity ruler
nlp.add_pipe("entity_ruler", before="ner")
```

repository: hiroshi-matsuda-rit/spaCy

repo_name: hiroshi-matsuda-rit/spaCy


text:
I have never previously programmed in PHP but I have given an effort to learn as much as I could in a short time.

I have written some javascript that will actually display the confetti on the screen and that will only be called
when the backend result is 1982. I also have a test function that is called via PHP and that was just used as
a test to prove that I can actually call that javascript function using PHP.

My plan was to collect the numbers that are inputted using the buttons and use the calculator class to do
the actual math with them, then check if the answer was 1982, all in PHP. If the answer is 1982, then I would use echo to
call the javascript function with a timeout to remove the confetti gif after 1 second. As it stands I am getting
an error saying that my checkAnswer function is not defined, which I do not understand.

At this point:
1. You can call showConfetti() in the console and it will work.
2. The calculator front end matches what was requested.
3. The page does not reload when the confetti is displayed as requested. (It should not reload when typing in numbers because
the plan is to add them to the window by updating the value of that readonly input.)

Thank you for the challenge! If you are looking for a developer with experience in PHP, then I might not be your best fit.
But if you are looking for someone who is willing to learn and understands some of the concepts required for web development, I
encourage you to take a chance on me. You will not be disappointed.


text:
Good commit messages serve at least three important purposes:

* To speed up the reviewing process.

* To help us write a good release note.

* To help the future maintainers of Plink

Structure your commit message like this:

<pre>
Summarize clearly in one line what the commit is about

Describe the problem the commit solves or the use
case for a new feature. Justify why you chose
the particular solution.

Finishes [ Commit # ]
</pre>

h2. DO

* Write the summary line and description of what you have done in the imperative mode, that is as if you were commanding someone. Write "fix", "add", "change" instead of "fixed", "added", "changed".

* Always leave the second line blank.

* Line break the commit message (to make the commit message readable without having to scroll horizontally in @gitk@).

* If applicable, add 'Finishes [Commit #]' to update Tracker

h2. DON'T

* Don't end the summary line with a period.

h2. Tips

* If it seems difficult to summarize what your commit does, it may be because it includes several logical changes or bug fixes, and are better split up into several commits using @git add<EMAIL_ADDRESS>

repository: tobocop/howhardwoulditbe

repo_name: tobocop/howhardwoulditbe


text:
---
title: Visiting Card
date: 2021-06-25 10:42:00 +0530
categories: [Products]
tags: [visiting card]     # TAG names should always be lowercase
comments: false
image:
  src: /assets/images/visiting-card-main.jpg
---
## Standard Visiting Card
---

### Specification
- Size: 89 x 51 mm
- 300 GSM Art Board
- Multi Colour Printing

### Pricing

| Print Sides | 100 Nos | 200 Nos | 300 Nos | 400 Nos | 500 Nos | 1000 Nos | 1500 Nos | 2000 Nos | 2500 Nos |
|-------------|--------:|--------:|--------:|--------:|--------:|---------:|---------:|---------:|---------:|
| Single Side | Rs. 180 | Rs. 260 | Rs. 340 | Rs. 450 | Rs. 500 |  Rs. 900 | Rs. 1300 | Rs. 1700 | Rs. 2100 |
| Both Side   | Rs. 240 | Rs. 380 | Rs. 520 | Rs. 660 | Rs. 800 | Rs. 1500 | Rs. 2200 | Rs. 2900 | Rs. 3600 |

### Design Guidelines
- Canvas size should be 93 x 55 mm (2 mm bleed on all edges).
It will be trimmed off to the final size of 89 x 51 mm.
- Contents should be within the safe margin (5 mm from the cutting line).
- Images must be in 300 dpi for best result.
- Colour profile must be in CMYK
- Export and share design file only in PDF format

Is your design file is ready? If yes, please send it to us<EMAIL_ADDRESS>We will take it up from there. If you are looking for a new design, please send us your business card details (Business Name, Logo, Individual Name, Job Title, Phone Number, Address, Email-Id and Website). Our design team will be glad to assist you.

## Customization

| Particulars        | Options                                                                                |
|--------------------|-----------------------------------------------------------------------------------------------------------------------------|
| Size               | Standard size is 3.5" x 2" (89 x 51 mm). *It fits in the wallet*.
But literally any reasonable shapes and size can be printed. |
| Paper Weight (GSM) | 250, 300, 350                                                                                |
| Paper Stock        | Art, Matt Art, Ivory, Needle Point, Linen White                                                                             |
| Lamination         | Gloss, Matt, Velvet                                                                                |
| Finishing          | Rounded Corner Cutting, Spot UV, Gold/Silver Foiling, Embossing                                                             |

## Enquiry
For estimate and enquiries related to printing and designing visiting cards, please fill out the below form. We will get back to you within a day. For faster response use the chat option in the bottom right corner or call us on<PHONE_NUMBER>

<form action="https://us-central1-vstpress-site.cloudfunctions.net/sendMail" method="POST">
  Name:

  <input type="text" id="fname" name="name">

  Phone:

  <input type="tel" id="phone" name="phone">

  Email:

  <input type="email" id="email" name="email">

  Message:

  <textarea name="message" cols="40" rows="5"></textarea>

  <input type="submit" value="Submit">
</form>

repository: sivarajan-software/vstpress-site

repo_name: sivarajan-software/vstpress-site


text:
Slow KDF
========

Slow KDF for serious key stretching based on Scrypt.

 :point_right: This is my new key stretching script: [Doubleslow Keystretcher](https://github.com/vstoykovbg/doubleslow)

You need to install `scrypt` before to use this program:
```
$ pip install --user scrypt
```

If you prefer the version written in Python 3:

```
$ sudo apt-get install python3-pip
```

```
$ pip3 install --user scrypt
```
Warning: your computer may freeze if you run this program (if you have too little available RAM). It requires about 1GB of free RAM.

See here how to take control of your Linux if X freezes.

Updated version
====

The difference between v1 and v2 digest is that v2 is protected against hypothetical loss of entropy (due to repeated hashing with scrypt) with additional `sha512(key+salt+digest_v1)`.

I recommend using the `Version 2 digest in base64 format` or `Version 1+2 digest in base64 format` (if your application allow such long passwords - GnuPG 2 does not allow, you may use GnuPG 1!).

Why?
====

The problem is: popular cryptographic tools have laughable key stretching functionality.

How many seconds after you enter wrong passwords you get response "bad password", while using LUKS, TrueCrypt, GnuPG? Less than a second? Two seconds? Do you think that this is enogh?

See this example:

```$ gpg -vv -c --force-mdc --s2k-mode 3 --s2k-count 65011712 --s2k-digest-algo SHA512 --cipher-algo TWOFISH MyFile.txt```

* hashes the password using SHA512
* key-stretches the hash 65 million times
* and encrypts the file data using the Twofish cipher with a 256-bit key

```
$ time gpg --output clear.txt  -vv -d MyFile.txt.gpg
:symkey enc packet: version 4, cipher 10, s2k 3, hash 10
        salt ef1ae7a1f536715c, count 65011712 (255)
gpg: TWOFISH encrypted data
:encrypted data packet:
        length: unknown
        mdc_method: 2
gpg: encrypted with 1 passphrase
:compressed packet: algo=1
:literal data packet:
        mode b (62), created<PHONE_NUMBER>, name="MyFile.txt",
        raw data: 7780 bytes
gpg: original file name='MyFile.txt'
gpg: decryption okay

real    0m2.684s
user    0m1.105s
sys     0m0.004s
```

It takes about 1 second to decrypt the file after I wrote correct password!

With wrong password it takes about 1 second:

```
:symkey enc packet: version 4, cipher 10, s2k 3, hash 10
        salt ef1ae7a1f536715c, count 65011712 (255)
gpg: TWOFISH encrypted data
:encrypted data packet:
        length: unknown
        mdc_method: 2
gpg: encrypted with 1 passphrase
gpg: decryption failed: bad key

real    0m2.769s
user    0m1.144s
sys     0m0.004s
```

From the `man gpg`:

``
 -s2k-count n
    Specify how many times the passphrase mangling is repeated.  This value may range between 1024 and
    65011712  inclusive.   The  default  is  inquired from gpg-agent.  Note that not all values in the
    1024-65011712 range are legal and if an illegal value is selected, GnuPG  will  round  up  to  the
    nearest legal value.  This option is only meaningful if --s2k-mode is 3.
``

So, you can't enter number greater than 65011712 without modifying the source code.

The same weak key stretching is used for protecting your GnuPG/PGP private keys!
=====

To see your `s2k-count`, type this command:

```
$ gpg --list-packets ~/.gnupg/secring.gpg | grep count
        protect count: 65536 (96)
        protect count: 65536 (96)
        protect count: 65536 (96)
        protect count: 65536 (96)
```

In this example the `s2k-count` is 65536 (default).

And it also do not work as documented!
=====

There is a huge bug in GnuPG - your private keys are not protected by the desired key stretching:

I know I replied to you above about this, but this really needs to be super clear for everyone to see. Due to a bug, GPG completely ignored your S2K settings and isn't giving you the protection you asked for. You can see this for yourself:

```
$ gpg2 --export-secret-key | gpg2 --list-packets > packets
```

Running your exact command and then inspecting the packets gives me this:

```
# off=0 ctb=95 tag=5 hlen=3 plen=966
:secret key packet:
        version 4, algo 1, created<PHONE_NUMBER>, expires 0
        pkey[0]: [2048 bits]
        pkey[1]: [17 bits]
        iter+salt S2K, algo: 7, SHA1 protection, hash: 2, salt: 8792DC994E02F5DB
        protect count: 16252928 (223)
        protect IV:  ff 9d 2c 80 7c e4 1d 0c 6a 41 17 7f 41 1e 03 51
        skey[2]: [v4 protected]
        keyid: F021CA39AB2D856D
```

Notice it's using AES-128 (algo 7), SHA-1 (hash 2), and 16 million iterations instead of 65 million. That's why it's able to process your passphrase so quickly rather than taking a few seconds like it should.

This security bug is about 3 years old now.

/u/skeeto

The reason you can't make an S2K count over 65,011,712 is because values higher than that cannot be encoded into a single byte which is what RFC 4880 requires.

https://tools.ietf.org/html/rfc4880#section-<IP_ADDRESS>

Your problem is not with GnuPG, but with OpenPGP.

You could propose or implement a new S2K specifier type however: https://tools.ietf.org/html/rfc4880#section-10.1

/u/mitchellrj

PBKDF2, just like TrueCrypt?  I hope you decide to have better password
security than those guys.  The PBKDF2 key stretching in TrueCrypt is a
joke.  You have to find an obscure option to even enable the 2000-round
SHA256 key stretching, which provides close to no security at all.  The
kinds of passwords your users on Facebook will actually use have very
few bits of entropy, and can be guessed by hardware based brute force
attacks very quickly.

(...)

Now for my tin hat conspiracy theories: Why is Microsoft so stupid about
where to perform key stretching?  Why wont the TrueCrypt team adopt
decent password security?  Why does PGP have 0 key stretching by
default, and why is my SSH private key encrypted with no key stretching
and no option to use it?  You want to encrypt Facebook traffic with real
security?  I just have to wonder if some tall suits with dark sunglasses
are going to convince you to do otherwise.  My other theory is people
are really that stupid, whether they write SSH encryption code, PGP, or
TrueCrypt.
Bill Cox

Changes to the key with GnuPG 1 are invisible to GnuPG 2!
========

I tested it. First, I changed the password of my key with gpg (GnuPG 1) and then tried to use gpg2. I confirmed that the two programs now require different passwords for the same key.

Then, I also changed the password with gpg2. These changes are seen only by gpg2.

Discussion on Reddit: https://www.reddit.com/r/sysadmin/comments/6rzrut/changes_to_the_key_with_gnupg_1_are_invisible_to/

Please make sure that you are not keeping or exporting (backing up) the other version of your key with a weaker password!

GnuPG 2.1.11 does not accept long passwords (UPDATE: GnuPG 2.2.4 on Ubuntu accept long passwords)
========

In my version of GnuPG it does not accept passwords longer than 255 characters (ASCII). Just tested it again. It says:

    Passphrase too long

My software versions:

    gpg (GnuPG) 2.1.11
    libgcrypt 1.6.5
    pinentry-gnome3 (pinentry) 0.9.7
    Ubuntu 16.04.3 LTS

The old version of GnuPG (`gpg (GnuPG) 1.4.20`) works fine.

Conspirologists would say that this is part of the conspiracy to undermine the security of the popular open source security-related applications.

Read more about the conspiracy-relate stuff on Google: https://www.google.com/search?q=NSA%27s%20Decade-Long%20Plan%20to%20Undermine%20Encryption

**UPDATE: GnuPG 2.2.4 on Ubuntu accept long passwords**

Tested it with software versions:

    gpg (GnuPG) 2.2.4
    libgcrypt 1.8.1
    pinentry-gnome3 (pinentry) 1.1.0
    Ubuntu 18.04.3 LTS
And it works with an ASCII password with 495 charsacters. If I enter more characters there is an error message:

    gpg: problem with the agent: Too much data for IPC layer
    gpg: error creating passphrase: Operation cancelled

The documentation says:

    There is no limit on the length of a passphrase, and it should be carefully chosen.

This is not the case on my system.

When I use gpg (GnuPG 2.2.4) with the option `--pinentry-mode loopback` it asks for the password in the console (without popping up a dialog box) and the limit is still 255 ASCII characters. When I tried to replace 4 characters with Unicode the limit became 251 characters. This means that **the limit is in bytes (255 bytes), not in characters.**

When using options `--passphrase-file passphrase.txt --pinentry-mode loopback` it allowed **larger than 255 bytes** passwords (for example, 27665 bytes). When tried with 30185 bytes password I got error message `gpg: Warning: using insecure memory!` but it worked. When tried with 34728 bytes password I got error message `gpg: Fatal: out of core in secure memory while allocating 32800 bytes` and the program refused to continue.

Writing the password in a file is not very secure, especially when the filesystem is not `tmpfs` and full disk encryption is not used.

Workaround of the password length problem
========

You can use some hash function like sha512:

    echo -n "my loooooooong password" | sha512sum

Don't forget the `-n` part, because it gives different result:

    valentin@computer:~$ echo -n "my loooooooong password" | sha512sum
    f3a56f8ec032ea6d27f4ceff68e36ce9d25fc7042c737f2e4785f47d8267b5f3836987a0714921a8cc3f61dc1861523e92425ecdb90b522a81e0c82f4c5a224b  -
    valentin@computer:~$ echo "my loooooooong password" | sha512sum
    d7cbfb4629b661eef421474817587be5a6c7d96599f43638d32252ec88ba657314ba2dcb9c47bd2eeb6a24c8a9620f37ce570496eb758a6ae19ac30ef988eea4  -
    valentin@computer:~$

Example
========

    valentin@computer:~$ slowkdf.py
    Passphrase:
    Repeat passphrase:
    Repeat passphrase (again):
    Salt: salt
    Number of iterations: 1
    Iteration 1 from 1...

     == Version 1 ==

    Digest in hex format: ea6a7a27acbb972a6c280d12cbd63fb60b37bc666bf368ea58edaac7e08a41063fa00aac7206e183dd2fdd64db01636668182fc2f145f67f45f4588f32742bfd24cf3131462792b228a11522a0d8d65609be1d47a8a1ccc4098b6491f2be0b769eafca253812bf408f493e05210cc0e80b3de496dcc854e98976c4b7763c6551

    Digest in base64 format: 6mp6J6y7lypsKA0Sy9Y/tgs3vGZr82jqWO2qx+CKQQY/oAqscgbhg90v3WTbAWNmaBgvwvFF9n9F9FiPMnQr/STPMTFGJ5KyKKEVIqDY1lYJvh1HqKHMxAmLZJHyvgt2nq/KJTgSv0CPST4FIQzA6As95JbcyFTpiXbEt3Y8ZVE=

     == Version 2 ==

    Version 2 digest in hex format: b3d17366ac96465cbcaa843950dd9d77999055861f3550a11d61cb895cb54fc9d78c56dd6d053093fa43cfaa2eed628effa3555ee4f7a2adc4d068dc4004b5cf

    Version 2 digest in base64 format: s9FzZqyWRly8qoQ5UN2dd5mQVYYfNVChHWHLiVy1T8nXjFbdbQUwk/pDz6ou7WKO/6NVXuT3oq3E0GjcQAS1zw==

     == Version 1+2 ==

    Version 1+2 digest in hex format: ea6a7a27acbb972a6c280d12cbd63fb60b37bc666bf368ea58edaac7e08a41063fa00aac7206e183dd2fdd64db01636668182fc2f145f67f45f4588f32742bfd24cf3131462792b228a11522a0d8d65609be1d47a8a1ccc4098b6491f2be0b769eafca253812bf408f493e05210cc0e80b3de496dcc854e98976c4b7763c6551b3d17366ac96465cbcaa843950dd9d77999055861f3550a11d61cb895cb54fc9d78c56dd6d053093fa43cfaa2eed628effa3555ee4f7a2adc4d068dc4004b5cf

    Version 1+2 digest in base64 format: 6mp6J6y7lypsKA0Sy9Y/tgs3vGZr82jqWO2qx+CKQQY/oAqscgbhg90v3WTbAWNmaBgvwvFF9n9F9FiPMnQr/STPMTFGJ5KyKKEVIqDY1lYJvh1HqKHMxAmLZJHyvgt2nq/KJTgSv0CPST4FIQzA6As95JbcyFTpiXbEt3Y8ZVGz0XNmrJZGXLyqhDlQ3Z13mZBVhh81UKEdYcuJXLVPydeMVt1tBTCT+kPPqi7tYo7/o1Ve5PeircTQaNxABLXP

    valentin@computer:~$

The password used in this example is `my loooooooong password`.

The length of the digest v2 is ok for the latest GnuPG version (with the password length limitation). The v1 base64 digest also should work fine. But the 'tinfoil hat edition' (version 1+2) works only with GnuPG 1 (tested it with `gpg (GnuPG) 1.4.20`). (Update: on GnuPG 2.2.4 the limit is lifted to 495 characters, however it is still 255 bytes if option `--pinentry-mode loopback` is used; the limit is bigger with `--passphrase-file passphrase.txt --pinentry-mode loopback`.)

Limiting the CPU load to prevent overheating
====

On some old computers too much CPU load may lead to overheating and automatic shutdown of the computer.

The script `cpulimit.bash` is searching for a python script with `slowkdf` in the name. There is a limitation - it slows down only the first script with such a name.

I found a bug - it says "Stopped" but it continue to run and return the data to `stdout`: https://www.youtube.com/watch?v=LN_DjgREXjU

Testing scrypt modules for speed
====

```
$ ./scrypt_test.py
**** Testing with Scrypt from scrypt...
File of locally installed scrypt: /home/valentin/.local/lib/python3.6/site-packages/scrypt/__init__.py
Iteration 1 from 1...
*** Time:  4.310204744338989 **********************************************
Digest in base64 format: R/5/ga5KTwm8WIs9FquqJxxoIACiN8PGSTv9GvDrJO9OOhjEcf95kLJgucAXSWK2cGq+kAW3JqENKybD8tw3WOFnvPVAMOo/Zku/8kizjmYGEuhd/tOKg2hKgnnbUrsQTAZ9JYxWhR7flK7ygSmaNjZOXNObDNwAN0oXqFpbQ4k=

**** Testing with Scrypt from scrypt (old version)...
File of scrypt: /usr/lib/python3/dist-packages/scrypt.py
Iteration 1 from 1...
*** Time:  4.284672498703003 **********************************************
Digest in base64 format: R/5/ga5KTwm8WIs9FquqJxxoIACiN8PGSTv9GvDrJO9OOhjEcf95kLJgucAXSWK2cGq+kAW3JqENKybD8tw3WOFnvPVAMOo/Zku/8kizjmYGEuhd/tOKg2hKgnnbUrsQTAZ9JYxWhR7flK7ygSmaNjZOXNObDNwAN0oXqFpbQ4k=

**** Testing with Scrypt from Cryptodome...
Iteration 1 from 1...
*** Time:  5.6815948486328125 **********************************************
Digest in base64 format: R/5/ga5KTwm8WIs9FquqJxxoIACiN8PGSTv9GvDrJO9OOhjEcf95kLJgucAXSWK2cGq+kAW3JqENKybD8tw3WOFnvPVAMOo/Zku/8kizjmYGEuhd/tOKg2hKgnnbUrsQTAZ9JYxWhR7flK7ygSmaNjZOXNObDNwAN0oXqFpbQ4k=

**** Testing with Scrypt from hashlib...
Iteration 1 from 1...
*** Time:  4.702378988265991 **********************************************
Digest in base64 format: R/5/ga5KTwm8WIs9FquqJxxoIACiN8PGSTv9GvDrJO9OOhjEcf95kLJgucAXSWK2cGq+kAW3JqENKybD8tw3WOFnvPVAMOo/Zku/8kizjmYGEuhd/tOKg2hKgnnbUrsQTAZ9JYxWhR7flK7ygSmaNjZOXNObDNwAN0oXqFpbQ4k=

**** Testing with Scrypt from hazmat...
Iteration 1 from 1...
*** Time:  4.770408630371094 **********************************************
Digest in base64 format: R/5/ga5KTwm8WIs9FquqJxxoIACiN8PGSTv9GvDrJO9OOhjEcf95kLJgucAXSWK2cGq+kAW3JqENKybD8tw3WOFnvPVAMOo/Zku/8kizjmYGEuhd/tOKg2hKgnnbUrsQTAZ9JYxWhR7flK7ygSmaNjZOXNObDNwAN0oXqFpbQ4k=

```

Looks like the scrypt module from Ubuntu and the module installed with `pip3 install scrypt` are the best. My CPU is `Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz`.

Collisions
====

These two passwords produce the same scrypt digiest (I learned it from [here](https://mathiasbynens.be/notes/pbkdf2-hmac#comment-3)):

```PBKDF2-HMAC-SHA256-fail-affects-scrypt-no-security-issue-bGoDFpr8```

```;`B3nR6wQ2-_LSg"mH #yszm`[#z8B&L```

The scrypt digest is (in hex format):

`2bbb625cbb0201756397c93fdee9d902fe92c176c77e50ab5016b9db85dacfd7c5478820a2b8148f8f3790a7a12e63469eeda87459d68ea505826d0bd94c245570e39da1784327b2eed33b81f2da8ca850c2707139df81d52bc6610c07f663bec69d2009e416985def67f73e8a15dbce832bbd0d53dececa1f83a04c94927cce`

This is true for any salt, N, r, p.

However, versions 2 and 1+2 of the output from the `SlowKDF` script are different.

repository: vstoykovbg/slowkdf

repo_name: vstoykovbg/slowkdf


text:
---
title: How
published: true
---

## Trust issues (with software)

<img src="http://www.publicdomainpictures.net/pictures/80000/t2/silhouette-boy-with-hands-up.jpg" align=left width=300>
Put your hand up if you are worried about the
increasing use of of
software (and especially data mining software) to control, well, everything; e.g.

- Nobel-prize winning chemists use software  [to make their discoveries](http://goo.gl/Lwensc);
- Engineers use software to [designs many thing]( http://goo.gl/qBMyIZ)
  including optical tweezers, radiation therapy, remote sensing,  chip design;
- Web analysts use software  to analyze clickstreams to [improve sales and marketing strategies](http://goo.gl/b26CfY);
- Stock traders write software to [simulate trading strategies](http://www.quantopian.com);
- Analysts write software  to mine   labor statistics data to [review proposed gov policies](http://goo.gl/X4kgnc);
- Journalists use software   to analyze economic data, make visualizations [of their news stories](http://fivethirtyeight.com);
- In London or New York,  ambulances wait for your call at a location [determined by a software model](http://goo.gl/8SMd1p).
- Etc etc etc

In this century, much of what we can see and what we can do is selected and mediated and
controlled  and restricted by software.  This is a worry since
humans screw things up, [all the
time](https://en.wikipedia.org/wiki/List_of_cognitive_biases),
particualrly when [designing or building or using
software](http://seclists.org/risks/).
So how can we  trust  all that software?
How can
we tune it to our specific needs? How can we recognize when we should not use it?

## Trusting what we learn from data miners

In the general case of all softwware, I have little to say since "all software" is a very wide range of stuff.
But in the specific case of data mining software, there are very specific (and simple and general) things
we can do to increase the odds that person2 can critique and improve the models build my person1.

## What is Missing in Data Repositories?

Consider an operating system that can schedule processes, but has
no editors, compilers, file system, network support, ports, memory
management, gui, etc. Would you use that operating?

Now consider a data repository used by scientists stored data and
code in order to repeat/ confirm/ improve or even refute old results.
I think such repositories should do more than just support setting
and getting data, with (perhaps) some limited search interface. I
want my data repositories to be forums within which communitie can
curate, critique and even improve the conclusions drawn from data.

What would that extra layer of tools look like?
Well there are many possible answers and the following are just some.
Students of this subject should be very critical of the following
implementation ideas-- they are just one approach and they should
feel free to explore many more.

## The Elbow Test

A model passes the elbow test if, when displayed,
analysts are elbowed out of the way by
users in their haste to point to flaws on the display.

For this to work, models must be _readable_. Readable things are _NOT_ tediously long. There are many ways to reduce the size of a model:

- Unsuerpvised discretion: replace long list sof numbers
  with just a few `ranges`
### Unsupervised Discretization

Numeric ranges can be approxiated with a few  straight
line segments.

For example, suppose we sorted 100 numbers drawn from
`R<sup>2</sup>`
where `R` is a random number drawn from 0..1 so
`R<sup>2</sup>`  has a mean of about 0.25.

As shown below, the red dots mark the verticies os a set
of points which, if connected with straight lines,
approximate that `R<sup>2</sup>` with a very high
correlation.

![](img/unsuper.png)

These dots were found using the following rules

- Let the standard deviation of the whole population be _sd_;
- Let "epsilon" be _sd*0.5_ (which is one of [Cohen's rules](https://en.wikipedia.org/wiki/Effect_size#Cohen.27s_d))
     - Cohen defines small,medium,large to be sd*d_ where
        _d ∈ \in (0.2,0.5,1)_.
- Let "enough" numbers be the number of values, rasised to 0.5
     - (which is one of [Webb's rules](); another of which is
       _enough=30_).
- Let the max and min value of each range be _≥ epsilon_;
- Let the number of items in  a range be _≥ enough_.
- Let the max value of one range be less than the min value
  of the next.

### Supervised Discretization

Suppose we are watching variable one to understand its
effect on variable two. Suppose further:

- we have used unsupervised
discretization to find ranges in variable one
- we find that variable two is very similar in some
  of the ranges of variable one
- then we could safely combine variable two ranges

apply unsupervised discretization to remove spurious numerics
2.  apply supervised discretization to combine

These days,
anyone can be a scientist (making generalizations from data)  by
just downloading a data mining toolkit, and running it over some data.

Now science is not just a way to build models, it is a way to evaluate
those models as well. Much of my work as a "sciencetist" is reviewing
papers, proposals, and the ideas of my colleagues.
So I say
it should be possible to audit the conclusions generated
in that way.
I
want to  mistrust the conclusions of citizen scientists in
much the same way as I mustrust
ane explore , review, explore, evolve the conclusions of any other scientist.

These days, anyone can act like a scientist- grab a data miner and build a model from data. But
should we trust the recommendations that come from such models? There are many examples of
people using data mining to arrive at misleading
conclusions or using models the wrong way (see Ekrem13](http://menzies.us/pdf/13distributed.pdf),  [Crateri13](https://ocw.mit.edu/courses/engineering-systems-division/esd-864-modeling-and-assessment-for-policy-spring-2013/projects/student-work/MITESD_864S13_NASA_Colbia.pdf)).

Standard scientific
to generate bad models

Consider a table of data with rows and columns.
Let some rows be numeric and some symbolic

- Numebrs are things we can add and sort.
- Symbols are things we can only test for equal or not equal.

Some columns are independent variables and some are dependend

- Task: find combinations of the independent to predict values in the dependent.
- Prediction = there is onyl one dependent variable, and we have to guess what it might be:
   - Classification = one symbolic dependent variable.
   - Regression = one numeric depedent variable
- Optimization =  N numeric dependents and we see ways to maximize, minize those values
   - Multi-objective optimization = N > 1
   - Many-objective optimization = N > 5

Somehow, cluster the rows into a dendogram (a tree of clusters and sub-clusters, etc).

For each leaf cluster, build a predictor (e.g. majority class or medium score in that leaf).

Feature selection

Instance selection


text:
# schoolMIPS

A small MIPS CPU core originally based on Sarah L. Harris MIPS CPU ("Digital Design and Computer Arhitecture" by David Money Harris and Sarah L Harris). The first version of schoolMIPS was written for [Young Russian Chip Architects](http://www.silicon-russia.com/2017/06/09/arduino-and-fpga/) summer school.

Next version of schoolMIPS is based on RISC-V architecture: [schoolRISCV](https://github.com/zhelnio/schoolRISCV)

The CPU have several versions (from simple to complex). Each of them is placed in the separate git branch:
- **00_simple** - the simplest CPU without data memory, programs compiled with GNU gcc;
- **01_mmio** - the same but with data memory, simple system bus and peripherals (pwm, gpio, als);
- **01_mmio_sv_dpi** - the same but with data memory, simple system bus and peripherals (pwm, gpio, als);
- **02_irq** - data memory, system timer, interrupts and exceptions (CP0 coprocessor);
- **03_pipeline** - the pipelined version of the simplest core with data memory
- **04_pipeline_irq** - the pipelined version of 02_irq

Examples of using this kernel as an element of a multiprocessor system:
1. A.E. Ryazanova, A.A. Amerikanov, E. V Lezhnev, Development of multiprocessor system-on-chip based on soft processor cores schoolMIPS, J. Phys. Conf. Ser. 1163 (2019) 012026. [doi:10.1088/1742-6596/1163/1/012026](https://iopscience.iop.org/article/10.1088/1742-6596/1163/1/012026).
2. [Innovate FPGA 2019 project: EM028 » NoC-based multiprocessing system prototype](http://www.innovatefpga.com/cgi-bin/innovate/teams.pl?Id=EM028)

For docs and CPU diagrams please visit the project [wiki](https://github.com/MIPSfpga/schoolMIPS/wiki)

![CPU diagram](../../wiki/img/schoolMIPS_diagram.gif)

repository: MIPSfpga/schoolMIPS

gha_language: Verilog

repo_name: MIPSfpga/schoolMIPS


text:
---
title: "Search Engines and Its Types"
path: "/searchengines"
date: "2020-02-22"
author: "Atul Sharma"
excerpt: 'Ever given a thought how this search engines work what actually happens behind...'
tags: ["crawler", "directory","Search engines","google"]
---

## What is a Search Engine

The purpose of a search engine is to extract requested information from the huge database of resources available on the internet. Search engines become an important day to day tool for finding the required information without knowing where exactly it is stored.

Internet usage has been tremendously increased in recent days with the easy to use search engines like Google, Bing and Yahoo! In this article, we will see different types of search engines and purpose of them.

## Why Search Engines are Important

Search engines are part of daily life for two types of people.

- Users who search and get information
- Site owners who try to optimize their websites for getting top rank in the search results.

User do more than billions of searches only on Google to find relevant information. This opens out a huge scope for businesses and online content publishers to attract people to their website for free.

Search engines follow guidelines and have their own algorithm to decide the ranking of websites in search results.Optimizing websites for Google and other search engines is an essential part of any website owner for reaching out the large audience.

The visitors can generate revenue for site owners either through advertisements displayed on the site or though purchasing products.

---

## Different Types of Search Engines

Search engines are classified into the following three categories based on how it works.

1. Crawler based search engines
2. Human powered directories
3. Hybrid search engines
4. Other special search engines

### Crawler Based Search Engines

All crawler based search engines use a crawler or bot or spider for crawling and indexing new content to the search database. There are four basic steps, every crawler based search engines follow before displaying any sites in the search results.

- Crawling
- Indexing
- Calculating Relevancy
- Retrieving the Result

Search engines **crawl** the whole web to fetch the web pages available. A piece of software called *crawler* or *bot* or *spider*, performs the crawling of the entire web. The crawling frequency depends on the search engine and it may take few days between crawls.

Example of crawler based search engines:

- Google
- Bing
- Yahoo!
- Baidu
- Yandex

### Hybrid Search Engines

Hybrid Search Engines use both crawler based and manual indexing for listing the sites in search results. Most of the crawler based search engines like Google basically uses crawlers as a primary mechanism and human powered directories as secondary mechanism.

As human powered directories are disappearing, hybrid types are becoming more and more crawler based search engines.

In this though the crawlers control the processes, the control is manual to monitor and show the search results naturally.

## Wrapping Up

During earlier days of Internet, human powered search engines were popular and important source of information. The technological world is moving very fast towards automation and artificial intelligence.

Big technology companies like Google and Microsoft, who own the popular search engines, are driving this change and getting benefit out of it.

None of the human powered search engines could survive this fast shift. Though people uses special purpose search engines in very small scale, at this point of time artificial or crawler based search engines acts as a primary source of information with minimal human intervention.

url: https://raw.githubusercontent.com/yashrajmishra/JU-eezy-Blog/4c2b959caeb72546c00bd421d3593a7b9ca7bb10/src/posts/Search Engines and Its Types!!.md

repository: yashrajmishra/JU-eezy-Blog

filename: Search Engines and Its Types!!.md

repo_name: yashrajmishra/JU-eezy-Blog


text:
# Installation
> `npm install --save @types/react-google-recaptcha`

# Summary
This package contains type definitions for React Google Recaptcha (https://github.com/dozoisch/react-google-recaptcha).

# Details
Files were exported from https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/react-google-recaptcha.

### Additional Details
 * Last updated: Mon, 20 Jul 2020 15:15:30 GMT
 * Dependencies: [@types/react](https://npmjs.com/package/@types/react)
 * Global values: none

# Credits
These definitions were written by [Koala Human](https://github.com/KoalaHuman), [Tom Sturge](https://github.com/tomsturge), [Max Bo](https://github.com/MaxwellBo), [Meir Keller](https://github.com/meirkl), [Florian Roher](https://github.com/RohrerF), and [Timotei Mocan](https://github.com/mocantimoteidavid).

repository: schedyool/schedyool

gha_language: TypeScript

repo_name: schedyool/schedyool


text:
//[time-core](../../index.md)/[com.chrynan.time](index.md)/[firstYearWeek](first-year-week.md)

# firstYearWeek

[common]\

@[ExperimentalTime](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.time/-experimental-time/index.html)

fun [Year](-year/index.md).[firstYearWeek](first-year-week.md)(weekFormat: [WeekFormat](-week-format/index.md) = WeekFormat()): [Int](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-int/index.html)

Retrieves the first year week number of this [Year](-year/index.md) using the provided [weekFormat](first-year-week.md).

repository: chRyNaN/time

repo_name: chRyNaN/time


text:
---
layout: post
title: "두 변수를 교환하는 SWAP"
author: "Amy"
---

### SWAP 방식을 활용하여 무조건 큰 수에서 작은수를 빼기

{% highlight swift %}
var number1: Int = 5
var number2: Int = 10

if number1 < number2
{
    var temp = number2
    number2 = number1
    number1 = temp
}

print(number1 - number2)
{% endhighlight %}

### In-Out Parameter 이용하기

##### resources from [Apple Document](https://developer.apple.com/library/content/documentation/Swift/Conceptual/Swift_Programming_Language/Functions.html#//apple_ref/doc/uid/TP40014097-CH10-ID173)

- Function parameters are constants by default. Trying to change the value of a function parameter from within the body of that function results in a compile-time error. This means that you can’t change the value of a parameter by mistake. If you want a function to modify a parameter’s value, and you want those changes to persist after the function call has ended, define that parameter as an in-out parameter instead.

- You write an in-out parameter by placing the inout keyword right before a parameter’s type. An in-out parameter has a value that is passed in to the function, is modified by the function, and is passed back out of the function to replace the original value. For a detailed discussion of the behavior of in-out parameters and associated compiler optimizations, see In-Out Parameters.

- You can only pass a variable as the argument for an in-out parameter. You cannot pass a constant or a literal value as the argument, because constants and literals cannot be modified. **You place an ampersand (&) directly before a variable’s name when you pass it as an argument to an in-out parameter, to indicate that it can be modified by the function.**

{% highlight swift %}
func swapTwoInts(_ a: inout Int, _ b: inout Int) {
    let temporaryA = a
    a = b
    b = temporaryA
}
{% endhighlight %}

{% highlight swift %}
var someInt = 3
var anotherInt = 107
swapTwoInts(&someInt, &anotherInt)
{% endhighlight %}

gha_language: CSS


text:
# CAOS-Ray
A java-based Ray Tracer as an introductory open source project.

Our team is made up of college students of all experience levels - from beginning to experienced programmers.
Our goals are to:
1) Gain experience through group collaboration;
2) Improve our ability to work effectively with others;
3) Create a Ray Tracer.

repository: RileyLadd/CAOS-Ray

repo_name: RileyLadd/CAOS-Ray


text:
react-bootstrap-datetimepicker
===============================

This project is a port of https://github.com/Eonasdan/bootstrap-datetimepicker for React.js

Contributions
===============================
There's still plenty of features missing compared to the original date time picker, hence contributions would be highly appreciated.

Aknowledgments
===============================

I took https://github.com/stevoland/react-bootstrap as a reference to get the build actually building.


text:
# Physics benchmarks using the Gazebo Simulator

Documentation for physics benchmarks in the [gazebo simulator](http://gazebosim.org).
This folder contains documentation for several multibody benchmark problems
in iPython notebook format.
These documents can be viewed online in the following locations
or on your own machine following the instructions given below:

* [Boxes: free-floating rigid bodies](http://nbviewer.ipython.org/github/scpeters/benchmark/blob/master/boxes_description.ipynb)

![boxes benchmark animation: simple trajectory](img/boxes_5_simple.gif)
![boxes benchmark animation: complex trajectory](img/boxes_5_complex.gif)

These benchmarks can be run for the physics engines supported by gazebo:

* [Open Dynamics Engine (ODE)](http://ode.org), (source code on [bitbucket](https://bitbucket.org/odedevs/ode))
* [Bullet](http://bulletphysics.org), (source code on [github](https://github.com/bulletphysics/bullet3))
* [Simbody](https://simtk.org/home/simbody), (source code on [github](https://github.com/simbody/simbody))
* [Dynamic Animation and Robotics Toolkit (DART)](http://dartsim.github.io), (source code on [github](https://github.com/dartsim/dart))

The benchmarks tests are hosted in the
[scpeters/benchmark](https://github.com/scpeters/benchmark)
repository on github.
There are several plots and presentations in the
[pdf folder](https://github.com/scpeters/benchmark/tree/master/pdf),
including presentations from
[ROSCon 2014](https://github.com/scpeters/benchmark/blob/master/pdf/roscon2014_scpeters.pdf)
([video](https://vimeo.com/107517366))
and [Multibody 2015](https://github.com/scpeters/benchmark/blob/master/pdf/multibody2015_scpeters.pdf).

# Instructions

To run the tests,
[build or install gazebo 6 or later
with the optional physics engines](http://gazebosim.org/tutorials/?tut=install#OptionalPhysicsEngines),
and then clone and build the benchmarks:

~~~
git clone https://github.com/scpeters/benchmark
cd benchmark
mkdir build
cd build
cmake ..
make
make test
~~~

Once the tests are completed,
they will create time-stamped csv files in the `test_results` folder of the git repository.

To load and visualize the test results, you should make sure ipython notebook, matplotlib, and numpy are installed on your machine:
~~~
# Ubuntu Precise: do this step first
# Trusty uses ipython 1.2.1, use ppa on Precise to get the newer version
sudo apt-add-repository ppa:jtaylor/ipython

# Ubuntu Precise and Trusty
sudo apt-get install ipython-notebook python-matplotlib python-numpy

# Mac
brew install matplotlib
# do the following in specific terminals when using ipython notebook
export PYTHONPATH=/usr/local/lib/python2.7/site-packages
~~~

With the proper dependencies installed run ipython notebook from the
root of the git repository and open the
[boxes_results notebook](http://nbviewer.ipython.org/github/scpeters/benchmark/blob/master/boxes_results.ipynb):

~~~
ipython notebook
~~~

repository: scpeters/benchmark

repo_name: scpeters/benchmark


text:
# PGStandardsKeeper

This tool is meant for ETL developers, programmers, and just about anyone to keep track of any database but with extensive access to PostgreSQL.

## Why a Standards Tool

The standards tool uses the scalatra framework and postgreSQL to maintain records. This extends the information schema and aims to provide a graphical
structure to it as well in the long run. For now, all I need is standards documentation. Yes, this is based on my own needs.

PostgreSQL currently does not have a decent standards tracking tool and as far as I can tell, no database currently has a tool that allows for developers
to track changes and standards for their databases.

## Tracking Standards

The standards tracking tool keeps records in the following format. It is bare bones. You insert the fields. When a graphical view of the metadata is created,
the column and table names will be used to grab meta data information.

The user inserts the following record via graphical interface. These will be better organized as time goes on. I really just need a simple standards tracker at
the moment.

- table : The table ame to use
- column : The column to use
- type : The database type (e.g. integer, varchar, NUMBER)
- standard : A regular expression standards format (in case you want to do validation) or a description
- description : Data type description

## Setting Up the Project

The project requires some extra setup in the form of an XML file. The path must be set in the ClientConfig class before building the project for deployment on a server. This will be changed to a more standard folder later. The following file must be filled in.

```
<beans
xsi:schemaLocation="http://www.springframework.org/schema/beans   http://www.springframework.org/schema/beans/spring-beans-4.0.xsd   http://www.springframework.org/schema/util    http://www.springframework.org/schema/util/spring-util-4.0.xsd         http://www.springframework.org/schema/mvc         http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd   http://www.springframework.org/schema/context   http://www.springframework.org/schema/context/spring-context-4.0.xsd" default-lazy-init="false" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util" xmlns:p="http://www.springframework.org/schema/p" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:context="http://www.springframework.org/schema/context" xmlns="http://www.springframework.org/schema/beans">
     <context:annotation-config/>
     <context:component-scan base-package="org.aevans.simplrtek.StandardsAppClient"/>
     <bean id="props" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
          <property name="locations">
               <list>
        		    <value>file:~/Production.properties</value>
               </list>
          </property>
     </bean>
     <bean id="dataSource" class="org.apache.commons.dbcp2.BasicDataSource">
          <property value="org.postgresql.Driver" name="driverClassName"/>
          <property value="${jdburl}" name="url"/>
          <property value="${jdbuser}" name="username"/>
          <property value="${jdbpass}" name="password"/>
          <property value="10" name="initialSize"/>
          <property value="true" name="defaultAutoCommit"/>
          <property value="50" name="maxTotal"/>
          <property value="5" name="minIdle"/>
          <property value="Select 1" name="validationQuery"/>

     </bean>
     <bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
          <property ref="dataSource" name="dataSource"/>
     </bean>
     <!--  DAO Mapper Goes Here-->
     <bean id="DatabaseHandler" class="org.aevans.simplrtek.jdbc.DatabaseHandler" init-method="initDbs">
     	<property name="table" value="fields.dbfields"/>
		<property name="keyTable" value="fields.keys"/>
		<property name="indexTable" value="fields.indices"/>
     </bean>
     <bean id="standardsAppClient" class="org.aevans.simplrtek.standards.StandardsAppClient">
     </bean>

</beans>
```

The database properties in the dataSource bean should be set appropriately. This example uses a properties file.


text:
# Codher Purely Functional Interfaces in Elm: Code Examples

## Running Examples

Make sure you have the latest stable version of [Node.js](https://nodejs.org/en/download/current/) installed.

```sh
npm install create-elm-app -g
```

Run [elm-reactor](https://github.com/elm-lang/elm-reactor) with Create Elm App:

```
elm-app reactor
```

repository: Codher/elm-workshop

repo_name: Codher/elm-workshop


text:
# influxdb-client-ruby

[![CircleCI](https://circleci.com/gh/influxdata/influxdb-client-ruby.svg?style=svg)](https://circleci.com/gh/influxdata/influxdb-client-ruby)
[![codecov](https://codecov.io/gh/influxdata/influxdb-client-ruby/branch/master/graph/badge.svg)](https://codecov.io/gh/influxdata/influxdb-client-ruby)
[![Gem Version](https://badge.fury.io/rb/influxdb-client.svg)](https://badge.fury.io/rb/influxdb-client)
[![License](https://img.shields.io/github/license/influxdata/influxdb-client-ruby.svg)](https://github.com/influxdata/influxdb-client-ruby/blob/master/LICENSE)
[![GitHub issues](https://img.shields.io/github/issues-raw/influxdata/influxdb-client-ruby.svg)](https://github.com/influxdata/influxdb-client-ruby/issues)
[![GitHub pull requests](https://img.shields.io/github/issues-pr-raw/influxdata/influxdb-client-ruby.svg)](https://github.com/influxdata/influxdb-client-ruby/pulls)
[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://www.influxdata.com/slack)

This repository contains the reference Ruby client for the InfluxDB 2.x.

#### Note: Use this client library with InfluxDB 2.x and InfluxDB 1.8+ ([see details](#influxdb-18-api-compatibility)). For connecting to InfluxDB 1.7 or earlier instances, use the [influxdb-ruby](https://github.com/influxdata/influxdb-ruby) client library.

- [Features](#features)
- [Installation](#installation)
    - [Install the Gem](#install-the-gem)
- [Usage](#usage)
    - [Creating a client](#creating-a-client)
    - [Writing data](#writing-data)
    - [Querying data](#queries)
    - [Delete data](#delete-data)
    - [Management API](#management-api)
- [Advanced Usage](#advanced-usage)
    - [Default Tags](#default-tags)
    - [Proxy configuration](#proxy-configuration)
- [Contributing](#contributing)
- [License](#license)

## Documentation

This section contains links to the client library documentation.

* [Product documentation](https://docs.influxdata.com/influxdb/latest/api-guide/client-libraries/), [Getting Started](#installation)
* [Examples](examples)
* [API Reference](https://influxdata.github.io/influxdb-client-ruby/InfluxDB2.html)
* [Changelog](CHANGELOG.md)

## Features

InfluxDB 2.x client consists of two packages

- `influxdb-client`
    - Querying data using the Flux language
    - Writing data
        - batched in chunks on background
        - automatic retries on write failures
- `influxdb-client-apis`
    - provides all other InfluxDB 2.x APIs for managing
        - buckets
        - labels
        - authorizations
        - ...
    - built on top of `influxdb-client`

## Installation

The InfluxDB 2 client is bundled as a gem and is hosted on [Rubygems](https://rubygems.org/gems/influxdb-client).

### Install the Gem

The client can be installed manually or with bundler.

To install the client gem manually:

```
gem install influxdb-client -v 2.9.0
```

For management API:

```
gem install influxdb-client-apis -v 2.9.0
```

## Usage

### Creating a client

Use **InfluxDB::Client** to create a client connected to a running InfluxDB 2 instance.

```ruby
client = InfluxDB2::Client.new('https://localhost:8086', 'my-token')

client.do_something

client.close!
```

the **InfluxDB::Client** can be also used as a resource:

```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token') do |client|
  client.do_something
end
```

#### Client Options

| Option                         | Description                                                                        | Type                                                       | Default  |
|--------------------------------|------------------------------------------------------------------------------------|------------------------------------------------------------|----------|
| bucket                         | Default destination bucket for writes                                              | String                                                     | none     |
| org                            | Default organization bucket for writes                                             | String                                                     | none     |
| precision                      | Default precision for the unix timestamps within the body line-protocol            | String                                                     | none     |
| open_timeout                   | Number of seconds to wait for the connection to open                               | Integer                                                    | 10       |
| write_timeout                  | Number of seconds to wait for one block of data to be written                      | Integer                                                    | 10       |
| read_timeout                   | Number of seconds to wait for one block of data to be read                         | Integer                                                    | 10       |
| max_redirect_count             | Maximal number of followed HTTP redirects                                          | Integer                                                    | 10       |
| redirect_forward_authorization | Pass Authorization header to different domain during HTTP redirect.                | bool                                                       | false    |
| use_ssl                        | Turn on/off SSL for HTTP communication                                             | bool                                                       | true     |
| verify_mode                    | Sets the flags for the certification verification at beginning of SSL/TLS session. | `OpenSSL::SSL::VERIFY_NONE` or `OpenSSL::SSL::VERIFY_PEER` | none     |
| logger                         | Logger used for logging. Disable logging by set to false.                          | Logger                                                     | `STDOUT` |
| debugging                      | Enable debugging for HTTP request/response.                                        | bool                                                       | false    |

```ruby
client = InfluxDB2::Client.new('https://localhost:8086', 'my-token',
  bucket: 'my-bucket',
  org: 'my-org',
  precision: InfluxDB2::WritePrecision::NANOSECOND)
```

### Queries

The result retrieved by [QueryApi](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/query_api.rb) could be formatted as a:

   1. Raw query response
   2. Flux data structure: [FluxTable, FluxColumn and FluxRecord](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/flux_table.rb)
   3. Stream of [FluxRecord](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/flux_table.rb)

#### Query raw

Synchronously executes the Flux query and return result as unprocessed String
```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token', org: 'my-org') do |client|

  result = client
    .create_query_api
    .query_raw(query: 'from(bucket:"my-bucket") |> range(start: 1970-01-01) |> last()')
  puts result
end
```
#### Synchronous query
Synchronously executes the Flux query and return result as a Array of [FluxTables](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/flux_table.rb)
```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token', org: 'my-org') do |client|

  result = client
    .create_query_api
    .query(query: 'from(bucket:"my-bucket") |> range(start: 1970-01-01) |> last()')

  result.each do |table|
    table.records.each { |record| puts record.values }
  end
end
```

#### Query stream
Synchronously executes the Flux query and return stream of [FluxRecord](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/flux_table.rb)
```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token', org: 'my-org') do |client|

  stream = client
    .create_query_api
    .query_stream(query: 'from(bucket:"my-bucket") |> range(start: 1970-01-01) |> last()')

  stream.each do |record|
    puts record.values
  end
end
```

#### Parameterized queries
InfluxDB Cloud supports [Parameterized Queries](https://docs.influxdata.com/influxdb/cloud/query-data/parameterized-queries/)
that let you dynamically change values in a query using the InfluxDB API. Parameterized queries make Flux queries more
reusable and can also be used to help prevent injection attacks.

InfluxDB Cloud inserts the params object into the Flux query as a Flux record named `params`. Use dot or bracket
notation to access parameters in the `params` record in your Flux query. Parameterized Flux queries support only `int`
, `float`, and `string` data types. To convert the supported data types into
other [Flux basic data types, use Flux type conversion functions](https://docs.influxdata.com/influxdb/cloud/query-data/parameterized-queries/#supported-parameter-data-types).

Parameterized query example:
> :warning: Parameterized Queries are supported only in InfluxDB Cloud, currently there is no support in InfluxDB OSS.

```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token', org: 'my-org') do |client|

    query = 'from(bucket: params.bucketParam) |> range(start: duration(v: params.startParam))'
    params = { 'bucketParam' => 'my-bucket', 'startParam' => '-1h' }

    query_api = client.create_query_api
    result = query_api.query(query: query, params: params)

    result[0].records.each { |record| puts "#{record.time} #{record.measurement}: #{record.field} #{record.value}" }
end
```

### Writing data
The [WriteApi](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/write_api.rb) supports synchronous and batching writes into InfluxDB 2.x. In default api uses synchronous write. To enable batching you can use WriteOption.

```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND) do |client|

  write_api = client.create_write_api
  write_api.write(data: 'h2o,location=west value=33i 15')
end
```

#### Batching
The writes are processed in batches which are configurable by `WriteOptions`:

| Property                 | Description                                                                                | Default Value |
|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|
| batchSize                | the number of data point to collect in batch                                                                                | 1_000         |
| flush_interval           | the number of milliseconds before the batch is written                                                                                | 1_000         |
| retry_interval           | the number of milliseconds to retry unsuccessful write. The retry interval is used when the InfluxDB server does not specify "Retry-After" header.                                                                                | 5_000         |
| jitter_interval          | the number of milliseconds to increase the batch flush interval by a random amount                                                                                | 0             |
| max_retries              | the number of max retries when write fails                                                                                | 5             |
| max_retry_delay          | maximum delay when retrying write in milliseconds                                                                                | 125_000       |
| max_retry_time           | maximum total retry timeout in milliseconds                                                                                | 180_000       |
| exponential_base         | the base for the exponential retry delay, the next delay is computed using random exponential backoff as a random value within the interval  ``retry_interval * exponential_base^(attempts-1)`` and ``retry_interval * exponential_base^(attempts)``. Example for ``retry_interval=5000, exponential_base=2, max_retry_delay=125000, total=5`` Retry delays are random distributed values within the ranges of ``[5000-10000, 10000-20000, 20000-40000, 40000-80000, 80000-125000]`` | 2             |
| batch_abort_on_exception | the batching worker will be aborted after failed retry strategy                                                                                | false         |
```ruby
InfluxDB2::Client.use('http://localhost:8086',
                      'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND,
                      use_ssl: false) do |client|

  write_options = InfluxDB2::WriteOptions.new(write_type: InfluxDB2::WriteType::BATCHING,
                                              batch_size: 10, flush_interval: 5_000,
                                              max_retries: 3, max_retry_delay: 15_000,
                                              exponential_base: 2)

  write_api = client.create_write_api(write_options: write_options)
  write_api.write(data: 'h2o,location=west value=33i 15')
end
```

#### Time precision

Configure default time precision:
```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND) do |client|

  client.do_something
end
```

Configure precision per write:
```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND) do |client|

  write_api = client.create_write_api
  write_api.write(data: 'h2o,location=west value=33i 15', precision: InfluxDB2::WritePrecision::SECOND)
end
```
Allowed values for precision are:
- `InfluxDB2::WritePrecision::NANOSECOND` for nanosecond
- `InfluxDB2::WritePrecision::MICROSECOND` for microsecond
- `InfluxDB2::WritePrecision::MILLISECOND` for millisecond
- `InfluxDB2::WritePrecision::SECOND` for second

#### Configure destination

Default `bucket` and `organization` destination are configured via `InfluxDB::Client`:
```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org') do |client|

  client.do_something
end
```

but there is also possibility to override configuration per write:

```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token') do |client|

  write_api = client.create_write_api
  write_api.write(data: 'h2o,location=west value=33i 15', bucket: 'production-data', org: 'customer-1')
end
```

#### Data format

The data could be written as:

1. `String` that is formatted as a InfluxDB's line protocol
1. `Hash` with keys: name, tags, fields and time
1. [Data Point](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/point.rb#L28) structure
1. `Array` of above items

```ruby
InfluxDB2::Client.use('https://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND) do |client|

  point = InfluxDB2::Point.new(name: 'h2o')
                          .add_tag('location', 'europe')
                          .add_field('level', 2)

  hash = { name: 'h2o',
           tags: { host: 'aws', region: 'us' },
           fields: { level: 5, saturation: '99%' }, time: 123 }

  write_api = client.create_write_api
  write_api.write(data: ['h2o,location=west value=33i 15', point, hash])
end
```

#### Default Tags

Sometimes is useful to store same information in every measurement e.g. `hostname`, `location`, `customer`.
The client is able to use static value, app settings or env variable as a tag value.

The expressions:
- `California Miner` - static value
- `${env.hostname}` - environment property

##### Via API

```ruby
InfluxDB2::Client.use('http://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND,
                      use_ssl: false,
                      tags: { id: '132-987-655' }) do |client|

  point_settings = InfluxDB2::PointSettings.new(default_tags: { customer: 'California Miner' })
  point_settings.add_default_tag('data_center', '${env.data_center}')

  write_api = client.create_write_api(write_options: InfluxDB2::SYNCHRONOUS,
                                      point_settings: point_settings)

  write_api.write(data: InfluxDB2::Point.new(name: 'h2o')
                                        .add_tag('location', 'europe')
                                        .add_field('level', 2))
end
```

### Delete data

The [DeleteApi](https://github.com/influxdata/influxdb-client-ruby/blob/master/lib/influxdb2/client/delete_api.rb) supports deletes [points](https://docs.influxdata.com/influxdb/latest/reference/key-concepts/data-elements/#point) from an InfluxDB bucket.

```ruby
InfluxDB2::Client.use('http://localhost:8086', 'my-token',
                      bucket: 'my-bucket',
                      org: 'my-org',
                      precision: InfluxDB2::WritePrecision::NANOSECOND) do |client|

  client.create_delete_api.delete(DateTime.rfc3339('2019-02-03T04:05:06+07:00'),
                                  DateTime.rfc3339('2019-03-03T04:05:06+07:00'),
                                  predicate: 'key1="value1" AND key2="value"')
end
```

The time range could be specified as:

1. String - `"2019-02-03T04:05:06+07:00"`
1. DateTime - `DateTime.rfc3339('2019-03-03T04:05:06+07:00')`
1. Time - `Time.utc(2015, 10, 16, 8, 20, 15)`

### Management API

The client supports following management API:

|                                                                                | API docs                                                              |
|-----------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------|
| [**AuthorizationsApi**](https://influxdata.github.io/influxdb-client-ruby/InfluxDB2/API/AuthorizationsApi.html) | https://docs.influxdata.com/influxdb/latest/api/#tag/Authorizations   |
| [**BucketsApi**](https://influxdata.github.io/influxdb-client-ruby/InfluxDB2/API/BucketsApi.html)               | https://docs.influxdata.com/influxdb/latest/api/#tag/Buckets          |
| [**LabelsApi**](https://influxdata.github.io/influxdb-client-ruby/InfluxDB2/API/LabelsApi.html)                 | https://docs.influxdata.com/influxdb/latest/api/#tag/Labels           |
| [**OrganizationsApi**](https://influxdata.github.io/influxdb-client-ruby/InfluxDB2/API/OrganizationsApi.html)   | https://docs.influxdata.com/influxdb/latest/api/#tag/Organizations    |
| [**UsersApi**](https://influxdata.github.io/influxdb-client-ruby/InfluxDB2/API/UsersApi.html)                   | https://docs.influxdata.com/influxdb/latest/api/#tag/Users            |

The following example demonstrates how to use a InfluxDB 2.x Management API to create new bucket. For further information see docs and [examples](/examples).

```ruby
#
# This is an example how to create new bucket with permission to write.
#
# You could run example via: `cd apis && bundle exec ruby ../examples/create_new_bucket.rb`
#
$LOAD_PATH.unshift File.expand_path('../lib', __dir__)
require 'influxdb-client'
$LOAD_PATH.unshift File.expand_path('../apis/lib', __dir__)
require 'influxdb-client-apis'

url = 'http://localhost:8086'
bucket = 'my-bucket'
org = 'my-org'
token = 'my-token'

InfluxDB2::Client.use(url,
                      token,
                      bucket: bucket,
                      org: org,
                      use_ssl: false,
                      precision: InfluxDB2::WritePrecision::NANOSECOND) do |client|

  api = InfluxDB2::API::Client.new(client)

  # Find my organization
  organization = api.create_organizations_api
                    .get_orgs
                    .orgs
                    .select { |it| it.name == 'my-org' }
                    .first

  #
  # Create new Bucket
  #
  retention_rule = InfluxDB2::API::RetentionRule.new(type: 'expire', every_seconds: 3600)
  bucket_name = 'new-bucket-name'
  request = InfluxDB2::API::PostBucketRequest.new(org_id: organization.id,
                                                  name: bucket_name,
                                                  retention_rules: [retention_rule])
  bucket = api.create_buckets_api
              .post_buckets(request)

  #
  # Create Permission to read/write from Bucket
  #
  resource = InfluxDB2::API::Resource.new(type: 'buckets',
                                          id: bucket.id,
                                          org_id: organization.id)
  authorization = InfluxDB2::API::Authorization.new(description: "Authorization to read/write bucket: #{bucket.name}",
                                                    org_id: organization.id,
                                                    permissions: [
                                                      InfluxDB2::API::Permission.new(action: 'read', resource: resource),
                                                      InfluxDB2::API::Permission.new(action: 'write', resource: resource)
                                                    ])
  result = api.create_authorizations_api
              .post_authorizations(authorization)

  print("The token: '#{result.token}' is authorized to read/write from/to bucket: '#{bucket.name}'.")
end

```
- sources - [create_new_bucket.rb](/examples/create_new_bucket.rb)

## Advanced Usage

### Check the server status

Server availability can be checked using the `client.ping` method. That is equivalent of the [influx ping](https://docs.influxdata.com/influxdb/latest/reference/cli/influx/ping/).

### Proxy configuration

You can configure the client to tunnel requests through an HTTP proxy. To configure the proxy use a `http_proxy` environment variable.

```ruby
ENV['HTTP_PROXY'] = 'http://my-user:my-password@my-proxy:8099'
```

Client automatically follows HTTP redirects. The default redirect policy is to follow up to 10 consecutive requests.
You can configure redirect counts by the client property: `max_redirect_count`.

Due to a security reason `Authorization` header is not forwarded when redirect leads to a different domain.
To overcome this limitation you have to set the client property `redirect_forward_authorization` to `true`.

### InfluxDB 1.8 API compatibility

[InfluxDB 1.8.0 introduced forward compatibility APIs](https://docs.influxdata.com/influxdb/v1.8/tools/api/#influxdb-2-0-api-compatibility-endpoints) for InfluxDB 2.x. This allow you to easily move from InfluxDB 1.x to InfluxDB 2.x Cloud or open source.

The following forward compatible APIs are available:

| API                                                 | Endpoint                                                                                | Description                                                                                |
|:----------------------------------------------------|:---------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [query_api.rb](lib/influxdb2/client/query_api.rb)   | [/api/v2/query](https://docs.influxdata.com/influxdb/latest/tools/api/#api-v2-query-http-endpoint) | Query data in InfluxDB 1.8.0+ using the InfluxDB 2.x API and [Flux](https://docs.influxdata.com/flux/latest/) _(endpoint should be enabled by [`flux-enabled` option](https://docs.influxdata.com/influxdb/latest/administration/config/#flux-enabled-false))_ |
| [write_api.rb](lib/influxdb2/client/write_api.rb)   | [/api/v2/write](https://docs.influxdata.com/influxdb/latest/tools/api/#api-v2-write-http-endpoint) | Write data to InfluxDB 1.8.0+ using the InfluxDB 2.x API                                                                                |
| [health_api.rb](lib/influxdb2/client/health_api.rb) | [/health](https://docs.influxdata.com/influxdb/latest/tools/api/#health-http-endpoint)             | Check the health of your InfluxDB instance                                                                                |

For detail info see [InfluxDB 1.8 example](examples/influxdb_18_example.rb).

## Local tests

```
brew install wget # on a mac, if not yet installed!
bin/influxdb-restart.sh
rake test
```

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/influxdata/influxdb-client-ruby.

## License

The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

repository: influxdata/influxdb-client-ruby

gha_language: Ruby

repo_name: influxdata/influxdb-client-ruby