text:
  Uber macOS •
  Mad lab •
  TITAN •
  FeSpinner

  iOS Awesome Starter Kit •
  FeSlideFilter •
  Responsive Interaction Control

Uber macOS
------------

![](https://i1.wp.com/nghiatran.me/wp-content/uploads/2017/07/uber_screenshot.jpg?w=1700&ssl=1)

Obviously, you've already known what it's, right? 🤣

⚠ Uber macOS's stilling on **sandbox** mode. It involves to be reviewed and approved by Uber before live on production.

Keep in touch with me at Homepage: https://nghiatran.me/uber-macos/

![](https://img.shields.io/badge/Swift-3.0-blue.svg?style=flat)
![License](https://img.shields.io/npm/l/express.svg?style=flat)
![Platform](https://img.shields.io/badge/platform-ios-green.svg?style=flat)
[![Build Status](https://travis-ci.com/NghiaTranUIT/Unofficial-Uber-macOS.svg?token=UeyzvGcPRtnd5oLvpQsY&branch=master)](https://travis-ci.com/NghiaTranUIT/Unofficial-Uber-macOS)

Roadmap
------------

- [x] Solid Foundation: macOS app and core framework
- [x] MVVM Swift
- [x] RxSwift
- [x] Uber functionalities
- [ ] Support fully Uber features
- [ ] 100% converage test
- [ ] Uber review

Contributor
------------
It would be greatly appreciated when you make a pull-quest  🤗

License
------------
Uber macOS is available under the MIT license. See the LICENSE file for more info.


text:
---
layout: default
---

# 1998-10-09 - RE: GPL & commercial software, the critical distinction (fwd)

## Header Data

From: Matthew James Gering<EMAIL_ADDRESS>To: "'Steve Mynott'"<EMAIL_ADDRESS>Message Hash: f571ea1fd97257f2b405ba44ce3595ec71293b1d4405a83c06d57be1745f646c

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1998-10-09 10:00:10 UTC

Raw Date: Fri, 9 Oct 1998 18:00:10 +0800

## Raw message

```
{% raw  %}From: Matthew James Gering<EMAIL_ADDRESS>Date: Fri, 9 Oct 1998 18:00:10 +0800
To: "'Steve Mynott'"<EMAIL_ADDRESS>Subject: RE: GPL & commercial software, the critical distinction (fwd)
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Not really, the state holds a monopoly on "legal" force, whereas the
Mafia and other black-market organizations do not. What is interesting
is they use force often because they cannot use the government legal
system to arbitrate their disputes.

	Matt

> -----Original Message-----
> From: Steve Mynott<EMAIL_ADDRESS>
> On Thu, Oct 08, 1998 at 04:14:32PM -0700, Matthew James Gering wrote:
> >
> > The Mafia has had two primary characteristics:
> >
> > a) the use of force
>
> wouldn't this characteristic define the mafia as a "state" to some
> degree?

{% endraw %}
```

## Thread

+ Return to [October 1998](/archive/1998/10)

+ Return to "["James A. Donald" <jamesd@echeque.com>](/authors/james_a_donald_jamesd_at_echeque_com_)"
+ Return to "[Matthew James Gering <mgering@ecosystems.net>](/authors/matthew_james_gering_mgering_at_ecosystems_net_)"

+ 1998-10-09 (Fri, 9 Oct 1998 18:00:10 +0800) - RE: GPL & commercial software, the critical distinction (fwd) - _Matthew James Gering<EMAIL_ADDRESS>  + 1998-10-10 (Sun, 11 Oct 1998 07:05:59 +0800) - [RE: GPL & commercial software, the critical distinction (fwd)](/archive/1998/10/1b8209b8f534b4f1b72406dd91507dc1809aaab586ec82fd218167fcf0c84cfe) - _"James A. Donald"<EMAIL_ADDRESS>

license: MIT

branch_name: refs/heads/master

extension: md

gha_license_id: MIT

license_type: permissive


text:
### Patch urllib3

```shell
2019-10-06 20:09:05,575 /usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py:_make_request:396 - DEBUG - https://localhost:8082 "GET /delay/1?page=0 HTTP/1.1" 200 302
```

```shell
2019-10-06 21:40:07,861 /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:_make_request:400 - DEBUG - https://localhost:8082 "GET /delay/1?page=2 HTTP/1.1" 200 302
```

```shell
vim /usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py
vim /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py
# Find where request is being sent
# Add debugging statement
```

repository: saurabh-hirani/slides

branch_name: refs/heads/gh-pages

gha_language: JavaScript

repo_name: saurabh-hirani/slides


text:
## Skk-vue
**skk-vue** is a project for learning vue. So it's not surprised if you find some similar or completely same code. The library implement most of functions that belong to vue including **__ bind methods、bind data twoway、computed data、customize component  __** and so on.

## Install
Recommended Install
```
  npm install skk-vue
```

## How Can I use
The HTML structure looks like this:
``` HTML
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <style>
        * {
            margin: 0;
            padding: 0;
        }
    </style>
</head>
<body>
<div id="container">
    <div id="my-app">
        <button type="button" v-on:click="changeName">click</button>
        <button type="button" v-on:click="getName">get</button>
        {{name}}
        <span v-text="name"></span>
        <input type="text" v-bind:placeholder="placeholder"
               v-model="temp"
        >

        <div v-html="myTemp"></div>
    </div>
</div>
<script src="../lib/vue.js"></script>
<script src="skk-vue.js"></script>
<script>
  var demo = new Sue({
        el: 'my-app',
        data () {
            return {
                placeholder: '请输入数字',
                myTemp: '<i>this is kaikai</i><button type="button">click</button>',
                name: 'kaikai',
                temp: 123,
                sname: {
                    a: '123',
                    b: '456'
                },
                sage: 22,
                arr: [1,2,3,4]
            }
        },
        methods: {
            changeName (e) {
                this.temp += 10;
            },
            getName (e) {
                console.log(this.temp);
            },
            changeAge (age) {
                this.sage = age;
            }
        }
    })
    demo.name = 'jiejie';
    console.log(demo);
</script>
</body>
</html>
```

## Support Directives
|dir|usage|
|---|---|
|v-html|`<div v-html="myTemplate"></div>`|
|v-text|`<div v-text="myText"></div>`|
|v-model|`<input v-model="myTemplate">`|
|v-bind|`<div v-bind:id="myId"></div>`|
|v-on|`<div v-on:click="myClick"></div>`|
|v-show|`<div v-show="show"></div>`|

## v-for
You can use it like following:
``` HTML
  <ul>
      <li v-for="(val, valIndex) of textArr">
          {{valIndex}}-----{{val}}
      </li>
  </ul>
```

## Support Coumputed
You can use computed property just as simple as vue, like below:

**Note:** don't use allow function in the computed property, which may cause unexpected errors
``` javascript
  var demo = new Sue({
      data: function() {
        return {
          firstName: 'xiao',
          lastName: 'ming'
        }
      }
      computed: {
        fullname: function() {
          return this.firstName + this.lastName;
        }
      }
    })
    console.log(demo.fullname) // xiao
```
## Customize filter
function Sue.filter(name, fn)

parameter: name {string} - the name of filter

parameter: fn {function} - a function that processes data and achieve data as the first argument.

If calling Sue.filter() successfully which will return Sue. So you can chain call filter like following:
``` javascript
  Sue.filter('toUpper', (val) => {
      return val.toUpperCase();
  }).filter('addMoreStr', (val) => {
      return val + '123456'
  });
```
## Customize Component
Support custom component :flags: You can use props just like vue easily.

The following is the js code:
``` javascript
  const FirstComponent = Sue.extend({
      template: '<div><strong>{{firstname}} 23</strong><hr><strong>{{lastname}} 45</strong></div>',
      data () {
          return {
              name: 'jiejie123',
              age: 22
          }
      },
      props: {
          firstname: '',
          lastname: ''
      },
      created () {
          this.$on('getName', function(name) {
              console.log('I get the data' + name);
          })
          setTimeout(() => {
              this.$emit('getAge', 333);
          }, 1000);
      },
      mounted () {
          console.log('This is component mounted hook');
      },
  })
  Sue.component('first-component', FirstComponent);
```
Here is the html code:
``` HTML
  <first-component v-bind:firstname="firstName" lastname="hong"></first-component>
```
## Hook Function
When data is all ready, a function named created will be called.

When dom is all ready and mounted, a function named mounted will be called.

``` javascript
  created () {
    console.log('This is created')
  }
  mounted () {
    console.log('This is mounted')
  }
```
## Event
How to register a event?
``` javascript
  // vm is a instance of the Sue
  vm.$on('name', (arg1, arg2) => {
    console.log('This is name event');

    // If you want to stop bubble of broadcast, you can return false;
    return false;
  })
```
How to broadcast a event to all children?
``` javascript
  // The first argument is the event name needed to call and the rest arguments
  // will pass to callback function.
  vm.$broadcast('name', 'arg1', 'arg2', ...);
```
How to emit a event to parent?
``` javascript
  // The first argument is the event name needed to call and the rest arguments
  // will pass to callback function.
  vm.$emit('name', 'arg1', 'arg2', ...);
```

## Use Tips
The library is just for learning and at the stage of experiment. So I can percent determine the library that lacks of **complete Compatibility and Function tests** must have many bugs that are strange and unexpected.Before you insist on using it, I hope you can realize what you will face. :chicken:

## License
MIT license

[http://www.opensource.org/licenses/MIT](http://www.opensource.org/licenses/MIT)

repository: softkaikai/skk-vue

repo_name: softkaikai/skk-vue


text:
### [CVE-2002-0970](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2002-0970)
![](https://img.shields.io/static/v1?label=Product&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=n%2Fa&color=brighgreen)

### Description

The SSL capability for Konqueror in KDE 3.0.2 and earlier does not verify the Basic Constraints for an intermediate CA-signed certificate, which allows remote attackers to spoof the certificates of trusted sites via a man-in-the-middle attack.

### POC

#### Reference
No PoCs from references.

#### Github
- https://github.com/ARPSyndicate/cvemon
- https://github.com/chnzzh/OpenSSL-CVE-lib

repository: trickest/cve

branch_name: refs/heads/main

gha_language: HTML

repo_name: trickest/cve


text:
---
title:  "Hacker News app in Elm"
date:   2017-05-23
categories: [programming,elm]
tags: [elm,hn]
---
About a year ago I put together a little [Hacker News][hn] reader in [Elm][elm]. I've now moved it to this blog for posterity and ease of accessibility.

[http://hn.codeninja.blog](http://hn.codeninja.blog)

The [sources][sources] are freely available on GitHub. If you are new to [Elm][elm] I also put together a little introduction to the Elm Architecture in the [README][readme]. It starts from "hello world" and drills down into the guts so you can really understand what's going on when making your own apps.

Enjoy!

[hn]:      http://news.ycombinator.com
[elm]:     http://elm-lang.org
[sources]: https://github.com/massung/elm-hn
[readme]:  https://github.com/massung/elm-hn#introduction-to-the-elm-architecture

repository: massung/codeninja

extension: markdown

repo_name: massung/codeninja


text:
The exponentiation algorithms in this section are based on performing exponentiation by means of repeated multiplication. In a similar way, one can perform integer multiplication by means of repeated addition. The following multiplication procedure (in which it is assumed that our language can only add, not multiply) is analogous to the expt procedure:

```scheme
(define (* a b)
  (if (= b 0)
      0
      (+ a (* a (- b 1)))))
```

This algorithm takes a number of steps that is linear in b. Now suppose we include, together with addition, operations double, which doubles an integer, and halve, which divides an (even) integer by 2. Using these, design a multiplication procedure analogous to fast-expt that uses a logarithmic number of steps.

```scheme
(define (* a b)
  (if (= b 0)
      0
      (+ a (* a (- b 1)))))

(define (even? n)
  (= (remainder n 2) 0))

(define (double a)
    (+ a a))

(define (halve x)
   (/ x 2))

(define (multi a b)
    (cond ((= b 1) a)
          ((even? b) (multi (double a) (halve b)))
          (else (+ a (multi a (- b 1))))))
```

repository: kenspirit/SICP

repo_name: kenspirit/SICP


text:
# Testing the Wi-Fi Jamming Attack

1. Establish your test environment:
   - Create secure wireless network with a unique SSID.
   - Setup network tracking or logging capabilities to collect and analyze information.
   - Set different levels of access for different users and/or roles.

2. Deploy your wireless network and begin tracking traffic:
   - Provide access to all authorized users and install appropriate security protocols to protect the network from unauthorized access.
   - Monitor the network and log all wireless traffic, noting the SSIDs of all access points seen by the network.

3. Use an attacker tool to test your security and detect potential SSID Tracking attacks:
   - Utilize an attack tool like [Aircrack-ng](https://www.aircrack-ng.org/) to simulate an attacker attempting to connect to the wireless network.
   - Use the attack tool to flood the network with SSID requests, and analyze the logs to see if any of them contain the unique SSID of the network.

4. Analyze results and adjust security accordingly:
   - If the SSID appears in the logs, the attack was successful and your security isn't sufficient to prevent tracking.
   - Adjust network security measures to ensure that unauthorized users cannot access the network and its resources.

## Testing Tools:

| Target Testing | Testing Technique | Test Analysis | Test Method | Test Tool | Mobile Platform |
| ------------- | ---------------- | ------------ | ---------- | -------- | --------------- |
| Wi-Fi SSID Tracking | White-box | Dynamic | Boundary Analysis | Qualys Network Inspector | iOS, Android, Windows |
| Wi-Fi SSID Tracking | Grey-box | Static | Source Code Analysis | Veracode | Android, Windows |
| Wi-Fi SSID Tracking | Black-box | Hybrid | Penetration Testing | Burp Suite | iOS, Android, Windows |

repository: SECURIoTESIGN/SECURIoTESIGN

repo_name: SECURIoTESIGN/SECURIoTESIGN


text:
# go-ipfs-keystore

> go-ipfs-keystore implements keystores for ipfs
[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://protocol.ai)
[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
[![Travis CI](https://travis-ci.com/ipfs/go-ipfs-keystore.svg?branch=master)](https://travis-ci.com/ipfs/go-ipfs-keystore)
[![Go Reference](https://pkg.go.dev/badge/github.com/ipfs/go-ipfs-keystore.svg)](https://pkg.go.dev/github.com/ipfs/go-ipfs-keystore)

## ❗ This repo is no longer maintained.
👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/keystore.
🏎 Good news!  There is [tooling and documentation](https://github.com/ipfs/boxo#migrating-to-boxo) to expedite a switch in your repo.

⚠ If you continue using this repo, please note that security fixes will not be provided (unless someone steps in to maintain it).

📚 Learn more, including how to take the maintainership mantle or ask questions, [here](https://github.com/ipfs/boxo/wiki/Copied-or-Migrated-Repos-FAQ).

go-ipfs-keystore provides the Keystore interface for key management.  Keystores support adding, retrieving, and deleting keys as well as listing all keys and checking for membership.

## Table of Contents

- [Install](#install)
- [Usage](#usage)
- [License](#license)

## Install

`go-ipfs-keystore` works like a regular Go module:
```
> go get github.com/ipfs/go-ipfs-keystore
```

## Usage
```
import "github.com/ipfs/go-ipfs-keystore"
```

## License

This project is dual-licensed under Apache 2.0 and MIT terms:

- Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)

repository: ipfs/go-ipfs-keystore

gha_language: Go

gha_license_id: NOASSERTION

repo_name: ipfs/go-ipfs-keystore


text:
# How to use Yandex.Metrika goals for generate analytics

Self-notes for reuse later.

See [index.md](src/index.md) for details.

### Uses
- Hastie, static file generator
  https://mkaz.github.io/hastie/
- Bulma, the modern CSS framework that just works.
  https://bulma.io/documentation/overview/start/
- Task - a task runner / build tool, quite simple and VERY fast
  https://taskfile.dev


text:
# Nix System Configuration

[![Build Status](https://api.cirrus-ci.com/github/kclejeune/system.svg?branch=master)](https://cirrus-ci.com/github/kclejeune/system)

This repository manages system configurations for all of my
macOS, nixOS, and linux machines.

## Structure

This repository is a [flake](https://nixos.wiki/wiki/Flakes). All system configurations are defined
in [flake.nix](./flake.nix). Platorm specific configurations are found defined in the flake outputs
`darwinConfigurations`, `nixosConfigurations` for macOS and NixOS respectively.

### Overlapping Nix-Darwin and NixOS

Nix-Darwin and NixOS configurations share as much overlap as possible in the common module, [./modules/common.nix](./modules/common.nix).
Platform specific modules add onto the common module in [./modules/darwin/default.nix](./modules/darwin/default.nix) and [./modules/nixos/default.nix](./modules/nixos/default.nix) for macOS and NixOS respectively.

### Decoupled Home Manager Configuration

My home-manager configuration is entirely decoupled from NixOS and nix-darwin configurations.
This means that all of its modules are found in [./modules/home-manager](./modules/home-manager).
These modules are imported into all other configurations in the common module similarly to this:

```nix
{ config, pkgs, ... }: {
  home-manager.users.kclejeune = import ./home-manager/home.nix;
}
```

This means that [home.nix](./modules/home-manager/home.nix) is fully compatible as a standalone configuration, managed with the `home-manager` CLI.
This allows close replication of any user config for any linux system running nix. These configurations are defined in the `homeConfigurations` output.

### User Customization

User "profiles" are specified in [./profiles](./profiles); these modules configure
contextual, identity-specific settings such as SSL certificates or work vs. personal email addresses.
When possible, home-manager functionality is extracted into [./profiles/home-manager](./profiles/home-manager), as mentioned previously

## Installing a Configuration

### Non-NixOS Prerequisite: Install Nix Package Manager

Run the installer script to perform a multi-user installation on darwin or linux:

```bash
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```

Note that this step is naturally skipped on NixOS since `nix` is the package manager by default.

## System Bootstrapping

### NixOS

Follow the installation instructions, then run

```bash
sudo nixos-install --flake github:kclejeune/system#phil
```

### Darwin/Linux

Clone this repository into `~/.nixpkgs` with

```bash
git clone https://github.com/kclejeune/system ~/.nixpkgs
```

You can bootstrap a new system with the `bootstrap` command:

```bash
nix run .#sysdo bootstrap
```

This will attempt to detect the host system and install nix-darwin or home-manager, but this behavior can be overridden using the `--darwin` or `--home-manager` flags.

## `sysdo` CLI

The `sysdo` utility is a python script that wraps `nix`, `darwin-rebuild`, `nixos-rebuild`,
and `home-manager` commands to provide a consistent interface across multiple platforms. It has some dependencies which are defined in the `devShell`
flake output. Documentation for this tool is found in [sysdo.md](./docs/sysdo.md).

repository: kclejeune/system

gha_language: Nix

repo_name: kclejeune/system


text:
#LGame (formal name : Loon)

源自中國本土的項目

International Entertainment Machines

#This Java implemented game framework(but will provide a cross-compiler contain C#,C/C++,Go,JavaScript or other programming language implemented),Support IOS\Android\WP7\PSM\Mac\Windows\Linux.

Built-in over 30 game example(Part screenshots):

![LGame](https://raw.github.com/cping/LGame/master/e0x.png "0")

![LGame](https://raw.github.com/cping/LGame/master/e1x.png "1")

![LGame](https://raw.github.com/cping/LGame/master/e2x.png "2")

![LGame](https://raw.github.com/cping/LGame/master/e3x.png "3")

![LGame](https://raw.github.com/cping/LGame/master/e4x.png "4")

License : http://www.apache.org/licenses/LICENSE-2.0

20130619:

Updated to version 0.4.0

Start rewriting JVM ......


text:
# zepto.js - modular branch #

This is a fork of [Thomas Fuchs](http://mir.aculo.us/) minimalistic JavaScript framework [zepto.js](http://github.com/madrobby/zepto).

It was based on a *early stage* of the framework but **branched into a completely different direction**, the core structure was totally refactored. **It may have extra/missing features** and/or **different API** than the original. **The code structure/implementations are diverging from the original at each day.**

To know more about why this branch was created and what should come next see this [pull request](http://github.com/madrobby/zepto/pull/5).

The main idea is to keep the API as close as possible to jQuery without bloating the code size but focusing on the most important/used methods, features will be added as needed and will try to unit test 100% of the API.

**Some changes implemented on this branch may be added to the main one but probably most of them won't**.

## Important ##

**This branch favors good practices and readability over file-size**, specially since modern compressors/minifiers does a good job at reducing code size, for more details read: [The Performance Dogma](http://blog.millermedeiros.com/2010/10/the-performance-dogma/).

## Targets ##

Modern mobile browsers.

## Syntax & features ##

### Basic call ###

    $('some css selector').html('set contents').css('set styles');

### Element functions ###

 - `.each(fn)` - Execute a function for each matched element.
 - `.get()` - Return array of all elements found.
 - `.get(0)` - Return first element found.
 - `.eq(1)` - Reduce matched elements to a single el0ement by index.
 - `.find('selector')` - Match descendant element(s) based on selector.
 - `.html('new html')` - Set the contents of the element(s).
 - `.css('css properties')` - Set styles of the element(s).
 - `.append`, `prepend`, `after`, `before` - Like `html`, but append/prepend to element contents or before/after element.
 - `.transition(transform, opacity, duration, delay, ease)` - Use -webkit-transform/opacity and do an animation.
 - `.delegate(selector, eventType, handler)` - Attach an Event listener to matched elements using the event delegation pattern.
 - `.hasClass(className)` - Check if any matched element has given class.
 - `.addClass(className)` - Add one or more classes (separated by spaces) into each matched element.
 - `.removeClass(className)` - Removes one or more class names (separated by spaces) from matched elements, removes all classes if `null`.
 - `.toggleClass(className, switch)` - Add or remove one or more classes from each element in the set of matched elements.
 - `.add(elementsArray)` - Add a set of elements to the matched elements set.
 - `.map(callbackFn(i, element))` - Pass each element in the current matched set through a function, producing a new zepto object containing the return values.
 - `.attr(name, value)` - Set/get attribute value.

### Helpers ###

 - `$.isDef(param)` - Check if parameter is different than `undefined`.
 - `$.isFunction(param)` - Check if parameter is a `Function`.
 - `$.isArray(param)` - Check if parameter is an `Array`.
 - `$.unique(array)` - Return a new array that contain only unique items.
 - `$.makeArray(obj)` - Convert an array-like Object into an real array.
 - `$.extend(firstObj, secondObj)` - Mixin, copy properties from one object into another, will extend `zepto` by default if second parameter is omitted.
 - `$.map(array, callbackFn)` - Translate all items in an array or array-like object to another array of items. (similar to similar to `jQuery.map` and not to `Array.prototype.map`)
 - `$.each(collection, callbackFn(i|key, value))` - Call method over each array/object item.
 - `$.noop` - Empty function.
 - `$.trim(str)` - Remove white spaces from begining and end of string.
 - `$.ready(fn)` - Specify a method to be called after DOM is "ready" (fully loaded).

### Event Handlers ###

#### bind/unbind ####

 - `.bind(eventType, eventHandler)` - Attach event listener onto each matched element.
 - `.unbind(eventType, eventHandler)` - Remove event listener from each matched element, *(both parameters are required)*.

##### example #####

    var handler = function(evt){
        $(this).unbind('touchstart', handler); //remove listener after first call, so it will be called only once per matched element
        alert("yay!");
    };

    $('#my_div').bind('touchstart', handler); //add listener

#### delegate ("live" event) ####

`.delegate(selector, eventType, eventHandler)` - Listen for events triggered by descendant nodes of matched elements - A.K.A "event delegation".

    //will trigger when any descendant element of "#my_div" that has the class ".touchable" is touched
    $('#my_div').delegate('.touchable', 'touchstart', function(evt){
        alert("I'm touched!");
    });

### Plugins ###

#### swipe ####

Custom "swipe" gesture event. (only works for touch enabled devices)

 - `.swipe(directionStr, endCallbackFn, [thresholdObj], [preventScroll])`
  - directionStr: 'up', 'down', 'left', 'right' - direction of swipe gesture.
  - endCallbackFn({direction: string, changeX: number, changeY: number}) - called on touch end if gesture matches threshold.
  - thresholdObj : {x: number, y:number} : defines how accurate the swipe gesture needs to be.
  - preventScroll : boolean - prevent browser scroll.

### Ajax: ###

 - `$.ajax(method, url, succesCallback, errorCallback)`
 - `$.get(url, succesCallback, errorCallback)`
 - `$.post(url, succesCallback, errorCallback)`
 - `$.getJSON(url, succesCallback, errorCallback)`

## Notes ##

### Future improvements ###

 - Add sniffing based on feature detection, maybe only load *zepto.min.js* and subsequent files if browser support required features.

### Recommendations ###

 - Make sure the site still works without JavaScript since many devices doesn't support it! **Use it as an enhancement and not as a prerequisite.**
 - Compress all your JavaScript files before deploying.

### Unit Test ###

 - **Tests may be created before features are implemented**, so don't get scared if a few of them throws errors telling that method doesn't exist (unless method is described on the README file).

repository: millermedeiros/zepto

repo_name: millermedeiros/zepto


text:
---
description: Metadata search results that contain summarized information about an asset.
seo-description: Metadata search results that contain summarized information about an asset.
seo-title: AssetSummary
solution: Experience Manager
title: AssetSummary
topic: Scene7 Image Production System API
uuid: 0ac8f900-c16c-409d-b83c-3bdf0ad28fac
---

# AssetSummary{#assetsummary}

Metadata search results that contain summarized information about an asset.

 Syntax

## Parameters {#section-ebc75cc024d94c439260d2e71873cc74}

|  Name  | Type  | Description  |
|---|---|---|
|  ` *`assetHandle`*`  | `xsd:string`  | Asset handle.  |
|  ` *`type`*`  | `xsd:string`  | Asset type. The "Asset Types" constant defines the possible values. Optional.  |
|  ` *`name`*`  | `xsd:string`  | Asset name. Optional.  |
|  ` *`folder`*`  | `xsd:string`  | The folder that contains the asset.  |
|  ` *`filename`*`  | `xsd:string`  | Asset's file name.  |
|  ` *`created`*`  | `xsd:dateTime`  | Asset creation date.  |
|  ` *`createUser`*`  | `xsd:string`  | The user who created the asset.  |
|  ` *`lastModified`*`  | `xsd:dateTime`  | The date that the asset was last updated.  |
|  ` *`lastModifyUser`*`  | `xsd:string`  | The last user who modified the asset.  |
|  ` *`metadataArray`*`  | `types:MetadataArray`  | Array of metadata values associated with the asset.  |
|  ` *`score`*`  | `xsd:double`  | Defines the precision in case of a similarity search (0 = no match, 1 = exact match).  |
|  ` *`scoreDetail`*`  | `xsd:string`  | Holds detailed information about similar areas as a result of a similarity search.  |


text:
## Diagnostics
### Ln 1, Col 2
```marko
> 1 | <button id="test"/>
    |  ^^^^^^ Fix any of the following:
  Document has active elements with the same id attribute: test
  2 | <button id="test"/>
  3 |
  4 | <!-- Should not break because these are not focusable -->
```

repository: marko-js/language-server

gha_language: TypeScript

repo_name: marko-js/language-server


text:
# About

rmHarmony is a drawing app for the remarkable tablet

![](https://i.imgur.com/KJlWdAAl.png)
![](https://i.imgur.com/KJ4rReu.gifv)

## Status

Approaching first release. Stay tuned!

### Implemented

* Tilt + pressure sensitive brushes
* Procedural brushes
* Drawing in 3 colors: white, gray & black
* Saving + loading pngs

## Installation

### Try it out

run `wget -O- https://rmkit.dev/try.sh -q | sh -` on the remarkable via SSH

To exit harmony, press the power button to bring up the exit dialog.

### Install

run `wget -O- https://rmkit.dev/install.sh -q | sh -` on the remarkable via SSH
to install the harmony launcher until the next time the device is updated

to launch harmony, hold the center button for 2+ seconds. To exit harmony,
press the power button to bring up the exit dialog.

### Manual Installation

* download or build the binaries for harmony (see docs/BUILDING.md)
* copy `harmony` to /home/root/harmony/harmony on the remarkable
* launch `/home/root/harmony/harmony` through SSH

## Compilation

see docs/BUILDING.md

## License

MIT except where noted otherwise

## acknowledgements

* [fontawesome](https://fontawesome.com) for their icons
* [harmony](https://github.com/mrdoob/harmony) for the idea of procedural brushes
* [libremarkable](https://github.com/canselcik/libremarkable) for reverse engineering remarkable IO
* [stb](https://github.com/nothings/stb) for image resizing and font rendering libraries

repository: Evidlo/rmkit

repo_name: Evidlo/rmkit


text:
Kerberos
========

The `kerberos` package is a C++ extension that requires a build environment to be installed on your system. You must be able to build node.js itself to be able to compile and install the `kerberos` module. Furthermore the `kerberos` module requires the MIT Kerberos package to correctly compile on UNIX operating systems. Consult your UNIX operation system package manager what libraries to install.

{{% note class="important" %}}
Windows already contains the SSPI API used for Kerberos authentication. However you will need to install a full compiler tool chain using visual studio C++ to correctly install the kerberos extension.
{{% /note %}}

### Diagnosing on UNIX

If you don’t have the build essentials it won’t build. In the case of linux you will need gcc and g++, node.js with all the headers and python. The easiest way to figure out what’s missing is by trying to build the kerberos project. You can do this by performing the following steps.

```
git clone https://github.com/christkv/kerberos.git
cd kerberos
npm install
```

If all the steps complete you have the right toolchain installed. If you get node-gyp not found you need to install it globally by doing.

```
npm install -g node-gyp
```

If correctly compiles and runs the tests you are golden. We can now try to install the kerberos module by performing the following command.

```
cd yourproject
npm install kerberos --save
```

If it still fails the next step is to examine the npm log. Rerun the command but in this case in verbose mode.

```
npm --loglevel verbose install kerberos
```

This will print out all the steps npm is performing while trying to install the module.

### Diagnosing on Windows

A known compiler tool chain known to work for compiling `kerberos` on windows is the following.

* Visual Studio c++ 2010 (do not use higher versions)
* Windows 7 64bit SDK
* Python 2.7 or higher

Open visual studio command prompt. Ensure node.exe is in your path and install node-gyp.

```
npm install -g node-gyp
```

Next you will have to build the project manually to test it. Use any tool you use with git and grab the repo.

```
git clone https://github.com/christkv/kerberos.git
cd kerberos
npm install
node-gyp rebuild
```

This should rebuild the driver successfully if you have everything set up correctly.

### Other possible issues

Your python installation might be hosed making gyp break. I always recommend that you test your deployment environment first by trying to build node itself on the server in question as this should unearth any issues with broken packages (and there are a lot of broken packages out there).

Another thing is to ensure your user has write permission to wherever the node modules are being installed.

repository: neurom-iot/naide

repo_name: neurom-iot/naide


text:
+++
title = "National And Regional Google Analytics Is Back"

date = 2017-10-20T00:00:00
lastmod = 2017-10-20T00:00:00
draft = false

# Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`.
authors = []

tags = ["seasonality", "Google", "concerts", "benchmarking", "bestweek", "ticket-sales"]
summary = "Google Trends data combined with regular, programmatic data extraction and surveys it is an extremely powerful tool to provide timely forecasts and to give your surveys an extra granularity in time, or down to city level."

# Projects (optional).
#   Otherwise, set `projects = []`.
# projects = ["Marketing"]

# Featured image
# To use, add an image named `featured.jpg/png` to your project's folder.
[image]
  # Caption (optional)
  caption = "Concert Interest Seasonality in the CEE"

  # Focal point (optional)
  # Options: Smart, Center, TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight
  focal_point = ""

  # Show image only in page previews?
  preview_only = false

+++

An important aspect of the relatively week CEE markets is the very strong seasonality aspect.  Ljubjlana, Zagreb, Budapest and Prague are full with life and concerting opportunities in December, but the difference in interest for concerts among peak months and low month can be as much as 40%, about the double of the all-year Viennese market.  Summer festivals obviously alter the picture, but it is essential for tour organizers as well as concert promoters to understand seasonality and to build longer tour seasons in these countries.  Cultural spending is not necessarily low in off-peak season but may be diverted to cinema, theatres or outdoor activities.

We use R program code to retrieve concert, cinema, theatre and other relevant search information from web searches and Youtube.  We produce data that is comparable across the whole CEE region. We make time-series, forecasting, benchmarking and other data analytics applications on the data. The main use is forecasting, because tickets sales are known when searched for and not after sales are reported. The granularity of the data can help to understand seasonality of months, weeks, best time for touring, for cinema scheduling.

Our [Google Trend Analytics](https://trends.google.com/trends/) is included in our private and public reports.  Google Trends is an interesting toy if you use it interactively.  Combined with regular, programmatic data extraction and surveys it is an extremely powerful tool to provide timely forecasts and to give your surveys an extra granularity in time, or down to city level.  For example, [combining seasonality analytics](http://www.ceemid.eu/music/comparing-the-demography-of-live-audiences/) with demographic analysis can explain most of the difficulties of market development in the Czech Republic, Croatia, Hungary, Poland or Slovakia.


text:
# covid19-daily-visualization-preprocessor

This code is just some simple node script(s) for preparing data for our [Covid-19 visualization dashboard](https://observablehq.com/@benracine/covid-19-cumulative-deaths-per-thousand-us-counties).

Some basics:
- This repo can be fetched with git or downloaded directly.
- Running the script is achieved atm using `node ./index.js -f data/datatest01.csv`. I'll make this more sophisticated soon.


text:
---
layout: doc_page
---

# About Druid

Druid is an open-source analytics data store designed for real-time exploratory queries on large-scale data sets (trillions of events, petabytes of data). Druid provides cost-effective and always-on realtime data ingestion and arbitrary data exploration.

-   Try out Druid with our Getting Started [Tutorial](./Tutorial%3A-A-First-Look-at-Druid.html)
-   Learn more by reading the [White Paper](http://static.druid.io/docs/druid.pdf)

Key Features
------------

-   **Designed for Analytics** - Druid is built for exploratory analytics for OLAP workflows. It supports a variety of filters, aggregators and query types and provides a framework for plugging in new functionality. Users have leveraged Druid’s infrastructure to develop features such as top K queries and histograms.
-   **Interactive Queries** - Druid’s low-latency data ingestion architecture allows events to be queried milliseconds after they are created. Druid’s query latency is optimized by reading and scanning only exactly what is needed. Aggregate and filter on data without sitting around waiting for results.
-   **Highly Available** - Druid is used to back SaaS implementations that need to be up all the time. Your data is still available and queryable during system updates. Scale up or down without data loss.
-   **Scalable** - Existing Druid deployments handle billions of events and terabytes of data per day. Druid is designed to be petabyte scale.

Why Druid?
----------

Druid was originally created to resolve query latency issues seen with trying to use Hadoop to power an interactive service. It's especially useful if you are summarizing your data sets and then querying the summarizations. Put your summarizations into Druid and get quick queryability out of a system that you can be confident will scale up as your data volumes increase. Deployments have scaled up to 2TB of data per hour at peak ingested and aggregated in real-time.

Druid is a system that you can set up in your organization next to Hadoop. It provides the ability to access your data in an interactive slice-and-dice fashion. It trades off some query flexibility and takes over the storage format in order to provide the speed.

We have more details about the general design of the system and why you might want to use it in our [White Paper](http://static.druid.io/docs/druid.pdf) or in our [Design](Design.html) doc.

When Druid?
----------

* You need to do interactive, fast, exploration on large amounts of data
* You need analytics (not a key-value store)
* You have a lot of data (10s of billions of events added per day, 10s of TB of data added per day)
* You want to do your analysis on data as it’s happening (in real-time)
* You need a data store that is always available, 24x7x365, and years into the future.

Druid vs…
----------

* [Druid-vs-Impala-or-Shark](Druid-vs-Impala-or-Shark.html)
* [Druid-vs-Redshift](Druid-vs-Redshift.html)
* [Druid-vs-Vertica](Druid-vs-Vertica.html)
* [Druid-vs-Cassandra](Druid-vs-Cassandra.html)
* [Druid-vs-Hadoop](Druid-vs-Hadoop.html)
* [Druid-vs-Spark](Druid-vs-Spark.html)
* [Druid-vs-Elasticsearch](Druid-vs-Elasticsearch.html)

About This Page
----------
The data infrastructure world is vast, confusing and constantly in flux. This page is meant to help potential evaluators decide whether Druid is a good fit for the problem one needs to solve. If anything about it is incorrect please provide that feedback on the mailing list or via some other means so we can fix it.

repository: msprunck/druid

repo_name: msprunck/druid


text:
# Duckweed Tasks - demo app

Demo app for the [Duckweed](https://github.com/foxbunny/duckweed) microframework.

[![Duckweed badge](src/shared/duckweed-badge.svg)](https://github.com/foxbunny/duckweed)

## Getting started

Install yarn globally:

```shell
npm i -g yarn
```

Init the project:

```shell
yarn
```

## Commands

- `yarn` - install all development dependencies
- `yarn start` - start the webpack dev server on port 8080
- `yarn run build` - to perform a production build
- `yarn run build:profile` - to perform a production build and analyze it
- `yarn test` - run tests
- `yarn run test:watch` - run tests in watch mode

## License

Copyright (c) 2017 Hajime Yamasaki Vukelic

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

repository: foxbunny/duckweed-tasks

repo_name: foxbunny/duckweed-tasks


text:
[中文文档](https://github.com/xiyuan-fengyu/ppspider/blob/master/README.md)

<!-- toc -->

- [Quick Start](#quick-start)
  * [Install NodeJs](#install-nodejs)
  * [Install TypeScript](#install-typescript)
  * [Prepare the development environment](#prepare-the-development-environment)
  * [Download And Run ppspider_example](#download-and-run-ppspider_example)
    + [Clone ppspider_example with IDEA](#clone-ppspider_example-with-idea)
    + [Install npm dependencies](#install-npm-dependencies)
    + [Run tsc](#run-tsc)
    + [Startup ppspider App](#startup-ppspider-app)
- [Deploy with docker](#deploy-with-docker)
- [Examples](#examples)
- [ppspider System Introduction](#ppspider-system-introduction)
  * [Decorator](#decorator)
    + [@Launcher](#launcher)
    + [@OnStart](#onstart)
    + [@OnTime](#ontime)
    + [@AddToQueue @FromQueue](#addtoqueue-fromqueue)
    + [@JobOverride](#joboverride)
    + [@Serializable @Transient](#serializable-transient)
    + [@RequestMapping](#requestmapping)
    + [@Bean @Autowired](#bean-autowired)
    + [@DataUi @DataUiRequest](#dataui-datauirequest)
  * [PuppeteerUtil](#puppeteerutil)
    + [PuppeteerUtil.defaultViewPort](#puppeteerutildefaultviewport)
    + [PuppeteerUtil.addJquery](#puppeteerutiladdjquery)
    + [PuppeteerUtil.jsonp](#puppeteerutiljsonp)
    + [PuppeteerUtil.setImgLoad](#puppeteerutilsetimgload)
    + [PuppeteerUtil.onResponse](#puppeteerutilonresponse)
    + [PuppeteerUtil.onceResponse](#puppeteerutilonceresponse)
    + [PuppeteerUtil.downloadImg](#puppeteerutildownloadimg)
    + [PuppeteerUtil.links](#puppeteerutillinks)
    + [PuppeteerUtil.count](#puppeteerutilcount)
    + [PuppeteerUtil.specifyIdByJquery](#puppeteerutilspecifyidbyjquery)
    + [PuppeteerUtil.scrollToBottom](#puppeteerutilscrolltobottom)
    + [PuppeteerUtil.parseCookies](#puppeteerutilparsecookies)
    + [PuppeteerUtil.useProxy](#puppeteerutiluseproxy)
    + [PuppeteerUtil example](#puppeteerutil-example)
  * [NetworkTracing](#networktracing)
  * [Database](#database)
  * [logger](#logger)
- [Debug](#debug)
- [Related Information](#related-information)
  * [jQuery](#jquery)
  * [puppeteer](#puppeteer)
  * [nedb](#nedb)
  * [mongodb](#mongodb)
  * [Angular](#angular)
  * [G2](#g2)
  * [bootstrap](#bootstrap)
- [WebUI](#webui)
- [Question](#question)
- [Update Note](#update-note)

<!-- tocstop -->

# Quick Start
## Install NodeJs
http://nodejs.cn/download/

## Install TypeScript
```
npm install -g typescript
```

## Prepare the development environment
Recommended IDEA(Ultimate version)
![Ultimate 版本截图](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/1.png)

Nodejs And Javascript Configuration in IDEA
![nodejs-config.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/2.png)
![javascript-config.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/3.png)

## Download And Run ppspider_example
ppspider_example github address
https://github.com/xiyuan-fengyu/ppspider_example
### Clone ppspider_example with IDEA
Warning: git is required and the executable file path of git should be set in IDEA
![IDEA git 配置](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/4.png)
![IDEA clone from git](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/5.png)
![IDEA clone from git](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/6.png)

### Install npm dependencies
Click "Terminal" on the bottom side of IDEA to open a terminal and run the following command
```
npm install
```

### Run tsc
Run 'tsc' in terminal
Or
ContextMenu on package.json -> Show npm Scripts -> Double click 'auto build'
tsc is a TypeScript compiler which can auto compile the ts file to js file after any ts file change

### Startup ppspider App
Run lib/quickstart/App.js
Open http://localhost:9000 in the browser to check the ppspider's status

# Deploy with docker
https://github.com/xiyuan-fengyu/ppspider_docker_deploy/blob/master/README.en.md

# Examples
1. Monitor website access speed, visualize real-time statistics, and view the details of all requests in the process of opening a web page [ppspider-webMonitor](https://github.com/xiyuan-fengyu/ppspider-webMonitor)
2. Dynamically set task parallel via cron expressions [DynamicParallelApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/DynamicParallelApp.ts)
3. If you wants some queues to not be executed immediately after the application is started, look this [QueueWaitToRunApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/QueueWaitToRunApp.ts)
4. Web page screenshot, super long page is supported [ScreenshotApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/dataUi/ScreenshotApp.ts)
5. Get video's infos and comments from https://www.bilibili.com [BilibiliApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/BilibiliApp.ts)
6. Set proxy for a page [Page Proxy](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/ProxyTest.ts)
7. QQ music info and comments [QQ Music](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/QqMusicApp.ts)
8. request + cheerio, crawling static web pages [CheerioApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/CheerioApp.ts)
9. Twitter get topic comments and user info [TwitterApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/TwitterApp.ts)
10. Depth-First-Search，DFS (By default, DefaultQueue is breadth-first search) [DepthFirstSearchApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/DepthFirstSearchApp.ts)
11. request + cheerio, crawling static web pages [QuotesToScrapeApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/QuotesToScrapeApp.ts)
12. [Music 163](https://music.163.com) download [Music163App](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/Music163App.ts)
    ![网易云音乐爬取结果](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/7.png)
13. [ProxyPool](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/ProxyPoolApp.ts)
14. [Bandcamp Music download](https://github.com/banbuchunqiu/ppspider_bandcamp)
15. Set max try num of job [SetMaxTryApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/SetMaxTryApp.ts)
16. Page.evaluate with async function [PuppeteerEvalAsyncApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/PuppeteerEvalAsyncApp.ts)
17. Set AddToQueue/FromQueue name with a regexp string to create a series of queues dynamically [AddToRegexQueue](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/AddToRegexQueue.ts)
18. Drag to complete the puzzle [PuppeteerUtil.dragJigsaw](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/DragJigsawTest1.ts)
19. Solve the problem that some requests work abnormally [HandlBadRequestOnHeadlessApp](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/examples/HandlBadRequestOnHeadlessApp.ts)

# ppspider System Introduction
## Decorator
Declare like
```
export function TheDecoratorName(args) { ... }
```
Usage
```
@TheDecoratorName(args)
```
Decorator is similar of look with java Annotation, but Decorator is stronger.  Decorator can provide meta data by parameters and modify the target or descriptor to change behavior of class or method
In ppspider, many abilities are provided by Decorator

### @Launcher
```
export function Launcher(theAppInfo: AppInfo) { ... }
```
Launcher of a ppspider app
Params type
```
export type AppConfig = {
    // all cache files and the db file will be saved to the workplace folder. You can save the data files to this folder too.
    workplace: string;

    // file path to save the running status，default is workplace + "/queueCache.json"
    queueCache?: string;

    // database url, nedb or mongodb is supported. When the app will generate a little amount of data, use nedb, the url format: nedb://nedbDirPath; Otherwise, mongodb is recommended, the url format: mongodb://username:password@host:port/dbName. The default value is "nedb://" + appInfo.workplace + "/nedb"
    dbUrl?: string;

    // import all task class
    tasks: any[];

    // import all DataUi class
    dataUis?: any[];

    // all workerFactory instances, only PuppeteerWorkerFactory, NoneWorkerFactory is provided at present
    workerFactorys: WorkerFactory<any>[];

    // the port for web ui，default 9000
    webUiPort?: number | 9000;

    // logger setting
    logger?: LoggerSetting;
}
```

### @OnStart
```
export function OnStart(config: OnStartConfig)
```
A job executed once after app start, but you can execute it again by pressing the button in webUI. The button will be found after the queue name in webUI's Queue panel.

Params type
```
export type OnStartConfig = {
    // urls to crawl
    urls: string | string[];

     // if set true, this queue will not run after startup
    running?: boolean;

    // config of max paralle num, can be a number or a object with cron key and number value
    parallel?: ParallelConfig;

    // the execute interval between jobs, all paralle jobs share the same exeInterval
    exeInterval?: number;

    // make a random delta to exeInterval
    exeIntervalJitter?: number;

    // Task timeout, in milliseconds, default: 300000, negative number means never timeout
    timeout?: number;

    maxTry?: number;

    // description of this sub task type
    description?: string;

    // default BloonFilter，the job won't execute again after save and restart. If you want to re-execute, use NoFilter
    filterType?: Class_Filter;

    // default content of job.datas
    defaultDatas?: any;
}
```
[@OnStart example](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/quickstart/App.ts)

### @OnTime
```
export function OnTime(config: OnTimeConfig) { ... }
```
A job executed at special times resolved by cron expression
Params type
```
export type OnTimeConfig = {
    urls: string | string[];
    cron: string; // cron expression
    running?: boolean;
    parallel?: ParallelConfig;
    exeInterval?: number;
    exeIntervalJitter?: number;
    timeout?: number;
    maxTry?: number;
    description?: string;
    defaultDatas?: any;
}
```
[@OnTime example](https://github.com/xiyuan-fengyu/ppspider_example/tree/master/src/ontime/App.ts)

### @AddToQueue @FromQueue
Those two should be used together, @AddToQueue will add the function's result to job queue, @FromQueue will fetch jobs from queue to execute

@AddToQueue
```
export function AddToQueue(queueConfigs: AddToQueueConfig | AddToQueueConfig[]) { ... }
```
@AddToQueue accepts one or multi configs
Config type：
```
export type AddToQueueConfig = {
    // queue name
    name: string;

    // queue provided: DefaultQueue(FIFO), DefaultPriorityQueue
    queueType?: QueueClass;

    // filter provided: NoFilter(no check), BloonFilter(check by job's key)
    filterType?: FilterClass;
}
```
You can use @AddToQueue to add jobs to a same queue at multi places, the queue type is fixed at the first place, but you can use different filterType at each place.

The method Decorated by @AddToQueue shuold return a AddToQueueData like.
```
export type CanCastToJob = string | string[] | Job | Job[];

export type AddToQueueData = Promise<CanCastToJob | {
    [queueName: string]: CanCastToJob
}>
```
If @AddToQueue has multi configs, the return data must like
```
Promise<{
    [queueName: string]: CanCastToJob
}>
```
PuppeteerUtil.links is a convenient method to get all expected urls, and the return data is just AddToQueueData like.

@FromQueue
```
export function FromQueue(config: FromQueueConfig) { ... }

export type FromQueueConfig = {
    // queue name
    name: string;

    running?: boolean;

    parallel?: ParallelConfig;

    exeInterval?: number;

    exeIntervalJitter?: number;

    // Task timeout, in milliseconds, default: 300000, negative number means never timeout
    timeout?: number;

    maxTry?: number;

    description?: string;

    defaultDatas?: any;
}
```
[@AddToQueue @FromQueue example](https://github.com/xiyuan-fengyu/ppspider_example/tree/master/src/queue)

### @JobOverride
```
export function JobOverride(queueName: string) { ... }
```
Modify job info before inserted into the queue.
You can set a JobOverride just once for a queue.

A usage scenario is: when some urls with special suffix or parameters navigate to the same page,
you can modify the job key to some special and unique id taken from the url with a JobOverride. After
that, jobs with duplicate keys will be filtered out.

Actually, sub task type OnStart/OnTime is also managed by queue whose name just likes OnStart_ClassName_MethodName
or OnTime_ClassName_MethodName, so you can set a JobOverride to it.
[JobOverride example](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/jobOverride/App.ts)

### @Serializable @Transient
```
export function Serializable(config?: SerializableConfig) { ... }
export function Transient() { ... }
```
@Serializable is used to mark a class, then the class info will keep during serializing and deserializing.
Otherwise, the class info will lose when serializing.

@Transient is used to mark a field which will be ignored when serializing and deserializing.
Warn: static field will not be serialized.
These two are mainly used to save running status. You can use @Transient to ignore fields which are not
related with running status, then the output file will be smaller in size.
[example](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/SerializeTest.ts)

### @RequestMapping
```
export function RequestMapping(url: string, method: "" | "GET" | "POST" = "") {}
```

@RequestMapping is used to declare the HTTP rest interface, providing the ability to dynamically add tasks remotely.
Returning the crawl results requires self-implementation (such as asynchronous url callbacks).
[RequestMapping example](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/requestMapping/App.ts)

### @Bean @Autowired
仿造 java spring @Bean @Autowired 的实现，提供实例依赖注入的功能
[example](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/BeanTest.ts)

### @DataUi @DataUiRequest
You can define you own tab page in UI(http://localhost:webPort) by this which can extend support for data visualization and user interaction
You should import the DataUiClass in @Launcher appConfig.dataUis

There is a built-in DataUi DbHelperUi which can support db search

[example 1 DataUi basic usage](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/dataUi/test.ts)
![nedbHelper.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/8.png)
![dataUiTest1.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/9.png)
![dataUiTest2.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/10.png)

[example 2 Add Dynamic Job On UI](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/dataUi/App.ts)
![动态任务.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/11.png)

[example 3 Web Page Screenshot](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/dataUi/ScreenshotApp.ts)
Long web page screenshot is also supported
![网页截图.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/12.png)

## PuppeteerUtil
### PuppeteerUtil.defaultViewPort
set page's view port to 1920 * 1080

### PuppeteerUtil.addJquery
inject jquery to page, jquery will be invalid after page refresh or navigate, so you should call it after page load.

### PuppeteerUtil.jsonp
parse json in jsonp string

### PuppeteerUtil.setImgLoad
enable/disable image load

### PuppeteerUtil.onResponse
listen response with special url, max listen num is supported

### PuppeteerUtil.onceResponse
listen response with special url just once

### PuppeteerUtil.downloadImg
download image with special css selector

### PuppeteerUtil.links
get all expected urls

### PuppeteerUtil.count
count doms with special css selector

### PuppeteerUtil.specifyIdByJquery
Find dom nodes by jQuery(selector) and specify random id if not existed，
finally return the id array.
A usage scenario is:
In puppeteer, all methods of Page to find doms by css selector finally call
document.querySelector / document.querySelectorAll which not support some
special css selectors, howerver jQuery supports. Such as: "#someId a:eq(0)", "#someId a:contains('next')".
So we can call specifyIdByJquery to specify id to the dom node and keep the special id returned,
then call Page's method with the special id.

### PuppeteerUtil.scrollToBottom
scroll to bottom

### PuppeteerUtil.parseCookies
Parse cookie string to SetCookie Array，than set coookie through page.setCookie(...cookieArr)
How to get cookie string?
open interested url in chrome-> press F12 to open devtools -> Application panel -> Storage:Cookies:<SomeUrl> -> cookie detail in the left panel -> choose all by mouse，press Ctrl+c to copy all
You will get something similar to the following
```
PHPSESSID	ifmn12345678	sm.ms	/	N/A	35
cid	sasdasdada	.sm.ms	/	2037-12-31T23:55:55.900Z	27
```

### PuppeteerUtil.useProxy
Set dynamic proxy for a single page

### PuppeteerUtil example
[PuppeteerUtil example](https://github.com/xiyuan-fengyu/ppspider_example/tree/master/src/puppeteerUtil)
[Single Page Proxy](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/ProxyTest.ts)

## NetworkTracing
Recode all requests during page opening [NetworkTracing example](https://github.com/xiyuan-fengyu/ppspider_example/blob/master/src/dataUi/NetworkTracingTest.ts)

## Database
Nedb is supported by NedbDao.
Mongodb is supported by MongodbDao.
You can set the "dbUrl" in @Launcher parameters, then use appInfo.db to visist db. All metheds defined in src/common/db/DbDao.
Nedb is a server-less database, no need to install the server end, the data will be persisted to the local file. Url format: nedb://nedbDirectoryPath. When the amount of data is large, the data query speed is slow, and it will take a lot of time to load data each time you restart the application, so use it only if the data is small.
Mongodb needs to install the mongo server. The url format: mongodb://username:password@host:port/dbName. It is recommended to save a large amount of data.

After the application is started, a job collection is automatically created to save the job info during execution.

[database example](https://github.com/xiyuan-fengyu/ppspider_example/tree/master/src/db)

## logger
Use logger.debug, logger.info, logger.warn or logger.error to print log.
Those functions are defined in src/common/util/logger.ts.
The output logs contain extra info: timestamp, log level, source file position.
```
logger.debugValid && logger.debug("test debug");
logger.info("test info");
logger.warn("test warn");
logger.error("test error");
```

# Debug
simple typescript/js code can be debugged in IDEA

The inject js code can be debugged in Chromium. When building the PuppeteerWorkerFactory instance, set headless = false, devtools = true to open Chromium devtools panel.
[Inject js debug example](https://github.com/xiyuan-fengyu/ppspider_example/tree/master/src/debug)
```
import {Launcher, PuppeteerWorkerFactory} from "ppspider";
import {TestTask} from "./tasks/TestTask";

@Launcher({
    workplace: __dirname + "/workplace",
    tasks: [
        TestTask
    ],
    workerFactorys: [
        new PuppeteerWorkerFactory({
            headless: false,
            devtools: true
        })
    ]
})
class App {

}
```

<html>
    <p>
    Add <span style="color: #ff490d; font-weight: bold">debugger;</span> where you want  to debug
    </p>
</html>

```
import {Job, OnStart, PuppeteerWorkerFactory} from "ppspider";
import {Page} from "puppeteer";

export class TestTask {

    @OnStart({
        urls: "http://www.baidu.com",
        workerFactory: PuppeteerWorkerFactory
    })
    async index(page: Page, job: Job) {
        await page.goto(job.url());
        const title = await page.evaluate(() => {
           debugger;
           const title = document.title;
           console.log(title);
           return title;
        });
        console.log(title);
    }

}
```

In addition, when developing DataUi, you can debug it in browser
![DataUiDebug.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/13.png)

# Related Information
## jQuery
http://www.runoob.com/jquery/jquery-syntax.html

## puppeteer
https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md

## nedb
https://github.com/louischatriot/nedb

## mongodb
https://docs.mongodb.com/manual/reference/method/js-collection/

## Angular
https://angular.io/

## G2
https://antv.alipay.com/zh-cn/g2/3.x/demo/index.html
G2 integrated in web ui for data visualization in DataUi

## bootstrap
https://v3.bootcss.com/css/
The web ui integrates bootstrap and jquery which make it easy to use the bootstrap and jquery to
write the ui interface directly in DataUi.

# WebUI
open http://localhost:9000 in browser

Queue panel: view and control app status
![Queue Help](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/14_en.png)

Job panel: search jobs and view details
![ppspiderJobs.cn.png](https://raw.githubusercontent.com/xiyuan-fengyu/ppspider/master/github/15.png)

# Question
1. When running an app in idea under debug mode after a long time, the application may get stucked
    on a code line just like a breakpoint stop. It is due to low memory, just add a node
    paramter "--max-old-space-size=8192" to solve it.
    This situation has often appeared in previous versions, mainly due to nedb write/read process or serialization/deserialization process of QueueManager.
  In the new version (v2.1.2), it has been optimized.

2. When using the v2.2.0+ version, the workerFactory property is removed from @OnStart, @OnTime, @FromQueue. If you want to use the puppeteer page to crawl the website,
  You can import the Page class via
    ```
    import {Page} from "ppspider";
    ```
    and then declare a page: Page parameter in the parameter list of the callback function. If use
  Import {Page} from "puppeteer" to import Page, the imported Page is just an interface, which cannot be determined at runtime by reflect-metadata,
  and the page instance will not be injected successfully. This error is checked during startup.

# Update Note
2020-12-07 v2.2.4-preview.1607350101966
1. Modify puppeteer's default userDataDir to prevent generating excessive temporary folders when puppeteer crashes unexpectedly
2. Add worker info in QueueManager_JobExecuted event
3. rewrite EventBus to await EventBus.emit

2020-04-07 v2.2.3
1. fix bug: in case of cross-domain, page.frames may not find some iframes.
2. optimize PuppeteerUtil.dragJigsaw implementation to improve accuracy.

2019-09-04 v2.2.3-preview.1578363288631
1. fix bug in job interruption

2019-09-04 v2.2.3-preview.1577332807380
1. fix bug: job is no longer retried after being interrupted on UI interface
2. add maxTry parameter for task and queue configuration, support always try when maxTry is less than 0
3. add defaultDatas parameter for task and queue configuration as the default value for job.datas
4. add the operation of switching the running state of the queue on UI interface
5. add the operation to edit timeout/maxTry/defaultDatas paramters on UI interface

2019-09-04 v2.2.3-preview.1574909694087
1. OnStart can be configured to use BloonFilter(default, will not re-run after saving status and restart) or NoFilter(will re-run after saving status and restart)
2. fix a bug in db search by page
3. add a new decorator called OnEvent, used to watch system event, there is only one use presently [OnEvent Example](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/OnEventTest.ts)

2019-09-04 v2.2.3-preview.1569208986875
1. fix bug: null pointer error in src/common/db/MongodbDao.ts#remove
2. add a new parameter to transfer the parent job info for JobOverride callback method
3. ignore classes which are not marked with @Serializable during the serialization process
4. fix bug: match with RegExp.toString
5. Add SocksProxyAgent support in RequestUtil

2019-07-31 v2.2.2-preview
1. @Bean @Autowired bug fixed, support type identification for @Autowired
2. Add new method RequestUtil.linesToHeaders to parse header-lines; add headerLines property to the parameter named options of
    RequestUtil.simple, add an optional parameter handler to watch the response
3. Add cookies automatically when using PuppeteerUtil.useProxy to proxy requests
4. Add async function support for Page.evaluate, evaluateOnNewDocument, evaluateHandle, $eval, $$eval
5. Page import fixed
6. Add UserAgents util to get random user-agent
7. Add regexp support for AddToQueue/FromQueue name([example](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/AddToRegexQueue.ts))
8. Set Page default viewport as 1920 * 1080, set navigator.webdriver=false
9. Add a new function PuppeteerUtil.drag to drag bar
10. Add a new PuppeteerUtil.triggerAndWaitRequest to trigger and watch a request
11. Add a new PuppeteerUtil.triggerAndWaitResponse to trigger a request and watch the response
12. Provides solutions for slider verification
    Drag from left to right [PuppeteerUtil.dragBar](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/DragBarTest.ts)
    Drag to the right place to complete the puzzle [PuppeteerUtil.dragJigsaw](https://github.com/xiyuan-fengyu/ppspider/blob/master/src/test/component/DragJigsawTest1.ts)

2019-06-22 v2.2.1
1. Rewrite the way to inject the worker instance through the reflection mechanism provided by typescript and reflect-metadata
   during calling method decorated by @OnStart, @OnTime, @FromQueue.
   The workerFactory property of @OnStart, @OnTime, @FromQueue is removed. The freamwork check the parameter types of the
   decorated method to determine whether the job parameter needs to be passed, whether the worker instance needs to be
   passed (if true, which worker type is the correct one). The order and number of parameters are no longer fixed.

   But there are also restrictions, in the parameters, at most one with the Job type, and at most one with the worker type which has the corresponding WorkerFactory definition (only Page is currently provided).
   Be careful that the class Page is provided in the ppspider package, not the interface Page defined in @types/puppeteer.

   Because of this change, some code needs to be upgraded. You need to remove the workerFactory property in @OnStart, @OnTime, @FromQueue.
   If you want to use page: Page in the method decorated by @OnStart, @OnTime, @FromQueue, you need to import {Page} from "ppspider" instead of "puppeteer",
   other parameters except job: Job should be removed. The order and name of the parameters can be defined freely.
   If the job: Job is not used in the method, you can also remove this parameter.

2. Fixed a bug: @AddToQueue does not work without @OnStart / @OnTime / @FromQueue.
3. Add deployment scheme based on docker

2019-06-13 v2.1.11
1. fix the once listener bug in PuppeteerWorkerFactory.overrideMultiRequestListenersLogic
2. add Buffer type to the parameter content in FileUtil.write
3. in process of QueueManager.loadFromCache, fix the lastExeTime of queue
4. change the dependencies of ui: bootstrap(3.4.1), g2(@antv/g2, @antv/data-set)
5. define window.__awaiter in ui, to support async and await in DataUi
6. define window.loadScript in ui, make it easy to load third-party script in DataUi
7. JobOverride supports async callback methods; change the timing that JobOverride calls callback method.

2019-06-06 v2.1.10
1. Override Add/Remove/Query request listener of Page to ensure that a page has only one request listener(theOnlyRequestListener).
    The user-added request listener will be called via theOnlyRequestListener.
    Whether to call request.continue is no longer confusing.
2. Override the process of AddToQueue to add the method result to queues, so that Filter supports asynchronous check.
3. Rewrite the proxy process of PuppeteerUtil.useProxy by request.
4. The description of fail / failed is added to the UI interface. The number of failed in the queue shows two numbers:
    the number of failed tries, and the number of tasks that still fail after the maximum number of tries.

2019-06-03 v2.1.9
1. Fix bug: app stoped if exception occurred during request proxy in PuppeteerUtil.useProxy

2019-06-02 v2.1.8
1. Add new method: PuppeteerUtil.useProxy, to support single page proxy
2. update puppeteer version to 1.17.0

2019-05-28 v2.1.6
1. Fix a wrong ref of URL model in MongodbDao
2. Fix a bug to parse date in PuppeteerUtil.parseCookie

2019-05-24 v2.1.3
1. On Job panel of ui interface, change "createTime" to text input, accurate to milliseconds
2. Fix bug during deleting job.
3. Rewrite the method to set parallel by cron expressions, then the parallel will be setted immediately after the application launch.

2019-05-21 v2.1.2
1. Rewrite the serialization and deserialization process to solve the problem of large object serialization failure.
2. Delete DefaultJob, change interface Job to class Job, and the methods are changed to corresponding fields. This change causes historical QueueCache data to be incompatible.
3. Override the process of loading and saving Nedb to support reading and writing large amounts of data.
4. Rewrite NedbDao, provide new mongodb support: MongodbDao. They two are compatible with reading and writing data. Users can directly use @Launcher dbUrl to configure the database, and operate the database through appInfo.db.

2019-05-09 v2.0.5
1. Change cron lib: later -> cron
2. Update the logic to calculate the next execution time of OnTime Job

2019-05-08 v2.0.4
1. Upgrade puppeteer: v1.15.0
2. Rewrite the serialization and deserialization process to solve the problem of large object serialization failure
    Due to this change, if you need to use the old queueCache.json which saved the running state, you can upgrade this file as following:
    Import UpgradeQueueCacheTask at "@Launcher tasks",
    Rename the old queueCache.json to queueCache_old.json and put it in the workplace directory,
    Start the app and the queueCache.txt will be generated in the workplace directory,
    Remove UpgradeQueueCacheTask at "@Launcher tasks".

2019-04-29 v2.0.3
1. When the ui interface switches tab pages, the previous tab page will not be destroyed.
2. fix bug： PuppeteerUtil.links can't find urls when passing Regex parameter.
3. fix bug： Nedb uses string concatenation to build new data content during the process of compressing data.
    Memory overflow occurs when the amount of data is large.

2019-04-29 v2.0.2
1. Output colorful logs using ansi-colors

2019-04-22 v2.0.1
1. add RequestUtil, Encapsulate request as a promise style call
2. Modify the declaration of NedbHelperUi.defaultSearchExp to avoid error reporting during ts compilation

2019-04-04 v2.0.0
1. Multiple task threads of the same queue in the QueueManager no longer share a single wait interval
2. The QueueManager can forcibly interrupt the execution of a running job (the UI Job panel provides an interactive button)
    By code
    ```
    appInfo.eventBus.emit(Events.QueueManager_InterruptJob, JOB_ID, "your interrupt reason");
    ```
3. Status can be saved when ppspider is running
4. Added @Bean @Autowired decorator to provide dependency injection programming
5. Added @DataUi @DataUiRequest decorator to provide custom UI tab page, user can customize data visualization page, interactive tool

2019-01-28 v0.1.22
1. Automatically clear jobs marked as filtered.
2. Add job (status: Fail | Success) to queue by clicking button on the Job UI .

2018-12-24 v0.1.21
1. Add timeout to task configuration, default 300000ms
2. Add logs to job during task execution to analysis execution time and failure reasons

2018-12-10 v0.1.20
1. Fix a bug: duplicate instances of the same user defined Task are created

2018-11-19 v0.1.19
1. Fix a bug: the logger always print error as '{}'.
2. You can select multiple items when the condition has a list of options on the UI interface.
3. Update puppeteer version to 0.10.0.

2018-09-19 v0.1.18
1. Change the reference address of the exported class in index.ts to the original path,
    so that the user can locate the source code location in the editor quickly.
2. Print job detail when an error occurred.
3. The number of job failures increases after all attempts fail.
4. The parameter list of several methods declared in the logger to print log is changed to an indefinite parameter list.
    The format parameter is no longer accepted. The indefinite parameter list is used as the message list, and the messages are automatically divided by '\n';
    The logger automatically converts obj to string using JSON.stringify(obj, null, 4).
5. Add a new property named "exeIntervalJitter" to OnStart, OnTime, FromQueue, it makes a random jitter in a range to the execution interval(exeInterval).
    default: exeIntervalJitter = exeInterval * 0.25
6. Add a new property named "running" to OnStart, OnTime, FromQueue to control the queue running status, default is true.
    Change the running status like following:
    ```
    mainMessager.emit(MainMessagerEvent.QueueManager_QueueToggle_queueName_running, queueNameRegex: string, running: boolean)
    ```
7. The nedb related code in JobManager is rewritten with NedbDao
8. Add a decorator @RequestMapping to declare the HTTP rest interface, providing the ability to dynamically add jobs remotely.
    Returning the crawl results requires self-implementation (such as asynchronous url callbacks).

2018-08-24 v0.1.17
1. In job detail modal, add support to recursively load the parent job detail,
    add target="_blank" to all 'a' elements to open a new tab when clicked
2. The push mode of spider system info is changed from periodic to event driven and delayed
3. Update puppeteer version to 1.7.0
4. Add NoneWorkerFactory whose init action is not required, it is used when puppeteer is not required
5. Complement the SSS of logger datetime with 0
6. The job with empty url is accepted now

2018-07-31 v0.1.16
1. Fix bug: no effect after setting maxParallelConfig=0
2. Update puppeteer version to 1.6.1-next.1533003082302 to solve the bug which causes loss of response in version 1.6.1 of puppeteer

2018-07-30 v0.1.15
1. Change the implement of PuppeteerUtil.addJquery
    Change the inject way of jQuery because Page.addScriptTag is not supported
    by some website
2. Fix bug: PuppeteerWorkerFactory.exPage compute a wrong position of js executing error
3. Fix bug: The CronUtil.next may generate a next execute time which equals to the last job

2018-07-27 v0.1.14
1. Add support to logger level check
    Add support to change logger config
    Logger config can be set in @launcher
2. Change the parameter of DefaultJob.datas to optional
3. Enhance the page created by PuppeteerWorkerFactory, methods such as
    $eval, $$eval, evaluate, evaluateOnNewDocument, evaluateHandle are improved,
    when the inject js executes with error, it will print the detail and position
    of error
4. Change the implement of serialize
5. Add comments to source code
6. Update puppeteer version

2018-07-24 v0.1.13
1. Import source-map-support, add logger to print log with source code position

2018-07-23 v0.1.12
1. Add @Transient Decorator to ignore field when serialization

2018-07-19 v0.1.11
1. Set the max wait time to 60 secondes when spider system shutdown,
    those jobs which are not complete will become fail once and add to queue
    to try again
2. Fix bug: img url match is incorrect

2018-07-16 v0.1.8
1. Implement a serialization util to solve the circular reference question

repository: xiyuan-fengyu/ppspider

repo_name: xiyuan-fengyu/ppspider


text:
---
layout: layout.pug
navigationTitle: Calico
title: Calico
menuWeight: 8
excerpt: Calico Container Networking Interface
beta: true
enterprise: false
---

<!-- markdownlint-disable MD004 MD007 MD025 MD030 -->

# Calico

The Container Network Interface (CNI) specification defines a standard plugin interface for Linux container networking. CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.

Konvoy ships with the latest version of [Calico][calico] as the CNI plugin to provide pod-to-pod network connectivity.

## Calico Customizations

- [BGP Route Reflectors](bgp-route-reflectors)
- [CIDR Pool](cidr-pools)
- [Calico Version](calico-version)
- [Encapsulation](encapsulation)
- [Network Policy](network-policy)

For more information, see:

- [Calico manifest for Kubernetes][calico_yaml]

[calico]: https://docs.projectcalico.org/introduction/
[calico_yaml]: https://docs.projectcalico.org/v3.13/manifests/calico.yaml


text:
---
title: Elasticsearch API 06 - Executing Aggregations
header:
  teaser: elasticsearch.png
categories:
  - Elasticsearch
tags:
  - elasticsearch
---

## Executing Aggregations

The aggregations framework helps provide aggregated data based on a search query. It is based on simple building blocks called aggregations, that can be composed in order to build complex summaries of the data.

## Structuring Aggregations

The following snippet captures the basic structure of aggregations:

```js
"aggregations" : {
    "<aggregation_name>" : {
        "<aggregation_type>" : {
            <aggregation_body>
        }
        [,"meta" : {  [<meta_data_body>] } ]?
        [,"aggregations" : { [<sub_aggregation>]+ } ]?
    }
    [,"<aggregation_name_2>" : { ... } ]*
}
```

This example groups all the accounts by state, and then returns the top 10 (default) states sorted by count descending (also default):

```js
POST localhost:9200/bank/_search?pretty
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state"
      }
    }
  }
}
```

In SQL, the above aggregation is similar in concept to:

```sql
SELECT state, COUNT(*) FROM bank GROUP BY state ORDER BY COUNT(*) DESC
```

Response:

```js
"hits" : {
    "total" : 1000,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "group_by_state" : {
      "buckets" : [ {
        "key" : "al",
        "doc_count" : 21
      }, {
        "key" : "tx",
        "doc_count" : 17
      }, {
        "key" : "id",
        "doc_count" : 15
      }, {
        "key" : "ma",
        "doc_count" : 15
      }, {
        "key" : "md",
        "doc_count" : 15
      }, {
        "key" : "pa",
        "doc_count" : 15
      }, {
        "key" : "dc",
        "doc_count" : 14
      }, {
        "key" : "me",
        "doc_count" : 14
      }, {
        "key" : "mo",
        "doc_count" : 14
      }, {
        "key" : "nd",
        "doc_count" : 14
      } ]
    }
  }
}
```

Building on the previous aggregation, this example calculates the average account balance by state (again only for the top 10 states sorted by count in descending order):

```js
POST localhost:9200/bank/_search?pretty
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state"
      },
      "aggs": {
        "average_balance": {
          "avg": {
            "field": "balance"
          }
        }
      }
    }
  }
}
```

Building on the previous aggregation, let’s now sort on the average balance in descending order:

```js
POST localhost:9200/bank/_search?pretty
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state",
        "order": {
          "average_balance": "desc"
        }
      },
      "aggs": {
        "average_balance": {
          "avg": {
            "field": "balance"
          }
        }
      }
    }
  }
}
```

This example demonstrates how we can group by age brackets (ages 20-29, 30-39, and 40-49), then by gender, and then finally get the average account balance, per age bracket, per gender:

```js
POST localhost:9200/bank/_search?pretty
{
  "size": 0,
  "aggs": {
    "group_by_age": {
      "range": {
        "field": "age",
        "ranges": [
          {
            "from": 20,
            "to": 30
          },
          {
            "from": 30,
            "to": 40
          },
          {
            "from": 40,
            "to": 50
          }
        ]
      },
      "aggs": {
        "group_by_gender": {
          "terms": {
            "field": "gender"
          },
          "aggs": {
            "average_balance": {
              "avg": {
                "field": "balance"
              }
            }
          }
        }
      }
    }
  }
}
```


text:
# Assignment_3
As a database you can use any SQL database( postgresql, mysql, sqlite)
Database should have at least one Table with name User, and columns: id, login, password, token
User Table should contain at least 3 records
# /login
Use as a baseline a code which was provided in a Week 6, Moodle
After successful login( if login and password matches with a record in User Table), as response route should return html text: token: <token value> and store that token in the User Table
If provided login and password does not exist in the User Table, as a response route should return html text: Could not found a user with login: <login>
# /protected
This route should receive as a parameter token value
Token value needs to be passed over URL,
e.g. http://127.0.01:5000/protected?token=24230ifdsjfjdsklfj43943ut943
This route should return html text: Hello, token which is provided is correct, if as a parameter RIGHT token value is passed
This route should return html text: Hello, Could not verify the token, if as a parameter WRONG token value is passed
# Example
![image](https://user-images.githubusercontent.com/80098706/139100353-db7d1248-10cc-4ef1-84ca-f75ac5ce9c45.png)
# after login, table is updated
![image](https://user-images.githubusercontent.com/80098706/139100453-55973230-80c8-47b4-a47e-664801be1181.png)
![image](https://user-images.githubusercontent.com/80098706/139100911-e908213e-897b-489e-8416-2c0442c008de.png)
![image](https://user-images.githubusercontent.com/80098706/139101043-d79fef14-f306-4a28-87c1-c1bcd87a6892.png)
![image](https://user-images.githubusercontent.com/80098706/139100240-5ce6f2ca-984f-4c8a-a5a7-79569835f044.png)


text:
# Covid-19 7-Day-Incidence for Düsseldorf
![Covid-19 Widget Preview light](https://raw.githubusercontent.com/ThisIsBenny/iOS-Widgets/main/Covid-19/previewLight.jpeg)
![Covid-19 Widget Preview dark](https://raw.githubusercontent.com/ThisIsBenny/iOS-Widgets/main/Covid-19/previewDark.jpeg)

**Notice: This widget isn't working anymore due to missing data source. Düsseldorf will not anymore provide any data on the open data platfrom. You can switch to this widget: https://gist.github.com/kevinkub/46caebfebc7e26be63403a7f0587f664**

This widget displays current 7-Day-Incidence for Düsseldorf. The incidence is based on the OpenData data of the city Düsseldorf. New data are published daily at around 12 pm.

[[Download]](https://raw.githubusercontent.com/ThisIsBenny/iOS-Widgets/main/Covid-19/Covid-19.js)

repository: bjoern-sellnau/iOS-Widgets

repo_name: bjoern-sellnau/iOS-Widgets


text:
# Bob
[![build](https://github.com/qoollo/bob/actions/workflows/build.yml/badge.svg)](https://github.com/qoollo/bob/actions/workflows/build.yml) [![Docker Pulls](https://img.shields.io/docker/pulls/qoollo/bob?color=brightgreen)](https://hub.docker.com/r/qoollo/bob/)

![Bob](logo/bob-git-docker-logo.svg)

Bob is a distributed storage system designed for byte data such as photos. It has decentralized architecture where each node can handle user calls. [Pearl](https://github.com/qoollo/pearl) is used as a backend storage.

More information can be found on [wiki](https://github.com/qoollo/bob/wiki).

# Installation
For Linux x86_64, you can use RPM or DEB packages from the [releases](https://github.com/qoollo/bob/releases) page.
There is also a ZIP archive with compiled x64 binaries.

Docker images can be found on DockerHub [qoollo/bob](https://hub.docker.com/r/qoollo/bob/).

Additionally, you can build Bob from sources. Instruction can be found [here](https://github.com/qoollo/bob/wiki/Build-from-source).

# API
Bob uses gRPC API with the following proto file: [bob.proto](bob-grpc/proto/bob.proto).
This means that you can automatically generate a protocol implementation for any platform.
Detailed gRPC API description can be found on [wiki](https://github.com/qoollo/bob/wiki/gRPC-API).

There is also a rich client for .NET: [https://github.com/qoollo/bob-client-net](https://github.com/qoollo/bob-client-net).

# Tools
You can use [bobc](https://github.com/qoollo/bob/blob/master/bob-apps/bin/bobc.rs) and [bobp](https://github.com/qoollo/bob/wiki/Tools#bob-benchmark-tool-bobp) to access data on a cluster. `brt` can be used to check and recover corrupted BLOBs.

There are also tools for removing old partitions, recovering aliens, expanding cluster and more, that can be found in [bob-tools](https://github.com/qoollo/bob-tools) repository.

# Performance

Bob can handle more than 10k RPS.
Detailed tests and hardware configuration can be found on [wiki](https://github.com/qoollo/bob/wiki/Performance-tests).

# Architecture
[cluster.yaml](https://github.com/qoollo/bob/wiki/Cluster-config) file describes the addresses of all nodes in the cluster and a set of directories/physical disks on each node. All data is logically distributed across virtual disks (vdisks). You should create a cluster with more virtual disks than physical ones (it would be reasonable to place 3..10 vdisks on 1 physical disk). Destination vdisk determs like this: `vdisk_id = data_id % vdisks_count`. Cluster writes data to all nodes that contain target vdisk.

Example config with 2 nodes and 3 vdisks:
```YAML
- nodes:
  - name: node1
    address: <IP_ADDRESS>:20000
    disks:
      - name: disk1
        path: /tmp/d1
  - name: node2
    address: <IP_ADDRESS>:20001
    disks:
      - name: disk1
        path: /tmp/d1
      - name: disk2
        path: /tmp/d2

- vdisks:
  - id: 0
    replicas:
      - node: node1
        disk: disk1
  - id: 1
    replicas:
      - node: node2
        disk: disk1
  - id: 2
    replicas:
      - node: node1
        disk: disk1
      - node: node2
        disk: disk2
```

As you can see, there are no replicated nodes, instead Bob replicates vdisks. Vdisks are distributed across nodes, which provides more flexibility. Some nodes with more powerful hardware can have more vdisks and therefore store more data.

`cluster.yaml` can be generated semi-automatically with [CCG](https://github.com/qoollo/bob/wiki/Tools#cluster-config-generator-ccg).

# Backend
Backend store data in 2 groups: "normal" and "alien" folders. "Normal" are vdisk folders described in cluster config . "Alien" is folder for data that cannot be written to its node due to node unavailability. It has the same structure that "normal" folder but also has "node name" in folder hierarchy.
Under vdisk info in folder it has timestamp folder info.
```
- disk 1
    - vdisk id 1
...
    - vdisk id N
        - timestamp 1
        ...
        - timestamp N
            - blob 1
            ...
            - blob N
...
- disk N
    - Alien
        - node name
            - vdisk id 1
                - timestamp 1
                    - blob 1
```

repository: qoollo/bob

gha_language: Rust

repo_name: qoollo/bob


text:
---
title: The state will dredge the
tags:
  - Feb 1913
---

The state will dredge the river if the silting is due to the Everglades drainage.

Newspapers: **Miami Morning News or The Miami Herald**

Page: **1**, Section: **N/A**


text:
Transform a number into the given base.

### Example

  For base = 2 and n = 13, output should be "1101"

[input] integer base

positive integer, 1 < base < 10
[input] integer n

positive integer
[output] string


text:
---
type: rule
archivedreason:
title: Messages - Do you use messages that are concise?
guid: f388b297-50e5-4df6-a8ff-dc9589133963
uri: messages-do-you-use-messages-that-are-concise
created: 2012-11-27T04:41:51.0000000Z
authors: []
related: []
redirects: []

---

It is important to use terminology that your users will understand. Do not to use technical terms that may confuse your users. Use consistent words and phrasing for similar situations. For example, the following phrases have the same meaning which is the best one?

<!--endintro-->

* Not enough memory.
* There is not enough memory.
* There is not enough free memory.
* Insufficient memory.
* No memory was available.
* Your computer does not have sufficient memory.
* Memory resource is not enough.
* Ran out of memory.
* You may be out of memory.

::: bad
![Figure: Bad Example - Is it OK to Cancel?](../../assets/Bad-MessageBoxZango.jpg)
:::

Microsoft uses this one:

::: good
![Figure: Good Example - Microsoft error message is concise](../../assets/NotEnoughMemory.gif)
:::

Some other message types that Microsoft uses are:

| Message type | Sample message |
| --- | --- |
| Not enough disk space | There is not enough disk space to complete the operation. Delete some unneeded files on your hard disk, and then try again |
| File not found | The program cannot find the file filename |
| Re-running setup | The filename file is missing. Run Setup again to reinstall the missing file. For more information about running Setup, press F1.  |

Consider using or adapting them in your application in similar scenarios. Only include the information that the user needs and will understand.

This also applies to general design principles, read our rule on [Less is more: do you know people scan, not read?](/less-is-more-do-you-know-people-scan-not-read)


text:
#### Master Build Status
[![Build status](https://ci.appveyor.com/api/projects/status/alfu0biu2cxy80a8?svg=true)](https://ci.appveyor.com/project/IB313184/transcendence) [![Build Status](https://travis-ci.org/IB313184/transcendence.svg?branch=master)](https://travis-ci.org/IB313184/transcendence)

### Transcendence is a fork of [Gossip Coin](https://github.com/g0ssipcoin) that forked [PIVX](https://github.com/PIVX-Project/PIVX) that forked [Dash](https://github.com/dashpay/dash) that forked [Bitcoin](https://github.com/bitcoin/bitcoinp)


text:
# Get overdue clients and send payment request by email
A trigger function (`start` function) that fills out a Queue (SQS) that is consumed (by a `mail` function) and makes a GET request (serial not parallel as we dont want to kill Zauru) to Zauru that automatically sends the payment request by email.

We separated this function in 2 because AWS APIGatewayProxyRequest only allow 30 seconds to generate a response and if the response is issued, no more code can run in the function. We connected both functions via SQS to make the second function to last the 300 seconds that are available.

## start function
Gets the params via HTTP GET to schedule the necesary GETs to send to Zauru

> ### params
> * ZauruUserEmail - required<EMAIL_ADDRESS>> * ZauruUserToken - required (SKD9lskjdf2923e)
> * ExcludeExclusiveSeller - optional
> * ExcludeCat - optional

## mail function

Gets the list of URLs to call from SQS (filled up by the other function `start`).

### Notices
 1 install dot_env node module to enable the env variables to be pushed to lambda with the serverless framework

repository: intuitiva/zauru-automation

repo_name: intuitiva/zauru-automation