text:
*By Cleuton Sampaio*

[Brazilian Portuguese ![Brazil](brasil.png)](README_PT-BR.md)

Angular 2 is not easy to learn, despite their efforts creating a tutorial, even this tutorial is quite difficult to understand. There are many new concepts, and a good deal of Accidental Complexity, which even frightens Java developers.

**Stupid Start** Shows a simple REST/Web app, which consumes a public RESTful service. It does not show you all the *bells and whistles* of a full-fledged Angular 2.x app,
but it will turn you up and running faster, allowing you to learn the basics and preparing you to make more complex apps latter.

Just follow the instructions. If you want, you can clone the full app on this repository, and just run it.

Oh, BTW, it will help a lot if you already know **Java** or other OOP language, like **C++**, **python** or **Ruby**.

## OMG! How to I begin?

Ok, your boss has sold a new and improved web interface to the Client! He said that it would use Angular 2
(or perhaps Angular 4.0). And you, with your *rough* experience of Angular 1.x, are not prepared.  Just
create a new folder, and download the ZIP into it. Do not clone the *Quickstart seed* repo. Relax! Don't bother with creating a Git repo or setup anything.
The only tools you will need, you (probably) already have: [NPM and Node.js](https://nodejs.org/en/download/).

Once you have downloaded the ZIP file, extract it into the folder you created.

Rename the extracted folder to "showMyIp", to be compatible with our app.

Of course, you can download **this app**, if you want. But this tutorial assumes that you have cloned the *Quickstart Seed* app.

Be sure you have at least the versions: node `v4.x.x` and npm `3.x.x`! Run `node -v` and `npm -v` to check the versions.

## About the app

The **Stupid Start** app is named "showMyIp", which does exactly this: Show Your External IP Address.

![Screenshot](showMyIp.png)

That's it! You click on a button and it shows your IP etc. This information comes from a public RESTful service:
**ipinfo.io**. Try this link using `curl` or `wget`:

`curl http://ipinfo.io`

You should see something like this:

```
curl http://ipinfo.io
{
  "ip": "<IP_ADDRESS>",
  "hostname": "No Hostname",
  "city": "Rio de Janeiro",
  "region": "Rio de Janeiro",
  "country": "BR",
  "loc": "-22.9876,-43.3207",
  "org": "VIVO"
}
```

We only need to get this JSON and navigate it using a `<tr>`.

Simple.

## Cut the crap

Ok, you probably noticed that there is a `package.json` file along with an `app` folder, right? So, open the main folder in a terminal
(I recommend [Visual Studio Code](https://code.visualstudio.com/)).

Ok, if you are using *VSCODE*, so open the folder containing the `package.json` file, and select menu `view / Integrated Terminal`.
If you are using a simple Terminal (or Console), just navigate to the folder you just unpacked (which contains `package.json`) and
type:

`npm install`

This may take a while. NPM is installing some files inside a folder named: `node_modules`. All you must know for now is that this folder is a library of components, needed by Angular.

Open folder "app", inside the main folder, and take a look at the files:

* **main.ts**: Angular application *bootstrap* script. It starts your app. And, BTW, it is written in [typescript language](https://www.typescriptlang.org/) (I forgot to tell you that... My bad!);
* **app.module.ts**: Your app's main module, which contains all the other components you created (also `typescript`);
* **app.component.ts**: Your app's main **Component**. There may be more components, if you need. Your app will be composed by Components like this (**OMG**! Everything is `typescript`!);

Run the current app by typing in the terminal:

`npm start`

A new browser window should open and show your app's main page:

![main page](fig2.png)

Note the URL: `http://localhost:3000` this is the address of your local web server, opened by NPM. It is an option of `typescript` compiler (`tsc -w`) which compiles
all your `typescript` source code into `javascript`, and serve the `index.html` file using TCP port 3000.

**What happened?**

Ok, open `index.html`. Note that it imports a lot of crap, none of your app's:

```
    <script src="node_modules/core-js/client/shim.min.js"></script>

    <script src="node_modules/zone.js/dist/zone.js"></script>
    <script src="node_modules/reflect-metadata/Reflect.js"></script>
    <script src="node_modules/systemjs/dist/system.src.js"></script>

    <script src="systemjs.config.js"></script>
```

But there are 2 things near the end of file:

```
    <script>
      System.import('app').catch(function(err){ console.error(err); });
    </script>

```

and

```
<my-app>Loading AppComponent content here ...</my-app>
```

Forget about the first one, just remember that it is necessary. Now, the last one is a custom tag `<my-app>`, and it is
    defined by your app, inside file `app.component.ts`:

```
@Component({
  selector: 'my-app',
  template: `<h1>Hello {{name}}</h1>`,
})
```

This strage declaration is a **decorator**, a kind of Annotation that `typescript` uses to **complement** a class' declaration. Let's see the entire file `app.component.ts`:

```
import { Component } from '@angular/core';

@Component({
  selector: 'my-app',
  template: `<h1>Hello {{name}}</h1>`,
})
export class AppComponent  { name = 'Show My IP'; }
```

We have a source file that **imports** an already existing module. You can import your own modules or modules created by others. In this case, we are importing an Angular module,
which is the `@Component` decorator itself. Inside the decorator, we declare some metadata about our **Component**: its **tagname** and its **template**.

Our template is using an Angular **expression** (surrounded by "{{" and "}}"). The entire expression will be substituted by a *property* named `name`.

And this property is declared inside our **Component's** class, named `AppComponent`.

**How is this Component loaded?**

Open file `app.module.ts` and you'll see:

```
import { NgModule }      from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';

import { AppComponent }  from './app.component';

@NgModule({
  imports:      [ BrowserModule ],
  declarations: [ AppComponent ],
  bootstrap:    [ AppComponent ]
})
export class AppModule { }
```

The main module for our app imports, declares and bootstraps our **Component's** class! If you have more components, they must be
declared inside `declarations` property (inside `@NgModule` decorator), but only one must be declared inside `bootstrap`, and is the initial component of your app.

**That's it**

And the file `app.component.spec.ts`? It is just for **Protractor** tests. Forget it by now.

## Let's build our app

Ok, now you know the general structure, right? Let's begin by changing the `app.component.ts` file to:

```
import { Component } from '@angular/core';

@Component({
  selector: 'my-app',
  template: `
      <h1>Welcome to: {{name}}</h1>
      <input type="button" value="Pesquisar" (click)="pesquisar()"/>
      <hr/>
  `,
})
export class AppComponent  {
  name = 'Show My IP';
  pesquisar(): Resultado {
    return null;
  }
}
```

There must be some errors right now in the browser and in the VSCODE. We added a `button` with a `(click)` event, pointing to a `pesquisar()` funtion.
This function is declared inside class `AppComponent`. Note that it returns something called `Resultado`, which does not exists yet.

The `typescript` sintax for functions is:

```
function add(x: number, y: number): number {
    return x + y;
}

let myAdd = function(x: number, y: number): number { return x+y; }
```

I think you've got it! Let's not loose time with this. If you want to know more, check [typescript docs](https://www.typescriptlang.org/docs/tutorial.html).

Back to the `app.component` file, we need to create a class definition for `Resultado`.

```
import { Component } from '@angular/core';

class Resultado {
  ip: string;
  hostname: string;
  city: string;
  region: string;
  country: string;
  loc: string;
  org: string;
}
...
```

This class has the same layout of our expected response. Note that all properties have types! That's the name **TYPE**script!

Now we need to get the data from `ipinfo`. The best way is to create a **Service** and get the external data there. But we want this app to be simple and **Stupid**, so, we are gonna get the data inside the same component file.
Add the import, just bellow the first import:

```
import { Http, Response } from '@angular/http';
```

We imported 2 things: `Http` module and `Response` class. Now, we need to inject the `http` service into our Component's class. This is done using a **Constructor**, which initializes a **Private** property. Add this line just bellow the `export class...` line:

```
    constructor(private http: Http) { }
```

But easy on that... The `HttpModule` which is the **Provider** for `Http` module, must be injected on our `AppModule` class! Open `app.module` and add the import, just bellow the first one:

```
import { HttpModule } from '@angular/http';
```

And change the `import` property of `@NgModule` decorator to import `HttpModule` into our class:

```
@NgModule({
  imports:      [ BrowserModule, HttpModule ],
```

Now the `HttpModule` is available to all components of our app. Let's return to `app.component` file. We need to import some files from [RxJS Reactive Extensions](https://github.com/Reactive-Extensions/RxJS) into our `app.component.ts` file:

```
import {Observable} from 'rxjs/Rx';
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/catch';
```

The `Http` service returns an instance of `Observable` class (which is a **template** class, like C++, or **Generics** like Java). And we need 2 methods: `map` and `catch`.

Now, let's add the code for the button to our Component's class. Add the code bellow, right after the "name" property declaration:

```
  resultado: Resultado;
  pesquisar(): void {
    this.getIpData()
      .subscribe(
          res => {
            console.log(JSON.stringify(res))
            this.resultado = res},
          err => {
              console.log(err);
          });
  }

  getIpData() : Observable<Resultado> {
    return this.http.get('http://ipinfo.io')
      .map((res:Response) => res.json())
      .catch((error:any) => Observable.throw(error.json().error || 'Server error'));
  }
```

**OMG!

Take a deep breath, and I will explain...

The `pesquisar()` function is invoked by the button we created, right? It is in the HTML template for our component.
This function uses another function, named `getIpData()`, which is also declared in our component.

The first function (`pesquisar()`) just invokes `getIpData()`, which return an instance of `Observable<Resultado>`. `Observable` is a **template** class, and it allows us to monitor asynchronously the results of getting an Ip Data.
    The method `subscribe()` of an `Observable` allow us to get the results using a **lambda** expression (and the error too).

So, the second function (`getIpData()`) uses `http.get()` to send a **GET** request to the RESTful service. Then, we use the `map` function (which we have just imported) to get the json() response. We also use the `catch` function, to get the error.

This is how we get data using `http` service. It will be easier to use `Promise`, but it is outside the scope for this demo.

**Let's change the HTML template**

We need to show the properties using a `<table>`. This is easy! Just change the `template` property of the `@Component` decorator to:

```
  template: `
      <h1>Welcome to: {{name}}</h1>
      <input type="button" value="Pesquisar" (click)="pesquisar()"/>
      <hr/>
      <div *ngIf="resultado">
          <table border="1">
              <thead>
                  <tr role="row">
                      <th>Propriedade</th>
                      <th>Valor</th>
                  </tr>
              </thead>
              <tbody>
                  <tr>
                      <td>ip</td>
                      <td>{{resultado.ip}}</td>
                  </tr>
                  <tr>
                      <td>hostname</td>
                      <td>{{resultado.city}}</td>
                  </tr>
                  <tr>
                      <td>region</td>
                      <td>{{resultado.region}}</td>
                  </tr>
                  <tr>
                      <td>country</td>
                      <td>{{resultado.country}}</td>
                  </tr>
                  <tr>
                      <td>loc</td>
                      <td>{{resultado.loc}}</td>
                  </tr>
                  <tr>
                      <td>org</td>
                      <td>{{resultado.org}}</td>
                  </tr>
              </tbody>
          </table>
      </div>
  `
```

The "`" signal allows us to use multiline strings. This ends our tutorial.

## Did it run?

If you left the terminal open, the webserver is updating at each file change, and you should see the final app running. But, if the results are different, try to compare your files with my files (in the repository).

This is the first Angular 2.x> tutorial. Excuse-me my bad english skills and cursewords. More tutorials ar coming, and remember: This is a **Stupid Start**. just to get you up and running faster.


text:
---
layout: post
title: IPTO to remove information on annulled bankruptcy orders from its online information search service -- Press Release.
permalink: /news/press-releases/ipto-to-remove-information-on-annulled-bankruptcy-orders-from-its-online-information-search-service
breadcrumb: Press Releases
collection_name: news
---

31 Mar 2010 Posted in [Press releases](/news/press-releases)

The report is in PDF format, to view [click here (0.03MB)](/files/news/press-releases/2010/03/linkclicka3b6.pdf).

Last updated on 25 Nov 2014

license: MIT

repository: chungcuiyi/isomerpages-mlaw

branch_name: refs/heads/master

extension: md

gha_license_id: NOASSERTION

license_type: permissive

repo_name: chungcuiyi/isomerpages-mlaw


text:
# Introduction

[[toc]]

## What is Anno.js

Anno.js is a practice of Typescript, Vue, Ant Design Vue and other open sources. Precisely, Anno.js is a web project template. We implemented it with functional style components, just like `React`. That's the main idea of the project.

## Getting Started

#### Step 1
Download the latest stable version of Node.js, and install it.

#### Step 2
Clone the project `https://github.com/stonychen/anno.js.git`.

#### Step 3
Run `npm install` to resolve dependencies.

#### Step 4
Start the project with `npm run serve`.

#### Others
Run `npm run docs` to view documents.

Run `npm run test` to execute unit test.

Run `npm run test:e2e` to execute e2e test.

::: tip
Before executing e2e, ensure that you have installed chromedriver with `npm install chromedriver` manually. We didn't add this dependency in `package.json`, because many chinese users cannot resolve dependencies, and because of GFW.
:::

## Dependencies

### Vue Related
- [vue](https://vuejs.org/)
- [Vue Cli](https://cli.vuejs.org/)
- [vue-router](https://router.vuejs.org/)
- [vuepress](https://vuepress.vuejs.org/)
- [Vue Class Component](https://class-component.vuejs.org/)
- [vue-property-decorator](https://github.com/kaorun343/vue-property-decorator)
- [ant-design-vue](https://github.com/vueComponent/ant-design-vue)

### CSS Related
- [Sass](https://sass-lang.com/)
- [Less](http://lesscss.org/)

### Test Related
- [vue-test-utils](https://vue-test-utils.vuejs.org/)
- [Mocha](https://mochajs.org/)
- [Mochapack](https://github.com/sysgears/mochapack/)
- [Chai](https://www.chaijs.com/)
- [Nightwatch](https://nightwatchjs.org/)

### Others
- [Typescript](https://www.typescriptlang.org/)
- [axios](https://github.com/axios/axios)
- [moment](https://momentjs.com/)
- [Webpack](https://webpack.js.org/)
- [Github Markdown Guide](https://guides.github.com/features/mastering-markdown/)
- [nanoid](https://github.com/ai/nanoid)
- [Prettier](https://prettier.io/docs/en/)

gha_language: JavaScript


text:
CloudStructures
===============
Redis Client based on BookSleeve. Features: connection management, serialize/deserialize to object, key distribute and configuration. And includes Redis Profiler for Glimpse. The concept is distributed collection inspired by Cloud Collection(System.Cloud.Collections.IAsyncList[T], etc...) of [ActorFx](http://actorfx.codeplex.com/).

Why use CloudStructures?
---
BookSleeve is pure, low level library. It is Redis driver like ADO.NET. CloudStructures is O/R(Object/Redis) Mapper like Dapper.

Install
---
using with NuGet, [CloudStructures](https://nuget.org/packages/CloudStructures/)
```
PM> Install-Package CloudStructures
```

Example
---
```csharp
// Server of Redis
public static class RedisServer
{
    public static readonly RedisSettings Default = new RedisSettings("<IP_ADDRESS>");
}

// a class
public class Person
{
    public string Name { get; set; }
    public int Age { get; set; }
}

// Redis String
var redis = new RedisString<Person>(RedisServer.Default, "test-string-key");
await redis.Set(new Person { Name = "John", Age = 34 });

var copy = await redis.GetValueOrDefault();

// Redis list
var redis = new RedisList<Person>(RedisServer.Default, "test-list-key");
await redis.AddLast(new Person { Name = "Tom" });
await redis.AddLast(new Person { Name = "Mary" });

var persons = await redis.Range(0, 10);

// and others - Set, SortedSet, Hash, Dictionary(Generic Hash), Class(Object-Hash-Mapping)
```

ConnectionManagement
---
```csharp
// Represents of Redis settings
var settings = new RedisSettings(host: "<IP_ADDRESS>", port: 6379, db: 0);

// BookSleeve's threadsafe connection
// keep single connection and re-connect when disconnected
var conn = settings.GetConnection();

// multi group of connections
var group = new RedisGroup(groupName: "Cache", settings: new[]
{
    new RedisSettings(host: "<IP_ADDRESS>", port: 6379, db: 0),
    new RedisSettings(host: "<IP_ADDRESS>", port: 6379, db: 0),
});

// key hashing
var conn = group.GetSettings("hogehoge-100").GetConnection();

// customize serializer(default as JSON, and option includes protocol-buffers)
new RedisSettings("<IP_ADDRESS>", converter: new JsonRedisValueConverter());
new RedisSettings("<IP_ADDRESS>", converter: new ProtoBufRedisValueConverter());
```

PubSub -> Observable
---
Experimental feature, CloudStructures with Reactive Extensions. RedisSubject is ISubject = IObservable and IObserver. Observer publish message to Redis PubSub Channnel. Observable subscribe to Redis PubSub Channel.

using with NuGet(Including PreRelease), [CloudStructures-Rx](https://nuget.org/packages/CloudStructures-Rx/)
```
PM> Install-Package CloudStructures-Rx -Pre
```

```csharp
// RedisSubject as ISubject<T>
var subject = new RedisSubject<string>(RedisServer.Default, "PubSubTest");

// subject as IObservable<T> and Subscribe to Redis PubSub Channel
var a = subject
    .Select(x => DateTime.Now.Ticks + " " + x)
    .Subscribe(x => Console.WriteLine(x));

var b = subject
    .Where(x => !x.StartsWith("A"))
    .Subscribe(x => Console.WriteLine(x), () => Console.WriteLine("completed!"));

// subject as IObserver and OnNext/OnError/OnCompleted publish to Redis PubSub Channel
subject.OnNext("ABCDEFGHIJKLM");
subject.OnNext("hello");
subject.OnNext("world");

Thread.Sleep(200);

a.Dispose(); // Unsubscribe is Dispose
subject.OnCompleted(); // if receive OnError/OnCompleted then subscriber is unsubscribed
```

Configuration
---
load configuration from web.config or app.config

```xml
<configSections>
    <section name="cloudStructures" type="CloudStructures.Redis.CloudStructuresConfigurationSection, CloudStructures" />
</configSections>

<cloudStructures>
    <redis>
        <group name="cache">
            <add host="<IP_ADDRESS>" />
            <add host="<IP_ADDRESS>" port="1000" />
        </group>
        <group name="session">
            <add host="<IP_ADDRESS>" db="2" valueConverter="CloudStructures.Redis.ProtoBufRedisValueConverter, CloudStructures" />
        </group>
    </redis>
</cloudStructures>
```

```csharp
// load configuration from .config
var groups = CloudStructuresConfigurationSection.GetSection().ToRedisGroups();
```

group attributes are "host, port, ioTimeout, password, maxUnsent, allowAdmin, syncTimeout, db, valueConverter, commandTracer".
It is same as RedisSettings except commandTracer.

Glimpse.CloudStructures.Redis
---
CloudStructures has Redis Profiler for [Glimpse](http://getglimpse.com/). - [Glimpse.CloudStructures.Redis](https://nuget.org/packages/Glimpse.CloudStructures.Redis/)

```
PM> Install-Package Glimpse.CloudStructures.Redis
```

Setup Glimpse and add config with RedisProfiler for example

```xml
<cloudStructures>
<redis>
    <group name="Demo">
    <add host="<IP_ADDRESS>" db="0" commandTracer="Glimpse.CloudStructures.Redis.RedisProfiler, Glimpse.CloudStructures.Redis" />
    </group>
</redis>
</cloudStructures>
```

You can see Redis Tab on Glimpse.

![](http://i.imgur.com/QZ7hZu6.jpg)

Command, Key, Sent/Received Object as JSON, Duration. If duplicate command and key then show warn(Icon and Orange Text).

And Timeline, can visualise parallel access.

![](http://i.imgur.com/yqzAIzk.jpg)

Sample is avaliable on this Repositry, [CloudStructures.Demo.Mvc](https://github.com/neuecc/CloudStructures/tree/master/CloudStructures.Demo.Mvc).

Who is using this?
---
CloudStructures is in production use at [Grani](http://grani.jp/)
Grani is top social game developer in Japan(and I'm CTO at Grani).
The game is developed by C# 5.0 + ASP.NET MVC 5 on AWS(Windows Server + RDS(MySQL) + Redis).
The game use redis massively heavy, hundreds of thousands of message per second.

History
---
2013-11-16 ver 0.6.1
* fix, Configuration's GetElementKey take uniq by Host, Port, Db.

2013-11-15 ver 0.6.0
* add, Glimpse.CloudStructures.Redis
* improved, connection waitOpen use syncTimeout.
* improved, can monitor RedisConnection event(Open/Close/Error/Shutdown) on RedisSettings.
* fix bugs, List.AddFirstAndFixLength is always trimed right length -1.
* breaking changes, ICommandTracer receive many extra key(settings, sentObject, receivedObject).

License
---
under [MIT License](http://opensource.org/licenses/MIT)

repository: skyilover/CloudStructures

repo_name: skyilover/CloudStructures


text:
Ui4j
====

[![Download](http://img.shields.io/badge/download-latest--jar-orange.svg)](https://repo1.maven.org/maven2/com/ui4j/ui4j-all/1.0.1/ui4j-all-1.0.1.jar) [![Version](http://img.shields.io/badge/version-1.0.1-green.svg)](https://github.com/ui4j/ui4j) [![License](http://img.shields.io/badge/license-MIT-blue.svg)](http://opensource.org/licenses/MIT)

Ui4j is a web-automation library for Java. It is a thin wrapper library around the JavaFx WebKit Engine, and can be used for automating the use of web pages and for testing web pages.

Supported Java Versions
-----------------------

Oracle Java 8.

Both the JRE and the JDK are suitable for use with this library.

Licensing
---------

Ui4j is released under the terms of the MIT License (MIT).

You are free to use Ui4j or any of its constituent parts in any other project (even commercial projects) so long as its copyright headers are left intact.

Stability
---------

This library is suitable for use in production systems.

If you have found a defect or you want to request a feature enhancement an issue report is the way to bring the attention to the Ui4j Community.

Integration with Maven
----------------------
[![Maven](http://img.shields.io/maven-central/v/com.ui4j/ui4j-all.svg)](http://search.maven.org/#search%7Cga%7C1%7Cui4j)

To use the official release of Ui4j, please use the following snippet in your pom.xml

```xml
    <dependency>
        <groupId>com.ui4j</groupId>
        <artifactId>ui4j-all</artifactId>
        <version>1.0.1</version>
    </dependency>
```

Using Ui4j without Maven
------------------------
If you use Ui4j without Maven, download [pre built jar file](https://repo1.maven.org/maven2/com/ui4j/ui4j-all/1.0.1/ui4j-all-1.0.1.jar).

Supported Platforms
-------------------

Ui4j has been tested under Windows 8.1 and Ubuntu 14.04, but should work on any platform where a Java 8 JRE or JDK is available.

Headless Mode
-------------

Ui4j can be run in "headless" mode using Xfvb.

Logging
-------
Both simple logger for java (SLF4J) and Java utility logger (JUL) is supported.
If slf4j is available on classpath com.ui4j.api.util.LoggerFactory use slf4j else java utility logger is used.

CSS Selector Engine
-------------------
Ui4j use W3C selector engine which is default selector engine of WebKit. Alternatively [Sizzle](http://http://sizzlejs.com) selector engine might be used.
Sizzle is the css selector engine of JQuery and it supports extra selectors like _:has(div)_, _:text_, _containts(text)_ etc.
Check the [Sizzle.java](https://github.com/ui4j/ui4j/blob/master/ui4j-sample/src/main/java/com/ui4j/sample/Sizzle.java) for using sizzle with Ui4j.

Usage Examples
--------------

Here is a very basic sample program that uses Ui4j to display a web page with a "hello, world!" message. See the [ui4j-sample](https://github.com/ui4j/ui4j/tree/master/ui4j-sample/src/main/java/com/ui4j/sample) project for more sample code snippets.

```java
package com.ui4j.sample;

import com.ui4j.api.browser.BrowserEngine;
import com.ui4j.api.browser.BrowserFactory;
import com.ui4j.api.browser.Page;

public class HelloWorld {

    public static void main(String[] args) {
        // get the instance of the webkit
        BrowserEngine browser = BrowserFactory.getWebKit();

        // navigate to blank page
        Page page = browser.navigate("about:blank");

        // show the browser page
        page.show();

        // append html header to the document body
        page.getDocument().getBody().append("<h1>Hello, World!</h1>");
    }
}
```

Here is another sampe code that list all front page news from Hacker News.

```java
package com.ui4j.sample;

import com.ui4j.api.browser.BrowserFactory;

public class HackerNews {

    public static void main(String[] args) {
        BrowserFactory
            .getWebKit()
            .navigate("https://news.ycombinator.com")
            .getDocument()
            .queryAll(".title a")
            .forEach(e -> {
                System.out.println(e.getText());
            });
    }
}
```

Building Ui4j
-------------
mvn install

Getting help and getting involved
---------------------------------

Please do not hesitate to submit pull requests.
We are happy to accept documentation and sample code contributions.

FAQ
---

#### How can i set the user agent string?

See [UserAgent.java](https://github.com/ui4j/ui4j/blob/master/ui4j-sample/src/main/java/com/ui4j/sample/UserAgent.java) sample.

#### How can i execute javascript?

See [JavaScriptExecution.java](https://github.com/ui4j/ui4j/blob/master/ui4j-sample/src/main/java/com/ui4j/sample/JavaScriptExecution.java) sample.

#### How can i handle browser login, prompt or confirmation dialog?

See [DialogTest.java](https://github.com/ui4j/ui4j/blob/master/ui4j-webkit/src/test/java/com/ui4j/test/DialogTest.java) for custom handlers or
use default handlers from [Dialogs.java](https://github.com/ui4j/ui4j/blob/master/ui4j-api/src/main/java/com/ui4j/api/dialog/Dialogs.java).

#### What is the easiest way clear all input elements?

Use [clear](https://github.com/ui4j/ui4j/blob/master/ui4j-api/src/main/java/com/ui4j/api/dom/Form.java#L13) method of the Form class.


text:
# v23
A v23 softmodem

# How to use
To build the software, make sure you have a suitable `gcc`, then run `make`.  The program will be built in the build directory.

The program can either modulate or demodulate a signal - not both at the same time.
If you want both, you'll need to run the program twice.

The program is set up with good channel isolation, so you can run the forward and backward channels on one audio line without,
for example, needing sidetone cancellation.

`v23` is now able to use ALSA directly, so you don't need to pipe data around yourself.  There are no other audio backends.

## Basics
By default, v23 either:
* Reads audio data from ALSA (for demodulation) and writes characters to STDOUT, and messages to STDERR
* Reads characters from STDIN (for modulation) and writes audio data to ALSA, and messages to STDERR

## Command-line arguments
Note the command-line interpretation is fairly dumb; each argument must be separate,
and any parameters must be specified without spaces in between.

### Default configuration
The default configuration is equivalent to specifying:
```shell
v23 -md -cb -r44100 -f10dddddddp1 -Ddefault -L100
```
This means v23 will demodulate the backward channel, using a 44100Hz sample rate, and no character
will be output when there is a parity error.  The frame format is equivalent to 7o1 - see below for
a description of the frame specifier.  The ALSA "default" device is used, with a 100ms latency.

Basic command-line options:
* `-m` selects whether `v23` should modulate or demodulate a signal.  Use `-mm` to modulate, and `-md` to demodulate.
* `-c` selects the channel `v23` should work on.  Use `-cf` for the forward channel, and `-cb` for the backward channel.
* `-d` increases debugging output.  Use `-d -d -d ...` for more debugging.
* `-q` increases quietness.  This disables some status messages.
* `-r` overrides the default sample rate - e.g. use `-r48000` for 48kHz sampling.
* `-f` overrides the default frame format.  See below for details.
* `-D` overrides the ALSA audio device.
* `-L` overrides the ALSA latency in ms.

The following command-line options are understood by `v23` for _modulation only_:
* `-A` specifies the amplitude of the output, in dB relative to full-scale.  Specify `-A6` for -6dB, for example.

The following command-line options are understood by `v23` for _demodulation only_:
* `-e` specifies the character to be output in the case of a parity error.  Use `-e?` to specify `?` as the error character.
* `-M` puts the program in monitor mode, for debugging the signal operations.  See below for details.

Note that you can't alter the FSK frequencies.  These are set within the code.
If you want to change them, pick the null frequencies for `init_modemcfg` carefully.

### Frame specifiers
The following characters can be used in frame format specifications:
* `1` or `0`: This bit must be in the correct state for the frame to be recognised.  Examples: start / stop bits.
* `d` or `D`: Data bits, LSb or MSb first, respectively.
* `p` or `P`: Parity bit, odd or even, respectively.

The first bit in one frame may overlap the last bit in the previous (assuming they're set the same).  This allows the
program to ensure the line was in the stop/idle state at the beginning and end of the frame, for alignment purposes.

Note that all data bits must be consecutive, or `v23` won't work as expected.

You'll only get the 8 least-significant bits of the data (in case you try to use a format such as 9n1) from stdout,
and you can only control the 8 least-significant bits at stdin.

Some examples of valid frame formats:
* `-f10dddddddp1` (7o1, the default)
* `-f10dddddddd1` (8n1, a common frame format with no parity)
* `-f10dddddddP1` (7e1, as used by viewdata)
* `-f100DDDDD11` (5 data bits, MSb first, with two start and stop bits)

### Monitor mode
If `v23` is put in monitor mode, the following things happen:
* Raw 16-bit signed audio data is written to STDOUT.  It contains one channel per signal monitored (at present, 8).
* Any demodulated data received is sent to STDERR (along with the usual messages).

## Example usage
For demodulation, use a command-line like:
```shell
build/v23 -e'?' -f10dddddddP1
```

For modulation, use a command-line like:
```shell
build/v23 -cf -mm -d -A6 -f10dddddddP1
```

You'll almost certainly want to build a phone line injector or simulator to hook this up to, but you can just
connect audio leads for testing, or to talk to another softmodem at the far end.

If you want to monitor the signals, use something like:
```shell
build/v23 -e'?' -f10dddddddP1 -M > dump.raw
```

The resulting raw audio can be imported into a tool such as Audacity.  The format is signed 16-bit little-endian, at the
same sample rate as the application was running (Default 44100Hz), with 8 channels (at present) containing the monitored
signals.


text:
# Open Track Data Standard
This document presents a simplified, generic [MIT Licensed](./LICENSE.md) data format for sensor systems that measure another object's position over time, also known as a "track". The name of this format is Open Track Data (OTD). It is meant to be agnostic to the sensing technology being used. Example systems that may be converted to this format include: primary radar, secondary radar, ADS-B, onboard GPS, etc.

The three main use-cases for the OTD are:

1. Data from a stationary sensor tracking a target vehicle. For example a ground based RF sensor detecting and tracking an aircraft.
2. Data from the target vehicle itself. For example the GPS track from a UAV.
3. Data from a sensor onboard one vehicle platform measuring another vehicle platform. The targets may be stationary or moving. For example a drone tracking a manned craft, or a ground sensor mounted to a vehicle on the move.

OTD is primarily intended for data at rest and uses [RFC-7464 jsonseq](https://tools.ietf.org/html/rfc7464) as its format.  It may also be useful as a streaming format in some situations.

The standard itself is comprised of 2 major documents:

1.  A markdown file containing the standard.
2.  A json schema file describing the JSON record structures in detail.

It also holds some basic examples of valid files used for parsing.

## Driving requirements
* Human readable
* Machine parsable
* Extensible, but with a required core
* Global, unambiguous coordinate system
* UTC time format

## Known Drawbacks
* Space intensive (when uncompressed)
* Slow serialization / deserialization of JSON (especially if also validating against the schema)
* Aux fields are a "wild west"

## Contributing
Community contributions (bug reports, merge requests, enhancements) are welcome.  Over time our goal is to build a healthy ecosystem of open source converters and parsers.

## Project History
OTD originally denoted "Offline Data Format" and developed as a "middle man" format for internal analysis tools used at Hidden Level.  Since a need for such a format has been identified in industry, in February of 2021 the format was open sourced so that other collaborators can make it better over time.

And yes, we are well aware of [XKCD 927](https://xkcd.com/927/).

### 1.0.0
This is simply an exact replica of the Offline Track Data format.  It is intended to be a baseline for expansion to future versions with the help of the community.

* [1.0.0 Standard](./versions/1.0.0/offline_track_data.md)
* [Record JSON Schema](./versions/1.0.0/schema.json)
* [Data example](./examples/1.0.0/multi_craft_sim.otd)

## Roadmap
See [ROADMAP.md](./ROADMAP.md)

repository: hidden-level/open-track-data

branch_name: refs/heads/main

repo_name: hidden-level/open-track-data


text:
# ng2-app-fetch-github-issues
An application which will fetch github repo's issues. This is an Angular 2.0 demo app

### [Live demo](http://www.voidcanvas.com/demo/ng2-app-fetch-github-issues/)

## More can be done:
	* Make better error handling.
	* I didn't use DI as it was taking time to understand DI in Angular 2.
	* The regX I used has a little hack. Would like to remove that.
	* I hardly used any styling.
	* Client Id, Client secret, urls etc should be in a config file.


text:
---
description: Welcome to Batman's Secret Server.
---

# Batman's Secret Server

## Getting Super Powers

Becoming a super hero is a fairly straight forward process:

1. Find the username and password in our secret discord server.
2. Go to [app.gitbook.com](https://app.gitbook.com)
3. Log-in with the provided username and password.

{% hint style="info" %}
You must be a verified member to access the password.
{% endhint %}

## Rules

1. Do not post any NSFW Content.
2. Do not reveal full names of members \(except for yourself if you so choose\).
3. Don't be evil. \(@Beth, @Gerry\)

## To-Do

* [ ] Create a list of good board games.
* [ ] Plan out the skydiving adventure.
* [ ] Plan out the Canada adventure.


text:
# 1415. The k-th Lexicographical String of All Happy Strings of Length n

Difficulty: Medium

https://leetcode.com/problems/the-k-th-lexicographical-string-of-all-happy-strings-of-length-n/

A happy string is a string that:

* consists only of letters of the set ['a', 'b', 'c'].
* s[i] != s[i + 1] for all values of i from 1 to s.length - 1 (string is 1-indexed).

For example, strings "abc", "ac", "b" and "abcbabcbcb" are all happy strings and strings "aa", "baa" and "ababbc" are not happy strings.

Given two integers n and k, consider a list of all happy strings of length n sorted in lexicographical order.

Return the kth string of this list or return an empty string if there are less than k happy strings of length n.

**Example 1:**
```
Input: n = 1, k = 3
Output: "c"
Explanation: The list ["a", "b", "c"] contains all happy strings of length 1. The third string is "c".
```

**Example 2:**
```
Input: n = 1, k = 4
Output: ""
Explanation: There are only 3 happy strings of length 1.
```

**Example 3:**
```
Input: n = 3, k = 9
Output: "cab"
Explanation: There are 12 different happy string of length 3 ["aba", "abc", "aca", "acb", "bab", "bac", "bca", "bcb", "cab", "cac", "cba", "cbc"]. You will find the 9th string = "cab"
```

**Example 4:**
```
Input: n = 2, k = 7
Output: ""
```

**Example 5:**
```
Input: n = 10, k = 100
Output: "abacbabacb"
```

**Constraints:**

* 1 <= n <= 10
* 1 <= k <= 100

url: https://raw.githubusercontent.com/jiadaizhao/LeetCode/4ddea0a532fe7c5d053ffbd6870174ec99fc2d60/1401-1500/1415-The k-th Lexicographical String of All Happy Strings of Length n/README.md

repository: jiadaizhao/LeetCode

gha_license_id: MIT

repo_name: jiadaizhao/LeetCode


text:
# detection3D
> 3D Object Detection and Localization

## Usage
```bash
mkdir build
cd build
cmake ..
make
./detection3D
```

## esources
[_A General Framework for Fast 3D Object Detection and Localization Using an Uncalibrated Camera_](http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7045976&punumber%3D7045624%26filter%3DAND(p_IS_Number%3A7045853)%26pageNumber%3D5).
[_Framework for natural landmark-based robot localization_](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6233133&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D6233133).


text:
# Wgit

[![Inline gem version](https://badge.fury.io/rb/wgit.svg)](https://rubygems.org/gems/wgit)
[![Inline downloads](https://img.shields.io/gem/dt/wgit)](https://rubygems.org/gems/wgit)
[![Inline build](https://travis-ci.org/michaeltelford/wgit.svg?branch=master)](https://travis-ci.org/michaeltelford/wgit)
[![Inline docs](http://inch-ci.org/github/michaeltelford/wgit.svg?branch=master)](http://inch-ci.org/github/michaeltelford/wgit)
[![Inline code quality](https://api.codacy.com/project/badge/Grade/d5a0de62e78b460997cb8ce1127cea9e)](https://www.codacy.com/app/michaeltelford/wgit?utm_source=github.com&utm_medium=referral&utm_content=michaeltelford/wgit&utm_campaign=Badge_Grade)

---

Wgit is a Ruby gem similar in nature to GNU's `wget` tool. It provides an easy to use API for programmatic web scraping, indexing and searching.

Fundamentally, Wgit is a WWW indexer/scraper which crawls URL's, retrieves and serialises their page contents for later use. You can use Wgit to copy entire websites if required. Wgit also provides a means to search indexed documents stored in a database. Therefore, this library provides the main components of a WWW search engine. The Wgit API is easily extended allowing you to pull out the parts of a webpage that are important to you, the code snippets or tables for example. As Wgit is a library, it supports many different use cases including data mining, analytics, web indexing and URL parsing to name a few.

Check out this [example application](https://search-engine-rb.herokuapp.com) - a search engine (see its [repository](https://github.com/michaeltelford/search_engine)) built using Wgit and Sinatra, deployed to Heroku. Heroku's free tier is used so the initial page load may be slow. Try searching for "Ruby" or something else that's Ruby related.

## Table Of Contents

1. [Installation](#Installation)
2. [Basic Usage](#Basic-Usage)
3. [Documentation](#Documentation)
4. [Practical Examples](#Practical-Examples)
5. [Practical Database Example](#Practical-Database-Example)
6. [Extending The API](#Extending-The-API)
7. [Caveats](#Caveats)
8. [Executable](#Executable)
9. [Change Log](#Change-Log)
10. [Development](#Development)
11. [Contributing](#Contributing)
12. [License](#License)

## Installation

Add this line to your application's `Gemfile`:

```ruby
gem 'wgit'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install wgit

## Basic Usage

Below shows an example of API usage in action and gives an idea of how you can use Wgit in your own code.

```ruby
require 'wgit'

crawler = Wgit::Crawler.new
url = Wgit::Url.new 'https://wikileaks.org/What-is-Wikileaks.html'

doc = crawler.crawl url

doc.class # => Wgit::Document
doc.stats # => {
# :url=>44, :html=>28133, :title=>17, :keywords=>0,
# :links=>35, :text_snippets=>67, :text_bytes=>13735
#}

# doc responds to the following methods:
Wgit::Document.instance_methods(false).sort # => [
#   :==, :[], :author, :base, :base_url, :css, :date_crawled, :doc, :empty?,
#   :external_links, :external_urls, :find_in_html, :find_in_object, :html,
#   :init_nokogiri, :internal_absolute_links, :internal_full_links,
#   :internal_links, :keywords, :links, :relative_absolute_links,
#   :relative_absolute_urls, :relative_full_links, :relative_full_urls,
#   :relative_links, :relative_urls, :score, :search, :search!, :size, :stats,
#   :text, :title, :to_h, :to_json, :url, :xpath
# ]

results = doc.search 'corruption'
results.first # => "ial materials involving war, spying and corruption.
              #     It has so far published more"
```

## Documentation

100% of Wgit's code is documented using [YARD](https://yardoc.org/), deployed to [Rubydocs](https://www.rubydoc.info/gems/wgit). This greatly benefits developers in using Wgit in their own programs. Another good source of information (as to how the library behaves) are the tests. Also, see the [Practical Examples](#Practical-Examples) section below for real working examples of Wgit in action.

## Practical Examples

Below are some practical examples of Wgit in use. You can copy and run the code for yourself.

### WWW HTML Indexer

See the `Wgit::Indexer#index_the_web` documentation and source code for an already built example of a WWW HTML indexer. It will crawl any external url's (in the database) and index their markup for later use, be it searching or otherwise. It will literally crawl the WWW forever if you let it!

See the [Practical Database Example](#Practical-Database-Example) for information on how to setup a database for use with Wgit.

### Website Downloader

Wgit uses itself to download and save fixture webpages to disk (used in tests). See the script [here](https://github.com/michaeltelford/wgit/blob/master/test/mock/save_site.rb) and edit it for your own purposes.

### Broken Link Finder

The `broken_link_finder` gem uses Wgit under the hood to find and report a website's broken links. Check out its [repository](https://github.com/michaeltelford/broken_link_finder) for more details.

### CSS Indexer

The below script downloads the contents of the first css link found on Facebook's index page.

```ruby
require 'wgit'
require 'wgit/core_ext' # Provides the String#to_url and Enumerable#to_urls methods.

crawler = Wgit::Crawler.new
url = 'https://www.facebook.com'.to_url

doc = crawler.crawl url

# Provide your own xpath (or css selector) to search the HTML using Nokogiri underneath.
hrefs = doc.xpath "//link[@rel='stylesheet']/@href"

hrefs.class # => Nokogiri::XML::NodeSet
href = hrefs.first.value # => "https://static.xx.fbcdn.net/rsrc.php/v3/y1/l/0,cross/NvZ4mNTW3Fd.css"

css = crawler.crawl href.to_url
css[0..50] # => "._3_s0._3_s0{border:0;display:flex;height:44px;min-"
```

### Keyword Indexer (SEO Helper)

The below script downloads the contents of several webpages and pulls out their keywords for comparison. Such a script might be used by marketeers for search engine optimisation for example.

```ruby
require 'wgit'

my_pages_keywords = ['Everest', 'mountaineering school', 'adventure']
my_pages_missing_keywords = []

competitor_urls = [
  'http://altitudejunkies.com',
  'http://www.mountainmadness.com',
  'http://www.adventureconsultants.com'
]

crawler = Wgit::Crawler.new competitor_urls

crawler.crawl do |doc|
  # If there are keywords present in the web document.
  if doc.keywords.respond_to? :-
    puts "The keywords for #{doc.url} are: \n#{doc.keywords}\n\n"
    my_pages_missing_keywords.concat(doc.keywords - my_pages_keywords)
  end
end

if my_pages_missing_keywords.empty?
  puts 'Your pages are missing no keywords, nice one!'
else
  puts 'Your pages compared to your competitors are missing the following keywords:'
  puts my_pages_missing_keywords.uniq
end
```

## Practical Database Example

This next example requires a configured database instance. Currently the only supported DBMS is MongoDB. See [MongoDB Atlas](https://www.mongodb.com/cloud/atlas) for a free (small) account or provide your own MongoDB instance.

`Wgit::Database` provides a light wrapper of logic around the `mongo` gem allowing for simple database interactivity and object serialisation. Using Wgit you can index webpages, store them in a database and then search through all that's been indexed. The use of a database is entirely optional however and isn't required for crawling/indexing.

The following versions of MongoDB are supported:

| Gem    | Database |
| ------ | -------- |
| ~> 2.9 | ~> 4.0   |

### Setting Up MongoDB

Follow the steps below to configure MongoDB for use with Wgit. This is only needed if you want to read/write database records.

1) Create collections for: `documents` and `urls`.
2) Add a [*unique index*](https://docs.mongodb.com/manual/core/index-unique/) for the `url` field in **both** collections.
3) Enable `textSearchEnabled` in MongoDB's configuration (if not already so).
4) Create a [*text search index*](https://docs.mongodb.com/manual/core/index-text/#index-feature-text) for the `documents` collection using:
```json
{
  "text": "text",
  "author": "text",
  "keywords": "text",
  "title": "text"
}
```
5) Set the connection details for your MongoDB instance (see below) using `Wgit.set_connection_details` (prior to calling `Wgit::Database#new`)

**Note**: The *text search index* (in step 4) lists all document fields to be searched by MongoDB when calling `Wgit::Database#search`. Therefore, you should append this list with any other fields that you want searched. For example, if you [extend the API](#Extending-The-API) then you might want to search your new fields in the database by adding them to the index above.

### Database Example

The below script shows how to use Wgit's database functionality to index and then search HTML documents stored in the database. If you're running the code for yourself, remember to replace the database [connection string](https://docs.mongodb.com/manual/reference/connection-string/) with your own.

```ruby
require 'wgit'
require 'wgit/core_ext' # => Provides the String#to_url and Enumerable#to_urls methods.

### CONNECT TO THE DATABASE ###

# Set your connection details manually (as below) or from the environment using
# Wgit.set_connection_details_from_env
Wgit.set_connection_details('DB_CONNECTION_STRING' => '<your_connection_string>')
db = Wgit::Database.new # Connects to the database...

### SEED SOME DATA ###

# Here we create our own document rather than crawling the web (which works in the same way).
# We pass the web page's URL and HTML Strings.
doc = Wgit::Document.new(
  'http://test-url.com'.to_url,
  "<html><p>How now brown cow.</p><a href='http://www.google.co.uk'>Click me!</a></html>"
)
db.insert doc

### SEARCH THE DATABASE ###

# Searching the database returns Wgit::Document's which have fields containing the query.
query = 'cow'
results = db.search query

search_result = results.first
search_result.class           # => Wgit::Document
doc.url == search_result.url  # => true

### PULL OUT THE BITS THAT MATCHED OUR QUERY ###

# Searching the returned documents gives the matching text from that document.
search_result.search(query).first # => "How now brown cow."

### SEED URLS TO BE CRAWLED LATER ###

db.insert search_result.external_links
urls_to_crawl = db.uncrawled_urls # => Results will include search_result.external_links.
```

## Extending The API

Indexing in Wgit is the means of downloading a web page and serialising parts of the content into accessible document attributes/methods. For example, `Wgit::Document#author` will return you the webpage's HTML tag value of `meta[@name='author']`.

By default, Wgit indexes what it thinks are the most important pieces of information from each webpage. This of course is often not enough given the nature of webpages and their differences from each other. Therefore, there exists a set of ways to extend the default indexing logic.

There are two ways to extend the indexing behaviour of Wgit:

1. Add the elements containing **text** that you're interested in to be indexed.
2. Define custom indexers matched to specific **elements** that you're interested in.

Below describes these two methods in more detail.

### 1. Extending The Default Text Elements

Wgit contains an array of `Wgit::Document.text_elements` which are the default set of webpage elements containing text; which in turn are indexed and accessible via `Wgit::Document#text`.

If you'd like the text of additional webpage elements to be returned from `Wgit::Document#text`, then you can do the following:

```ruby
require 'wgit'
require 'wgit/core_ext'

# Let's add the text of links e.g. <a> tags.
Wgit::Document.text_elements << :a

# Our Document has a link whose's text we're interested in.
doc = Wgit::Document.new(
  'http://some_url.com'.to_url,
  "<html><p>Hello world!</p>\
<a href='https://made-up-link.com'>Click this link.</a></html>"
)

# Now all crawled Documents will contain all visible link text in Wgit::Document#text.
doc.text # => ["Hello world!", "Click this link."]
```

**Note**: This only works for textual page content. For more control over the indexed elements themselves, see below.

### 2. Defining Custom Indexers Via Document Extensions

If you want full control over the elements being indexed for your own purposes, then you can define a custom indexer for each type of element that you're interested in.

Once you have the indexed page element, accessed via a `Wgit::Document` instance method, you can do with it as you wish e.g. obtain it's text value or manipulate the element etc. Since the returned types are plain [Nokogiri](https://www.rubydoc.info/github/sparklemotion/nokogiri) objects, you have the full control that the Nokogiri gem gives you.

Here's how to add a Document extension to index a specific page element:

```ruby
require 'wgit'
require 'wgit/core_ext'

# Let's get all the page's table elements.
Wgit::Document.define_extension(
  :tables,                  # Wgit::Document#tables will return the page's tables.
  '//table',                # The xpath to extract the tables.
  singleton: false,         # True returns the first table found, false returns all.
  text_content_only: false, # True returns one or more Strings of the tables text,
                            # false returns the tables as Nokogiri objects (see below).
) do |tables|
  # Here we can manipulate the object(s) before they're set as Wgit::Document#tables.
end

# Our Document has a table which we're interested in.
doc = Wgit::Document.new(
  'http://some_url.com'.to_url,
  '<html><p>Hello world!</p>\
<table><th>Header Text</th><th>Another Header</th></table></html>'
)

# Call our newly defined method to obtain the table data we're interested in.
tables = doc.tables

# Both the collection and each table within the collection are plain Nokogiri objects.
tables.class        # => Nokogiri::XML::NodeSet
tables.first.class  # => Nokogiri::XML::Element
```

**Note**: Wgit uses Document extensions to provide much of it's core functionality, providing access to a webpages text or links for example. These [default Document extensions](https://github.com/michaeltelford/wgit/blob/master/lib/wgit/document_extensions.rb) provide examples for your own.

**Extension Notes**:

- Any page links should be mapped into `Wgit::Url` objects; Url's are treated as Strings when being inserted into the database.
- Any object (like a Nokogiri object) will not be inserted into the database, it's up to you to map each object into a primitive type e.g. `Boolean, Array` etc.

## Caveats

Below are some points to keep in mind when using Wgit:

- All absolute `Wgit::Url`'s must be prefixed with an appropiate protocol e.g. `https://` etc.
- By default, up to 5 URL redirects will be followed; this is configurable however.
- IRI's (URL's containing non ASCII characters) are supported and will be normalised/escaped prior to being crawled.

## Executable

Currently there is no executable provided with Wgit, however...

In future versions of Wgit, an executable will be packaged with the gem. The executable will provide a `pry` console with the `wgit` gem already loaded. Using the console, you'll easily be able to index and search the web without having to write your own scripts.

This executable will be similar in nature to `./bin/console` which is currently used for development and isn't packaged as part of the `wgit` gem.

## Change Log

See the [CHANGELOG.md](https://github.com/michaeltelford/wgit/blob/master/CHANGELOG.md) for differences (including any breaking changes) between releases of Wgit.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

## Contributing

Bug reports and feature requests are welcome on [GitHub](https://github.com/michaeltelford/wgit/issues). Just raise an issue, checking it doesn't already exist.

The current road map is rudimentally listed in the [TODO.txt](https://github.com/michaeltelford/wgit/blob/master/TODO.txt) file. Maybe your feature request is already there?

## Development

For a full list of available Rake tasks, run `bundle exec rake help`. The most commonly used tasks are listed below...

After checking out the repo, run `bundle exec rake setup` to install the dependencies (requires `bundler`). Then, run `bundle exec rake test` to run the tests. You can also run `bundle exec rake console` for an interactive (`pry`) REPL that will allow you to experiment with the code.

To generate code documentation run `bundle exec yardoc`. To browse the generated documentation in a browser run `bundle exec yard server -r`. You can also use the `yri` command line tool e.g. `yri Wgit::Crawler#crawl_site` etc.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, see the *Gem Publishing Checklist* section of the `TODO.txt` file.

repository: pablitosanchez/wgit

repo_name: pablitosanchez/wgit


text:
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.5.0] - (2023-08-24)

### Added

-   Added classifier support for KMCP profiles (#129).
-   Added a command-line option `--add-rank-lineage` to the `standardise` and
    `merge` commands, which inserts a new column `rank_lineage` to results that
    contains semi-colon-separated strings with the ranks (#130).
-   Added a taxonomy table to the BIOM output format when the `--taxonomy` option is
    used (#134).

### Changed

-   Reversed the order of lineages printed to output files (#131).

## [0.4.1] - (2023-07-13)

### Fixed

-   Forced reading the `duplicates` and `coverage` columns of KrakenUniq profiles as
    float (#123).

## [0.4.0] - (2023-07-02)

### Added

-   Provided curl commands in the intro to quickly fetch example files to try out (#97).
-   Added new tutorial on merging across classifiers, and warnings why taxpasta
    does not currently do this natively (#98).
-   Added classifier support for ganon report files (#109).

### Changed

-   Made profile validation stricter, such that providing an input from another
    than the chosen profiler will usually result in an error (#101). The only
    exception is that Kraken2 standard profiles can be provided to the
    Centrifuge reader, and vice versa, since they are identical in format.

### Fixed

-   Fixed a few broken internal links (#89).
-   Simplified getting started tutorial and moved original to complex tutorial (#95).
-   Improved help text for `--output-format` option to clearly state that it disables
    any automatic detection of the output file format (#96).
-   Extended the MetaPhlAn profile reader to support version 3 & 4 profiles (#107).

## [0.3.0] - (2023-04-01)

### Added

-   Inserted columns with taxonomic information like the name, rank, or lineage of a
    taxon at the beginning of the table rather than the end (#86).

### Fixed

-   Enabled handling of MetaPhlAn profiles with taxonomy identifier -1 and combined all
    unclassified entries into one with ID 0 (#85).
-   Generally increased the tolerance in the compositionality checks. Up to 1%
    deviation are now allowed; 2% for Bracken profiles (#84).

## [0.2.3] - (2023-03-12)

### Fixed

-   Fixed the version string according to hatch-vcs output (#74).
-   Increased the tolerance when comparing floats in the Bracken profile validation
    (#76).

## [0.2.2] - (2023-03-08)

### Fixed

-   Fixed how the `taxopy.TaxDb` is loaded. It was previously deleting local files and
    downloading a new copy from NCBI instead (#67).

## [0.2.1] - (2023-03-01)

### Fixed

-   Handled metaphlan profiles with unclassified taxa in the lineage (#61).

## [0.2.0] - (2023-02-23)

### Added

-   Created command line options to expand the standard profile with name, rank,
    and/or lineage taxonomic information (#60).

### Fixed

-   Made the Krakenuniq reader more accepting (#57).
-   Documented _which_ columns are used in final standardised output profiles.

## [0.1.1] - (2023-01-28)

### Fixed

-   Documentation on where the taxpasta-supported Kaiju output comes from (i.e.,
    `kaiju2table` rather than Kaiju itself) (#54).
-   Corrected the assumption that the first two entries in a kraken2 profile are
    always unclassified and root node percentage of abundances. This is not the
    case for profiles where 100% of reads are assigned to taxa. Kraken2 profile
    validation now looks for the `'U'` and `'R'` taxonomy levels explicitly and
    checks their sum.

## [0.1.0] - (2023-01-19)

-   First release

repository: taxprofiler/taxpasta

branch_name: refs/heads/dev

gha_language: Python

repo_name: taxprofiler/taxpasta


text:
# Health
#### DEF CON for N00bs - Essential

DEF CON, like any conference or convention, can be a stressful environment both physically and mentally.

## General conference health tips:

- **Stay hydrated:** Conferences are really exciting, but it involves a lot of moving around, hot rooms, and waiting in line. It is
important that attendees stay hydrated. It is not uncommon for someone to have issues not heeding this advice, so drink some
friggin water.

- **Stay hygenic:** There are a lot of people, from a lot of places, and almost certainly someone less hygenic than you currently
are (except for that one attendee... that person is the worst). Con Flu (a.k.a., "Con Crud") is an actual thing, and it is likely you will get it if you aren't hygenic and have never been to a conference before. Bring some hand sanitizer and actually use it every now and then (or when you want that slimy grease off your hand from shanking another). Don't forget to shower.  Good attendees shower.  Be a good attendee.

- **Take a break:** Seriously. It can be a lot of fun, but rest once in a while! What happens if you don't rest? You pass out and
wake up in bed with a furry... so yeah, don't do that. Okay, okay... in all seriousness it is just a health thing. You likely will not wake up in bed with a furry (though I cannot speak for everyone on that).


text:
## The String Calculator Kata

### The Kata

1) Create a simple String calculator with a method `int add(String numbers)`

The method can take 0, 1 or 2 numbers, and will return their sum.
For an empty string it will return 0.

example inputs include: `""`, `"1"` or `"1,2"`.

* Start with the simplest test case of an empty string and move to 1 and two numbers.
* Remember to solve things as simply as possible so that you force yourself to write tests you did not think about.
* Remember to refactor after each passing test.

2) Allow the `add()` method to handle any number of numbers. eg: `"1,2,3,4"`

3) Allow the `add()` method to also handle new lines between numbers instead of simply commas.
* the following input is ok:  `"1\n2,3"`, the result is 6.
* the following input is NOT ok:  `"1,\n"`, nor is `"1\n,"`. In short, empty tokens are not allowed, aside from the empty string.

4)  Support different delimiters.
* To change a delimiter, the beginning of the string will contain a separate line that looks like this:   `//[delimiter]\n[numbers…]` for example `//;\n1;2` should return three where the default delimiter is `;`.
* The first line is optional. all existing scenarios should still be supported

5) Calling `add()` with a negative number will throw an exception `negatives not allowed` - and the negative that was passed. If there are multiple negatives, show all of them in the exception message.

6) Numbers bigger than 1000 should be ignored, so adding 2 + 1001  = 2

repository: Hydragyrum/programming-katas

repo_name: Hydragyrum/programming-katas


text:
# The Proto Company
<img src="https://fec-image-bucket.s3-us-west-2.amazonaws.com/Screen+Shot+2020-10-05+at+8.33.31+PM.png" />
## Table of Contents

- [Product Overview](#product-overview-page)
- [Features](#features)
- [Feedback](#feedback)
- [Build Process](#build-process)
- [Tech Stack](#tech-stack)

## Product Overview Page
The product overview page displays relevant information for a single product in the catalogue. The Proto Company catalogue is organized by *products*. One single product can be associated with many sizes and styles which each result in unique SKUs (stock keeping units). The product overview page presents items at the product level. Further breakdown by style or size is only reflected within the product detail page.

The same product detail page is displayed for every product in the catalogue. Upon navigating to the product detail page, using the searchbar to find a product, or selecting a new product to display, the contents of the page updates to show information relevant to the selected product.

## Features

The item detail page is comprised of distinct modules. Each module displays information related to the product being displayed.
The Overview module is the top-most module on the Product Detail page of The Proto Company website. The functionality contained within this module can be divided into several pieces:

	1. Image gallery
		1. Default View
			* Dynamic Image
			* Overlay of Dynamic Thumbnails
			* Navigation Arrows for Thumbnails and main Image
			* Magnifying Glass
		2. Expanded View
			* Dynamic Image Overlay
			* Navigation Arrows
			* Image Icons
			* Zoom In / Zoom Out Icons
	2. Product information
		1. Star Rating
			* Average Rating
			* 5 Stars
			* Reviews Link
		2. Product Category
		3. Product Title
		4. Price
			* Dynamic Price
			* Prices Linked to SKUs
			* Default Price / Style
			* Idea: Enter promo code to see adjusted price
		5. Product Overview
			* Text Field
		6. Social Share
			* Pinterest
			* Twitter
			* Facebook
			* Email
	3. Style Selector
		* Thumbnails
		* Rows
		* Default First Style / Price
		* Selected Style Checkmark Overlay
	4. Add to Cart
		1. Size Dropdown
			* Default Text “Select Size”
			* In Stock / Out of Stock
		2. Quantity Selector
			* Range from 1 to max  in stock
			* If no size selected “-“
			* If size selected 1
		3. Button
			* Places item into cart
			* If the default ‘Select Size’ is currently selected: Clicking this button should open the size dropdown, and a message should appear above the dropdown stating “Please select size”
			* If there is no stock: This button should be hidden
			* If both a valid size and valid quantity are selected: Clicking this button will add the product to the user’s cart.

## Feedback

Feel free to [file an issue](https://github.com/callback-kids/product-detail/issues/new). Feature requests are always welcome. If you wish to contribute, please submit a [pull request](https://github.com/callback-kids/product-detail/issues)

## Build Process
## Build Process

To learn React, check out the [React documentation](https://reactjs.org/).

In the project directory, you can run:

### `npm start`

Runs the app in the development mode.\
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.

The page will reload if you make edits.\
You will also see any lint errors in the console.

### `npm test`

Launches the test runner in the interactive watch mode.\
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.

### `npm run build`

Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.\
Your app is ready to be deployed!

See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.

## Tech Stack
- JavaScript
- HTML5
- CSS3
- ReactJS and React Hooks
- NodeJS
- ExpressJS
- Jest
- Enzyme
- React-Bootstrap

repository: callback-kids/product-detail

repo_name: callback-kids/product-detail


text:
# Cypress Enterprise Security Middleware Library

### Overview
This library provides the capability for Cypress's best in class Wi-Fi enabled
PSoC6 devices to connect to enterprise Wi-Fi networks. This library implements
a collection of the most commonly used Extensible Authentication Protocolx (EAP)
that are used in enterprise networks.

### Features
This section provides details on the list of enterprise security Wi-Fi features
supported by this library:
* Supports following EAP security protocols:
    * EAP-TLS
    * PEAPv0 with MSCHAPv2
    * EAP-TTLS with EAP-MSCHAPv2
* Supports TLS session (session ID based) resumption.
* Supports 'PEAP Fast reconnect' (this is applicable only for PEAP protocol).
* Supports roaming across APs in the enterprise network (vanilla roaming)
* Supports TLS versions 1.0, 1.1, and 1.2

### Supported Platforms
This middleware library and it's features are supported on following Cypress platforms:
* [PSoC6 WiFi-BT Prototyping Kit (CY8CPROTO-062-4343W)](https://www.cypress.com/documentation/development-kitsboards/psoc-6-wi-fi-bt-prototyping-kit-cy8cproto-062-4343w)

### Additional Information
* [Enterprise Security RELEASE.md](./RELEASE.md)
* [Enterprise Security API reference guide](./docs/api_reference_manual.html)
* [Enterprise Security library version](./version.txt)


text:
# Welcome to Ghostwriter's Parser/Markov Chain Generator
---

## The Team
- Ken Saganski: [saganawski](https://github.com/saganawski)
- Armando Dollia: [armandodollia](https://github.com/armandodollia)
- Mike Bonetti: [MikeyB303](https://github.com/MikeyB303)

## Purpose
The purpose of this application is to generate Markov chains and their n-grams from a given corpus for use by the [Ghostwriter web-app](https://github.com/MikeyB303/ghostwritr-web)

## Dependencies
To run this web app on your own, it is required that you have ruby installed. For instructions on ruby installation, visit their website [here](https://www.ruby-lang.org/en/)

## Getting Started
1. Clone this repository to your local computer

    `$ git clone https://github.com/saganawski/ghostwriter.git`

2. Navigate to the newly cloned repo.

    `$ cd wherever-you-cloned-the-repo-to/ghostwriter`

3. Using your favorite text-editor, open the `markov_chain.rb` file and scroll to the bottom. You'll see a line of code that looks like this:

    ```ruby
    MarkovChain.new('text-file-to-be-parsed.txt', 0.24).return_probability_chain(2, 'parsed_file_location/parsed-file-name.json')
    ```

4. Replace the following values:
    * text-file-to-be-parsed.txt -- file name of your corpus
    * 0.24 -- probability floor (this number and all probabilities smaller than it will not be included.)
    * 2 -- The amount of additional words included in the key value (integers only, use 0 for a n = 1 gram)
    * parsed_file_location/parsed-file-name.json -- The desired directory and filename where the markov chain JSON file will be saved

5. Move the JSON files to the `ghostwritr-web/db/seeds/corpus_files/` directory of your Ghostwriter web-app.

## Contributing
If you'd like to contribute to Ghostwriter, simply fork this repo, make your desired changes, and create a pull request for us to review. If all is good, we'll accept your changes!

## License
Ghostwriter is released under the [MIT License](https://opensource.org/licenses/MIT)

repository: armandodollia/ghostwriter

repo_name: armandodollia/ghostwriter


text:
---
title: '#Dear ID Episode 47: Svein-Tore on H5P'
path: "/blog/dear-id-episode-47-svein-tore-on-h5p"
date: '2017-06-29'
featured: "../featured/twitter.jpg"
categories: ["Dear ID Show", "elearning"]
tags: ["authoring tools", "elearning", "free", "interactive video", "open source"]
---

In this episode, I sit down with Svein-Tore, the founder of H5P, an amazing open source authoring tool for creating elearning interactions. We discuss the story behind H5P, the benefits of using an open-source tools and the opportunities that H5P brings newbie IDs.

<iframe src="https://simplecast.com/e/74665?style=medium-light" width="100%" height="94px" frameborder="0" scrolling="no" seamless=""></iframe>

This is an amazing set of tools bundled together FOR FREE! It's constantly being worked on and updated and added to. If you're an ID newbie looking to create interesting things, including interactive video, give H5P a try! Just a heads up to all you fabulous listeners: Dear ID will be on hiatus until I can record some more episodes. I'm looking for IDs willing to share their journey. If that's you, please get in touch! Also, as always, [leaving a rating and review for us on iTunes ](itms://itunes.apple.com/us/podcast/dear-instructional-designer/id1111995456?mt=2)would be much appreciated!

repository: anthkris/portfolio-site

gha_language: HTML

repo_name: anthkris/portfolio-site


text:
#8.5 More Schemas

##Release 2: Create a One-to-one Schema
![One to One](imgs/one-to-one.png)

##Release 4: Refactor
![Many to Many](imgs/many-to-many.png)

##Release 6: Reflect

###What is a one-to-one database?
####A one-to-one database is a one where the different table connect based on one similar column or piece of information.

###When would you use a one-to-one database? (Think generally, not in terms of the example you created).
####When you want to simplify your large confusing table into smaller easier to navigate tables that can be linked by similar pieces of information.

###What is a many-to-many database?
####A many-to-many database is one that will associate to categories that could be a property of each other and relates those instances with a join table.

###When would you use a many-to-many database? (Think generally, not in terms of the example you created).
####When you have two properties that could have many items of each other, but not always. It's better to put them in separate databases and then join them when there is overlap.

###What is confusing about database schemas? What makes sense?
####The many to many is a bit confusing and I think I need to see it in use and practice more to really understand.

gha_language: Ruby


text:
# CodeGenPattern constructor

Creates an instance.

```csharp
public CodeGenPattern(string namePattern, string requiredSubstring)
```

## See Also

* class [CodeGenPattern](../CodeGenPattern.md)
* namespace [Facility.Definition.CodeGen](../../Facility.Definition.md)

<!-- DO NOT EDIT: generated by xmldocmd for Facility.Definition.dll -->

repository: FacilityApi/Facility

gha_language: C#

repo_name: FacilityApi/Facility


text:
---
permalink: /curriculum-learning/
---
# Curriculum Learning

[Graves et al. (2017)](https://arxiv.org/abs/1704.03003) consider loss-driven and complexity-driven signals to reward learning progress, then use a nonstationary multi-armed bandit algorithm to automatically generate learning curriculum.

[Sukhbaatar et al. (2017)](https://arxiv.org/abs/1703.05407) used self-play where one agent builds a curriculum of increasingly more difficult tasks to challenge an identical agent.

## References

* 2017 April 19, Sainbayar Sukhbaatar, Ilya Kostrikov, Arthur Szlam, and Rob Fergus. [Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play](https://arxiv.org/abs/1703.05407). *arXiv:1703.05407*.
* 2017 April 10, Alex Graves, Marc G. Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. [Automated Curriculum Learning for Neural Networks](https://arxiv.org/abs/1704.03003). *arXiv:1704.03003*.
* 2016 November 6, Jacob Andreas, Dan Klein, and Sergey Levine. [Modular Multitask Reinforcement Learning with Policy Sketches](https://arxiv.org/abs/1611.01796). *arXiv:1611.01796*.
* 2016 September 22, Coline Devin, Abhishek Gupta, Trevor Darrell, Pieter Abbeel, and Sergey Levine. [Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer](https://arxiv.org/abs/1609.07088). *arXiv:1609.07088*.


text:
malu - mechanical alu
=====================

I have been fascinated by the idea of mechanical computing for many years. This project contains notes, diagrams, and other bits related to this.

A lot of this is based on Ralph C. Merkle's article "Two Types of Mechanical Reversible Logic" at http://www.zyvex.com/nanotech/mechano.html. While this article focuses primarily on computing at the nano level, I am interested in bigger - much bigger. If I could figure out how to build these using massive I-Beams, I would.

repository: d-gold/malu

repo_name: d-gold/malu


text:
---
layout: post
title: Analyzing My Spotify Top Songs 2016 - 2022 Part 2
---

Hi! Welcome to part 2 of my series on analyzing my Spotify top songs from 2016 to 2022. If you're reading this series for the first time today, on Spotify there are audio features that every song has: acousticness, danceability, duration, energy, instrumentalness, key, liveness, loudness, mode, speechiness, tempo, time signature, and valence. Last time I talked about the acousticness of my top songs. Now let's discuss danceability and duration.

### Danceability

Per Spotify's official [documentation](https://developer.spotify.com/documentation/web-api/reference/#/operations/get-audio-features), danceability is the confidence that a track is suitable for dancing (0.0 - least danceable, 1.0 - most danceable). This metric combines several musical elements including tempo, rhythm stability, beat strength, and overall regularity.

Figure 1. Danceability of Will Pisani's top songs as a function of time. The confidence that a song is suitable for dancing increases as the value increases.

### Duration
Duration is the length of the track.

Figure 2. Duration of Will Pisani's top songs as a function of time.

### Conclusion


text:
+++
title = "Storage classes"
date = 2020-12-23T16:07:26+01:00
weight = 10
chapter = true
pre = "<b>10. </b>"
+++

### Chapter 10

# Storage classes

Storage classes describe the features of a variable/function. These features basically include the scope, visibility and lifetime. They are used to trace the existence of a certain variable during the runtime of a program.

*Scope* refers to where in a program a variable may be accessed. *Lifetime* refers to how long a variable will exist or retain its value.


text:
# Quora-Question-Pair-Similarity

## 1. Business Problem
## 1.1 Description

Quora is a place to gain and share knowledge—about anything. It’s a Platform to ask questions and connect with people who contribute unique insights and quality answers. This empowers people to learn from each other and to better understand the world.

Over 100 million people visit Quora every month, so it's no surprise that many people ask similarly worded questions. Multiple questions with the same intent can cause seekers to spend more time finding the best answer to their question, and make writers feel they need to answer multiple versions of the same question. Quora values canonical questions because they provide a better experience to active seekers and writers, and offer more value to both of these groups in the long term.

> Credits: Kaggle
__ Problem Statement __

- Identify which questions asked on Quora are duplicates of questions that have already been asked.
- This could be useful to instantly provide answers to questions that have already been answered.
- We are tasked with predicting whether a pair of questions are duplicates or not.

## 1.2 Sources/Useful Links
- Source : https://www.kaggle.com/c/quora-question-pairs

____ Useful Links ____
- Discussions : https://www.kaggle.com/anokas/data-analysis-xgboost-starter-0-35460-lb/comments
- Kaggle Winning Solution and other approaches: https://www.dropbox.com/sh/93968nfnrzh8bp5/AACZdtsApc1QSTQc7X0H3QZ5a?dl=0
- Blog 1 : https://engineering.quora.com/Semantic-Question-Matching-with-Deep-Learning
- Blog 2 : https://towardsdatascience.com/identifying-duplicate-questions-on-quora-top-12-on-kaggle-4c1cf93f1c30
## 1.3 Real world/Business Objectives and Constraints
1. The cost of a mis-classification can be very high.
2. You would want a probability of a pair of questions to be duplicates so that you can choose any threshold of choice.
3. No strict latency concerns.
4. Interpretability is partially important.
## 2. Machine Learning Probelm
### 2.1 Data
### 2.1.1 Data Overview
- Data will be in a file Train.csv
- Train.csv contains 5 columns : qid1, qid2, question1, question2, is_duplicate
- Size of Train.csv - 60MB
- Number of rows in Train.csv = 404,290

## 2.1.2 Example Data point

"id","qid1","qid2","question1","question2","is_duplicate"

"0","1","2","What is the step by step guide to invest in share market in india?","What is the step by step guide to invest in share market?","0"

"1","3","4","What is the story of Kohinoor (Koh-i-Noor) Diamond?","What would happen if the Indian government stole the Kohinoor (Koh-i-Noor) diamond back?","0"

"7","15","16","How can I be a good geologist?","What should I do to be a great geologist?","1"

"11","23","24","How do I read and find my YouTube comments?","How can I see all my Youtube comments?","1"

## 2.2 Mapping the real world problem to an ML problem
### 2.2.1 Type of Machine Leaning Problem
It is a binary classification problem, for a given pair of questions we need to predict if they are duplicate or not.

### 2.2.2 Performance Metric
Source: https://www.kaggle.com/c/quora-question-pairs#evaluation

Metric(s):

- log-loss : https://www.kaggle.com/wiki/LogarithmicLoss
- Binary Confusion Matrix
## 2.3 Train and Test Construction
We build train and test by randomly splitting in the ratio of 70:30 or 80:20 whatever we choose as we have sufficient points to work with.

## step by step procedure to solve this case study
- First load the train.csv data set into data frame
- We find number of data points : 404290
- We find there are 6 columns in data
- Those are [id , qid1, qid2, question1, question2. is_duplicate ]
- clearly observe that is_dupicate is labeled column
- Question pairs are not Similar (is_duplicate = 0): 63.08%
- Question pairs are Similar (is_duplicate = 1): 36.92%
- We done some EDA to understand data much more rigorous
- We find Total number of Unique Questions are: 537933
- Number of unique questions that appear more than one time : 111780 (20.77953945937505%)
- Max number of times a single question is repeated : 157
- Number of duplicate questions 0
- Maximum number of times a single question is repeated: 157
- Checking whether there are any rows with null values
- There are two rows with null values in question2
## Basic Feature Extraction (before cleaning)
- Hear we done some basic feature extraction
- freq_qid1 = Frequency of qid1's
- freq_qid2 = Frequency of qid2's
- q1len = Length of q1
- q2len = Length of q2
- q1_n_words = Number of words in Question 1
- q2_n_words = Number of words in Question 2
- word_Common = (Number of common unique words in Question 1 and Question 2)
- word_Total =(Total num of words in Question 1 + Total num of words in Question 2)
- word_share = (word_common)/(word_Total)
- freq_q1+freq_q2 = sum total of frequency of qid1 and qid2
- freq_q1-freq_q2 = absolute difference of frequency of qid1 and qid2

## Analysis of some of the extracted features
- we find some questions have only one single words.

- Minimum length of the questions in question1 : 1

- Minimum length of the questions in question2 : 1

- Number of Questions with minimum length [question1] : 67

- Number of Questions with minimum length [question2] : 24

- When we perform univariat analysis on Feature : word_shapre

- The distributions for normalized word_share have some overlap on the far right-hand side, i.e., there are quite a lot of questions with high word similarity

- The average word share and Common no. of words of qid1 and qid2 is more when they are duplicate(Similar)

- Feature: word_Common

- The distributions of the word_Common feature in similar and non-similar questions are highly overlapping

## EDA : Advanced Feature Extraction
- we done some advanced feature extraction on data
## Preprocessing of Text
- Removing html tags
- Removing Punctuations
- Performing stemming
- Removing Stopwords
- Expanding contractions etc
- Preprocessing of Text

## Definition:

- Token: You get a token by splitting sentence a space
- Stop_Word : stop words as per NLTK.
- Word : A token that is not a stop_word

## Features:

- cwc_min : Ratio of common_word_count to min lenghth of word count of Q1 and Q2
cwc_min = common_word_count / (min(len(q1_words), len(q2_words))

- cwc_max__ : Ratio of common_word_count to max lenghth of word count of Q1 and Q2
cwc_max = common_word_count / (max(len(q1_words), len(q2_words))

- csc_min__ : Ratio of common_stop_count to min lenghth of stop count of Q1 and Q2
csc_min = common_stop_count / (min(len(q1_stops), len(q2_stops))

- csc_max__ : Ratio of common_stop_count to max lenghth of stop count of Q1 and Q2
csc_max = common_stop_count / (max(len(q1_stops), len(q2_stops))

- ctc_min__ : Ratio of common_token_count to min lenghth of token count of Q1 and Q2
ctc_min = common_token_count / (min(len(q1_tokens), len(q2_tokens))

- ctc_max : Ratio of common_token_count to max lenghth of token count of Q1 and Q2
ctc_max = common_token_count / (max(len(q1_tokens), len(q2_tokens))

- last_word_eq : Check if First word of both questions is equal or not
last_word_eq = int(q1_tokens[-1] == q2_tokens[-1])

- first_word_eq : Check if First word of both questions is equal or not
first_word_eq = int(q1_tokens[0] == q2_tokens[0])

- abs_len_diff : Abs. length difference
abs_len_diff = abs(len(q1_tokens) - len(q2_tokens))

- mean_len : Average Token Length of both Questions
mean_len = (len(q1_tokens) + len(q2_tokens))/2

- fuzz_ratio : https://github.com/seatgeek/fuzzywuzzy#usage http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/

- fuzz_partial_ratio : https://github.com/seatgeek/fuzzywuzzy#usage http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/

- Token_sort_ratio : https://github.com/seatgeek/fuzzywuzzy#usage http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/

- Token_set_ratio : https://github.com/seatgeek/fuzzywuzzy#usage http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/

- longest_substr_ratio : Ratio of length longest common substring to min lenghth of token count of Q1 and Q2
- longest_substr_ratio = len(longest common substring) / (min(len(q1_tokens), len(q2_tokens))

- append the new features as a columns to data set

## Plotting Word clouds:

- Creating Word Cloud of Duplicates and Non-Duplicates Question pairs
- We can observe the most frequent occuring words
## Visualization :

- We visualize the data using TSNE
- We find some labels are well separated and some labels are overlaped
## Vectorizer & train and test data
- we drop some columns [qid1, qid2]

- we merge three data frames by using id column

- take X = total data

- take Y = labels

- split the data into train and test data

- Applying tfidf vectorizer on train data

- Transform the vectorizer to X_train_q1 and X_train_q2

- we find most of columns/values are zeros

- so we convert train and test vectorized data into sparse matrix

- Then merge X_train_q1 and X_train_q2 using hstack function

- done both train and test data

- save those train and test data as .npz files

## Machine Learning Models
- our loss function is logloss
- In order to find best logloss first we build a random model to find log-loss
- We observe log_loss on test data using random mode : 0.887
- In order to reduce log loss we apply some machine learnign algorithams
## Logistic Regression
- When we apply logistic regression we done hyperparameter tuning
- we find best hyperparameter C / (1/lambda) is : 1
- log-loss its reduced is : 0.5146
- we are using l1 Reglarization because the features are very high Then We plot confusion matrix and Presision and Recall matrix
## Linear SVM
- When we apply linear SVM we done hyperparameter tuning
- We find best hyperparameter alpha is : 0.00001
- Log-loss its reduced is : 0.5638
- We are using l1 reglarization because the features are very high
- Then we plot confusion matrix and Rresision and recall matrixs
## XgBoost
- When we apply XgBoost we done hyperparameter tuning
- We find best hyperparameter n_estimators : 80 and max_depth : 100
- log-loss its reduced is : 0.5914
- Then we plot confusion matrix and Rresision and recall matrixs


text:
# Alexandria Core

This library contains Alexandria's core functions like kernel configuration, exceptions, query-parameters utilities, validators, etc...

This was made specifically for Go.

**IMPORTANT: Currently migrating to common-go and many more independent packages to avoid unused dependencies**


text:
# No arrests yet after Cape Town teen hit by stray bullet

Published at: **2019-11-05T16:08:00+00:00**

Author: ****

Original: [News24](https://www.news24.com/SouthAfrica/News/no-arrests-yet-after-cape-town-teen-hit-by-stray-bullet-20191105)

No arrests have yet been made after a 13-year-old girl was hit and injured by a stray bullet in Hanover Park, Western Cape police said on Tuesday.
The girl had been walking from the shop in Athburg Walk around 14:00 on Monday when she was wounded in the arm and stomach, said Sergeant Noloyiso Rwexana.
She was rushed to a medical facility for treatment.
Rwexana said the circumstances surrounding the shooting were being investigated.
Last month, a 15-year-old girl was injured after being hit by a stray bullet just a few roads away in Rywood Road.
She had apparently been standing on the street corner with two people - one an alleged gangster - when a rival group spotted him and opened fire, Eyewitness News reported.
Philippi community police forum chair Ebrahim Abrahams said the girl was recovering.
He said it had been a gang shooting and that the area was a hotspot for this activity, with two law enforcement officers shot dead in the same vicinity.
The Philippi policing precinct was identified as one of 10 crime hotspots and the South African National Defence Force was deployed in July to help curb the high murder rate.
According to police crime statistics for 2018/2019, the Philippi police station saw a 55.3% increase in attempted murder cases.

        Compiled by Jenna Etheridge.

license: Unlicense

repository: dsadlollioygjmzvcxv/tdc-exported

repo_name: dsadlollioygjmzvcxv/tdc-exported


text:
# _MyReads_

#### _A reading list project for Udacity's React Nanodegree, 9.14.2017_

#### By _**Wolfgang Warneke**_

## About this MyReads project

This is my submission for the React Fundamentals course project. It is a reading list app which uses a limited sandbox API for its book data. A user may add book to their current reading list, a reading wish list, or a list of books already read. A user may click on a book for more information and search for new books to add.

## Getting Started

If you would like to check out this project for grading purposes or otherwise...
* Clone project
* Run `npm install` on the command line to install dependencies
* Run `npm install start` on the command line to start the development server
* The development server should automatically open localhost:3030/

### Prerequisites

```
- NPM
```
If NPM is not installed, please check out this [get NPM](https://www.npmjs.com/get-npm) page and follow their instructions.

### Usage

The *BooksAPI* in use for this project provides data with some limits. It will always return 20 results (unless there are only less than that amount available) and will only accept certain terms.  The following is a list of acceptable search terms for testing the API.

```
'Android', 'Art', 'Artificial Intelligence', 'Astronomy', 'Austen', 'Baseball', 'Basketball', 'Bhagat', 'Biography', 'Brief', 'Business', 'Camus', 'Cervantes', 'Christie', 'Classics', 'Comics', 'Cook', 'Cricket', 'Cycling', 'Desai', 'Design', 'Development', 'Digital Marketing', 'Drama', 'Drawing', 'Dumas', 'Education', 'Everything', 'Fantasy', 'Film', 'Finance', 'First', 'Fitness', 'Football', 'Future', 'Games', 'Gandhi', 'Homer', 'Horror', 'Hugo', 'Ibsen', 'Journey', 'Kafka', 'King', 'Lahiri', 'Larsson', 'Learn', 'Literary Fiction', 'Make', 'Manage', 'Marquez', 'Money', 'Mystery', 'Negotiate', 'Painting', 'Philosophy', 'Photography', 'Poetry', 'Production', 'Programming', 'React', 'Redux', 'River', 'Robotics', 'Rowling', 'Satire', 'Science Fiction', 'Shakespeare', 'Singh', 'Swimming', 'Tale', 'Thrun', 'Time', 'Tolstoy', 'Travel', 'Ultimate', 'Virtual Reality', 'Web Development', 'iOS'
```

### Features

Here are some extra features beyond the scope of the original rubric:

* Individual book details view
* Search by author or related terms from details view
* "Bookmarkable" search URLs (*press Enter while inputing search query*)

## Built With

* [React](https://github.com/facebook/react) - JavaScript UI framework
* [Create React App](https://github.com/facebookincubator/create-react-app) - Tool to bootstrap a simple React boilerplate
* [React Router](https://github.com/ReactTraining/react-router) - URL rooting package for React
* [Prop Types](https://github.com/facebook/prop-types) - Used to verify component property types

## License

MIT License

Copyright (c) [2017] [Wolfgang Warneke]

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

## Acknowledgments
Thank you...
* [Udacity](https://www.udacity.com/)
* [React Training](https://www.reacttraining.com/)


text:
# Artsy
![Screenshot](./client/src/icons/Artsy.png)

## Description

Artsy is a real time miltiplayer drawing game. One person draws a word given to them and the other plays have to guess what is being drawn.

## How to play:
    1. Log in or create an account with a username
    2. Click Join a game from the Home page
    3. Enter a Room and Name to join a game
    4. Once joined there will be a random word seen on the screen for someone to draw and the others have to guess.

## Future Updates
    1.The first person to join a room will be set to the host and get a random word to draw. Currently everyone who joins will be able to see a random word and draw on the page.
    2.The person to correct the right word will gain points and be set to the drawer for the next round.
    3. Points will be awarded to the person who guesses the correct drawing.
    4. Leader Board on past wins/score
    5. Add a timer for the drawer
## Technology Used
 - React
 - Canvas
 - Socket.io
 - Node.js

## Credits
  - Marissa Adams (@mrenadams)
  - Joseph Magdaleno (@josephmagdaleno)
  - Eddie Aguilar (@Reigningeddie)

## Table of Contents

* [Installation](#installation)

* [Usage](#usage)

* [License](#license)

* [Contributing](#contributing)

* [Tests](#tests)

* [Questions](#questions)

## Installation

To install necessary dependencies, run the following command:

```
npm i
```
## Tests

To run tests, run the following command:

```
npm test
```

## Questions

If you have any questions about the repo, open an issue or contact [Reigningeddie, mrenadams, josephmagdaleno](undefined) directly at .


text:
What’s the best practice for “pre-loading” data into the store (on application start) in a Redux application?

- The most 'redux-like' way of handling the pre-loading of data would be to fire off the asynchronous action in the lifecycle method (probably componentWillMount )
of a Higher Order Component that wraps your app.

[Link](https://stackoverflow.com/questions/39356517/correct-way-to-pre-load-component-data-in-reactredux#:~:text=1%20Answer&text=The%20most%20'redux%2Dlike',Component%20that%20wraps%20your%20app.)

When using a thunk/async action that dispatches the actual action, which do you export from your reducer?

- The dispatched action updateDone mutates state before post and profile data are in the store.
This makes async/await unpredictable since we don't know when a dispatch with response data executes.

[Link](https://blog.jscrambler.com/async-dispatch-chaining-with-redux-thunk/)

middleware- software that lies between an operating system and the applications running on it

thunk- middleware that allows you to return functions, rather than just actions, within Redux


text:
# Practical Pedestal/Swagger: Slack integration
How the combination of Pedestal, Swagger and other useful patterns allowed the low friction implementation of a RESTful extension for the messaging service Slack to provide Memes-as-a-Service

Built with [reveal.js](https://github.com/hakimel/reveal.js)

### Basic slideshow

Open `index.html` in any browser.

### Full slideshow

1. Install [Node.js](http://nodejs.org/)

2. Install [Grunt](http://gruntjs.com/getting-started#installing-the-cli)

3. Clone the repo
   ```sh
   $ git clone git@github.com:oliyh/slacky-pres.git
   ```

4. Navigate to the presentation folder
   ```sh
   $ cd slacky-pres
   ```

5. Install dependencies
   ```sh
   $ npm install
   ```

6. Serve the presentation and monitor source files for changes
   ```sh
   $ grunt serve
   ```

7. Open <http://localhost:8083> to view your presentation

   You can change the port by using `grunt serve --port 8001`.

## License

MIT licensed

Copyright (C) 2015 Oliver Hine

repository: oliyh/slacky-pres

repo_name: oliyh/slacky-pres


text:
[![ZenHub] (https://raw.githubusercontent.com/ZenHubIO/support/master/zenhub-badge.png)] (https://zenhub.io)
# kujira-graphite
Daemon to cache graphite statistics in Redis database.
To be able to use library you have to copy config file to /etc :
* `cd kujira-api/kujira_graphite`
* `cp config/kujira-graphite.cfg /etc/kujira-graphite.cfg`

It is also possible to start a caching service:
* install `kujira_graphite` folder in `/opt`
* `cp service/kujira-graphite.service /etc/systemd/system/kujira-graphite.service`
* change config file `/etc/kujira-graphite.cfg` if necessary
* `systemctl daemon-reload` - to reload systemctl
* `systemctl enable kujira-graphite` - to start service during the system boot
* `systemctl start kujira-graphite` - to start service

Now you can check if kujira-graphite is active `systemctl status kujira-graphite` and check redis database.

repository: KujiraProject/kujira-graphite

repo_name: KujiraProject/kujira-graphite