text:
# FFmpeg-Web-Client

[![made-with-Go](https://img.shields.io/badge/Made%20with-Go-1f425f.svg)](https://golang.org/)

This program registered in the Windows service is executed as a server, which transcodes (encodes) an video using the resources of a web client and transmits the video to a ftp server.

## Recently

- Added the function to display the remaining time.

## Demo

![화면](https://user-images.githubusercontent.com/43510811/65562539-c1fefe80-df81-11e9-95c4-722a1ba6ac84.png)

## Browsers support

| [](http://godban.github.io/browsers-support-badges/)
IE / Edge | [](http://godban.github.io/browsers-support-badges/)
Chrome |
| --------- | --------- |
| IE10, IE11| last version

## Install

    go get github.com/Hunet-edutech/ffmpeg-web-client

## Run

1. To run the program, you need to download below libraries.

    #### for service

    * [golang/sys](https://godoc.org/golang.org/x/sys) : windows service
    * [gorilla/websocket](https://github.com/gorilla/websocket) : websocket
    * [jinzhu/copier](https://github.com/jinzhu/copier) : deep copy
    * [dutchcoders/goftp](https://github.com/dutchcoders/goftp) : ftp client
    * [ffmpeg.exe / ffprobe.exe](https://ffmpeg.zeranoe.com/builds/) : ffmpeg and ffprobe .exe file

    #### for test

    * [goftp/server](https://github.com/goftp/server) : ftp server for test

2. Change location of ffmpeg.exe / ffprobe.exe to directory of main.go

3. Run your ftp server for testing

        exampleftpd -root /tmp

4. Check your ftp server's connection

        ftp > open localhost 2121

5. Check your websocket connection after openning 'sample.html'

## API Reference

* ftp server

    | Domain      | Port           |
    | ----------  | ---------------|
    | /           | 2121           |

* default : localhost:5050

* To connect websocket

    * Request

            Get /echo

    * Progress Json Data

        | Field            | Description                                    | Optional   |
        | ---------------- | -----------------------------------------------| ---------- |
        | `fileIndex`      | file index                                     | no         |
        | `fileName`       | file name                                      | yes        |
        | `progressTime`   | current encoding time                          | yes        |
        | `progressStatus` | current encoding status (continue / end)       | no         |
        | `totalTime`      | file total time                                | yes        |
        | `timeRemaining`  | encoding time emaining (일, 시/ 분, 분/초, 초) | yes        |
        | `progressRate`   | encoding progress rate ( 00.00)                | no         |

* To check a File existence of ftp server

    * Request

            Get /fileExist

        Query parameters :

        | Field            | Description                            | Optional   |
        | ---------------- | ---------------------------------------| ---------- |
        | `uploadFileName` | file name to check(encoded file name)   | no         |
        | `courseCd`       | course code                            | no         |

    * Response

            Content-Type : text/plain;
            exmaple : true(not exist) / false(exist) / error

* To requst encoding and uploading a file

    * Request

            Post /transcoding
            Content-Type : multipart/form-data;

       - Form Data :

            | Field               | Description                            | Optional   |
            | ------------------- | ---------------------------------------| ---------- |
            | `uploadFileData`    | file data                              | no         |
            | `uploadFileInfo`    | information of 'uploadFileData'        | no         |
            | `uploadEncodingInfo`| encoding information                   | no         |

       - 'uploadFileInfo' Json Data :

            | Field             | Description                               | Optional   |
            | ----------------- | ------------------------------------------| ---------- |
            | `fileIndex`       | index                                     | no         |
            | `courseCd`        | course code                               | no         |
            | `courseNo`        | course number(file name)                  | no         |
            | `chapterNo`       | chapter number                            | yes        |
            | `courseName`      | course name                               | yes        |
            | `courseDetailName`| course detail name                        | yes        |
            | `uploadFileName`  | filename ('courseNo' + 'uploadFileExt')   | no         |
            | `uploadFileSize`  | file size                                 | yes        |
            | `uploadFileExt`   | filename extension                        | yes        |

       - 'uploadEncodingInfo' Json Data :

            | Field             | Description                               | Optional   |
            | ----------------- | ------------------------------------------| ---------- |
            | `videoCodec`      | video codec                               | no         |
            | `videoBitrate`    | video bitrate                             | no         |
            | `audioCodec`      | audio codec                               | no         |
            | `audioBitrate`    | audio bitrate                             | no         |
            | `resolution`      | resolution                                | no         |

    * Response

            Content-Type : text/plain;

* To stop trancoding

    * Request

            DELETE /transcoding

       - Form Data :

            | Field          | Description                            | Optional   |
            | -------------- | ---------------------------------------| ---------- |
            | `fileIndex`    | index                                  | no         |

    * Response

             Content-Type : text/plain;
             exmaple : true / false

* To check if service program is running

    * Request

            Get /serviceCheck

    * Response

            Content-Type : text/plain;
            exmaple : 1(running) / error

## Notice

* If you want to check logs, you must use Init() method in Run() method.
* Due to the security issue(fakepath) on the web, the process will copy a stored data received from the web client, then encode it and send it to the ftp server.
 Therefore, this process must need to delete a stored data and encoded data on local.
* You can stop to encoding a file, but you can't stop during uploading.
* You can set the output directly to the ftp server during the ffmpeg encoding process, but it is not recommended.

* the ftp server's file structure is

        /GoTest/cource_code/cource_number.mp4

## References

 * [ffmpeg command](https://ffmpeg.org/ffmpeg.html)


text:
jquery-parse-js
============

jQuery wrapper for the [Parse.com Javascript API](https://www.parse.com/docs/js_guide#javascript_guide)
If you're looking for the jQuery REST wrapper for Parse you'll be wanting [jQuery-Parse](https://github.com/srhyne/jQuery-Parse)

##Current Functionality##
 - Basic Object Retrieval
 - Getting basic relationships
 - Basic queries with some operators

##TODO##
 - Create, Update, Delete operations
 - more fully featured Query operations
 - TESTS
 -

##Basic Usage##
All methods have optional callback arguments for `successCallBack`, `errorCallBack`, and `callbackArgs`. If no callbacks are passed in the method will return the object returned by the Parse API.

    $.ParseGet('MyClass', 'abc12311', wootItWorked, crapItFailed, someUsefulArgs);
    $.ParseGetRelation(myParseObject, 'awesomeRelationshipProperty', wootItWorked, crapItFailed, someUsefulArgs);
    $.ParseQuery('MyClass', [{ 'operator': 'equalTo', 'key': 'moarProperty', 'value': 'RAWR', wootItWorked, crapItFailed, someUsefulArgs }];

##Contributing##
I absolutely admit that [I am the coding horror](http://blog.codinghorror.com/on-the-meaning-of-coding-horror/). This is my first Github project so cut me a little slack ;)

If something sucks, could be done better, or needs to be done feel free to [create an issue](https://github.com/ultimatemonty/jQuery.Parse/issues/new) or fix it and [submit a PR](https://github.com/ultimatemonty/jQuery.Parse/compare/). Or fork and make it better. Whatever suits you.

license: MIT

repository: ultimatemonty/jquery-parse-js

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: ultimatemonty/jquery-parse-js


text:
# slimfish

Minimal, customizable fast and elegant [fish](https://github.com/fish-shell/fish-shell) prompt. Displays the right information at the right time.

Features:
- sleek look
- customizable non-blocking git information display
- prompt symbol indicates if prompt background tasks are running
- exit code of last command if the exit code is not zero
- runtime of executed command if it exceeds a threshold
- username and host name if different than current user or connected to a ssh server
- very customizable

With most information (connected to ssh server, runtime and exit status from last command):

For a zsh compatible version of this theme have a look at [slimline](https://github.com/mgee/slimline).

## Table Of Contents

- [Requirements](#requirements)
    - [Optional](#optional)
- [Installation](#installation)
- [Options](#options)
    - [Prompt Symbol](#prompt-symbol)
    - [Current Working Directory](#current-working-directory)
    - [Exit Status](#exit-status)
    - [Execution Time](#execution-time)
    - [User and Host Info](#user-and-host-info)
    - [AWS Profile Info](#aws-profile-info)
    - [Auto Correction](#auto-correction)
    - [Git Information](#git-information)
        - [Gitline](#gitline)
    - [Async Mode](#async-mode)
- [Example](#example)
- [Thanks](#thanks)
- [License](#license)

## Requirements

* fish

### Optional

* python 2.6+ to enable git information display

## Installation

With [fisher](https://github.com/jorgebucaran/fisher)

```fish
fisher add mengelbrecht/slimfish
```

## Options

Slimfish can be customized using a variety of environment variables.
For an example on how to do so see the [example](#example).

### Prompt Symbol

##### `SLIMFISH_PROMPT_SYMBOL`

Defines the symbol of the prompt. Default is `∙`.

##### `SLIMFISH_PROMPT_SYMBOL_COLOR_WORKING`

Defines the color of the prompt when asynchronous tasks are running. Default is `red`.

##### `SLIMFISH_PROMPT_SYMBOL_COLOR_READY`

Defines the color of the prompt when all asynchronous tasks are finished. Default is `white`.

### Current Working Directory

##### `SLIMFISH_CWD_COLOR`

Defines the color of the current working directory. Default is `cyan`.

##### `SLIMFISH_CWD_ROOT_COLOR`

Defines the color of the current working directory if it equals the root directory `/`. Default is `red`.

### Exit Status

##### `SLIMFISH_DISPLAY_EXIT_STATUS`

Defines whether the exit status is displayed if the exit code is not zero. Default is `1`.

##### `SLIMFISH_EXIT_STATUS_SYMBOL`

Defines the symbol of the exit status glyph. Default is `↵`.

##### `SLIMFISH_EXIT_STATUS_COLOR`

Defines the color of the exit status information. Default is `red`.

### Execution Time

##### `SLIMFISH_DISPLAY_EXEC_TIME`

Defines whether the runtime of a process is displayed if it exceeds the maximum execution time
specified by the option below. Default is `1`.

##### `SLIMFISH_MAX_EXEC_TIME`

Defines the maximum execution time of a process until its run time is displayed on exit.
Default is `5` seconds.

##### `SLIMFISH_EXEC_TIME_COLOR`

Defines the color of the execution time. Default is `yellow`.

### User and Host Info

##### `SLIMFISH_DISPLAY_USER_HOST_INFO`

Defines whether the `user@host` part is displayed if the user differs from the default user or if connected to a ssh server. Default is `1`.

##### `SLIMFISH_USER_COLOR`

Defines the color of the user. Default is `green`.

##### `SLIMFISH_USER_ROOT_COLOR`

Defines the color of the user if the user is root. Default is `red`.

##### `SLIMFISH_HOST_COLOR`

Defines the color of the host. Default is `yellow`.

### AWS Profile Info

##### `SLIMFISH_DISPLAY_AWS_INFO`

Defines whether value of `AWS_PROFILE` environment variable should be displayed. Default is `0`.

##### `SLIMFISH_AWS_COLOR`

Defines the color of aws profile info. Default is `blue`.

### Async Mode

#### `SLIMFISH_ENABLE_ASYNC`

Defines whether the prompt data should be obtained asynchronously (if possible).
This currently only includes the gitline data. Default is `1`.

### Git Information

##### `SLIMFISH_ENABLE_GIT`

Defines whether git information shall be displayed (requires python). Default is `1`.

#### Gitline

slimfish uses [gitline](https://github.com/mgee/gitline) to display git information.
gitline can be extensively customized. Have a look at the [gitline options](https://github.com/mgee/gitline#options).

## Example

Here is an example for customizing the prompt symbol as well as the git repository indicator and
branch format:

```shell
set -x SLIMFISH_PROMPT_SYMBOL '$'
# If you have a powerline compatible font you can also use the alternative repo indicator ''.
set -x GITLINE_REPO_INDICATOR '${reset}git'
set -x GITLINE_BRANCH '[${blue}${branch}${reset}]'
```

## Thanks

- [sindresorhus/pure](https://github.com/sindresorhus/pure)
- [sorin-ionescu/prezto](https://github.com/sorin-ionescu/prezto.git)

## License

Released under the [MIT License](LICENSE)

repository: edisile/slimfish

gha_license_id: MIT

repo_name: edisile/slimfish


text:
# Labs Sample Stream

> The labs sample stream is still in preview is likely to become a replacement of the current sample stream in the future. ([twitter doc](https://developer.twitter.com/en/docs/labs/sampled-stream/overview))

Tweetinvi does not properly support labs yet.\
Here is a workaround to work with the labs sample stream v1.

``` c#
// You need to use ConsumerOnly credentials to run a sample stream from labs
var appCredentials = new ConsumerOnlyCredentials("CONSUMER_KEY", "CONSUMER_SECRET")
{
    BearerToken = "APP_BEARER_TOKEN"
};

var appClient = new TwitterClient(appCredentials);
```

``` c#
var stream = appClient.Streams.CreateTweetStream();
stream.StallWarnings = null; // this is required as this parameter does not exist for this endpoint

stream.EventReceived += (sender, eventReceived) =>
{
    Console.WriteLine(eventReceived.Json);
};

await stream.StartAsync("https://api.twitter.com/labs/1/tweets/stream/sample");
```

Labs sample stream only support application credentials (with a bearer token).

Learn more about Application Only Credentials


text:
---
title: "Rhinoceros"
playwright: "Eugène Ionesco"
period: Spring
season: "In House"
season_sort: 410
date_start: 2015-06-10
date_end: 2015-06-13
venue: New Theatre

cast:
  - role: Berenger
    name: Eoin Buckley
  - role: Jean
    name: Jake Leonard
  - role: Daisy
    name: Penny Bainbridge
  - role: Mr Papillon
    name: Joe Hincks
  - role: Dudard
    name: Max Miller
  - role: Botard
    name: Omid Faramarzi
  - role: Logician
    name: Emma Kendall
  - role: Little Old Man / Grocer
    name: Tom Tolond
  - role: Old Gentleman
    name: Bernie Kerr
  - role: Little Old Woman
    name: Niamh Caines
  - role: Mrs Bouef / The Housewife
    name: Lou Knapp
  - role: Cafe Proprietor / Fireman
    name: Daniella Finch

crew:
  - role: Director
    name: Chris Trueman
  - role: Producer
    name: Molly McGee
  - role: Assistant Director
    name: Daniel Stranks
  - role: Technical Director
    name: Sam Osborne
  - role: Technical Director
    name: Joanne Blunt
  - role: Lighting Designer
    name: Sam Osborne
  - role: Lighting Designer
    name: Joanne Blunt
  - role: Set Construction
    name: Joseph Heil
  - role: Poster Designer
    name: Max Miller
  - role: Artwork
    name: Felix Taylor
  - role: Technical Operator
    name: Daniel O'Connor
  - role: Technical Operator
    name: Emma Fearon
  - role: Stage Manager
    name: Ollie Shortt
  - role: Stage Manager
    name: Olivia Rose French
  - role: Stage Manager
    name: Tom Proffitt
  - role: Stage Manager
    name: Beth Wilson
  - role: Stage Manager
    name: Jessica Millott
  - role: Stage Manager
    name: Rachel Connolly
  - role: Stage Manager
    name: Matt Standen
  - role: Stage Manager
    name: Darcey Graham

assets:
  - type: poster
    image: r8xWsHz
  - type: poster
    image: SVhSc3c
  - type: poster
    image: Lg2x2Cm
  - type: backstage
    video: HGWNzQB
    title: Backstage At
  - type: programme
    title: Rhinoceros Programme
    filename: rhino.pdf

links:
  - type: Review
    href: http://kevcastletheatrereviews.blogspot.co.uk/2015/06/rhinoceros-by-eugene-ionesco-at-new.html
    snapshot: M5pwF
    publisher: Kev Castle Reviews
    title: "\"Rhinoceros\" by Eugene Ionesco"
    date: 2015-06-10
    quote: "Some lovely comic touches in the script, well performed by the major as well as the lesser characters."
  - type: Review
    href: https://impactnottingham.com/2015/06/rhinoceros-nnt/
    snapshot: HoVhn
    publisher: Impact Nottingham
    author: Amy Wilcockson
    title: "Rhinoceros @ NNT"
    date: 2015-06-11
    rating: 8/10
    quote: "This satirical, seemingly nonsensical play at times, was overall a great performance to watch."

prod_shots: Tz3wCC

trivia:
  - quote: As can be seen in some of the prod shots, we attempted to install a functional proscenium curtain that lowered and rose at the top of each act. Ended up getting cut during the dress run as it buckled under its own weight.
    name: Sam Osborne
    submitted: 2019-06-25
  - quote: |
      On one of the final performances of the show, one of the doors on the set fell off.

      During the sequence which followed this event, Eoin Buckley (playing Berenger) was meant to turn up at the character of Jean's house and shout out the line "Your door's open".

      Thinking on his feet, Eoin shouting instead "Your door's gone" to which Jake Leonard (who played Jean) responded with the rather hilarious ad lib "Stupid cowboy builders" and the play continued.
    name: Tom Proffitt
    submitted: 2019-09-19
---

Written by one of the founding fathers of the ‘the Theatre of the Absurd’ / ‘Rhinoceros’ follows Berenger / a disenfranchised and disillusioned anti-hero / as he refuses to change whilst the rest of the characters turn into rhinoceroses. A dark / dystopian / satirical phase / ‘Rhinoceros’ is fundamentally a play about conformity and the cult of opinion. However / the play touches on many ideas / such as the meaningless of language / the ridiculousness of everyday convention and alienation. This chilling representation of humanity attacks the arrogance of our everyday moral convictions.

repository: newtheatre/history-project

gha_language: HTML

repo_name: newtheatre/history-project


text:
# route-66

A Clojure library for routing web requests.

I created Route-66 to solve some problems that I had with Compojure. In
particular, I wanted to separate the matching of routes from the handling of
requests. With route-66, routes are matched using Clout, as in Compojure, but
the handler is not immediately called; instead, it is inserted into the request
map under the `:handler` key to be handled later.

In addition to adding the `:handler` key, the route-matcher can insert other
data into the request-map, for use by middleware. With Compojure, this is not
possible because route-matching is always immediately followed by a call to the
handler - you cannot insert middleware that comes after route-matching.

Another goal of route-66 was to use significantly less macro-magic when defining
routes. Route-definitions are simply data - no macros invovled. This means that
you can use the full power of Clojure to build and transform your
route-definitions.

## Usage

```clojure
(ns my.awsome.webapp
  (:require [route-66 [core :as r66]
                      [compojure :refer :all]))

(defroutes app-routes
  (GET "/" [] "hello")
  (GET "/user/:id" [id] (show-user id))
  (context "/foo"
    (GET "/:id" [id] (show-foo id))
    (PUT "/" [] new-foo)))

(def handler
  (-> r66/handler                      ;; More-or-less just calls
                                       ;; ((:handler request) request)

      some-middleware                  ;; This middleware can make decisions
                                       ;; that depend on which route was matched.

      (r66/wrap-routing app-routes)    ;; Matches the url, picks a handler, and
                                       ;; adds it to the request map.

      other-middleware                 ;; Normal (pre-match) middleware
      ))

```

```clojure

(def app-routes
  [{:method :get :path "/" :handler (fn [] "hello")}
   {:method :get :path "/user/:id" :handler (fn [id] (show-user id))}
   {:method :any :path "/foo" :handler ???}])

```

## License

Copyright © 2013 Chris Perkins

Distributed under the MIT License.


text:
# Migrate ECS from a shared cloud hosting server to a DDH {#task_smm_jnn_tdb .task}

To meet your business requirements for flexible deployment, you can migrate ECS instances from a shared cloud hosting server to a DDH under your account.

Before migrating an ECS instance from a shared cloud hosting server to a DDH, make sure the following:

-   You have at least one DDH that can run the instance of the specified type. For more information, see [create a DDH](../../../../intl.en-US/Quick Start/Create a DDH.md#).
-   The destination DDH has sufficient available resources for the ECS instance. For more information, see [view resources on a DDH](intl.en-US/User Guide/View resources on a DDH.md#).
-   The billing method of the ECS instance is supported.

Before you migrate an ECS instance from a shared cloud hosting server to a DDH, consider the following:

-   The ECS instance and the destination DDH must be owned by one account and in the same region and zone. Besides, the ECS instance type must be eligible on the DDH. For more information, see [dedicated host types](../../../../intl.en-US/Product Introduction/Dedicated Host types.md#).

    **Note:** ECS instances with local SSD disks cannot be migrated.

-   The ECS instance must be in the **Stopped** status.

    **Note:** An ECS instance must be in the Stopped status when migrating. Stopping an ECS instance may interrupt your business operations. Proceed with caution.

-   The billing method of the ECS instance depends on the billing method of the destination DDH:

    [Subscription](../../../../intl.en-US/Pricing/Subscription.md#) On a Subscription DDH, both Subscription and Pay-As-You-Go ECS instances can run. The Subscription ECS instance to be migrated must expire earlier than the destination DDH.

1.   Log on to the [ECS console](https://ecs.console.aliyun.com/#/home).
2.   In the left-side navigation pane, click **Instances**.
3.   Select a region.
4.  In the upper-right corner of the instance list, click the Set Display Items icon, and on the dialog box, click **Dedicated Host**, and then click **OK**. You can view the**Dedicated Host** column in the table, which helps you locate the host that an ECS instance is hosted on.
5.   Find an ECS instance, and in the **Actions** column, select **More** \> **Stop**, to stop the instance.
6.   When the instance is in the **Stopped** status, in the **Actions** column, select **More** \> **Modify DDH Deployment**.
7.   On the dialog box, select a DDH and click **OK**.

Refresh the instance list until the Dedicated Host information is updated. The instance starts automatically. Finally It is in the **Running** status.

url: https://raw.githubusercontent.com/junjunfun/ddh/58edb6866dd7c53e187b373a63e64bbdb0ba07e7/intl.en-US/User Guide/Migrate ECS from a shared cloud hosting server to a DDH.md

repository: junjunfun/ddh

repo_name: junjunfun/ddh


text:
# No. Phone. WTF!

## Contribute

Let's keep the line items short. If you have a good ideas, corrections, and edits, please send in your changes via a [Pull Request](https://github.com/oinam/no.phone.wtf/pulls).

## License

Licensed under MIT.

branch_name: refs/heads/main


text:
# Changelog for [0.2.0] (2020-12-07)

The following sections list the changes in ownCloud File picker 0.2.0.

[0.2.0]: https://github.com/owncloud/file-picker/compare/v0.1.0...v0.2.0

## Summary

* Bugfix - Reset resource selection when opening folder: [#11](https://github.com/owncloud/file-picker/pull/11)
* Enhancement - Bootstrap Vue: [#19](https://github.com/owncloud/file-picker/pull/19)
* Enhancement - Do not import Vue: [#16](https://github.com/owncloud/file-picker/pull/16)

## Details

* Bugfix - Reset resource selection when opening folder: [#11](https://github.com/owncloud/file-picker/pull/11)

   We've added reset of selected resources when a folder is opened. This prevents from having
   selected resources which are not visible.

   https://github.com/owncloud/file-picker/pull/11

* Enhancement - Bootstrap Vue: [#19](https://github.com/owncloud/file-picker/pull/19)

   We've bootstrapped Vue to add access to the file picker via window.

   https://github.com/owncloud/file-picker/pull/19

* Enhancement - Do not import Vue: [#16](https://github.com/owncloud/file-picker/pull/16)

   We've stopped importing Vue because it is not bundled during the build process and needs to be
   included as a global variable in the target web application.

   https://github.com/owncloud/file-picker/pull/16

# Changelog for [0.1.0] (2020-09-01)

The following sections list the changes in ownCloud File picker 0.1.0.

[0.1.0]: https://github.com/owncloud/file-picker/compare/f9d6d96e52b9fa0c275e9971a11e02d11e9ecd85...v0.1.0

## Summary

* Enhancement - Add basic implementation of file picker: [#2](https://github.com/owncloud/file-picker/pull/2)

## Details

* Enhancement - Add basic implementation of file picker: [#2](https://github.com/owncloud/file-picker/pull/2)

   We've added a basic implementation of ownCloud file picker. File picker is a web component
   which can be integrated into existing products and is firing event providing array of selected
   resources

   https://github.com/owncloud/file-picker/pull/2

repository: gavinwang-au/file-picker

repo_name: gavinwang-au/file-picker


text:
# Simple calculator

This example is application that performs basic mathematical calculation.
The input is hard coded into the application, if you want to change it just rewrite the *run.py* file.

## Usage

You need grammpy library installed before running the example.
Type one the following command.
```text
pip install -r requirements.txt
```

You can then run the application by executing *run.py* script.

```text
python ./run.py
```

It will output result of `10 + (5 * 2 + 4) / 2 * 4` statement.
You can change the statement in the source code.

-----

Author: Patrik Valkovič

Licence: MIT

repository: PatrikValkovic/grammpy

gha_language: Python

repo_name: PatrikValkovic/grammpy


text:
---
layout: post
title: "[LeetCode 123] Best Time to Buy and Sell Stock III"
comments: true
category: Leetcode

---

### Question
[link](https://oj.leetcode.com/problems/best-time-to-buy-and-sell-stock-iii/)

Say you have an array for which the ith element is the price of a given stock on day i.

Design an algorithm to find the maximum profit. You may complete at most two transactions.

Note:

You may not engage in multiple transactions at the same time (ie, you must sell the stock before you buy again).

### Stats

		Frequency
		1

		Difficulty
		4

		Adjusted Difficulty
		4

		Time to use
		--------

Ratings/Color = 1(white) 2(lime) 3(yellow) 4/5(red)

### Analysis

__This is much harder to solve then previous 2 question combined__.

### Solution

The best solution is explained in [this blog](http://rleetcode.blogspot.sg/2014/02/best-time-to-buy-and-sell-stock-iii-java.html).

> we can just dived the whole prices array at every point, try to calculate max profit from left and from right respectively...

> However, there are many repeat calculations. So we can apply DP to record max profits for each left side and right side. then add them together at each point.

A detailed illustration with examples can be found [here](http://yucoding.blogspot.sg/2012/12/leetcode-question-10-best-time-to-buy.html).

However, the coding part is slightly difficult at least for me. __I made a terrible mistake with the variable 'leftMin' and 'rightMax'__. I declared it as __'rightMin'__ instead, and then the program logic is wrong. We should avoid this kind of mistakes.

### Code

    public int maxProfit(int[] prices) {
        int len = prices.length;
        if (len == 0) return 0;
        int leftMin = Integer.MAX_VALUE, rightMax = Integer.MIN_VALUE;
        int leftPro[] = new int[len], rightPro[] = new int[len];
        for (int j = 0; j <= len - 1; j ++) {
            leftMin = Math.min(leftMin, prices[j]);
            if (j == 0) leftPro[0] = 0;
            else leftPro[j] = Math.max(leftPro[j-1], prices[j] - leftMin);
        }
        for (int k = len - 1; k >= 0; k --) {
            rightMax = Math.max(rightMax, prices[k]);
            if (k == len - 1) rightPro[len-1] = 0;
            else rightPro[k] = Math.max(rightPro[k+1], rightMax - prices[k]);
        }
        int totalPro = 0;
        for (int i = 0; i < len; i ++) {
            totalPro = Math.max(totalPro, leftPro[i] + rightPro[i]);
        }
        return totalPro;
    }

branch_name: refs/heads/source


text:
# Draft RawFS Specification
RawFS is an extremely rudimentary file system intended for boot media that will
never be written to by the booted environment. This is not to say that it can't
be written to. It is indended to be a light weight, minimal implementation to
assist with rapid development of _vboot_ and _vkernel_.

**It should never be used for a production environment!**

### Layout
The layout of RawFS is basic, and is illustrated below.

```

  +-----+-----+---------
  |  A  |  B  |   ...
  +-----+-----+---------

 	A : 512 byte bootsector
 	B : 512 byte meta-data block
 	C : Contiguous array of "files"

```

The bootsector is like any other bootsector. It should contain a BIOS Parameter
Block, and have a signature of `AA55` to allow the BIOS to recognise the disk as
bootable.

The Metadata Block is a binary blob of configuration specific information. It
has the following structure:

| Name         | Offset           | Size |
| ------------ | ---------------- | ---- |
| boot_string  | 0                | 32   |
| stage2_off   | 32               | 2    |
| stage2_len   | 34               | 2    |
| kernel_off   | 36               | 4    |
| kernel_len   | 38               | 4    |
| module_count | 42               | 4    |
| module_off   | 46 + (index * 8) | 4    |
| module_len   | 50 + (index * 8) | 4    |

repository: tjhancocks/vboot

gha_language: Assembly

repo_name: tjhancocks/vboot


text:
# Rinodina teichophila (Nyl.) Arnold SPECIES

#### Status
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Flora, Jena 46: 329 (1863)

#### Original name
Lecanora teichophila Nyl.

### Remarks
null

repository: mdoering/backbone

repo_name: mdoering/backbone


text:
---
sidebarDepth: 0
---

# GridLayout

`GridLayout` is a layout container that lets you arrange its child elements in a table-like manner.

The grid consists of rows, columns, and cells. A cell can span one or more rows and one or more columns. It can contain multiple child elements which can span over multiple rows and columns, and even overlap each other.

By default, `GridLayout` has one column and one row. You can add columns and rows by configuring the columns and the rows properties. In these properties, you need to set the number of columns and rows and their width and height. You set the number of columns by listing their widths, separated by a comma. You set the number of rows by listing their heights, separated by a comma.

You can set a fixed size for column width and row height or you can create them in a responsive manner:

An absolute number: Indicates a fixed size.
auto: Makes the column as wide as its widest child or makes the row as tall as its tallest child.
*: Takes as much space as available after filling all auto and fixed size columns or rows.

## Live Demo

<DocExampleBox :liveDemoMode="true">
  <GridLayoutLiveDemo />
</DocExampleBox>

The following example creates a simple 2-by-2 grid with fixed column widths and row heights.

<DocExampleBox >

```html
<GridLayout class="grid" width="200" height="220" columns="115, 115" rows="115, 115">
  <Label text="0,0" row="0" col="0" backgroundColor="#43b883"/>
  <Label text="0,1" row="0" col="1" backgroundColor="#1c6b48"/>
  <Label text="1,0" row="1" col="0" backgroundColor="#289062"/>
  <Label text="1,1" row="1" col="1" backgroundColor="#43b883"/>
</GridLayout>
```

```scss
.grid {
  margin-bottom: 12px;
}
```

<GridLayoutDoc />
</DocExampleBox>

The following example creates a grid with responsive design, where space is alotted proportionally to child elements.

<DocExampleBox >

```html
<GridLayout width="200" height="200" columns="*, 2*" rows="2*, 3*" backgroundColor="#3c495e">
  <Label text="0,0" row="0" col="0" backgroundColor="#43b883"/>
  <Label text="0,1" row="0" col="1" backgroundColor="#1c6b48"/>
  <Label text="1,0" row="1" col="0" backgroundColor="#289062"/>
  <Label text="1,1" row="1" col="1" backgroundColor="#43b883"/>
</GridLayout>
```

<GridLayoutResponsiveDoc />
</DocExampleBox>

The following example creates a complex grid with responsive design, mixed width and height settings, and some merged cells.

<DocExampleBox >

```html
<GridLayout width="200" height="200" columns="40, auto, *" rows="40, auto, *" backgroundColor="#3c495e">
  <Label text="0,0" row="0" col="0" backgroundColor="#43b883"/>
  <Label text="0,1" row="0" col="1" colSpan="2" backgroundColor="#1c6b48"/>
  <Label text="1,0" row="1" col="0" rowSpan="2" backgroundColor="#289062"/>
  <Label text="1,1" row="1" col="1" backgroundColor="#43b883"/>
  <Label text="1,2" row="1" col="2" backgroundColor="#289062"/>
  <Label text="2,1" row="2" col="1" backgroundColor="#1c6b48"/>
  <Label text="2,2" row="2" col="2" backgroundColor="#43b883"/>
</GridLayout>
```

<GridLayoutComplexGridDoc />
</DocExampleBox>

## Props

| Name    | Type   | Default | Web | Mobile | Description |
| ------- | ------ | ------- | --- | ------ | ----------- |
| columns | String |    -    | ✔   | ✔      | A string value representing column widths delimited with commas.
Valid values: an absolute number, auto, or \*.
 A number indicates an absolute column width. auto makes the column as wide as its widest child. \* makes the column occupy all available horizontal space. The space is proportionally divided over all star-sized columns. You can set values such as 3\* and 5\* to indicate a ratio of 3:5 in sizes. |
| rows    | String |    -    | ✔   | ✔      | A string value representing row heights delimited with commas.
 Valid values: an absolute number, auto, or \*.
 A number indicates an absolute row height. auto makes the row as tall as its tallest child. \* makes the row occupy all available vertical space. The space is proportionally divided over all star-sized rows. You can set values such as 3\* and 5\* to indicate a ratio of 3:5 in sizes. |

## Additional Children Props

| Name    | Type   | Default | Web | Mobile | Description |
| ------- | ------ | ------- | --- | ------ | ----------- |
| row     | Number | `0`     | ✔   | ✔      | Specifies the row for this element. Combined with a col property, specifies the cell coordinates of the element.
 The first row is indicated by 0. |
| col     | Number | `0`     | ✔   | ✔      | Specifies the column for the element. Combined with a row property, specifies the cell coordinates of the element.
 The first column is indicated by 0. |
| rowSpan | Number | `1`     | ✔   | ✔      | Specifies the number of rows which this element spans across. |
| colSpan | Number | `1`     | ✔   | ✔      | Specifies the number of columns which this element spans across. |

repository: vuexp/vuexp

gha_language: JavaScript

repo_name: vuexp/vuexp


text:
# Any

This is a implementation of [N4562](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4562.html) std::experimental::any (merged into C++17) for C++11 compilers.

It contains a small object optimization for objects with a size of up to 2 words (such as  `int`, `float` and `std::shared_ptr`). Storing those objects in the container will not trigger a dynamic allocation.

For a easy to understand documentation, see [cppreference](http://en.cppreference.com/w/cpp/experimental/any), except our namespace is `linb`.

## Defines

You may additionally define the following preprocessor symbols (making the implementation non-standard):

  + `ANY_IMPL_NO_EXCEPTIONS`: This disables code paths that throw exceptions.
  + `ANY_IMPL_NO_RTTI`: This disables `any::type()`, thus removing all code paths using RTTI.
  + `ANY_IMPL_FAST_TYPE_INFO_COMPARE`: When checking if two `typeid` are the same, performs just a pointer comparison instead of the actual `type_info::operator==` comparison. Be aware this isn't recommended since objects passed across library boundaries mayn't compare equal. This has no effect if `ANY_IMPL_NO_RTTI` is defined.


text:
---
title: Fundamentals of Data Engineering
author: Week 10 - sync session
...

---

#
## Assignment Review
- Review your Assignment 09
- Get ready to share
- `docker pull midsw205/base:latest`
- `git pull` in `~/w205/course-content`

::: notes
- Seriously, took me about 5 min the other day

#### Breakout at about 5 after the hour:
- Check in with each group
- have students share screen
:::

#
## { data-background="images/pipeline-simple-steel-thread.svg" }

::: notes
What we need to have an end to end pipeline is:
- something up front generating events
- something to query out the back end
:::

## { data-background="images/pipeline-steel-thread-for-mobile-app.svg" }

::: notes
Let's walk through this
- user interacts with mobile app
- mobile app makes API calls to web services
- API server handles requests:
    - handles actual business requirements (e.g., process purchase)
    - logs events to kafka
- spark then:
    - pulls events from kafka
    - filters/flattens/transforms events
    - writes them to storage
- presto then queries those events
:::

#
## Project 3 options

- All: Essential game shopping cart data for homework
- Advanced option 1: Generate and filter more types of items
- Advanced option 2: Enhance the API to accept parameters for purchases (sword/item type)
- Advanced option 3: Shopping cart data & track state (e.g., user's inventory)

::: notes
- All: Game shopping cart data used for homework (flask app)
- Advanced option 1: Generate (in flask) and filter (in spark) more types of items.
- Advanced option 2: Enhance the API (in flask) to accept parameters for purchases (sword/item type) and filter (in spark)
- Advanced option 3: Shopping cart data & track state (e.g., user's inventory) and filter (in spark)

:::

#
## Flask with Kafka and Spark

## Set up directory, get docker-compose
```
mkdir ~/w205/flask-with-kafka-and-spark/
```
```
cd ~/w205/flask-with-kafka-and-spark/
```
```
cp ~/w205/course-content/10-Transforming-Streaming-Data/docker-compose.yml .
```

## The `docker-compose.yml`

```
---
version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
    expose:
      - "2181"
      - "2888"
      - "32181"
      - "3888"
    extra_hosts:
      - "moby:<IP_ADDRESS>"

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    expose:
      - "9092"
      - "29092"
    extra_hosts:
      - "moby:<IP_ADDRESS>"

  spark:
    image: midsw205/spark-python:0.0.5
    stdin_open: true
    tty: true
    volumes:
      - ~/w205:/w205
    expose:
      - "8888"
    ports:
      - "8888:8888"
    extra_hosts:
      - "moby:<IP_ADDRESS>"
    command: bash

  mids:
    image: midsw205/base:0.1.8
    stdin_open: true
    tty: true
    volumes:
      - ~/w205:/w205
    expose:
      - "5000"
    ports:
      - "5000:5000"
    extra_hosts:
      - "moby:<IP_ADDRESS>"
```

::: notes
- no need for a datafile on this one.

#
## Web-app

```python
#!/usr/bin/env python
from kafka import KafkaProducer
from flask import Flask
app = Flask(__name__)
producer = KafkaProducer(bootstrap_servers='kafka:29092')
topic = 'events'

@app.route("/")
def default_response():
    producer.send(topic, 'default'.encode())
    return "This is the default response!\n"

@app.route("/purchase_a_sword")
def purchase_a_sword():
    producer.send(topic, 'purchased_sword'.encode())
    return "Sword Purchased!\n"
```

::: notes
Take our instrumented web-app from before
:::

## More informative events

```python
#!/usr/bin/env python
import json
from kafka import KafkaProducer
from flask import Flask

app = Flask(__name__)
producer = KafkaProducer(bootstrap_servers='kafka:29092')

def log_to_kafka(topic, event):
    producer.send(topic, json.dumps(event).encode())

@app.route("/")
def default_response():
    default_event = {'event_type': 'default'}
    log_to_kafka('events', default_event)
    return "This is the default response!\n"

@app.route("/purchase_a_sword")
def purchase_a_sword():
    purchase_sword_event = {'event_type': 'purchase_sword'}
    log_to_kafka('events', purchase_sword_event)
    return "Sword Purchased!\n"
```

::: notes
let's get that to generate slightly more realistic events
:::

## Run it

```
docker-compose exec mids \
  env FLASK_APP=/w205/flask-with-kafka-and-spark/game_api_with_json_events.py \
  flask run --host <IP_ADDRESS>
```

::: notes
```
docker-compose exec mids env FLASK_APP=/w205/flask-with-kafka-and-spark/game_api_with_json_events.py flask run --host <IP_ADDRESS>
```
:::

## Test it by generating events

```
docker-compose exec mids curl http://localhost:5000/
```
```
docker-compose exec mids curl http://localhost:5000/purchase_a_sword
```

::: notes
- test it out by generating some events
```
docker-compose exec mids curl http://localhost:5000/
```
```
docker-compose exec mids curl http://localhost:5000/purchase_a_sword
```
- (repeat those a few times)
:::

## Read from kafka

```
docker-compose exec mids \
  kafkacat -C -b kafka:29092 -t events -o beginning -e
```

::: notes
and then reading them from kafka
```
docker-compose exec mids kafkacat -C -b kafka:29092 -t events -o beginning -e
```
:::

## Should show

```
{"event_type": "default"}
{"event_type": "default"}
{"event_type": "default"}
{"event_type": "purchase_sword"}
{"event_type": "purchase_sword"}
{"event_type": "purchase_sword"}
{"event_type": "purchase_sword"}
...
```

#

## Add more events

- Let's add more stuff to the events we're sending.
- Will do this over 2 breakouts.

## Breakout 1 Discussion

- Discuss business requirements for the project
- Which fields should we include?

::: notes
- What are the kinds of events that I need to track as data scientist at the game company?
  * List events
  * Give business reasons for tracking them
  * Groups report out
      * e.g., one event is "join a guild", what sort of business reason would you be tracking "join a guild"?
      * What other events other than the ones we're tracking now would you want to track?
:::

## Breakout 2 Discussion

- What info is available with these events?
- How should the events be structured?

::: notes
- 2nd breakout is the tech version -
  * what flask request objects?
  * what info do they have?
  * what is the schema?
:::

#

## Even _more_ informative events

```python
#!/usr/bin/env python
import json
from kafka import KafkaProducer
from flask import Flask, request

app = Flask(__name__)
producer = KafkaProducer(bootstrap_servers='kafka:29092')

def log_to_kafka(topic, event):
    event.update(request.headers)
    producer.send(topic, json.dumps(event).encode())

@app.route("/")
def default_response():
    default_event = {'event_type': 'default'}
    log_to_kafka('events', default_event)
    return "This is the default response!\n"

@app.route("/purchase_a_sword")
def purchase_a_sword():
    purchase_sword_event = {'event_type': 'purchase_sword'}
    log_to_kafka('events', purchase_sword_event)
    return "Sword Purchased!\n"
```

## Run it
```
docker-compose exec mids \
  env FLASK_APP=/w205/flask-with-kafka-and-spark/game_api_with_extended_json_events.py \
  flask run --host <IP_ADDRESS>
```

::: notes
```
docker-compose exec mids env FLASK_APP=/w205/flask-with-kafka-and-spark/game_api_with_extended_json_events.py flask run --host <IP_ADDRESS>
```
:::

## Test it - generate events
```
docker-compose exec mids curl http://localhost:5000/
```
```
docker-compose exec mids curl http://localhost:5000/purchase_a_sword
```

::: notes
```
    docker-compose exec mids curl http://localhost:5000/
    docker-compose exec mids curl http://localhost:5000/purchase_a_sword
```
:::

## Read from kafka
```
docker-compose exec mids \
  kafkacat -C -b kafka:29092 -t events -o beginning -e
```

::: notes
```
docker-compose exec mids kafkacat -C -b kafka:29092 -t events -o beginning -e
```
:::

## Should see

```
{"Host": "localhost:5000", "event_type": "default", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
{"Host": "localhost:5000", "event_type": "default", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
{"Host": "localhost:5000", "event_type": "default", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
{"Host": "localhost:5000", "event_type": "purchase_sword", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
{"Host": "localhost:5000", "event_type": "purchase_sword", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
{"Host": "localhost:5000", "event_type": "purchase_sword", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
{"Host": "localhost:5000", "event_type": "purchase_sword", "Accept": "*/*", "User-Agent": "curl/7.47.0"}
...
```

#
## Spark it up

## Run a spark shell
```
docker-compose exec spark pyspark
```

::: notes
check out `course-content/tutorials/tutorial-alternative-spark-clients.md` to
spin up the pyspark shell with `ipython` or `jupyter`!
:::

## Read from kafka

```
raw_events = spark \
  .read \
  .format("kafka") \
  .option("kafka.bootstrap.servers", "kafka:29092") \
  .option("subscribe","events") \
  .option("startingOffsets", "earliest") \
  .option("endingOffsets", "latest") \
  .load()
```

::: notes
```
 raw_events = spark.read.format("kafka").option("kafka.bootstrap.servers", "kafka:29092").option("subscribe","events").option("startingOffsets", "earliest").option("endingOffsets", "latest").load()
```
:::

## Explore our events
```
events = raw_events.select(raw_events.value.cast('string'))
```
```
extracted_events = events.rdd.map(lambda x: json.loads(x.value)).toDF()
```
```
extracted_events.show()
```

::: notes
maybe turn on streaming?
(add the console appender as a sink)
`import json`
- Cache this to cut back on warnings
```
raw_events.cache()
```
:::

## down

    docker-compose down

#

## Summary

## { data-background="images/pipeline-steel-thread-for-mobile-app.svg" }

::: notes
(repeat from earlier)

Let's walk through this
- user interacts with mobile app
- mobile app makes API calls to web services
- API server handles requests:
    - handles actual business requirements (e.g., process purchase)
    - logs events to kafka
- spark then:
    - pulls events from kafka
    - filters/flattens/transforms events
    - writes them to storage
- presto then queries those events
:::

#

<img class="logo" src="images/berkeley-school-of-information-logo.png"/>


text:
# 👨💻 QA Java, Cucumber and Selenium Challenge

Java, Cucumber and Selenium front-end test in the search functionality of a website (`https://www.wexinc.com/`) for the tech QA challenge from [WEX](https://www.wexinc.com/).

The features of the tests are inside `src/test/resources/wexsearchfunctionality/search_functionality.feature` and the step definitions are inside `src/test/java/wexsearchfunctionality/StepDefinitions.java`

## 🎥 Tests Execution Demonstration

![Cucumber execution](execution.gif)

## 📅 Test plan
- [Link to Test Plan](Beck_Matheus_QATask.pdf)

## 💾 Development Dependencies Used

- "Apahce Maven": "3.6.3"
- "Java":  "1.8.0"
- "Cucumber": "6.7.0"
- "Selenium Web Driver": "3.141.59"
- "Chrome Driver": "85.0.4183.87"
- "Google Chrome": "85.0.4183.121"

## 🎬 Getting Started

1. Clone the project into your machine, install all dependencies described above and compile the project using:

```console
mvn install
```

2. Now, to run the tests:

```console
mvn test
```

3. Verify the tests results output:

![Tests results output](tests_results.png)

:pencil: License

This project is under the MIT license. See the LICENSE for more information.

---

Made with ❤ by Matheus Beck 👋 [Get in touch!](https://www.linkedin.com/in/matheus-beck/)

repository: matheus-beck/java-cucumber-selenium-wex

repo_name: matheus-beck/java-cucumber-selenium-wex


text:
---
title: "Library 1.3 Released"
date: 2013-12-10 21:54
author: saguiitay
categories: [Development, Windows Phone]
tags: [.Net, Library, Windows Phone]
redirect_from:
 - /2013-12-10-library-1-3-released/
 - /development/library-1-3-released/
---
Version 1.3 of [Library]({{site.url}}/windows-phone/library/) is live in the Store. You can go ahead and download it
from [here](http://www.windowsphone.com/s?appid=01f350f2-01d1-4210-a83b-9874b71e9496).

Versions 1.2 and 1.3 contain several improvements:

- Added ability to export to Skydrive as an Excel file
- Added [LibraryThing](http://www.librarything.com/) as a source for books and authors information
- Various bug fixes
- Several UI improvements


text:
# IFCP Contest 2021

The repository contains almost all of the source code used to run ICFP Contest
2021.  There are a number of components:

 -  [`website/`](#website): This is the source for the website at
    <https://icfpcontest2021.github.io>.
 -  [`spec/`](#spec): The source code of the specification, as Pandoc markdown.
 -  `problems/`: The problems released during the contest.
 -  [`toolchain/`](#toolchain): The toolchain contains the bulk of our code.
    It is a Haskell project containing the database, judge, webserver, and
    numerous other utilities.
 -  [`infra/`](#infra): Some deployment stuff, most importantly contains a nix
    file to build docker images that we can deploy in a pretty
    straightforward way.

## Website

### Development

This site only uses a couple standard utilities, like `bash`, `sed`, `date`,
`make`.  It should work out of the box on most linux systems.

To build the website, simply use:

    make website

This produces a `_site` directory with static files.  You can preview those
directly using a browser, e.g.:

    firefox website/_site/index.html

Or you can use a simple HTTP server, e.g:

    python -m http.server --directory website/_site

## Spec

### Building

    make spec

## Toolchain

### Building and installing

    make toolchain

### Database

For some tools, you need to have a Postgres instance running somewhere.
First, create a database, then run the migrations that set up the schema.
For example:

```bash
sudo createuser jasper                            # set up local user
createdb icfpc                                    # create database
for i in toolchain/db/*; do psql icfpc <$i; done  # apply migrations
```

### Configuring

 -  `BRAINWALL_PG_CONNECTION_STRING`: where to find the database.
 -  `BRAINWALL_BIND_PORT`: port to serve the web app on.
 -  `BRAINWALL_BIND_ADDRESS`: address to listen on for the web app.

### Tools

After installing, you should have a number of executables in your `$PATH`.

Noteworthy:

 -  `brain-wall-web`: Worker to run the web portal.
 -  `brain-wall-prosecutor`: Worker to judge solutions in the database.
 -  `brain-wall-judge`: CLI utility to judge a pose.
 -  `brain-wall-render-svg`: Render an SVG for a problem (and possibly a pose).
 -  `brain-wall-parse-svg`: Takes an SVG that has a elements with IDs `hole` and
    `figure` and tries to turn this into a problem.
 -  `brain-wall-random-problem`: Generate a random problem.

Miscellaneous:

 -  `brain-wall-animate`: Renders an animation given a problem and pose.
 -  `brain-wall-download-sources`: Download the source code submitted by
    participants.
 -  `brain-wall-estimate-epsilon`: Try to find a reasonable _eps_ value for a
    problem.
 -  `brain-wall-gallery`: Generates an HTML gallery of the best solutions per
    problem.
 -  `brain-wall-generate-bonus`: A utility to assign some random bonuses.
 -  `brain-wall-generate-figure`: Generates a random figure.
 -  `brain-wall-generate-hole`: Generates a random hole.
 -  `brain-wall-insert-problems`: Submits problems to the database so they will
    show up in the web portal.
 -  `brain-wall-jasper-gen`: Fairly dumb problem generator.
 -  `brain-wall-jasper-solver`: Fairly dumb solver.
 -  `brain-wall-random-many`: Generate multiple random problems.
 -  `brain-wall-reset-password`: Reset the password for a participant.
 -  `brain-wall-scoreboard`: Renders the scoreboard in plain text.
 -  `brain-wall-texture`: Texturing experiment for planar figures.

Most of these have an `-h` flag to show supported options.

## Infra

### Building

    make images

gha_language: Haskell


text:
# Data Structure - Linked List #

> ### Information about the project
> An application to demonstrate both a singly and doubly linked list in Javascript. Aiming to create a console application where the user can interact with both of these lists and do the normal operations such as add, remove and so on.

> ### Technologies Used ###
* Javascript
* Typescript
* NodeJS

### Build Process Used ###

> Typescript: Dev files in 'src' which was then compiled down to es3 which is located in the dist folder.

### Installation ###

```javascript
    // Install the required packages
    npm install

    // Build the project
    npm run build

    // Start the app.js file using npm start
    npm start
```

repository: luvuong-le/data-structure-linked-list

gha_language: TypeScript

repo_name: luvuong-le/data-structure-linked-list


text:
## Example Project ##

Clone an example Java project.

`git clone https://github.com/javajon/code-analysis`{{execute}}

This project includes a microservice directory that contains a small Spring Boot application built with Gradle. Go into the application source root directory.

`cd code-analysis/microservice`{{execute}}

## Analysis ##

Analyze this microservices project using SonarQube as the rule engine and dashboard where the results are published.

Get your SonarQube engine location.

`SONAR_SERVICE=https://[[HOST_SUBDOMAIN]]-31111-[[KATACODA_HOST]].environments.katacoda.com`{{execute}}

Run the analysis.

`./gradlew -Dsonar.host.url=$SONAR_SERVICE sonarqube`{{execute}}

The analysis will take about 1-2 minutes. Once complete, navigate to the SonarQube portal and observe the analyzed project. The portal can be reached from the tab labeled _SonarQube Portal_ above this command-line or this link: https://[[HOST_SUBDOMAIN]]-31111-[[KATACODA_HOST]].environments.katacoda.com/.

repository: litmusbot/katacoda-courses

repo_name: litmusbot/katacoda-courses


text:
# edit-python.el

Facilities to improve Python editing in Emacs.

## imports

`edit-python-import-from` inserts an unqualified
"`from module import name`" for the identifier at point;

`edit-python-import-qualified` inserts a qualified "`import module`"
or "`import module as alias`". Both functions look up files of the
current project to suggest module names.

repository: astanin/emacs-edit-python

repo_name: astanin/emacs-edit-python


text:
# v0.6

This document contains all release notes pertaining to the `v0.6.x` releases of Vela.

## Full release notes available on [github.com/go-vela](https://github.com/go-vela)

* [go-vela/ui](https://github.com/go-vela/ui/releases)
* [go-vela/server](https://github.com/go-vela/server/releases)
* [go-vela/worker](https://github.com/go-vela/worker/releases)
* [go-vela/cli](https://github.com/go-vela/cli/releases)
* [go-vela/pkg-executor](https://github.com/go-vela/pkg-executor/releases)
* [go-vela/pkg-runtime](https://github.com/go-vela/pkg-runtime/releases)
* [go-vela/pkg-queue](https://github.com/go-vela/pkg-queue/releases)
* [go-vela/compiler](https://github.com/go-vela/compiler/releases)
* [go-vela/sdk-go](https://github.com/go-vela/sdk-go/releases)
* [go-vela/mock](https://github.com/go-vela/mock/releases)
* [go-vela/types](https://github.com/go-vela/types/releases)

## v0.6.2

Thank you to everyone for trying Vela! This new release contains an important security bug fix!

### Bug Fixes

> NOTE: More information can be found here:
>
> https://github.com/go-vela/compiler/security/advisories/GHSA-gv2h-gf8m-r68j

* disable ability to expose sensitive configuration set in the [Vela server](https://github.com/go-vela/server)

## v0.6.1

Thank you to everyone for trying Vela! This new release contains an important bug fix!

### Bug Fixes

> NOTE: More information can be found here:
>
> https://github.com/go-vela/server/pull/191

* (second attempt) fix race condition causing builds to error with `pq: duplicate key value violates`

## v0.6.0

Thank you to everyone for trying Vela! This new release contains bug fixes, enhancements to existing features and new features!

### Breaking Changes

With the new pull policy changes steps, services, or secret containing `pull: true` will be converted into `pull: always` during compile time.

It is **strongly recommended** customers read the [documentation](https://go-vela.github.io/docs/usage/pull_policies/) to identify if it's necessary to tell Vela to pull. Once you have identified the new policy that will work for you modify your pipeline with one of the new available pull policies types.

### Features

* new `pull` policy options for images within pipelines:
  * `always` - policy type for a service, secret or step to always pull an image.
  * `not_present` - policy type for a service, secret or step to only pull an image if it doesn't exist.
  * `on_start` - policy type for a service, secret or step to only pull an image before the container starts.
  * `never` - policy type for a service, secret or step to never pull an image.
* new log actions while viewing log output:
  * jump to bottom
  * jump to top
  * download logs

### Enhancements

* update bool flags in CLI to be strings
* add ability to tail logs
* tweak step log presentation
* enhanced secret breadcrumb navigation
* refactor `add-repositories` page to be `source-repositories` and include all repos instead of admin only ones

### Bug Fixes

* fix JSON schema requiring stages to have a field name
* properly set default value for `allow_command` field within secrets
* correct CLI flag syntax usage in init log output
* fix issue with improperly configured alternative secret naming was exposing secrets
* fix race condition causing builds to error with `pq: duplicate key value violates`
* fix bug causing commit messages with special characters to crash workers
* fix init logs not showing image hashes

repository: go-vela/community

gha_language: Go

repo_name: go-vela/community


text:
---
title: DEADLOCKS
summary: Learn the `DEADLOCKS` information_schema table.
---

# DEADLOCKS

The `DEADLOCKS` table shows the information of the several deadlock errors that have occurred recently on the current TiDB node.

{{< copyable "sql" >}}

```sql
USE information_schema;
DESC deadlocks;
```

```sql
+-------------------------+---------------------+------+------+---------+-------+
| Field                   | Type                | Null | Key  | Default | Extra |
+-------------------------+---------------------+------+------+---------+-------+
| DEADLOCK_ID             | bigint(21)          | NO   |      | NULL    |       |
| OCCUR_TIME              | timestamp(6)        | YES  |      | NULL    |       |
| RETRYABLE               | tinyint(1)          | NO   |      | NULL    |       |
| TRY_LOCK_TRX_ID         | bigint(21) unsigned | NO   |      | NULL    |       |
| CURRENT_SQL_DIGEST      | varchar(64)         | YES  |      | NULL    |       |
| CURRENT_SQL_DIGEST_TEXT | text                | YES  |      | NULL    |       |
| KEY                     | text                | YES  |      | NULL    |       |
| KEY_INFO                | text                | YES  |      | NULL    |       |
| TRX_HOLDING_LOCK        | bigint(21) unsigned | NO   |      | NULL    |       |
+-------------------------+---------------------+------+------+---------+-------+
```

The `DEADLOCKS` table uses multiple rows to show the same deadlock event, and each row displays the information about one of the transactions involved in the deadlock event. If the TiDB node records multiple deadlock errors, each error is distinguished using the `DEADLOCK_ID` column. The same `DEADLOCK_ID` indicates the same deadlock event. Note that `DEADLOCK_ID` **does not guarantee global uniqueness and will not be persisted**. It only shows the same deadlock event in the same result set.

The meaning of each column field in the `DEADLOCKS` table is as follows:

* `DEADLOCK_ID`: The ID of the deadlock event. When multiple deadlock errors exist in the table, you can use this column to distinguish rows that belong to different deadlock errors.
* `OCCUR_TIME`: The time when the deadlock error occurs.
* `RETRYABLE`: Whether the deadlock error can be retried. For the description of retryable deadlock errors, see the [Retryable deadlock errors](#retryable-deadlock-errors) section.
* `TRY_LOCK_TRX_ID`: The ID of the transaction that tries to acquire lock. This ID is also the `start_ts` of the transaction.
* `CURRENT_SQL_DIGEST`: The digest of the SQL statement currently being executed in the lock-acquiring transaction.
* `CURRENT_SQL_DIGEST_TEXT`: The normalized form of the SQL statement that is currently being executed in the lock-acquiring transaction.
* `KEY`: The blocked key that the transaction tries to lock. The value of this field is displayed in the form of hexadecimal string.
* `KEY_INFO`: The detailed information of `KEY`. See the [KEY_INFO](#key_info) section.
* `TRX_HOLDING_LOCK`: The ID of the transaction that currently holds the lock on the key and causes blocking. This ID is also the `start_ts` of the transaction.

<CustomContent platform="tidb">

To adjust the maximum number of deadlock events that can be recorded in the `DEADLOCKS` table, adjust the [`pessimistic-txn.deadlock-history-capacity`](/tidb-configuration-file.md#deadlock-history-capacity) configuration in the TiDB configuration file. By default, the information of the recent 10 deadlock events is recorded in the table.

</CustomContent>

<CustomContent platform="tidb-cloud">

The information of the recent 10 deadlock events is recorded in the `DEADLOCKS` table.

</CustomContent>

> **Warning:**
>
> * Only users with the [PROCESS](https://dev.mysql.com/doc/refman/8.0/en/privileges-provided.html#priv_process) privilege can query this table.
> * The information (SQL digest) in the `CURRENT_SQL_DIGEST` column is the hash value calculated from the normalized SQL statement. The information in the `CURRENT_SQL_DIGEST_TEXT` column is internally queried from statements summary tables, so it is possible that the corresponding statement cannot be found internally. For the detailed description of SQL digests and the statements summary tables, see [Statement Summary Tables](/statement-summary-tables.md).

## `KEY_INFO`

The `KEY_INFO` column shows the detailed information of the `KEY` column. The information is shown in the JSON format. The description of each field is as follows:

* `"db_id"`: The ID of the schema to which the key belongs.
* `"db_name"`: The name of the schema to which the key belongs.
* `"table_id"`: The ID of the table to which the key belongs.
* `"table_name"`: The name of the table to which the key belongs.
* `"partition_id"`: The ID of the partition where the key is located.
* `"partition_name"`: The name of the partition where the key is located.
* `"handle_type"`: The handle type of the row key (that is, the key that stores a row of data). The possible values are as follows:
    * `"int"`: The handle type is int, which means that the handle is the row ID.
    * `"common"`: The handle type is not int64. This type is shown in the non-int primary key when clustered index is enabled.
    * `"unknown"`: The handle type is currently not supported.
* `"handle_value"`: The handle value.
* `"index_id"`: The index ID to which the index key (the key that stores the index) belongs.
* `"index_name"`: The name of the index to which the index key belongs.
* `"index_values"`: The index value in the index key.

In the above fields, if the information of a field is not applicable or currently unavailable, the field is omitted in the query result. For example, the row key information does not contain `index_id`, `index_name`, and `index_values`; the index key does not contain `handle_type` and `handle_value`; non-partitioned tables do not display `partition_id` and `partition_name`; the key information in the deleted table cannot obtain schema information such as `table_name`, `db_id`, `db_name`, and `index_name`, and it is unable to distinguish whether the table is a partitioned table.

> **Note:**
>
> If a key comes from a table with partitioning enabled, and the information of the schema to which the key belongs cannot be queried due to some reasons (for example, the table to which the key belongs has been deleted) during the query, the ID of the partition to which the key belongs might be appear in the `table_id` field. This is because TiDB encodes the keys of different partitions in the same way as it encodes the keys of several independent tables. Therefore, when the schema information is missing, TiDB cannot confirm whether the key belongs to an unpartitioned table or to one partition of a table.

## Retryable deadlock errors

<CustomContent platform="tidb-cloud">

> **Note:**
>
> This section is not applicable to TiDB Cloud.

</CustomContent>

<CustomContent platform="tidb">

> **Note:**
>
> The `DEADLOCKS` table does not collect the information of retryable deadlock errors by default. If you want the table to collect the retryable deadlock error information, you can adjust the value of [`pessimistic-txn.deadlock-history-collect-retryable`](/tidb-configuration-file.md#deadlock-history-collect-retryable) in the TiDB configuration file.

</CustomContent>

When transaction A is blocked by a lock already held by transaction B, and transaction B is directly or indirectly blocked by the lock held by the current transaction A, a deadlock error will occur. In this deadlock, there might be two cases:

+ Case 1: Transaction B might be (directly or indirectly) blocked by a lock generated by a statement that has been executed after transaction A starts and before transaction A gets blocked.
+ Case 2: Transaction B might also be blocked by the statement currently being executed in transaction A.

In case 1, TiDB will report a deadlock error to the client of transaction A and terminate the transaction.

In case 2, the statement currently being executed in transaction A will be automatically retried in TiDB. For example, suppose that transaction A executes the following statement:

{{< copyable "sql" >}}

```sql
update t set v = v + 1 where id = 1 or id = 2;
```

Transaction B executes the following two statements successively.

{{< copyable "sql" >}}

```sql
update t set v = 4 where id = 2;
update t set v = 2 where id = 1;
```

Then if transaction A locks the two rows with `id = 1` and `id = 2`, and the two transactions run in the following sequence:

1. Transaction A locks the row with `id = 1`.
2. Transaction B executes the first statement and locks the row with `id = 2`.
3. Transaction B executes the second statement and tries to lock the row with `id = 1`, which is blocked by transaction A.
4. Transaction A tries to lock the row with `id = 2` and is blocked by transaction B, which forms a deadlock.

For this case, because the statement of transaction A that blocks other transactions is also the statement currently being executed, the pessimistic lock on the current statement can be resolved (so that transaction B can continue to run), and the current statement can be retried. TiDB uses the key hash internally to determine whether this is the case.

When a retryable deadlock occurs, the internal automatic retry will not cause a transaction error, so it is transparent to the client. However, if this situation occurs frequently, the performance might be affected. When this occurs, you can see `single statement deadlock, retry statement` in the TiDB log.

## Example 1

Assume that the table definition and the initial data are as follows:

{{< copyable "sql" >}}

```sql
create table t (id int primary key, v int);
insert into t values (1, 10), (2, 20);
```

Two transactions are executed in the following order:

| Transaction 1                               | Transaction 2                               | Description                 |
|--------------------------------------|--------------------------------------|----------------------|
| `update t set v = 11 where id = 1;`  |                                      |                      |
|                                      | `update t set v = 21 where id = 2;`  |                      |
| `update t set v = 12 where id = 2;`  |                                      | Transaction 1 gets blocked.          |
|                                      | `update t set v = 22 where id = 1;`  | Transaction 2 reports a deadlock error.  |

Next, transaction 2 reports a deadlock error. At this time, query the `DEADLOCKS` table:

{{< copyable "sql" >}}

```sql
select * from information_schema.deadlocks;
```

The expected output is as follows:

```sql
+-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+-----------------------------------------+----------------------------------------+----------------------------------------------------------------------------------------------------+--------------------+
| DEADLOCK_ID | OCCUR_TIME                 | RETRYABLE | TRY_LOCK_TRX_ID    | CURRENT_SQL_DIGEST                                               | CURRENT_SQL_DIGEST_TEXT                 | KEY                                    | KEY_INFO                                                                                | TRX_HOLDING_LOCK   |
+-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+-----------------------------------------+----------------------------------------+----------------------------------------------------------------------------------------------------+--------------------+
|           1 | 2021-08-05 11:09:03.230341 |         0 |<PHONE_NUMBER>45406216 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000002 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"2"} |<PHONE_NUMBER>45406217 |
|           1 | 2021-08-05 11:09:03.230341 |         0 |<PHONE_NUMBER>45406217 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000001 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"1"} |<PHONE_NUMBER>45406216 |
+-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+-----------------------------------------+----------------------------------------+----------------------------------------------------------------------------------------------------+--------------------+
```

Two rows of data are generated in the `DEADLOCKS` table. The `DEADLOCK_ID` field of both rows is `1`, which means that the information in both rows belongs to the same deadlock error. The first row shows that on the key of `"7480000000000000355F728000000000000002"`, the transaction of the ID `"426812829645406216"` is blocked by the transaction of the ID `"426812829645406217"`. The second row shows that on the key of `"7480000000000000355F728000000000000001"`, the transaction of the ID `"426812829645406217"` is blocked by the transaction of the ID `426812829645406216`, which constitutes mutual blocking and forms a deadlock.

## Example 2

Assume that you query the `DEADLOCKS` table and get the following result:

```sql
+-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+-----------------------------------------+----------------------------------------+----------------------------------------------------------------------------------------------------+--------------------+
| DEADLOCK_ID | OCCUR_TIME                 | RETRYABLE | TRY_LOCK_TRX_ID    | CURRENT_SQL_DIGEST                                               | CURRENT_SQL_DIGEST_TEXT                 | KEY                                    | KEY_INFO                                                                                | TRX_HOLDING_LOCK   |
+-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+-----------------------------------------+----------------------------------------+----------------------------------------------------------------------------------------------------+--------------------+
|           1 | 2021-08-05 11:09:03.230341 |         0 |<PHONE_NUMBER>45406216 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000002 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"2"} |<PHONE_NUMBER>45406217 |
|           1 | 2021-08-05 11:09:03.230341 |         0 |<PHONE_NUMBER>45406217 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000001 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"1"} |<PHONE_NUMBER>45406216 |
|           2 | 2021-08-05 11:09:21.252154 |         0 |<PHONE_NUMBER>17809412 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000002 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"2"} |<PHONE_NUMBER>17809413 |
|           2 | 2021-08-05 11:09:21.252154 |         0 |<PHONE_NUMBER>17809413 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000003 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"3"} |<PHONE_NUMBER>17809414 |
|           2 | 2021-08-05 11:09:21.252154 |         0 |<PHONE_NUMBER>17809414 |<PHONE_NUMBER>1edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | update `t` set `v` = ? where `id` = ? ; |<PHONE_NUMBER>000000355F728000000000000001 | {"db_id":1,"db_name":"test","table_id":53,"table_name":"t","handle_type":"int","handle_value":"1"} |<PHONE_NUMBER>17809412 |
+-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+-----------------------------------------+----------------------------------------+----------------------------------------------------------------------------------------------------+--------------------+
```

The `DEADLOCK_ID` column in the above query result shows that the first two rows together represent the information of a deadlock error, and the two transactions that wait for each other form the deadlock. The next three rows together represent the information of another deadlock error, and the three transactions that wait in a cycle form the deadlock.

## CLUSTER_DEADLOCKS

The `CLUSTER_DEADLOCKS` table returns information about the recent deadlock errors on each TiDB node in the entire cluster, which is the combined information of the `DEADLOCKS` table on each node. `CLUSTER_DEADLOCKS` also includes an additional `INSTANCE` column to display the IP address and port of the node to distinguish between different TiDB nodes.

Note that, because `DEADLOCK_ID` does not guarantee global uniqueness, in the query result of the `CLUSTER_DEADLOCKS` table, you need to use the `INSTANCE` and `DEADLOCK_ID` together to distinguish the information of different deadlock errors in the result set.

{{< copyable "sql" >}}

```sql
USE information_schema;
DESC cluster_deadlocks;
```

```sql
+-------------------------+---------------------+------+------+---------+-------+
| Field                   | Type                | Null | Key  | Default | Extra |
+-------------------------+---------------------+------+------+---------+-------+
| INSTANCE                | varchar(64)         | YES  |      | NULL    |       |
| DEADLOCK_ID             | bigint(21)          | NO   |      | NULL    |       |
| OCCUR_TIME              | timestamp(6)        | YES  |      | NULL    |       |
| RETRYABLE               | tinyint(1)          | NO   |      | NULL    |       |
| TRY_LOCK_TRX_ID         | bigint(21) unsigned | NO   |      | NULL    |       |
| CURRENT_SQL_DIGEST      | varchar(64)         | YES  |      | NULL    |       |
| CURRENT_SQL_DIGEST_TEXT | text                | YES  |      | NULL    |       |
| KEY                     | text                | YES  |      | NULL    |       |
| KEY_INFO                | text                | YES  |      | NULL    |       |
| TRX_HOLDING_LOCK        | bigint(21) unsigned | NO   |      | NULL    |       |
+-------------------------+---------------------+------+------+---------+-------+
```


text:
# Real Estate Market Analysis

#### Task:
- Using the data of the Yandex.Real estate service, determine the market value of real estate objects and typical parameters of apartments

#### Description:
- Research analysis and data preprocessing for a dataset with advertisements for the sale of apartments in St. Petersburg has been carried out. The influence of the area, ceilings, the number of rooms, the date of announcement on the prices of apartments in all the presented settlements and the center of St. Petersburg for the construction of an automated system for determining prices in order to avoid fraud and anomalies were revealed. Based on data from the Yandex.Real estate service, the market value of various types of real estate, typical parameters of apartments, depending on the distance from the center, was determined. Data preprocessing has been done. Added new data. Constructed histograms, boxplots, scatter diagrams.

#### Scope of Activities:
- Internet services, Advertising platforms, Services for business [b2b] (outsourcing consulting audit), Startups, IT company

#### Used Libraries:
- Python, Pandas, Matplotlib, exploratory data analysis, data visualization, data preprocessing, math

#### tags:
- data processing, histogram, boxplot, scattermatrix, categorization, scatterplot, marketing analyst, fraud analyst, fraud analyst, data analyst, data analyst, analyst, analyst, fraud monitoring


text:
# PyPut: `a better version of python's native input function`

> no dependencies :)

## importing the pyput module

```python
import pyput
from pyput.validators import * # validators

# imports below are not required
from pprint import PrettyPrinter # for pretty printing

printer = PrettyPrinter()
pprint = printer.pprint
```

## Usage:

##### collect `string` from user

```python
name = pyput.CliInput("enter your name: ").result(str)
print(name, "is of type", type(name))
```

    enter your name: rubbie kelvin
    rubbie kelvin is of type <class 'str'>

##### collect an `integer` from user

```python
age = pyput.CliInput("enter your age: ").result(int)
print(age, "is of type", type(age))
```

    enter your age: 18
    18 is of type <class 'int'>

##### collect a `boolean` from user

```python
is_active_on_media = pyput.CliInput("are you active on media? ").result(bool)
print(is_active_on_media, "is of type", type(age))
```

    are you active on media?
    False is of type <class 'int'>

##### collect a `list` from user

```python
friends = pyput.CliInput("enter your friends name").result(str, str, str)  # collects 3 strings and returns list
print(friends, "is of type", type(friends))
```

    enter your friends name[0]rezzy
    enter your friends name[1]brumeh
    enter your friends name[2]middo
    ['rezzy', 'brumeh', 'middo'] is of type <class 'list'>

```python
# in the above example, the length of the list is determined by the types passed into pyput.CliInput.result
# somthing like this could help :)

n = pyput.CliInput("how many friends do you have? ").result(int)
friends = pyput.CliInput("enter thier names").result([str]*n)
print(friends)
```

    how many friends do you have? 3
    enter thier names[0]brumeh
    enter thier names[1]kandy
    enter thier names[2]jerome
    ['brumeh', 'kandy', 'jerome']

##### working with a `dictionary`

```python
name_dict = pyput.CliInput("what's your name? ").result(firstname=str, lastname=str)
print(name_dict)
```

    what's your name? [firstname]rubbie
    what's your name? [lastname]kelvin
    {'firstname': 'rubbie', 'lastname': 'kelvin'}

```python
# working with large dictionaries
# lets say we have a default data template for a user
userdata = {
    "name": {"first":"Anonymous", "last":""},
    "gender":"male",
    "email":"no email",
    "numbers":[0, 0]
}

# we wouldnt need to start specifying types
# there's an inbuilt function to deal with that for us; puput.typed
userdata = pyput.CliInput("enter you details:").result(pyput.typed(userdata))
pprint(userdata)
```

    enter you details:[name][first]rubbie
    enter you details:[name][last]kelvin
    enter you details:[gender]male
    enter you<EMAIL_ADDRESS>    enter you details:[numbers][0]2348141342356
    enter you details:[numbers][1]0
    {'email'<EMAIL_ADDRESS>     'gender': 'male',
     'name': {'first': 'rubbie', 'last': 'kelvin'},
     'numbers': [2348141342356, 0]}

### What does `typed()` do?

typed() takes in a container, either a dictionary or a list, then iterates through it to change integers, strings, and booleans to thier respective datatypes

```python
# look at this:
typed_example = pyput.typed([1, 2, dict(name="rubbie", age=18, ismale=True)])
pprint(typed_example)
```

    [<class 'int'>,
     <class 'int'>,
     {'age': <class 'int'>, 'ismale': <class 'bool'>, 'name': <class 'str'>}]

### collecting 2D List

```python
matrix = [
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0]
]

matrix = pyput.CliInput("enter matrix number").result(pyput.typed(matrix))
pprint(matrix)
```

    enter matrix number[0][0]1
    enter matrix number[0][1]2
    enter matrix number[0][2]3
    enter matrix number[1][0]4
    enter matrix number[1][1]5
    enter matrix number[1][2]6
    enter matrix number[2][0]7
    enter matrix number[2][1]8
    enter matrix number[2][2]9
    [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

## Validating input

```python
# lets make sure the user enters a string with greater than 8 characters
def validator(value):
    return len(value) > 8

data = pyput.CliInput("enter a string: ", validator=validator).result(str)
```

    enter a string: this is valid

```python
# using lambdas
data = pyput.CliInput("enter a string: ", validator=lambda x:len(x)>8).result(str)
```

    enter a string: this is valid

### Validator Classes
#### Inbuilt Validator classes

* StringSizeValidator
* EmailValidator
* PasswordValidator
* RangeValidator

#### Simple password validator

```python
password = pyput.CliInput("enter your password: ", validator=PasswordValidator()).result(str)
```

    enter your password: qwertyA2!

#### Creating validator classes
all validator classes should have a `validate` method that takes in a string argument that returns a bool value :). you might also want to inherit the `pyput.validators.Validator` class (may not be neccessary).

```python
class SquareIntValidator(Validator):
    """this class validates if a number is a perfect square
    """
    def validate(self, value):
        root = value**0.5
        return value == root*root

perfect_square = pyput.CliInput("enter a perfect square: ", validator=SquareIntValidator()).result(int)
```

    enter a perfect square: 9

```python

```

repository: rubbieKelvin/pyput

repo_name: rubbieKelvin/pyput


text:
# Docker/Rails_app Chatterbox

 # For Linux Ubuntu 18.04:

### 1) Clone the project
``` git clone https://github.com/dremerten/chatter_box.git```

### 2) Change directories into 'chatter_box'
``` cd chatter_box```

### 3) Install docker-ce and docker-compose
``` bash ubuntu18.04-installer.sh```

### 4) Follow on-screen instructions

***************************************************************************************************************************
# For Linux Ubuntu 19.04

### 1) Clone the project
``` git clone https://github.com/dremerten/chatter_box.git```

### 2) change directories into chatterbox
```cd chatter_box```

### 3) Install docker-ce and docker-compose
``` bash ubuntu19.04-installer.sh```

### 4) Follow the on-screen instructions
**************************************************************************************************************************

# Mac OsX docker install

### 1)
https://download.docker.com/mac/stable/Docker.dmg

## 2) Have Postgres installed and running
https://postgresapp.com/

## Docker-ce and postgres services should be running

### 3) To get the containers up and running change into scripts directory and execute setup-env.sh
```$ cd chatter_box/scripts/```

### 4) Run setup
```$ ./setup-env.sh```

***************************************************************************************************************************

# For Windows
 https://hub.docker.com/editions/community/docker-ce-desktop-windows

### Install Chocolately ---> https://chocolatey.org/docs

###  Use the command prompt to install git.
https://chocolatey.org/packages/git

```# MUST RUN CMD AS ADMINISTRATOR
# /GitOnlyOnPath - Puts git install\cmd on path. This is also done by default if no package parameters are set.

# To install Git
choco install git.install --params "/GitOnlyOnPath"
```

### After installing git be sure to close the Command Prompt/Terminal to allow the changes to take effect.

 ### Clone this repo, from with the "Git Bash" Command Prompt
   ### To access click on the windows icon and select "Git" from there locate the Git Bash Command Prompt, right-click
   ### and run as Administrator.
```
git clone https://github.com/dremerten/chatter_box.git
```

### Locating the Git Bash Command Window.
![alt text](https://i.stack.imgur.com/soecn.png)

### To get the containers up and running change into scripts directory and execute setup-env.sh

# Windows(Git Bash Command Prompt)
```cd scripts```

```$ sh setup-env.sh```

****************************************************************************************************************

# For All OS Platforms

### Check the status of current running containers
```
docker container ps
```
### Enter the container run:
```
docker exec -it <container_Id> /bin/zsh
```
### Check out the app
```localhost:3000```

### Copy a file from a container to the host, you can use the command
```
docker cp <containerId>:/file/path/within/container /host/path/target
```

### To stop and kill the running containers(make sure to stop first)
```
docker stop $(docker ps -a -q)
docker kill $(docker ps -a -q)
```

### view all images running
```
$ docker images
```

### To additionally remove any stopped containers and all unused images (not just dangling images), add the -a flag to the command:
```
docker system prune -a
```
## For additional Docker commands please refer to "docker_notes.txt"

gha_language: Ruby


text:
*Deprecated in favor of [Nuimo emulator for iOS](https://github.com/getsenic/nuimo-emulator-ios) and [Nuimo emulator for Android](https://github.com/getsenic/nuimo-emulator-android)*

Web application to simulate Nuimo controllers over web sockets

#Requirements
Java 8+ JDK

#To compile and run the server execute
Linux/MacOS:
```
./start.sh
```
Windows:
```
gradlew shadowJar
java -jar build/libs/WebSocketNuimo-3.0.0-fat.jar
```
(Easier to handle start script coming soon)

#Build only
```
./gradlew shadowJar
```

#Run only
```
java -jar build/libs/WebSocketNuimo-3.0.0-fat.jar
```

repository: mpietruschka/nuimo-websocket-simulator

repo_name: mpietruschka/nuimo-websocket-simulator


text:
# Installation

## Requirements

- Node.js 14+
- PostGIS
- GDAL including python-gdal
- imposm3

## Get sources

```bash
git clone https://github.com/FreemapSlovakia/freemap-mapnik.git
```

## Installing prerequisites

1. Get OpenStreetMap data (eg. from [Geofabrik](http://download.geofabrik.de/))
1. Create a postgis database (see below for details)
1. Import OpenStreetMap data (see below for details)
1. Get digital elevation data and import it to database (optional; see below for details)
1. Change directory to project directory
1. Run `npm i`
1. Create `config/development.json5` where you can override settings from `config/default.json5` for your local environment
1. Run `npm run watch`
1. Open [preview.html](../preview.html) in your browser

## Building

To build sources for production run:

```bash
npm i
npm run build
```

## Importing OSM data to PostGIS

In following commands replace `<you>` with your username.

### Prepare database

1. `sudo su - postgres` (skip this on MacOS)
1. `createdb <you>`
1. `createuser <you>`
1. `psql <you>`
1. `CREATE EXTENSION postgis;`
1. `CREATE EXTENSION postgis_topology;`
1. `GRANT CREATE ON DATABASE <you> TO <you>;`
1. `ALTER USER <you> WITH PASSWORD '<your_password>';`

### Import OpenStreetMap to database

You must have imposm3 installed. For instructions how to build it see https://github.com/omniscale/imposm3.

To import the data use following command (with correct pbf filename):

```bash
imposm import -connection postgis://<you>:<your_password>@localhost/<you> -mapping mapping.yaml -read slovakia-latest.osm.pbf -write
```

Then deploy the import to production:

```bash
imposm import -connection postgis://<you>:<your_password>@localhost/<you> -mapping mapping.yaml -deployproduction
```

Import `additional.sql` to Postgresql.

See also [instructions to compute and import peak isolations](./PEAK_ISOLATION.md).

#### Building Imposm on MacOS

```bash
brew install golang leveldb geos
```

and then execute the commands [here](https://github.com/omniscale/imposm3/#compile)

## Contours and shaded relief (optional)

1. Obtain digital elevation data from [EarthExplorer](https://earthexplorer.usgs.gov/)
   - recommended Data Set is _SRTM 1 Arc-Second Global_
   - use GeoTIFF format; then convert it to HGT or modify `shading/Makefile` ;-)
1. To generate shaded relief and contours run `npm i -g dem-iron shp-polyline-splitter && cd shading && make -j 8`

## Setting up minutely diff applying

Follow these steps also after database reimport, which is required if you've updated `mapping.yaml`:

1. Download `europe-latest.osm.pbf` from Geofabrik
1. Extract area of focus with Osmium:
   ```bash
   osmium extract -p limit.geojson -s smart -S types=multipolygon,route,boundary europe-latest.osm.pbf -o extract.pbf
   ```
1. Stop imposm service
   ```bash
   systemctl stop imposm3
   ```
1. update and compile freemap-mapnik
   ```bash
   git pull && npm run build
   ```
1. Import the extract:
   ```bash
   imposm import -connection postgis://freemap:freemap@localhost/freemap -mapping mapping.yaml -read extract.pbf -diff -write -cachedir ./cache -diffdir ./diff -overwritecache
   ```
1. Delete pbf files
   ```bash
   rm europe-latest.osm.pbf extract.pbf
   ```
1. Stop prerendering
   ```bash
   systemctl stop freemap-mapnik-prerender
   ```
1. Deploy the import to production:
   ```bash
   imposm import -connection postgis://freemap:freemap@localhost/freemap -mapping mapping.yaml -deployproduction
   ```
1. Import `additional.sql` (or apply only its changes) to Postgresql
1. Update `./diff/last.state.txt` to reflect timestamp and sequence number of the imported map.
   See https://planet.openstreetmap.org/replication/minute/ for finding sequence number.
1. Start imposm and wait to catch it up
   ```bash
   systemctl start imposm3
   ```
1. Now you can optionally stop imposm service to prevent it from interrupring prerendering.
   You can start it later after pre-rendering has been finished.
   You can also start it somewhere during pre-rendering to catch-up and stop again, to apply some recent updates.
1. Delete content of `./expires` directory
1. Edit `./config/prerender.json5` and change `rerenderOlderThanMs` to current time plus a minute or more
1. Stop ondemand rendering
   ```bash
   systemctl stop freemap-mapnik-ondemand
   ```
1. Delete cached highzoom tiles and indexes
   ```bash
   cd ./tiles
   rm -rf 15 16 17 18 19
   cd ..
   find ./tiles/14 -name '*.index' -delete
   ```
1. Start ondemand rendering
   ```bash
   systemctl start freemap-mapnik-ondemand
   ```
1. Start prerendering
   ```bash
   systemctl start freemap-mapnik-prerender
   ```

## Nginx as front tier

```nginx
server {
    ...

    location /pdf {
      proxy_pass http://localhost:4000/pdf;
    }

    location / {
      rewrite ^(.*)$ $1.png break;
      root    /home/freemap/freemap-mapnik/tiles/;
      error_page 404 = @fallback;
    }

    location @fallback {
      proxy_pass http://localhost:4000;
      error_page 404 /40x.html;
    }

    ...
}
```

## Notes

Fixing 1201x3601 SRTM (skips `dem-iron`ing):

TODO: integrate to `shading/Makefile`.

```bash
for x in `seq 11 22`; do
  for y in `seq 50 51`; do
    gdalwarp N${y}E0${x}.hgt -multi -r cubicspline \
      -te $((x-1)).99986111 $((y-1)).99986111 $((x+1)).00013889 $((y+1)).00013889 \
      -tr 0.000277777777778 -0.000277777777778 \
      N${y}E0${x}.aa.tiff
  done
done
```

EPSG:3857: +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over

```bash
gdalwarp -overwrite -of GTiff -ot Float32 -co NBITS=16 -co TILED=YES -co PREDICTOR=3 -co COMPRESS=ZSTD -co NUM_THREADS=ALL_CPUS -multi -wo NUM_THREADS=ALL_CPUS -srcnodata -9999 -r cubicspline -order 3 -tr 19.109257071294062687 19.109257071294062687 -tap -t_srs "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over" -s_srs "+proj=krovak +lat_0=49.5 +lon_0=24.83333333333333 +alpha=30.28813972222222 +k=0.9999 +x_0=0 +y_0=0 +ellps=bessel +towgs84=542.5,89.2,456.9,5.517,2.275,5.516,6.96 +pm=greenwich +units=m +nadgrids=slovak +no_defs" sk.tiff sk_w.tiff

# -s_srs "+proj=krovak +ellps=bessel +nadgrids=slovak"
```

gdal_translate -of AAIGrid square.tif square.asc

Convert HGT to tiff

```bash
for i in *.HGT; do gdalwarp -overwrite -of GTiff -ot Float32 -co "NBITS=16" -co "TILED=YES" -co "PREDICTOR=3" -co "COMPRESS=ZSTD" -co "NUM_THREADS=ALL_CPUS" -r cubicspline -order 3 -tr 20 20 -multi -dstnodata NaN -t_srs "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over" $i ${i%.HGT}.tiff; done
```

Fill nodata in tiffs

```bash
for i in N*.tiff; do gdal_fillnodata.py -of GTiff -co "NBITS=16" -co "TILED=YES" -co "PREDICTOR=3" -co "COMPRESS=ZSTD" -co "NUM_THREADS=ALL_CPUS" $i ${i%.tiff}.filled.tiff; done
```

```bash
gdal_merge.py -of GTiff -co "NBITS=16" -co "TILED=YES" -co "PREDICTOR=3" -co "COMPRESS=ZSTD" -co "NUM_THREADS=ALL_CPUS" -n NaN -a_nodata NaN -o merged.tiff N*.filled.tiff sk_w.tiff
```

repository: FreemapSlovakia/freemap-mapnik

branch_name: refs/heads/develop

repo_name: FreemapSlovakia/freemap-mapnik