text:
# Kul

A unique textual notation that can be used as both a data format and a markup
language and that has powerful extensibility of both lexical syntax and
semantics, and a Rust library for parsing it.  It is inspired by the
little-known [Curl language][curl], but is only a notation like Curl's, not a
programming language like Curl.

The syntax is very minimal and exists only to delimit forms of nested text that
you sometimes define your own parsing and semantics of by binding *operator*
sub-forms to *combiner* functions which are similar to macros.  As such, by
itself, it is not a complete format for most applications and they will define
their own different extensions.  However, it enables having a common base syntax
that can still be parsed, without any extensions, into the basic structure which
can be used across applications that do not need to fully handle the inner
syntax of all of the nested text forms.  Furthermore, shared or standardized
extensions could be created in the future.

See the [motivation](#motivation) section for why the notation is designed the
way it is.

See the [objectives](#objectives) section for what the library design achieves.

This is only an experimental exploration and is my first Rust project, but the
implementation is fairly capable and featureful.

## Examples of the notation

```
As markup of free-form text for embedding structured data types, such as a
{date 2019-03-05 15:15 PST}, which might be rendered as widgets; or for {italic
{bold styling} the text}.  Only the \{, \}, and \\ characters require escaping.
```

```
{config {# As a data structure. (This form is a comment.)}

    {things {list 9, "blah", {map foo: 8.7, bar: asdf}}}

    {logging
        {err = yes}
        {warn = {maybe, if 1 + x = 3}}
    }

    {greeting: We can have text in structures without ugly quoting!}

    {knobs this := {that}; other := {}}

    {{compound-operator with arguments} 7/2 > 𝜋}
}
```

```
Using non-default delimiters:

⟪⟪source-code Rust⟫
    use kul::common::inmem::parse_str;

    fn main() {
        let input = "Escaped the {bold non-default} delimiters: ␛⟪, ␛⟫, ␛␛";
        dbg!(parse_str(input));
    }
⟫
```

## Motivation

The notation has the same advantage as Curl's design of this level, which is to
have a **very extensible** language for both markup **and** data with a minimum
of syntax.  Unlike most markup languages, you can have your own markup forms
with their own syntax, and unlike most data formats, you can have your own rich
data types with their own syntax, instead of being limited to only what is
provided by most others.

The flip side of this is that you have to implement your own parsing and
handling of your custom forms, but premade libraries that implement form
extensions could be provided by others.  Because most other notations are
limited and fixed, applications sometimes end up adding their own custom
processing to achieve what they want anyway, which can feel like inconsistent
hacks.  Instead, we design the notation to fundamentally support extension so it
is much cleaner and more capable in what you can express directly in it.  This
helps you create different domain-specific languages (DSLs) with their own
lexical syntax, instead of being locked into one particular syntax like with
other DSL designs.  This also enables embedding multiple DSLs' different
syntaxes within the same outer form or file.  (Lisp languages are similar in
allowing embedding multiple DSLs but they all have to use the same S-expression
lexical syntax.)

At first, this might seem like it would lead to being overwhelmed with a huge
variety of syntaxes and dialects.  But the inherent complexity of reality that
we want to model cannot be avoided and when you push that complexity out of the
notation it often re-manifests in how you later try to express complex
compositions.  Such as jamming DSLs into the string type, or into other types
like arrays and maps, or into identifiers and function signatures, because your
primary notation is too limited, and then you are back to having to parse and
handle your extensions but with a syntax and approach that was not designed for
such extension.

The Kul notation and parser is an experiment of trying the Curl approach that is
designed for extension.

For more about the motivation, see [the linked paper about Curl][curl] and the
other published papers about it.

## Objectives

The library is designed to offer:

- Zero-copy parsing of both entirely-in-memory inputs and of streamed inputs (if
  you arrange buffering into chunks appropriately), including zero-copy
  exclusion of the escape character.  This minimizes memory usage.  All achieved
  via a generic design of chunked text.

- Premade `char`-and-position iterator used for all chunked text types that only
  borrows their internal state (to avoid copying).  This greatly simplifies
  defining your own `Text` trait `impl`s (if needed) and assists the premade
  ones provided.  Achieved via a clever design of generically borrowing
  associated types with the correct lifetimes without using Rust's unstable
  `generic_associated_types` feature.

- Support for very-deep huge trees of the AST type (e.g. long lists and deep
  nests).  This enables working with deep trees produced by parsing or by your
  own construction.  Achieved via custom `Drop` and `PartialEq` implementations
  that avoid stack overflows (which would otherwise happen) when dropping or
  comparing deep trees.

- No `unsafe` code.

- No external dependencies.

- Non-panicking API. (`panic!`s should never happen but this is not proven
  currently.)

- A `no_std` core crate usable on its own with only stack allocation.  The
  parser's dynamic allocation can be done from fixed-size arrays.  This can also
  be used to enforce memory-consumption limiting.

- A `std`-using crate with more convenient `Vec`s, `String`s, `Box`es, etc. that
  builds on the core crate.

- Very generically parameterized in most aspects to allow maximal reuse for
  diverse applications.

## Status

Version `0.1.3`: experimental and unstable.  Builds fine and passes all tests
and lints.

## Rust version

At least `1.53` required.  This library will always require only the stable
version of Rust (not the nightly one).

## Usage of `kul`

It would require some length to describe the full range of possible usages,
given how very generically parameterized the library is.

See the `README` of the `kul_core` crate for a usage example relevant to
`no_std` applications where all allocation is done from the stack only.

For common basic applications, the following example shows how input strings can
be parsed, with and without custom extensions.  It prints the resulting AST
(abstract syntax tree) structures, collected into a vector per parse, that you
would work with, so you can see the `Datum` variants that correspond to the
different forms of the notation.  (As an introduction, this seems clearer than
showing a lot of matching and destructuring of the variants.)

```rust
use std::{time::SystemTime, str::FromStr, iter::FromIterator};

use kul::{
    common::inmem::{
        parse_str, parse_str_with,
        Text, OperatorBindings, DatumAllocator,
    },
    Combiner, Datum, datum::{BoxDatum, DatumBox}, Error, Text as _,
};

/// Parse without any bound operators and print results.  This shows that the
/// common base syntax can always be parsed without knowing about possible
/// extensions.
fn no_extension() {
    dbg!(parse_str("λ"));
    dbg!(parse_str(r"es\{c\}ap\\es"));
    dbg!(parse_str("{}"));
    dbg!(parse_str("{▷}"));
    dbg!(parse_str("Surrounding {{▷} λ {}} text."));
}

/// Parse with some bound operators and print results.  This shows that the
/// syntax and semantics of particular forms can be extended in custom ways.
fn with_extensions()
{
    /// Extends the types that may occur in the returned ASTs.
    #[derive(Hash, Eq, PartialEq, Debug)]
    enum MyDatumVariants {
        Time(SystemTime),
        Integer(i128),
    }

    /// Extends the types that may occur in errors returned by our custom form
    /// processing.
    #[derive(Debug)]
    enum MyCombinerError<'input> {
        Oops,
        Darnit(MyDatum<'input>),
    }

    // Convenient type aliases.

    type MyDatum<'input> = BoxDatum<Text<'input>, MyDatumVariants>;

    type MyOperatorBindings<'input> =
        OperatorBindings<'input, MyDatumVariants, MyCombinerError<'input>>;

    type MyDatumAllocator<'input> = DatumAllocator<'input, MyDatumVariants>;

    type AllocArg<'a> = &'a mut MyDatumAllocator<'static>;

    // The functions that process our custom forms.  Using closures can be nicer
    // because at least some type inference of the argument and return types can
    // be gained.  The operator and allocator arguments are always ignored for
    // this example, as they often are in real programs.

    let comment = |_operator, _operands, _: AllocArg<'_>| {
        Ok(None)
    };

    let pass_thru = |_operator, operands, _: AllocArg<'_>| {
        Ok(Some(operands))
    };

    let current_time = |_operator, operands, _: AllocArg<'_>| {
        if let Datum::EmptyList = operands {
            Ok(Some(Datum::Extra(MyDatumVariants::Time(SystemTime::now()))))
        } else {
            Err(Error::FailedCombiner(MyCombinerError::Darnit(operands)))
        }
    };

    let int = |_operator, operands: Text<'_>, _: AllocArg<'_>| {
        // Must convert the operands text into a `&str`, to be able to use other
        // parsing functions/libraries that take string slices.  (When the other
        // parsing functionality can instead take `Iterator`s of `char`s, this
        // conversion is unneeded.)
        let i = i128::from_str(&String::from_iter(operands.chars()))
                    .map_err(|_| Error::FailedCombiner(MyCombinerError::Oops))?;
        Ok(Some(Datum::Extra(MyDatumVariants::Integer(i))))
    };

    // Establish bindings of particular operator sub-forms to our processing
    // functions.  Other more declarative and concise ways of doing this are
    // possible, but, for this example, this shows the basic nature that other
    // ways could build on.

    let mut bindings = MyOperatorBindings::default();
    bindings.hashmap.insert(Datum::Text(Text::from_str("#")),
                            Combiner::Operative(Box::new(comment)));
    bindings.hashmap.insert(Datum::Text(Text::from_str("current-time")),
                            Combiner::Applicative(Box::new(current_time)));
    bindings.hashmap.insert(Datum::Text(Text::from_str("int")),
                            Combiner::Operative(Box::new(int)));
    let compound_operator_form =
        Datum::Combination {
            operator: DatumBox::new(Datum::Text(Text::from_str("compound"))),
            operands: DatumBox::new(Datum::EmptyList),
        };
    bindings.hashmap.insert(compound_operator_form,
                            Combiner::Applicative(Box::new(pass_thru)));

    // Parse a string that uses all of the above and print results.

    dbg!(parse_str_with(
        "{{compound} {current-time} {# removed} {unbound form} {int -42}}",
        bindings)
    );
}

fn main() {
    no_extension();
    with_extensions();
}
```

The above example can be run by doing:

```
cargo run --example common_basic
```

## Documentation

The source-code has many doc comments, which are rendered as the API
documentation.

View online at: http://docs.rs/kul and http://docs.rs/kul_core

Or, you can generate them yourself and view locally by doing:

```
cargo doc --open
```

See the documentation at the top of the `kul_core` crate for some further
overview of the Kul language and design.

(`TODO`: Some of the less-important doc comments' links are broken currently due
to refactorings changing their relative directories and names, and some just
need links to be made in the first place.  I'm hoping that the
[`intra_rustdoc_links` feature (RFC 1946)][intra_rustdoc_links] will become
stable soon to make fixing these be much easier and much more maintainable.)

[intra_rustdoc_links]: https://github.com/rust-lang/rfcs/blob/master/text/1946-intra-rustdoc-links.md

## Tests

The crates have many unit tests and integration tests, which can be run by doing:

```
cargo test --all
```

That uses around 2.1G of memory running them with `--test-threads=4` on my
64-bit 4-core computer.  If you want it to use less, you can do:

```
cargo test --all -- "" tree-size=$((2**N))  # where N < 21
```

221 is the default for the size of the trees used to test the `Drop`
and `PartialEq` implementations that prevent stack overflows for deep trees
(e.g. long lists and deep nests) of `kul::Datum`.  Below some size, it will no
longer be properly testing this because overflows would not happen without our
implementations anyway.  Alternatively, you may increase `N > 21` as much as you
can, to test very-deep trees.  To see overflows happen, comment-out the `Drop`
`impl`s in `src/drop.rs`.

## Unresolved

The following aspects are unresolved:

- Some of the terminology chosen might be improvable.

- The `Datum` enum type, used for the returned AST nodes and for the arguments
  given to extension functions, is designed to be multi-purpose so constrained
  applications, that do not use heap allocation and use only the core `no_std`
  crate, only need to provide an allocator of this one simple type, which
  enables just using a basic array of them on the stack.  But this results in
  some of the API being non-ideal because `Datum` is used where only a subset of
  its variants are possible, requiring matching to destructure with a branch
  that has to be made `unreachable!()`.  I couldn't figure out a way to improve
  this while keeping support for constrained allocation from basic arrays of a
  single type.  It is unresolved whether this can be improved without causing
  difficulties for no-heap constrained applications.

- The `Text` trait, used to represent logical concatenation of chunks of
  sequences of `char`s, is designed to be multi-purpose so it can be used for
  AST output, and for input of streamed or in-memory sources, and for
  efficiently excluding escape characters, and used generically with various
  underlying representations.  But this results in having to use iteration of
  its logical `char` sequence for all operations that want to know its content,
  which doesn't fit perfectly with applications that want to parse its content
  by using other libraries that take string slices (`&str`) as their inputs to
  parse (e.g. the `std` library's `FromStr` implementations).  Creating
  temporary `String`s from `Text` content is a reasonable way to use such
  libraries, but this doesn't feel great given the project's goal of supporting
  zero-copy as much as possible, and `String` (nor any dynamic string types) is
  not available to no-heap constrained applications.  It is unresolved whether
  this can be improved while keeping all the desired uses and qualities of the
  `Text` trait.  (The Kul library itself preserves zero-copy while using
  iterators of `char`s by using its own special iterator and related traits and
  types.)

- The name Kul was chosen after abandoning the name Kruvi.  Kruvi is the Lojban
  word for curve, in homage to Curl, but it sounds too similar to the name
  Groovy (an existing software project) and is too long to use as a filename
  extension.  The name Kul seems to be unclaimed as a project name and `.kul`
  unclaimed as a filename extension.  I'm imagining that applications might use
  filename extensions like `.whatever.kul` to indicate their particular
  extensions of the format as well as indicate that it can also always be parsed
  as the basic `.kul` structure.  I'm not very attached to any name and am open
  to changing it but do like that Kul is cool.

[curl]: http://people.csail.mit.edu/ward/curl-ijwet.pdf

repository: DerickEddington/kul

repo_name: DerickEddington/kul


text:
# Harvard-Wayfair NLP

This repo contains public source code and demo notebooks for a data science capstone project
at Harvard (Fall 2020). The project aimed to help Wayfair, a major online retailer selling
furniture and home goods, to examine whether free-text reviews contain signals that improve
the prediction of return rates. Methods and findings of the project are summarized in
[this](https://towardsdatascience.com/read-the-reviews-analyzing-nlp-signals-of-wayfair-products-7c31b63cd369) blog post.

## Team Members

* Sangyoon Park
* Alex Spiride
* Erin Yang
* Jason Zhang

## Directory Structure
The repo contains two subdirectories: `src/` and `demo/`. `src/` contains source code for classes and functions to perform feature extraction and modeling. Building on this source code, `demo/` uses toy (fake) data to demonstrate how to perform feature extraction and modeling. We recommend you start with demo notebooks in `demo/notebooks/`.
```
├── README.md          <- The top-level README for developers using this project.
│
├── pyproject.toml     <- The file that defines build system, project metadata, and other
│                         install requirements.
│
├── poetry.lock        <- The file that resolves and downloads dependencies in `pyproject.toml`.
│
├── src
│   ├── features       <- Defines classes that can be used to extract NLP features.
│   └── models         <- Defines classes that can be used to predict product return rates.
│
├── demo               <- Materials showing how to use major resources in the repo.
│   ├── data
│   ├── models
│   └── notebooks
```

## Getting Started
The current project repo uses [`poetry`](https://python-poetry.org/docs/) to manage
dependencies among different Python packages, which is essential to reproducibility.
Following are steps for setting up and getting started:

First, ensure you are using the right version of Python (`^3.8`). We recommend you
use [`pyenv`](https://github.com/pyenv/pyenv) to effectively manage multiple versions
of Python installation. You can then install `poetry`:
```
$ pip install poetry
```

Once you clone the current repo into your local machine, you can go inside the repo and run:
```
$ poetry install
```
to install the right versions of packages for running scripts in the project repo.

To use the new Python configuration that has been installed, you need to run:
```
$ poetry shell
```
which will activate the virtual environment for the project repo.

You can simply type:
```
$ exit
```
to exit from the virtual environment and return to the global (or system) Python installation.

Once you set up the virtual environment using `poetry`, you can create the corresponding `jupyter` kernel as follows:
```
$ poetry run ipython kernel install --user --name=[desired-kernel-name]
```
Running a notebook on this new kernel (`[desired-kernel-name]`) will enable you to use the project-specific packages installed in the virtual environment.

license: MIT

branch_name: refs/heads/main

extension: md

license_type: permissive


text:
# rspec-tapas

rspec-tapas is a set of small helpers and extensions we often use in [Selleo](https://selleo.com/) when testing our ruby apps.

## Installation

To install rspec-tapas just add following line to your `Gemfile`s `:test` group

```ruby
gem 'rspec-tapas'
```

then just `bundle install` and require it in `rails_helper.rb`

```ruby
require 'rspec_tapas/all'
```

## Helpers

### StubEnv

`StubEnv` is an extension that allows you to quickly stub values that should be retrieved from `ENV`.
It is assumed here, that you will use `fetch` method to retrieve such value (so it yields an exception when given ENV is not defined what sounds like a reasonable default).

*Example*

```ruby
it 'sends a notification after checkout' do
    stub_env('CHECKOUT_TIME', '13:00')
    #...
end
```

To include this extension only, call `require rspec_tapas/stub_env`

`StubEnv` is accessible in all types of specs

### DateHelpers

`DateHelpers` is a set of two methods that facilitate using `ActiveSupport::Testing::TimeHelpers`. Therefore you need to include `ActiveSupport::Testing::TimeHelpers` in your config if you want to use `DateHelpers`. It is as simple as

```ruby
RSpec.configure do |config|
  config.include ActiveSupport::Testing::TimeHelpers
end
```

#### at_date

`at_date` runs contents of the block at given date, provided in a format acceptable by `Date.new`. It is basically a shortcut to `travel_to(Date.new(*args)){}`.

*Example*

```ruby
context 'one day before deadline' do
    it 'sends a reminder' do
        #...
        at_date(2018, 1, 12) do
            SendPendingReminders.call
        end
        #...
    end
end
```

#### freeze_time

`freeze_time` freezes the time and runs contents of the block at. It is basically a shortcut to `travel_to(Time.current){}`.

*Example*

```ruby
it 'discards records created at the same time' do
    #...
    freeze_time do
      user_1 = create(:user)
      user_2 = create(:user)
    end
    #...
end
```

To include this extension only, call `require rspec_tapas/date_helpers`

`DateHelpers` are accessible in all types of specs

### ViewPage

`ViewPage` is a small helper that allows testing views using capybara helpers, usually limited only to feature specs.

*Example*

```ruby
RSpec.describe 'users/show.html.erb', type: :view do
  it 'renders user name' do
    user = create(:user, name: "Tony")
    allow(view).to receive(:user) { user }

    render

    expect(page).to have_content("Tony")
  end
end
```

To include this extension only, call `require rspec_tapas/view_page`

`ViewPage` is accessible only in view specs

### InvokeTask

`InvokeTask` is a small helper that facilitates calling rake tasks when testing them.

*Example*

```ruby
RSpec.describe 'db:materialize', type: :rake do
  it 'materializes db view by name' do
    database = double(:database)
    allow(Scenic).to receive(:database) { database }
    allow(database).to receive(:refresh_materialized_view)

    invoke_task('db:materialize', view_name: 'sample_view')

    expect(database).to \
      have_received(:refresh_materialized_view).with('sample_view')
  end
end
```

To include this extension only, call `require rspec_tapas/invoke_task`

`InvokeTask` is accessible only in rake specs

### GetMessagePart

`GetMessagePart` is a set of helpers to facilitate extraction of html/plain parts of emails sent. To extract plaintext part from email, use `text_part(mail)`. To extract HTML part, use `html_part(mail)`.

```ruby
RSpec.describe ReportMailer, type: :mailer do
  describe '#summary' do
    it 'creates email containing orders summary' do
      create_list(:order, 2)
      mail = ReportMailer.summary

      expect(mail.subject).to eq('Orders summary')
      expect(html_part(mail)).to include_html(
        <<~HTML
            <h1>Orders summary</h1>
            <strong>Total orders created:</strong> 2 <br />
        HTML
      )
      expect(text_part(mail)).to include('Total orders created: 2')
    end
  end
end
```

To include this extension only, call `require rspec_tapas/get_message_part`

`GetMessagePart` is accessible only in mailer specs

### JsonResponse

`JsonResponse` is a simple helper that transforms JSON encoded response body to `HashWithIndifferentAccess` or an array of those. This is to facilitate using regular matchers to assert such response.

*Example*

```ruby
it 'returns user names and emails' do
  create(:user, name: 'Tony', email<EMAIL_ADDRESS>  create(:user, name: 'Bruce', email<EMAIL_ADDRESS>
  get '/v1/users'

  expect(json_response).to match(
    data: [
      { name: 'Tony', email<EMAIL_ADDRESS>},
      { name: 'Bruce', email<EMAIL_ADDRESS>},
    ]
  )
end
```

To include this extension only, call `require rspec_tapas/json_response`

`JsonResponse` is accessible only in request and controller specs

### DownloadsHelpers

`DownloadsHelpers` is a set of two helpers oriented on facilitating testing downloads. First, we need to allow downloading file in given feature spec by calling `allow_file_downloads(page)`. Then, to get downloaded file contents, we need to use `downloaded_file_contents(file_name)` helper.

To allow downloads with headless-chrome capybara driver, we need to configure it in specific way. See example below:

```ruby
require 'selenium/webdriver'

Capybara.register_driver :headless_chrome do |app|
  options = Selenium::WebDriver::Chrome::Options.new
  options.add_argument('--headless')
  options.add_argument('--no-sandbox')
  options.add_argument('--disable-gpu')
  options.add_argument('--disable-popup-blocking')
  options.add_argument('--window-size=1920,1200')
  options.add_preference(
    :download,
    directory_upgrade: true,
    prompt_for_download: false,
    default_directory: RspecExtensions::DownloadsHelpers::DOWNLOADS_PATH
  )
  options.add_preference(:browser, set_download_behavior: { behavior: 'allow' })

  Capybara::Selenium::Driver.new(app, browser: :chrome, options: options)
end

Capybara.javascript_driver = :headless_chrome
```

Then, testing downloads is just a matter of few lines of code.

*Example*

```ruby
allow_file_downloads(page)

click_on 'Download report'

expect(downloaded_file_contents('daily_report.csv')).to eq("Name, Total revenue\nBatman suits, 1000")
```

To include this extension only, call `require rspec_tapas/downloads_helpers`

`DownloadsHelpers` are accessible only in feature specs

### FeatureHelpers

`FeatureHelpers` module contains a couple of methods useful in features specs.

- `display_logs` will display browser logs in console. This is useful for debugging javascript errors in feature specs.
- `ss` is a shortcut for `page.save_and_open_screenshot`
- `reload_page` is a handy shortcut for reloading current page.

To include this extension only, call `require rspec_tapas/feature_helpers`

`FeatureHelpers` are accessible only in feature specs

### BehaviorDSL

`BehaviorDSL` module has two roles. First one is to add some syntactic sugar to feature specs, by introducing additional level of describing context, but without interrupting test. This way you can group actions and assertions in readable blocks.

Second role of `BehaviorDSL` is to add capability of asserting feature - controller integration. This way we can ensure, that given block of capybara interactions were using particular controller action.

*Example*

```ruby
RSpec.describe 'Users management' do
  scenario do
    behavior 'Admin browses existing users', using: 'Admin::UsersController#index' do
      create(:user, name: 'Luke Cage')
      create(:unit, name: 'Jessica Jones')

      visit '/admin/users'

      expect(page).to have_row_content('Full name' => 'Luke Cage')
      expect(page).to have_row_content('Full name' => 'Jessica Jones')
    end

    behavior 'Admin updates existing user' do
      #...
    end
  end
end
```

To include this extension only, call `require rspec_tapas/behavior_dsl`

`BehaviorDSL` is accessible only in feature specs

## Matchers

### have_table_row

`have_table_row` is a matcher dedicated to finding table rows by their values correlated with specific headers. As so, this is mostly useful in feature specs.

*Examples*

```ruby
# Asserting headers by their names
expect(page).to have_table_row('First name' => 'Tony', 'Last name' => 'Stark', 'Rating' => 4.5)

# Asserting headers by their indices
expect(page).to have_table_row(1 => 'Tony', 2 => 'Stark', 3 => 4.5)

# Asserting headers by their indices - shorter version
expect(page).to have_table_row('Tony', 'Stark', 4.5)

# Composing matchers
expect(page).to have_table_row(have_content('ny'), 'Stark')
```

To include this matcher use: `config.include RSpecTapas::Matchers`

## About Selleo

![selleo](https://raw.githubusercontent.com/Selleo/selleo-resources/master/public/github_footer.png)

Software development teams with an entrepreneurial sense of ownership at their core delivering great digital products and building culture people want to belong to. We are a community of engaged co-workers passionate about crafting impactful web solutions which transform the way our clients do business.

All names and logos for [Selleo](https://selleo.com/about) are trademark of Selleo Labs Sp. z o.o. (formerly Selleo Sp. z o.o. Sp.k.)

repository: Selleo/rspec-tapas

branch_name: refs/heads/master

gha_language: Ruby

gha_license_id: MIT

repo_name: Selleo/rspec-tapas


text:
# Network Topologies in kOps

kOps supports a number of pre-defined network topologies. They are separated into commonly used scenarios, or topologies.

Each of the supported topologies are listed below, with an example on how to deploy them.

# Supported Topologies

kOps supports the following topologies:

| Topology        | Value   | Description                                                               |
|-----------------|---------|---------------------------------------------------------------------------|
| Public Cluster  | public  | All nodes will be launched in a subnet accessible from the internet.      |
| Private Cluster | private | All nodes will be launched in a subnet with no ingress from the internet. |

# Types of Subnets

## Public Subnet

A subnet of type `Public` accepts incoming traffic from the internet.

## Private Subnet

A subnet of type `Private` does not route traffic from the internet.

If a cluster is IPv6, then `Private` subnets are IPv6-only.

If the subnet is capable of IPv4, it typically has a CIDR range from private IP address space.
Egress to the internet is typically routed through a Network Address Translation (NAT) device,
such as an AWS NAT Gateway.

If the subnet is capable of IPv6, egress to the internet is typically routed through a
connection-tracking firewall, such as an AWS Egress-only Internet Gateway. Egress to the
NAT64 range `64:ff9b::/96` is typically routed to a NAT64 device, such as an AWS NAT Gateway.

## DualStack Subnet

A subnet of type `DualStack` is like `Private`, but supports both IPv4 and IPv6.

On AWS, this subnet type is used for nodes, such as control plane nodes and bastions,
which need to be instance targets of a load balancer.

## Utility Subnet

A subnet of type `Utility` is like `Public`, but is not used to provision nodes.

Utility subnets are used to provision load balancers that accept ingress from the internet.
They are also used to provision NAT devices.

# Defining a topology on create

To specify a topology use the `--topology` or `-t` flag as in :

```
kops create cluster ... --topology public|private
```

You may also set a [networking option](networking.md), with the exception that the
`kubenet` option does not support private topology.

Newly created clusters with private topology *will* have public access to the Kubernetes API and an (optional) SSH bastion instance
through load balancers. This can be changed as described below.

## Changing the Topology of the API Server

To change the load balancer that fronts the API server from internet-facing to internal-only there are a few steps to accomplish:

AWS load balancers do not support changing from internet-facing to internal. However, we can manually delete it and have kOps recreate the ELB for us.

### Steps to Change the Load Balancer from Internet-Facing to Internal

- Edit the cluster: `kops edit cluster $NAME`
- Change the api load balancer type from: Public to Internal. It should look like this when done:
```yaml
 spec:
    api:
      loadBalancer:
        type: Internal
```
 - Save and exit the edit
 - Run the update command to check the config: `kops update cluster $NAME`
 - BEFORE DOING the same command with the `--yes` option, go into the AWS console and DELETE the api load balancer
 - Run: `kops update cluster $NAME --yes`
 - Run a rolling update so that the control plane nodes register with the new internal load balancer.
 Run `kops rolling-update cluster --cloudonly --force --instance-group-roles master --yes` command.
 We have to use the  `--cloudonly` option because we deleted the API load balancer, leaving no way to talk to the Kubernetes API.
 The `--force` option is there because otherwise kOps doesn't know that we need to update the control plane nodes.
 Once the rolling update has completed you have an internal only load balancer that has the control plane nodes registered with it.

repository: kubernetes/kops

gha_language: Go

repo_name: kubernetes/kops


text:
Welcome to Dev Connector 👋

> Social network for developers

It is a small social network app that includes authentication, profiles and forum posts.

To practice at building NodeJS app, I built it on top of this [MERN Stack Front To Back](https://www.udemy.com/course/mern-stack-front-to-back/) course on Udemy.

## Version info

- NodeJS v12.16.1

## Getting started

> Set private token

```
$ cd config
$ touch default.json
```

```
{
  "mongoURI": "xxx",
  "jwtSecret": "xxx",
  "githubToken": "xxx"
}
```

> Install dependencies and Run dev

```
# Install server dependencies
npm install

# Install client dependencies
cd client
npm install
cd ..

# Run both Express & React from root
npm run dev
```

> Build for production

```
# Build for production
cd client
npm run build
```

### ✨ Live Demo

[herokuapp](https://suh-dev-connector.herokuapp.com/)

- Version: 1.0.0
- Author: Suhyeon Jang, [Brad Traversy MERN stack](https://www.udemy.com/course/mern-stack-front-to-back/) course

## Resource

You can get an Github token by following [these instructions](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line)

- [jwt authentication](https://jwt.io/)
- [transitioning from rdbms to mongodb](https://www.mongodb.com/blog/post/transitioning-from-relational-databases-to-mongodb)
- [prettier options](https://prettier.io/docs/en/options.html)


text:
---
layout: default
---

# 1996-03-04 - Re: Duress

## Header Data

From<EMAIL_ADDRESS>(Bill Frantz)

To<EMAIL_ADDRESS>Message Hash: 0f778167ce8eec2e62c87e92930829a0e30e877abaaeb8e3ae8683740308a10a

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1996-03-04 23:17:25 UTC

Raw Date: Tue, 5 Mar 1996 07:17:25 +0800

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Bill Frantz)
Date: Tue, 5 Mar 1996 07:17:25 +0800
To<EMAIL_ADDRESS>Subject: Re: Duress
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

At  2:14 PM 3/4/96 -0500<EMAIL_ADDRESS>(Jean-Francois Avon (JFA
Technologies, QC, wrote:
>The transponder analogy might not be very accurate.
>
>When you fly in a controled airspace, the *controller* asks you to
>dial in certain digits on the transponder.

The analogy may be reaching its elastic limit although you can use those
transponder codes when flying VFR, not in radio contact.  What I really
wanted to suggest is that we try to design (when we can) to minimize the
possibility of bad things happening to people who use duress codes.

Regards - Bill

------------------------------------------------------------------------
Bill Frantz       | The CDA means  | Periwinkle  --  Computer Consulting
(408)356-8506     | lost jobs and  | 16345 Englewood Ave.
<EMAIL_ADDRESS>| dead teenagers | Los Gatos, CA 95032, USA

{% endraw %}
```

## Thread

+ Return to [March 1996](/archive/1996/03)

+ Return to<EMAIL_ADDRESS>(Bill Frantz)](/authors/frantz_at_netcom_com_bill_frantz_)"

+ 1996-03-04 (Tue, 5 Mar 1996 07:17:25 +0800) - Re: Duress -<EMAIL_ADDRESS>(Bill Frantz)_


text:
+++
title = "2021 Workshop"
chapter = false
weight = 700
+++

Welcome to the Service Catalog tools network capability workshop - 2021 edition.

The average completion time for the workshop is between 2 - 3 hours.

## What are we going to be doing?

In this workshop you will be building out the networking capability for your multi-account environment.

You will learn about some of the essential building blocks the Service Catalog tools provide you when creating your
multi account workflow:
- provisioning resources into accounts using AWS CloudFormation, AWS Service Catalog and Hashicorp Terraform
- executing AWS Lambda functions
- Sharing Service Catalog portfolios

You will be using these building blocks to:

- Remove the default networking resources from your AWS accounts
- Provision a new AWS VPC using AWS CloudFormation
- Provision a new subnet using AWS Service Catalog
- Provision a new subnet using Hashicorp Terraform
- Share a Service Catalog portfolio so end users can create their own subnets

## Let's get going!

When you are ready, click the right arrow to begin!

{{% notice tip%}}
You can use the left and right arrows to navigate through the Workshop
{{% /notice %}}

gha_language: CSS


text:
---
layout: default
title: Upgrading from Yap 1.x to Yap 2.x
nav_order: 4
parent: Miscellaneous
---

## Upgrading from Yap 1.x to Yap 2.x

---

Follow the upgrade steps as you normally do above.  You will also need to follow item #3 & #4 under [Setup](../../general/setup/), to add the Twilio credentials.

You can check that everything is functioning by going to the upgrade advisor http://your-instance/upgrade-advisor.php

repository: GuillaumeIsabelleX/yap

repo_name: GuillaumeIsabelleX/yap


text:
Linux Desktop Icon Themes library
=================================

An [**Erlang**][1] library to access desktop themes data as specified
[**Icon Theme Specification**][2]

Build
-----

    $ rebar3 compile

[1]: http://www.erlang.org/
[2]: https://specifications.freedesktop.org/icon-theme-spec/latest/

repository: isvilen/themes

repo_name: isvilen/themes


text:
---
layout: post
title: SQS to the rescue
---

**tl;dr** [Amazon SQS](https://aws.amazon.com/sqs/) is currently one of the best options for a queue service. It's [inexpensive
](https://aws.amazon.com/sqs/pricing/) (first 1 million requests per month are free, thereafter $0.50 per 1 million requests), fast, reliable, scalable, fully managed message queuing service.  It just works.

For the following examples I'm using the [aws-sdk-ruby](https://github.com/aws/aws-sdk-ruby), which is a great SDK because it's transparent to the [official API documentation](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/Welcome.html), no abstractions, no different method names etc, what you see in the documentation is what you get on the SDK.

## Getting your hands dirty

Let's begin by bootstrapping a queue using the [create](http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/SQS/QueueCollection.html#create-instance_method) method:

```ruby
sqs.queues.create 'myqueue', visibility_timeout: 600, message_retention_period: 345600
```

Then let's send a message using the [send_message](http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/SQS/Queue.html#send_message-instance_method) method:

```ruby
sqs.queues.named('myqueue').send_message 'HELLO'
```

Finally let's consume a message using the [receive_message](http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/SQS/Queue.html#receive_message-instance_method) method:

```ruby
sqs_msg = sqs.queues.named('myqueue').receive_message
do_something(sqs_msg)
sqs_msg.delete
```

Catching things up - we created a queue that holds messages for 4 days `message_retention_period: 345600`, which means: if we don't consume a message within 4 days the message will be lost.

The `visibility_timeout` is the time in seconds we have to consume a message after its reception. When we send a message to SQS `send_message`, the message is listed on "Messages Available", when we receive a message `receive_message` the message is moved to "Messages in Flight", if we don't `delete` the message within 600 seconds (which is the visibility timeout we configured above) the message will be moved back to "Messages Available" and become ready to be consumed again. [Read more about the SQS Message Lifecycle](https://aws.amazon.com/sqs/details/#Amazon_SQS_Message_Lifecycle).

Be generous while configuring the default `visibility_timeout` for a queue. If your worker in the worst case takes 2 minutes to consume a message, set the `visibility_timeout` to at least 4 minutes. It doesn't hurt and it will be better than having the same message being consumed more than the expected.

## Auto retrying errors

```ruby
begin
  sqs_msg = sqs.queues.named('myqueue').receive_message
  do_something(sqs_msg)
  sqs_msg.delete
rescue => e
  error e
end
```

In the code above if `do_something` raises an exception, the message will become available again as soon as its `visibility_timeout` expires, so other workers will be able to re-attempt the message.

This behavior will also cover in case your process or server crashes. Basically, if you don't call `sqs_msg.delete`, the message will become available again no matter what :)

## Dead Letter Queues

Let's retry stuff, but not forever, right? Sometimes no matter how many times we re-attempt a message, it will continue failing. To do not discard or retry forever these messages addicted to failure, we can let them rest in peace using a [Dead Letter Queue
](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/SQSDeadLetterQueue.html).

The code below creates `myqueue_failures` and associates it with `myqueue` as a dead letter queue:

```ruby
dl_queue = sqs.queues.create('myqueue_failures')

sqs.queues.create('myqueue', redrive_policy: %Q{ {"maxReceiveCount":"3", "deadLetterTargetArn":"#{dl_queue.arn}"}" })
```

The `redrive_policy` above tells SQS to move a message from `myqueue` to `myqueue_failures` if its receive count reaches 3.

## Delaying a message `perform_in`

SQS supports delaying a message up to 15 minutes, before it becomes available to be consumed using the  [delay_seconds](http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/SQS/Queue.html#send_message-instance_method) option:

```ruby
sqs.queues.named('myqueue').send_message 'HELLO', delay_seconds: 60
```

But as we are all [hackerz](http://www.imdb.com/title/tt0113243/) and we know that the `visibility_timeout` [supports up to 12 hours](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ChangeMessageVisibility.html), we can use it to extend the delay.

### Sending a message with extended delay

```ruby
perform_at = Time.now + 1.hour # max: 12.hours

sqs.queues.named('myqueue').send_message 'H311O',
  message_attributes: {
    'perform_at' => { string_value: perform_at.to_s,
                      data_type: 'String' }
```

### Receiving a message with extended delay

```ruby
sqs_msg = sqs.queues.named('myqueue').receive_message, message_attribute_names: ['perform_at']

delay = Time.parse(sqs_msg.message_attributes['perform_at'][:string_value]) - Time.now

if delay > 0
  sqs_msg.visibility_timeout = delay.to_i
else
  do_something(sqs_msg)
  sqs_msg.delete
end
```

Be careful with this workaround, because it will increase the message receive count.

## 1 million != 1 million

> [first 1 million requests per month are free, thereafter $0.50 per 1 million requests](https://aws.amazon.com/sqs/pricing/))

1 million requests don't mean 1 million jobs consumed, as we need at least 3 requests to fully consume a message:

1. `send_message`
2. `receive_message`
3. `sqs_msg.delete`

Although these requests can be executed in batches up to 10 messages, you will need all of them to consume a message, and even if you are using the [poll](http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/SQS/Queue.html#poll-instance_method) method with a high `wait_time_seconds`, you will probably make some empty requests while looking for new messages.

## SNS to SQS

[SNS](https://aws.amazon.com/sns/) distributes messages across multiple SQS queues [and more](http://docs.aws.amazon.com/sns/latest/dg/SNS_Scenarios.html).

Sending the same `msg` to `my_queue1` and `my_queue2` using SQS:

```ruby
sqs.queues.named('my_queue1').send_message(msg)
sqs.queues.named('my_queue2').send_message(msg)
```

Using [SNS to SQS](http://docs.aws.amazon.com/sns/latest/dg/SendMessageToSQS.html):

```ruby
topic.publish(msg)
# sends to my_queue1 and my_queue2
```

SNS will fanout the message to both queues, consequently, you will pay ([SNS to SQS is free](https://aws.amazon.com/sns/pricing/)) and wait for only one request to SNS, instead of two to SQS.

## Trouble in Paradise

SQS is really good, the Ruby SDK is great, but… how to consume messages continuously?

```ruby
# myqueue_worker.rb
sqs.queues.named('myqueue').poll { | sqs_msg | do_something(sqs_msg) }
```

Then:

```bash
ruby myqueue_worker.rb
```

Doesn't seem to be very reliable, right? No threads, consumes messages one by one, and if you need to consume from another queue you will need to start a new process `ruby myotherqueue_worker.rb`, which is a waste of resources when one queue is empty and others have messages available.

### Introducing Shoryuken

[Shoryuken](https://github.com/phstc/shoryuken) is a project based on [Sidekiq](https://github.com/mperham/sidekiq), that uses all the cool stuff Sidekiq did with processes, [Celluloid](https://github.com/celluloid/celluloid), CLI etc, but for SQS.

No more `ruby myotherqueue_worker.rb`, [Shoryuken will handle that for you](https://github.com/phstc/shoryuken#start-shoryuken), including [signal handling](https://github.com/phstc/shoryuken/wiki/Signals).

A single Shoryuken process can consume messages from multiple queues, [load balancing the consumption](https://github.com/phstc/shoryuken#load-balancing).

It's transparent to SQS, it passes the [ReceivedMessage](docs.aws.amazon.com/AWSRubySDK/latest/AWS/SQS/ReceivedMessage.html) to the [workers](https://github.com/phstc/shoryuken#worker-class), no abstractions, no Rescue compatibility, SQS as is!

It accepts [multiple workers per queue](https://github.com/phstc/shoryuken/wiki/Sending-a-message#multiple-workers-for-the-same-queue).

Check more on the [project's README](https://github.com/phstc/shoryuken).

## Conclusion

SQS is cheap, you don't need to worry about managing your queue services, setup Redis etc.

Keep your focus on your workers/jobs (which should be the most important to you) and let Amazon take care of the infrastructure, it works!


text:
# Change Log

All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.

## [1.1.4](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.1.3<EMAIL_ADDRESS>(2021-06-27)

**Note:** Version bump only for package @sapphire/stopwatch

## [1.1.3](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.1.2<EMAIL_ADDRESS>(2021-06-06)

### Bug Fixes

-   remove peer deps, update dev deps, update READMEs ([#124](https://github.com/sapphiredev/utilities/issues/124)) ([67256ed](https://github.com/sapphiredev/utilities/commit/67256ed43b915b02a8b5c68230ba82d6210c5032))

## [1.1.2](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.1.1<EMAIL_ADDRESS>(2021-05-20)

### Bug Fixes

-   **stopwatch:** mark package as side effect free ([332ee5c](https://github.com/sapphiredev/utilities/commit/332ee5c2992af062f1d81de78f45046689288129))

## [1.1.1](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.1.0<EMAIL_ADDRESS>(2021-05-02)

### Bug Fixes

-   drop the `www.` from the SapphireJS URL ([494d89f](https://github.com/sapphiredev/utilities/commit/494d89ffa04f78c195b93d7905b3232884f7d7e2))
-   update all the SapphireJS URLs from `.com` to `.dev` ([f59b46d](https://github.com/sapphiredev/utilities/commit/f59b46d1a0ebd39cad17b17d71cd3b9da808d5fd))

# [1.1.0](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.8<EMAIL_ADDRESS>(2021-04-21)

### Features

-   add @sapphire/embed-jsx ([#100](https://github.com/sapphiredev/utilities/issues/100)) ([7277a23](https://github.com/sapphiredev/utilities/commit/7277a236015236ed8e81b7882875410facc4ce17))

## [1.0.8](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.7<EMAIL_ADDRESS>(2021-04-19)

### Bug Fixes

-   change all Sapphire URLs from "project"->"community" & use our domain where applicable 👨🌾🚜 ([#102](https://github.com/sapphiredev/utilities/issues/102)) ([835b408](https://github.com/sapphiredev/utilities/commit/835b408e8e57130c3787aca2e32613346ff23e4d))

## [1.0.7](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.6<EMAIL_ADDRESS>(2021-04-03)

**Note:** Version bump only for package @sapphire/stopwatch

## [1.0.6](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.5<EMAIL_ADDRESS>(2021-03-16)

### Bug Fixes

-   **stopwatch:** add tslib dependency ([5e7f690](https://github.com/sapphiredev/utilities/commit/5e7f6903475ccfec2147b97f8aa56fb7c037a860))

## [1.0.5](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.4<EMAIL_ADDRESS>(2021-02-16)

**Note:** Version bump only for package @sapphire/stopwatch

## [1.0.4](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.3<EMAIL_ADDRESS>(2021-01-16)

**Note:** Version bump only for package @sapphire/stopwatch

## [1.0.3](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.2<EMAIL_ADDRESS>(2021-01-01)

**Note:** Version bump only for package @sapphire/stopwatch

## [1.0.2](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.1<EMAIL_ADDRESS>(2020-12-26)

**Note:** Version bump only for package @sapphire/stopwatch

## [1.0.1](https://github.com/sapphiredev/utilities/compare/@sapphire/stopwatch@1.0.0<EMAIL_ADDRESS>(2020-12-22)

**Note:** Version bump only for package @sapphire/stopwatch

# 1.0.0 (2020-12-22)

### Features

-   add stopwatch package ([#44](https://github.com/sapphiredev/utilities/issues/44)) ([cb4370b](https://github.com/sapphiredev/utilities/commit/cb4370b034bced94142755316e933108b7b15a17))

repository: jurienhamaker/utilities

repo_name: jurienhamaker/utilities


text:
# Top 20 #HashTags in tweets by YOU

## Introduction

We're picking up top X **#hashtags** used in all tweets by this user & plotting them as barchart.

## Example

Below is a demonstrative plot for this category.

![top20HashTagsInTweetsByAnjan_Roy](../plots/top20HashTagsInTweetsByAnjan_Roy.png)

repository: itzmeanjan/twiz

gha_language: Python

repo_name: itzmeanjan/twiz


text:
---
title: "Configure GCP account"
chapter: true
weight: 2
---
# Configure GCP account

1. Get the project ID:

![GCP project ID](/images/gcloud-project-id.png?width=40pc  "GCP project ID")

1. Set the default project (replace **PROJECT_ID** with the project during the event workshop or your own project):

```
export PROJECT_ID=srecon19-workshop-250603
```

```
gcloud config set project $PROJECT_ID
```

1. Set the default compute region and zone:

```
export REGION_ID=europe-west1
```

```
gcloud config set compute/region $REGION_ID
```

```
export ZONE_ID=europe-west1-b
```

```
gcloud config set compute/zone $ZONE_ID
```

1. Check the configuration:
```
gcloud config list
```

```
[compute]
region = europe-west1
zone = europe-west1-b
[core]
account =<EMAIL_ADDRESS>disable_usage_reporting = True
project = srecon19-workshop-250603
```

Now, everything is prepared to create a Kubernetes cluster.


text:
---
id: data-loader
title: Data Loaders
---
Data Loaders are a popular caching pattern from the [JavaScript GraphQL implementation](https://github.com/graphql/dataloader).
`graphql-java` provides [support for this pattern](https://www.graphql-java.com/documentation/v16/batching/)
using the `DataLoader` and `DataLoaderRegistry`.

Since `graphql-kotlin` allows you to abstract the schema generation and data fetching code, you may not even need
data loaders if instead you have some persistant cache on your server.

```kotlin
class User(val id: ID) {
    // The friendService and userService, which have nothing to do with GraphQL,
    // should be concerned with caching and batch calls instead of your schema classes
    fun getFriends(): List<User> {
        val friends: List<ID> = friendService.getFriends(id)
        return userService.getUsers(friends)
    }

}
```

If you still want to use data loaders though, they are supported through the common interfaces.

`graphql-kotlin-dataloader` module provides convenient abstractions over the [java-dataloader](https://github.com/graphql-java/java-dataloader).

## `KotlinDataLoader`

To help in the registration of `DataLoaders`, we have created an interface `KotlinDataLoader`:

```kotlin
interface KotlinDataLoader<K, V> {
    val dataLoaderName: String
    fun getDataLoader(graphQLContext: GraphQLContext): DataLoader<K, V>
}
```

This allows for library users to still have full control over the creation of the `DataLoader` and its various configuration
options but also allows common server code to handle the registration, generation and execution of the request.

Additionally, the `getDataLoader` method provides access to the `graphQLContext` that was created from the [GraphQLContextFactory](../graphql-context-factory.md),
allowing you to use the `graphQLContext` as [context of the DataLoader](https://www.graphql-java.com/documentation/batching/#passing-context-to-your-data-loader).

## `KotlinDataLoaderRegistryFactory`

The [GraphQLRequestHandler](../graphql-request-handler.md) accepts an optional `KotlinDataLoaderRegistryFactory`.
which generates a new `KotlinDataLoaderRegistry` on every request. The registry is a map of a unique data loader names to a `DataLoader` object that handles the cache for an output type in your graph.
A `DataLoader` caches the types by some unique value, usually by the type id, and can handle different types of batch requests.

```kotlin
class UserDataLoader : KotlinDataLoader<ID, User> {
    override val dataLoaderName = "UserDataLoader"
    override fun getDataLoader(graphQLContext: GraphQLContext) =
        DataLoaderFactory.newDataLoader<ID, User> { ids ->
            CompletableFuture.supplyAsync {
                ids.map { id -> userService.getUser(id) }
            }
        }
}

class FriendsDataLoader : KotlinDataLoader<ID, List<User>> {
    override val dataLoaderName = "FriendsDataLoader"
    override fun getDataLoader(graphQLContext: GraphQLContext) =
        DataLoaderFactory.newDataLoader<ID, User>(
            { ids, batchLoaderEnvironment ->
                val context = batchLoaderEnvironment.getContext<GraphQLContext>()
                // do something with graphQLContext
                CompletableFuture.supplyAsync {
                    ids.map { id ->
                        val friends: List<ID> = friendService.getFriends(id)
                        userService.getUsers(friends)
                    }
                }
            },
            DataLoaderOptions.newOptions()
                .setCachingEnabled(false)
                .setBatchLoaderContextProvider { graphQLContext }
        )
}

val dataLoaderRegistryFactory = KotlinDataLoaderRegistryFactory(
    UserDataLoader(), FriendsDataLoader()
)

val dataLoaderRegistry = dataLoaderRegistryFactory.generate()
```

## `KotlinDataLoaderRegistry`

[KotlinDataLoaderRegistry](https://github.com/ExpediaGroup/graphql-kotlin/blob/master/executions/graphql-kotlin-dataloader/src/main/kotlin/com/expediagroup/graphql/dataloader/KotlinDataLoaderRegistry.kt)
is a decorator of the original `graphql-java` [DataLoaderRegistry](https://github.com/graphql-java/java-dataloader/blob/master/src/main/java/org/dataloader/DataLoaderRegistry.java)
that keeps track of all underlying `DataLoader`s futures. By keeping track of to cache map containing returned futures,
we get more granular control when to dispatch data loader calls.

## `getValueFromDataLoader`

`graphql-kotlin-server` includes a helpful extension function on the `DataFetchingEnvironment` so that you can easily retrieve values from the data loaders in your schema code.

```kotlin
class User(val id: ID) {
    @GraphQLDescription("Get the users friends using data loader")
    fun getFriends(dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<List<User>> {
        return dataFetchingEnvironment.getValueFromDataLoader("FriendsDataLoader", id)
    }
}
```

## DataLoaders and Coroutines

`graphql-java` relies on `CompletableFuture`s for scheduling and asynchronously executing GraphQL operations.
While we can provide native support for coroutines for data fetchers (aka field resolvers) because they are resolved
independently, we cannot easily provide native support for the `DataLoader` pattern as it relies
on `CompletableFuture` state machine internals and we cannot update it to use coroutines without fully rewriting
GraphQL Java execution engine.

If you would like to use `DataLoader` pattern in your project, you have to update your data fetchers (aka field resolvers) to return
`CompletableFuture` from the invoked `DataLoader`.

### Example

Consider the following query:

```graphql
fragment UserFragment on User {
    id
    name
}
query GetUsersFriends {
    user_1: user(id: 1) {
        ...UserFragment
    }
    user_2: user(id: 2) {
        ...UserFragment
    }
}
```

And the corresponding code that will autogenerate schema:

```kotlin
class MyQuery(
    private val userService: UserService
) : Query {
    suspend fun getUser(id: Int): User = userService.getUser(id)
}

class UserService {
    suspend fun getUser(id: Int): User = // async logic to get user
    suspend fun getUsers(ids: List<Int>): List<User> = // async logic to get users
}
```

When we execute the above query, we will end up calling `UserService#getUser` twice which will result in two independent
downstream service/database calls. This problem is called N+1 problem. By using `DataLoader` pattern,
we can solve this problem and only make a single downstream request/query.

Lets create the `UserDataLoader`:

```kotlin
class UserDataLoader : KotlinDataLoader<ID, User> {
    override val dataLoaderName = "UserDataLoader" // 1
    override fun getDataLoader(graphQLContext: GraphQLContext) = // 2
        DataLoaderFactory.newDataLoader<Int, User>(
            { ids, batchLoaderEnvironment ->
                val coroutineScope = // 3
                    batchLoaderEnvironment.getContext<GraphQLContext>()?.get<CoroutineScope>()
                        ?: CoroutineScope(EmptyCoroutineContext) // 4

                coroutineScope.future { // 5
                    userService.getUsers(ids)
                }
            },
            DataLoaderOptions.newOptions()
                .setBatchLoaderContextProvider { graphQLContext }
        )
}

```

There are some things going on here:

1. We define the `UserDataLoader` with name "UserDataLoader".
2. The `KotlinDataLoader#getDataLoader(GraphQLContext)` method returns a `DataLoader<Int, User>`, which `BatchLoader` function should return a `List<User>` and `BatchLoaderEnvironment` context is the `GraphQLContext`.
3. Given that we **don't want** to change our `UserService` async model that is using coroutines, we need a `CoroutineScope`, [which is conveniently available](../../schema-generator/execution/async-models/#coroutines) in the `GraphQLContext` and accessible through the `BatchLoaderEnvironment`.
4. After retrieving the `CoroutineScope` from the `batchLoaderEnvironment` we will be able to execute the `userService.getUsers(ids)` suspendable function.
5. We interoperate the suspendable function result to a `CompletableFuture` using [coroutineScope.future](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.future/future.html).

Finally, we need to update `user` field resolver, to return the `CompletableFuture<User>` from the invoked `DataLoader`.
Make sure to update method signature to also accept the `dataFetchingEnvironment` as you need to pass it to `DataLoader#load` method to be able to execute the request in appropriate coroutine scope.

```kotlin
class MyQuery(
    private val userService: UserService
) : Query {
    fun getUser(id: Int, dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<User> =
        dataFetchingEnvironment
            .getDataLoader<Int, Mission>("UserDataLoader")
            .load(id, dataFetchingEnvironment)
}

class UserService {
    suspend fun getUser(id: Int): User {
        // logic to get user
    }
    suspend fun getUsers(ids: List<Int>): List<User> {
        // logic to get users, this method is called from the DataLoader
    }
}
```

repository: ExpediaGroup/graphql-kotlin

gha_language: Kotlin

repo_name: ExpediaGroup/graphql-kotlin


text:
---
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

layout: docpage
title: User Interface (UI)
description: Where the user works with your app
permalink: /user-interface
---

# User Interface (UI)

Where the user works with your app

Developers know that the data layer, the business layer, and the user interface of an apploication all need to be fast, reliable, and robust. End-users tend to focus on the user interface. It is where they interact with all the strength and subtlety of an application, and where they tend to form their opinion of an app.

A good "user experience" includes not just getting the process result you want, but how comfortable you were working with the UI. How easy is it to do what you want to do? How "pleasant" and intuitive is the display? It is not for nothing that the title of one of the key books on user experience is _Don't Make Me Think_.

Apache Royale provides tools, structures, features, and component sets that help you create a user interface that helps your end-users do what they want to do.

* [Component Sets](component-sets)
  * [Basic](component-sets/basic)
  * [Jewel](component-sets/jewel)
* [Layouts](user-interface/layouts)
* [View States](/features/view-states)
* Alerts
* Styles and CSS
* Localization
* Internationalization
* Graphics
* Rich media

repository: apache/royale-docs

gha_language: Sass

repo_name: apache/royale-docs


text:
---
layout: post
title: Hadoop Database access experiment
tags: [MySQL, Apache Hadoop]
use_math: true
---
Over a couple of weeks I was reading and practicing the book ["Hadoop in Action"](https://www.manning.com/books/hadoop-in-action). After getting some insight on Hadoop and Map Reduce I worked out a couple of examples from the book and some example problems which I created too. Then I was discussing about features of Hadoop with some of my colleagues over a cup of tea. One of the guy asked a question regarding accessing database from Hadoop and process the data. I saw some discussions related to Hadoop and database access some where in the internet. Finally I digged-out the article "Database Access with Hadoop" for Cloudera blog. After reading the same I decided to work with a sample problem.

To workout the Hadoop database access sample program.  I extracted the Tweets for some social media analysis practice. The extraction was done using Twitter4j API. The data is stored in MySQL database. The database table contains one table called NewGamil with following structure.

```sql
        +-----------------+--------------+------+-----+---------+----------------+
        | Field           | Type         | Null | Key | Default | Extra |
        +-----------------+--------------+------+-----+---------+----------------+
        | TweetId        | int(11)      | NO   | PRI | NULL    | auto_increment |
        | Tweet           | varchar(240) | YES  |     | NULL    |                |
        +-----------------+--------------+------+-----+---------+----------------+

```

The problem which selected to workout is fetch all the tweets from the table 'NewGamil' and perform a word count. The word count result has to be stored in HDFS. In-fact there are ways to write data back to database itself. But I decided first experiment with read from database ;-).

Hadoop provides a handy API for accessing database; the DBInputformat API. The API allows us to read data from RDBMS like MySQL, PostgreSQL of Oracle . To access the data from DB we have to create a class to define the data which we are going to fetch and write back to DB.  In my project I created a class namely GetTweets to accomplish the same.

```java
    public static class GetTweets implements Writable, DBWritable {
        String strTweet;

        public GetTweets() {

        }

        public void readFields(DataInput in) throws IOException {

            this.strTweet = Text.readString(in);
        }

        public void readFields(ResultSet resultSet) throws SQLException {
            // this.id = resultSet.getLong(1);
            this.strTweet = resultSet.getString(1);
        }

        public void write(DataOutput out) throws IOException {

        }

        public void write(PreparedStatement stmt) throws SQLException {

        }

    }

```
Since I am accessing only one field from the table I defined the same in readFields() method. The write() methods are kept blank because the project does not aims to write back the data to DB. I'll experiment with writing data and post it soon.  In the readFileds() method we have to define how the data had to be extracted from the DB table. Since 'Tweet'  the data which I extractes for processing is VARCHAR() I am reading it as string and casting it to Text() data in hadoop. This class "GetTweets" will be used in our Mapper and Reducer class.

Now lets write our Mapper class:

```java

    public static class TweetWordCountMapper extends MapReduceBase implements
            Mapper<LongWritable, GetTweets, Text, IntWritable> {
        private final static IntWritable intTwordsCount = new IntWritable(1);
        private Text strTwoken = new Text();

        public void map(LongWritable key, GetTweets value,
                OutputCollector<Text, IntWritable> output, Reporter reporter)
                throws IOException {
            GetTweets tweets = new GetTweets();
            tweets.strTweet = value.strTweet;
            TwitterTokenizer twokenizer = new TwitterTokenizer();
            List<String> twokens = twokenizer.twokenize(value.strTweet
                    .toString());

            for (int i = 0; i < twokens.size(); i++) {
                output.collect(new Text(twokens.get(i)), intTwordsCount);
            }

        }

    }

```

In the mapper class ```'TweetWordCountMapper'``` I used the ```'GetTweets'``` class to fetch the values for processing. Then we can access the data by creating object of the class inside the Mapper class.
NB: The code for TwitterTokenizer is taken from https://github.com/vinhkhuc/Twitter-Tokenizer.

Now we can write our reducer class :

```java
    public static class TweetWordCountReducer extends MapReduceBase implements
            Reducer<Text, IntWritable, Text, IntWritable> {
        public void reduce(Text key, Iterator<IntWritable> values,
                OutputCollector<Text, IntWritable> output, Reporter reporter)
                throws IOException {
            int intTwokenCount = 0;
            while (values.hasNext()) {
                intTwokenCount += values.next().get();
            }
            output.collect(key, new IntWritable(intTwokenCount));
        }
    }
```

This reducer is responsible to sum the word count and produce the final output.

After this we have to configure the job with database connection details and driver class.

```java
        JobConf twokenJobConf = new JobConf(TweetWordCount.class);
        twokenJobConf.setJobName("twoken_count");

        twokenJobConf.setInputFormat(DBInputFormat.class); //Set input format here
        twokenJobConf.setOutputFormat(TextOutputFormat.class);// Sets the output format

        Object out = new Path("twokens");

        twokenJobConf.setMapperClass(TweetWordCountMapper.class);
        twokenJobConf.setCombinerClass(TweetWordCountReducer.class);
        twokenJobConf.setReducerClass(TweetWordCountReducer.class);

        twokenJobConf.setOutputKeyClass(Text.class);
        twokenJobConf.setOutputValueClass(IntWritable.class);

        DBConfiguration.configureDB(twokenJobConf, "com.mysql.jdbc.Driver",
                "jdbc:mysql://localhost/GmailTrend", "jaganadhg", "jagan123"); //Specifies the DB configuration

        String[] fields = { "Tweet" }; //Specifies the Fields to be fetched from DB
        DBInputFormat.setInput(twokenJobConf, GetTweets.class, "NewGamil",
                null /* conditions */, "Tweet", fields); // Specifies the DB table and fields

        SequenceFileOutputFormat.setOutputPath(twokenJobConf, (Path) out);

        JobClient.runJob(twokenJobConf);
```

Before compiling and running the program we have to some additional setup in the Hadoop ecosystem. The MySQL connector library has to be put in ``$HADOOP_HOME/lib`` folder. To download the connector .jar file go to ```MySQL Connector/J``` download folder. I used the ```mysql-connector-java-3.1.14-bin.jar``` file in my program. After putting the jar in ```$HADOOP_HOME/lib``` restart the hadoop ecosystem. Viola !! now you are ready to run the program. Convert the code to .jar file and run it.

The complete project is available in my [bitbucket repository](https://bitbucket.org/jaganadhg/hadooexamples/src/840db7af9abc/HadoopMySQLRead/).

Happy hacking !!!!!!!!!!!!

Migrated from my [old blog jaganadhg.freeflux.net](https://web.archive.org/web/20160323193721/http://jaganadhg.freeflux.net/blog)


text:
Welcome to another issue of Haskell Weekly!

This is a weekly summary of what's going on in its community.

## Featured

- [GHC 9.6.2 is now available](https://discourse.haskell.org/t/ghc-9-6-2-is-now-available/6308?u=taylorfausak) by Ben Gamari
  > The GHC developers are happy to announce the availability of GHC 9.6.2. This release is primarily a bug-fix release addressing a few issues found in 9.6.1.

- [Stack version 2.11.1](https://discourse.haskell.org/t/ann-stack-2-11-1/6287?u=taylorfausak) by Mike Pilgrem

- [Haskell Language Server <IP_ADDRESS> is now available](https://discourse.haskell.org/t/haskell-language-server-2-0-0-0-is-now-available/6289?u=taylorfausak) by Zubin Duggal
  > This is a fairly small release, marking the start of a new versioning scheme where HLS and all the included packages in the distribution will be versioned in lockstep.

- [All Cats Are Grey](https://github.com/JeanHuguesdeRaigniac/effects-landscape/blob/55b7e179eb8501b994372f62f47458c792a87d08/v1/README.md) by Jean-Hugues de Raigniac
  > In the previous article, we described how to use 10 effect libraries to run a sample application with common effects: writer, state, reader and error. By comparing them, we were able to highlight a pattern in their usage. Could we standardize it further and erase library details? Yes!

- [Competitive programming in Haskell: parsing with an NFA](https://byorgey.wordpress.com/2023/05/24/competitive-programming-in-haskell-parsing-with-an-nfa/) by Brent Yorgey
  > In this problem, we have to decide which words can be made by concatenating atomic element symbols.

- [Parsing Diff Output in Haskell](https://www.prborges.com/2023/parsing-diff-in-haskell/) by Pedro Borges
  > I describe here a parser for diff output written in Haskell using Attoparsec, a monadic parser combinator library designed to be fast and efficient.

- [Refactoring error messages in Haskell programs. Who cares about K9 composers?](https://rpeszek.github.io/posts/2023-05-15-FP-refactor-errors-p0.html) by Robert Peszek
  > In this post I will talk about improving error messages. We will also discuss dogs which compose music.

- [Sixteen Days of Haskell](https://codelyrical.com/sixteen-days-of-haskell/) by Matt Gregg
  > I had pre-ordered Effective Haskell on a beta program, but had forgotten about it until now. But, I'd said it, it was out there, I was doing Advent of Code in Haskell for 2022.

- [A Vulkan-based 3D Chess Game + Libraries](https://np.reddit.com/r/haskell/comments/13m7ow3/a_vulkanbased_3d_chess_game_libraries/) by Michael Szvetits
  > Seeing people publishing their Tic Tac Toe games here, I decided to show my fully functional, documented, local 3D chess game written in Haskell.

## Jobs

- [Haskell Developer at Gizra](https://www.gizra.com/jobs/candidates-info-ihp/)

Trying to hire a Haskell developer?
You should [advertise with us](https://haskellweekly.news/advertising.html)!

## In brief

- [IOG GHC Update #10](https://engineering.iog.io/2023-05-18-ghc-update/) by IOG
  > Biweekly update from the GHC DevX team at IOG.

- [MuniHac 2023](https://munihac.de/2023.html)
  > MuniHac is an annual three-day hackathon and conference in Munich that brings together Haskell developers and enthusiasts from across the globe.

- [The State of PureScript Survey 2023 - The Results Are In!](https://discourse.purescript.org/t/the-state-of-purescript-survey-2023-the-results-are-in/3523) by Mike Solomon
  > The results are in! PureScript's second annual survey revealed some key trends about the language and the broader context of functional programming.

## Show & tell

- [monoidmap](https://github.com/jonathanknowles/monoidmap/tree/56cde2c6a0cfd6ced19deb0d16103633d1aab24e) by Jonathan Knowles
  > This library provides a `MonoidMap` type that: models a total function with finite support from keys to monoidal values, with automatic minimal encoding.

- [Ogma version 1.0.9](https://discourse.haskell.org/t/ann-nasas-ogma-1-0-9/6301?u=taylorfausak) by Ivan Perez
  > Ogma is a NASA tool that facilitates integrating runtime monitors or runtime verification applications into other systems.

## Call for participation

- [postgrest: die on "fe_sendauth: no password supplied"](https://github.com/PostgREST/postgrest/issues/2801)
- [unison: expose current codebase schema version](https://github.com/unisonweb/unison/issues/4022)

repository: haskellweekly/haskellweekly

branch_name: refs/heads/haskellweekly

extension: markdown

gha_language: Haskell

repo_name: haskellweekly/haskellweekly


text:
# July 1, 2019 - Maximum value

Write a function that takes an array of numbers as input and outputs the
maximum value in the array.

You may assume that the input array will always have a length greater than 0.

Example:
```
Input : [1, 4, 2, 7, 10, 8]
Output: 10
```

repository: ErikHumphrey/daily-questions

repo_name: ErikHumphrey/daily-questions


text:
# CS-survival-guidance
CS专业生存指南

这是帮助正在读计算机本科的同学们提供的课程学习推荐，里面包含了本科课程所对应的国外优秀大学的课程的公开课和必要课程资源如作业，PPT和课程链接。
希望各位能享受学习带来的乐趣，同时通过完成课程对应的实验课提高的自己的动手能力。
Enjoy yourself!!

## 基础数学课程
Linear Algebra
18.06 from MIT By Gilbert Strang
Comment:
Highly recommended for the beiginners of learning Linear Algebra. It's a very classic linear algebra course which has been viewed over 10 million times on youtube.

[//]: <> (TProf. Strang will get you a well understanding of the key concepts in linear algebra including 4 fundamental subspaces of the matrix and
)

Here's the course link on OCW(open courseware) of MIT: [MIT 1806 Linear Algebra](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/)

The assignment of the course in recent years which includes programming assignments to help you get
an intuitive understanding of the abstract concepts:
[1806 Linear Algebra github](https://github.com/mitmath/1806)

Statistics
[Harvard statistic 110](https://projects.iq.harvard.edu/stat110/about)

Calculus
[MIT 1801 single variable calculus](https://ocw.mit.edu/courses/mathematics/18-01-single-variable-calculus-fall-2006/)

Discrete mathematic (mathematics for computer science)
[MIT 6.042j](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-042j-mathematics-for-computer-science-fall-2010/index.htm)

Stanford CS103, I only got the lecture book.

## 基础程序设计
Data strcuture and algorithm

[Introduction to Algorithm 6.006 from MIT](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-006-introduction-to-algorithms-fall-2011/)

## 计算机核心课程
Operating system
UC berkely
MIT
Stanford

Architecture of computer sytem
CMU 15-213

Computer network

Compiler
Stanford

## 热门课程
Machien learning
Stanford


text:
# Opinion Lexicon: Negative

This file contains a list of NEGATIVE opinion words (or sentiment words).

This file and the papers can all be downloaded from
http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html

If you use this list, please cite one of the following two papers:

Minqing Hu and Bing Liu. "Mining and Summarizing Customer Reviews."
Proceedings of the ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining (KDD-2004), Aug 22-25, 2004, Seattle,
Washington, USA,
Bing Liu, Minqing Hu and Junsheng Cheng. "Opinion Observer: Analyzing
and Comparing Opinions on the Web." Proceedings of the 14th
International World Wide Web conference (WWW-2005), May 10-14,
2005, Chiba, Japan.

## Notes:

    1. The appearance of an opinion word in a sentence does not necessarily
       mean that the sentence expresses a positive or negative opinion.
       See the paper below:

       Bing Liu. "Sentiment Analysis and Subjectivity." An chapter in
          Handbook of Natural Language Processing, Second Edition,
          (editors: N. Indurkhya and F. J. Damerau), 2010.

    2. You will notice many misspelled words in the list. They are not
       mistakes. They are included as these misspelled words appear
       frequently in social media content.

---

# Opinion Lexicon: Positive

This file contains a list of POSITIVE opinion words (or sentiment words).

This file and the papers can all be downloaded from
http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html

If you use this list, please cite one of the following two papers:

Minqing Hu and Bing Liu. "Mining and Summarizing Customer Reviews."
Proceedings of the ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining (KDD-2004), Aug 22-25, 2004, Seattle,
Washington, USA,
Bing Liu, Minqing Hu and Junsheng Cheng. "Opinion Observer: Analyzing
and Comparing Opinions on the Web." Proceedings of the 14th
International World Wide Web conference (WWW-2005), May 10-14,
2005, Chiba, Japan.

## Notes:

    1. The appearance of an opinion word in a sentence does not necessarily
       mean that the sentence expresses a positive or negative opinion.
       See the paper below:

       Bing Liu. "Sentiment Analysis and Subjectivity." An chapter in
          Handbook of Natural Language Processing, Second Edition,
          (editors: N. Indurkhya and F. J. Damerau), 2010.

    2. You will notice many misspelled words in the list. They are not
       mistakes. They are included as these misspelled words appear
       frequently in social media content.

repository: nhunzaker/speakeasy

gha_language: JavaScript

repo_name: nhunzaker/speakeasy


text:
# AngularParallaxJS
AngularParallaxJS is an AngularJS module with a goal to provide all the parallax functionality you could possibly dream of.

## Current Parallax Features

- Move element horizontally on horizontal mouse movement.
- Move element vertically on page scroll.

## TODO Parallax Features

- Move element vertically on mouse movement.
- Move element horizontally on page scroll.

## Usage

Add the ```parallax``` attribute to the element you are wanting to use parallax on.

```
<div class="redBox" parallax></div>
```

Add the ```x``` attribute to the element.  Set it equal to the desired x-position of the element.

```
<div class="redBox" parallax x="100"></div>
```

Add the ```y``` attribute to the element.  Set it equal to the desired y-position of the element.

```
<div class="redBox" parallax x="100" y="100"></div>
```

Add the ```paratype``` attribute to the element.  Set it equal to the desired parallax effect (See the [API](https://github.com/michaeldistler/angular-parallaxJs#api) for details).

```
<div class="redBox" parallax x="100" y="100" paratype="mousemove"></div>
```

Add the ```move``` attribute to the element.  Set it equal to the speed that you want the element to move (See the [API](https://github.com/michaeldistler/angular-parallaxJs#api) for details).

```
<div class="redBox" parallax x="100" y="100" paratype="mousemove" move=".05"></div>
```

## API

### Attributes

##### x="[x-position]"

The **x** attribute is the desired x-position of the element, from the left side of the window, that you are adding parallax functionality to.

##### y="[y-position]"

The **y** attribute is the desired y-position of the element, from the top of the window, that you are adding parallax functionality to.

##### paratype="[parallax type]"

The **paratype** attribute is the type of parallax functionality you are looking to use on the element.  Currently, AngularParallaxJS offers parallax functionality
on a horizontal ```mousemove``` and vertically ```onscroll```.

##### move

The **move** attribute is the speed that you want the parallax element to move.  This attribute should be set as a decimal ranging from ```.05``` - ```2```.
Anything higher than ```2``` moves the element too fast but feel free to experiment.

#### License

MIT

repository: michaeldistler/angular-parallaxJs

repo_name: michaeldistler/angular-parallaxJs


text:
# Create the remote git repo

All alba packages must have an associated remote git repo. If it does not yet exist, it should be created **according to the conventions given in [Appendix 4](Appendix_4.md)**.

You can either create it manually using the web interface og [git.cells](https://git.cells.es) or using the script `create_ctpkg_project`.

If you create the remote git repo manually using the web interface, remember to set the visibility level of the project to *public*.

For creating it automatically, use the following script:
`create_ctpkg_project <SRC_NAME>_deb "<description>" "<tagslist>"`

Notes: *\<SRC_NAME\>* must correspond to the source package name, and therefore comply with the naming convention from [Appendix 4](Appendix_4.md). *\<description\>* and *\<tagslist\>* are mandatory in order to filter the projects.

e.g.:

`create_ctpkg_project testDS_deb "Repo for packaging testDS on debian" "DS, ALL, cfalcon"`

repository: ALBA-Synchrotron/apg

repo_name: ALBA-Synchrotron/apg


text:
# Credits

Where-to-fly uses several components and resources.

## Components

The app uses [CesiumJS](https://cesiumjs.org/) to visualize a 3D map.
The component is licensed using the
[Apache License 2.0](https://github.com/CesiumGS/cesium/blob/master/LICENSE.md).

The app uses the [Microsoft App Center](https://appcenter.ms/) SDK packages to
distribute the app and to collect crash information.
The components are licensed using the
[MIT License](https://github.com/Microsoft/AppCenter-SDK-DotNet/blob/master/license.txt)

The app uses the [Newtonsoft.Json](https://www.newtonsoft.com/json) library to
format and parse JSON objects.
The component is licensed using the
[MIT License](https://github.com/JamesNK/Newtonsoft.Json/blob/master/LICENSE.md).

The app uses the [Xamarin.Essentials](https://github.com/xamarin/Essentials) library to
access all sorts of platform specific APIs.
The library is licensed using the
[MIT License](https://github.com/xamarin/Essentials/blob/master/LICENSE).

The app uses the [Xamarin Community Toolkit](https://github.com/xamarin/XamarinCommunityToolkit)
library. The library is licensed using the
[MIT License](https://github.com/xamarin/XamarinCommunityToolkit/blob/main/LICENSE).

The app uses the following [Xamarin Plugins](https://github.com/jamesmontemagno/Xamarin.Plugins):
- Xam.Plugin.Geolocator: Geo-location of user

All plugins are licensed using the
[MIT License](https://github.com/jamesmontemagno/GeolocatorPlugin/blob/master/LICENSE).

The app uses the [Markdig](https://github.com/lunet-io/markdig)
library to format MarkDown as HTML.
The component is licensed using the
[BSD 2-clause "Simplified" License](https://github.com/lunet-io/markdig/blob/master/license.txt).

The app uses the [SharpKML](https://github.com/samcragg/sharpkml) library to
load Google Earth .kml and .kmz files.
The library is licensed using the
[MIT License](https://github.com/samcragg/sharpkml/blob/master/LICENSE).

The app uses the [Refit](https://paulcbetts.github.io/refit/) library to send
REST based web service requests.
The library is licensed using the
[MIT License](https://github.com/paulcbetts/refit/blob/master/COPYING).

The app uses the [Html Agility Pack](https://html-agility-pack.net/) to parse
HTML content.
The library is licensed using the
[MIT License](https://github.com/zzzprojects/html-agility-pack/blob/master/LICENSE).

The app uses the [Rg.Plugins.Popup](https://github.com/rotorgames/Rg.Plugins.Popup)
library to show popup dialogs.
The library is licensed using the
[MIT License](https://github.com/rotorgames/Rg.Plugins.Popup/blob/master/LICENSE.md).

The app uses the [Xamarin.FFImageLoading](https://github.com/luberda-molinet/FFImageLoading)
library to show and cache SVG images.
The library is licensed using the
[MIT License](https://github.com/luberda-molinet/FFImageLoading/blob/master/LICENSE.md).

The app uses the [sqlite-net](https://github.com/praeclarum/sqlite-net)
library to store all data.
The library is licensed using the
[MIT License](https://github.com/praeclarum/sqlite-net/blob/master/LICENSE.txt).

The app uses the [SkiaSharp](https://github.com/mono/SkiaSharp)
library to draw custom controls.
The library is licensed using the
[MIT License](https://github.com/mono/SkiaSharp/blob/master/LICENSE.md).

The app uses the [Chart.js](https://www.chartjs.org/)
library to display height profiles. The library is licensed using the
[MIT License](https://github.com/chartjs/Chart.js/blob/master/LICENSE.md).

The app uses the [chartjs-adapter-moment](https://github.com/chartjs/chartjs-adapter-moment)
library to integrate the Moment.js library into Chart.js.
The library is licensed using the
[MIT License](https://github.com/chartjs/chartjs-adapter-moment/blob/master/LICENSE.md).

The app uses the [Moment.js](https://momentjs.com/)
library to format date and time values in Chart.js.
The library is licensed using the
[MIT License](https://github.com/moment/moment/blob/develop/LICENSE).

The app uses the [chartjs-plugin-crosshair](https://github.com/abelheinsbroek/chartjs-plugin-crosshair/)
library to provide a crosshair for height profiles. The library is licensed using the
[MIT License](https://github.com/AbelHeinsbroek/chartjs-plugin-crosshair/blob/master/LICENSE).

The app uses the [chartjs-plugin-zoom](https://github.com/chartjs/chartjs-plugin-zoom/)
library to provide zooming into height profiles. The library is licensed using the
[MIT License](https://github.com/chartjs/chartjs-plugin-zoom/blob/master/LICENSE.md).

The app uses the [Hammer.js](https://hammerjs.github.io/)
library for touch gesture handling in the chartjs-plugin-zoom plugin. The library is licensed using the
[MIT License](https://github.com/hammerjs/hammer.js/blob/master/LICENSE.md).

The backend uses the [QuickGraph](https://github.com/oldrev/Sandwych.QuickGraph)
library for tour planning.
The library is licensed using the
[Microsoft Public License](https://github.com/oldrev/Sandwych.QuickGraph/blob/master/LICENSE.md).

## Services

The app uses the Thermal Skyways from [https://thermal.kk7.ch/](https://thermal.kk7.ch/),
with kind permission from Michael of https://kk7.ch/.

The app uses the Waymarked Trails Hiking overlay imagery from
[https://waymarkedtrails.org/](https://waymarkedtrails.org/), licensed under the
[CC BY-SA 3.0 DE](https://creativecommons.org/licenses/by-sa/3.0/de/deed.de)
license.

## Resources

### Icons

#### Google Material

The following icons are from the Google Material icon set:

- autorenew
- bus
- close
- close-circle-outline
- compass
- crosshairs-gps
- delete
- delete-forever
- directions
- dots-vertical
- eye
- format-list-bulleted
- information-outline
- layers-outline
- magnify
- magnify-plus-outline
- map
- map-marker
- menu
- navigation
- parking
- playlist-plus
- settings
- settings-outline
- share-variant
- train
- camera
- refresh
- export-variant
- bookmark-plus-outline
- close
- arrow-expand-horizontal
- camera-outline
- cloud-upload-outline (modified)

The Google Material Icons are licensed under the
[Apache License 2.0](https://github.com/google/material-design-icons/blob/master/LICENSE).

#### Material Design Icons

The following icons in the App are from the
[Material Design Icons](https://materialdesignicons.com/) page:

- ruler - distance.png - Austin Andrews @Templarian
- compass-rose - Michael Richins @MrGrigri
- church - church.png - Austin Andrews @Templarian
- castle - castle.png - Austin Andrews @Templarian
- bridge - bridge.png - Austin Andrews @Templarian
- arrow-up - up.png - Austin Andrews @Templarian
- arrow-down - down.png - Austin Andrews @Templarian
- weather-partlycloudy - Austin Andrews @Templarian
- border-none-variant - Austin Andrews @Templarian
- calendar-clock - Austin Andrews @Templarian
- map-marker-distance - Michael Richins @MrGrigri
- map-marker-plus - Cody @XT3000
- eye-off-outline - Austin Andrews @Templarian
- layers-plus - Austin Andrews @Templarian
- gesture-spread - Michael Richins @MrGrigri
- gesture-tap - Michael Richins @MrGrigri
- timeline-clock-outline - Austin Andrews @Templarian
- magnify-scan - Sascha Wohlgemuth @whlgmth

All custom icons are licensed under the
[Apache 2.0 license](https://github.com/Templarian/MaterialDesign/blob/master/LICENSE).

#### Mapbox Maki Icons

Some icons used in the App are used from the Mapbox Maki icon set:
[https://www.mapbox.com/maki-icons/](https://www.mapbox.com/maki-icons/)

The Maki icon set is licensed using the CC0 license:
[https://creativecommons.org/publicdomain/zero/1.0/](https://creativecommons.org/publicdomain/zero/1.0/)

#### Other Icons

The following icons are from the onlinewebfonts web page:

- Paragliding icon: Icon made from [https://www.onlinewebfonts.com/icon/](https://www.onlinewebfonts.com/icon/ "Icon Fonts").
  The icon is licensed by CC BY 3.0.

The following icons are from Wikimedia Commons:
- Cave icon: Icon made by Carport; the icon is licensed under the CC BY-SA 3.0
  license.
  [https://commons.wikimedia.org/wiki/File:Gfi-set01-cave.svg](https://commons.wikimedia.org/wiki/File:Gfi-set01-cave.svg)

repository: vividos/WhereToFly

gha_language: C#

repo_name: vividos/WhereToFly


text:
When using dark mode, you'll read slower and spot fewer errors.

<!--more-->

This video was inspired by someone calling me a freak for using a light color theme while coding. In the end, I think it’s all about preference. I enjoy using light mode during the day and dark mode at night. But then again, bugs are attracted to light, so maybe I should only use dark mode when coding...

Anyway, don't think this personal if you prefer dark mode all the time.

## Sources

{% bibtex %}
@article{Piepenbrock2013PositiveDP,
  title={Positive display polarity is advantageous for both younger and older adults},
  author={Cosima Piepenbrock and S. Mayr and Iris Mund and A. Buchner},
  journal={Ergonomics},
  year={2013},
  volume={56},
  pages={1116 - 1124}
}

@article{doi:10.1177/0018720813515509,
	author = {Cosima Piepenbrock and Susanne Mayr and Axel Buchner},
	title ={Positive Display Polarity Is Particularly Advantageous for Small Character Sizes: Implications for Display Design},
	journal = {Human Factors},
	volume = {56},
	number = {5},
	pages = {942-951},
	year = {2014},
	doi = {10.1177/0018720813515509},
  note ={PMID: 25141597},
	URL = { https://doi.org/10.1177/0018720813515509 },
	eprint = { https://doi.org/10.1177/0018720813515509 },
}

@article{Piepenbrock2014SmallerPS,
  title={Smaller pupil size and better proofreading performance with positive than with negative polarity displays},
  author={Cosima Piepenbrock and Susanne Mayr and Axel Buchner},
  journal={Ergonomics},
  year={2014},
  volume={57},
  pages={1670 - 1677}
}

@inproceedings{10.1145/3458864.3467682,
author = {Dash, Pranab and Hu, Y. Charlie},
title = {How Much Battery Does Dark Mode Save? An Accurate OLED Display Power Profiler for Modern Smartphones},
year = {2021},
isbn = {9781450384438},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3458864.3467682},
doi = {10.1145/3458864.3467682},
booktitle = {Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services},
pages = {323–335},
numpages = {13},
keywords = {display power profiler, OLED display, dark mode, power modeling},
location = {Virtual Event, Wisconsin},
}

@online{src,
    title={The Dark Side of Dark Mode},
    url={https://tidbits.com/2019/05/31/the-dark-side-of-dark-mode/},
    author={Adam Engst},
    organization={TidBITS},
    year={2019},
    month={5},
    day={31}
}

@online{src,
    title={Is dark mode better or worse for your eyes?},
    url={https://www.allaboutvision.com/digital-eye-strain/is-dark-mode-better-for-eyes/},
    organization={All About Vision},
    author={Autumn Sprabary},
    year={2020},
    month={3},
    day={30}
}

@online{src,
    title={Dark Mode vs. Light Mode: Which Is Better?},
    url={https://www.nngroup.com/articles/dark-mode/},
    organization={Nielsen Norman Group},
    author={Raluca Budiu},
    year={2020},
    month={2},
    day={2}
}

@online{src,
    title={Dark mode can save battery, but only if your device has an OLED screen — here's what you should know},
    url={https://www.businessinsider.com/guides/tech/does-dark-mode-save-battery},
    author={Abigail Abesamis Demarest},
    year={2021},
    month={2},
    day={16}
}
{% endbibtex %}

gha_language: HTML


text:
## THIS PROJECT IS DEPRECATED AND DISCONTINUED

Its functionality is integrated now as part of [vertx-web-accesslog](https://github.com/romanpierson/vertx-web-accesslog)

# vertx-web-accesslog-elasticsearch-appender

An appender implementation to be used with [vertx-web-accesslog](https://github.com/romanpierson/vertx-web-accesslog).

Writes its data to a [vertx-elasticsearch-indexer](https://github.com/romanpierson/vertx-elasticsearch-indexer) instance. For details check out the documentation there.

## Technical Usage

The artefact is published on bintray / jcenter (https://bintray.com/romanpierson/maven/com.mdac.vertx-web-accesslog-elasticsearch-appender)

Just add it as a dependency to your project (gradle example)

```xml
dependencies {
	compile 'com.mdac:vertx-web-accesslog-elasticsearch-appender:1.4.0'
}
```

## Usage

### Configure route

```java
Router router = Router.router(vertx);

JsonObject config = .... load or create your configuration json

router.route().handler(AccessLoggerHandler.create(config));

```

As configuration is now done by plain JsonObject its very simple to use and inject configuration eg by yaml, see as an example `ServerSetupStarter`

```yaml
configurations:
  - identifier: accesslog-plain
    logPattern: "%{msec}t %D cs-uri"
    appenders:
      - appenderClassName : com.mdac.vertx.web.accesslogger.appender.elasticsearch.impl.ElasticSearchAppender
        config:
          instanceIdentifier: accesslog
          fieldNames:
            - timestamp
            - duration
            - uri
```

### Conventions

By default all log elements configured will be sent to the indexer to be interpreted as message to be indexed. However the timestamp is passed to the indexer as meta data as the indexer potentially requires that raw value eg to determinate the target index name.
By default you should put as first element plain timestamp %{msec} and the appender will remove that element from the message sent to the indexer.

The instance identifier tells the indexer verticle to what ES instance the data should be indexed. All the detailed configuration of this is done directly on the indexer verticle itself (so the appender does not have to know about this).

### Field Names Definition

As for each field to be indexed a specific name needs to be used this has to be explicitly set by configuration property `fieldNames`. Be aware that you need to put a name for all fields of your logpattern even for eg the timestamp one that is actually skipped at the end. The real fieldname used in the indexer for the ES timestamp field is configured in the indexer verticle itself.

## Changelog

### 1.3.0 (2019-02-14)

* Initial version

### 1.4.0 (2020-12-24)

* Upgrade to Vertx 4

gha_language: Java