text:
# Exasol plugin for AWS Managed Airflow (MWAA)
(Airflow 1.10.12)

## Requirements
1. Create Exasol connection in Airflow UI with a connection type of MySQL (as we don't have Exasol by default)
2. Clone repository
3. Create zip of plugin code:

  * Change directories to your `$AIRFLOW_HOME/plugins` folder:
    `myproject$ cd $AIRFLOW_HOME/plugins`
  * Run the following command to ensure that the contents have executable permissions (macOS and Linux only).
    `$AIRFLOW_HOME/plugins$ chmod -R 755 .`
  * Zip the contents within your `$AIRFLOW_HOME/plugins` folder.
    `$AIRFLOW_HOME/plugins$ zip -r plugins.zip .`

4. Upload zip to s3 bucket and restart MWAA with picking up the version of just uploaded plugin.

## Usage example
```python
from airflow import DAG
from airflow.utils.dates import days_ago
from airflow.sensors.exasol_plugin import ExasolSensor
from airflow.operators.exasol_plugin import ExasolOperator

DEFAULT_ARGS = {
    'owner': 'Airflow'
}

with DAG(
        dag_id='exasol_plugin_example',
        default_args=DEFAULT_ARGS,
        start_date=days_ago(1),
        schedule_interval='@once'
) as dag:

    # Set path to your SQL files which will be executed by operator
    project_path = '/usr/local/airflow/dags/projects/exasol_plugin_template'

    ###########################
    # ExasolOperator Examples #
    ###########################

    # Get SQL from file in your project
    with open(f'{project_path}/query_example.sql', 'r') as f:
        sql = f.read()

    # Run operator to execute SQL on Exasol with exasol_default connection
    run_sql1 = ExasolOperator(
        task_id='sql_example1',
        exasol_conn_id='exasol_default',
        autocommit=True,
        sql=sql,
        dag=dag
    )

    # Run operator to execute SQL statement on Exasol with exasol_default connection
    run_sql2 = ExasolOperator(
        task_id='sql_example2',
        exasol_conn_id='exasol_default',
        autocommit=True,
        sql='select 1',
        dag=dag
    )

    #########################
    # ExasolSensor Examples #
    #########################

    # Set SQL sensor query
    # If query returns 0, '0' or None then we assume criteria in not met and wait <poke_interval> seconds till <timeout>
    # otherwise assume criteria is met stop sensor and continue DAG execution
    # Default values:
    # poke_interval=60
    # timeout=60 * 60 * 24 * 7

    sensor_query = '''
    SELECT count(*) from test_schema.test_table
    '''

    # Define sensor task with default poke interval and timeout
    run_sensor1 = ExasolSensor(
        task_id='sensor1',
        conn_id='exasol_default',
        sql=sensor_query,
        dag=dag
    )

    # Define sensor task with custom poke interval and timeout and query from file
    with open(f'{project_path}/sensor_query_example.sql', 'r') as f:
        sql = f.read()

    run_sensor2 = ExasolSensor(
        task_id='sensor2',
        conn_id='exasol_default',
        sql=sql,
        poke_interval=60,
        timeout=60*5,
        dag=dag
    )

    (
        run_sensor1 >>
        run_sql1 >>
        run_sql2 >>
        run_sensor2
    )
```

repository: KaterynaBabich/apache-airflow

repo_name: KaterynaBabich/apache-airflow


text:
GoogleScript
============

This repository is dedicated to the Open Source community for Google Scripts

This Google Script web app is designed as a web portal for new hires

The program takes general inputs such as name, email, etc along with multiple file uploads

It then validates that all fields are filled before activating the submit button and enabling the doPost method.

doPost appends all information to a spreadsheet, checks a drive folder to make sure the user's folder does not exisit, if it does it will create a new folder with their name on it and create all the files submitted in it.

repository: Delphi-US/GS-Form-Upload

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: Delphi-US/GS-Form-Upload


text:
# Go Fish

Go Fish is a stream processor built in Golang. It provides the capability to write rules in Go as Go Plugins, which are dynamically linked at runtime.

Go Fish seeks to implement similar functionality to Apache Samza, without tight coupling to Kafka or YARN. Currently it implements reading streams, writing output events and state management. Future versions may include Checkpointing, Windowing, Metrics and a cluster implementation to co-ordinate message routing between nodes.

**Development Status**: No longer in active development. [Benthos](https://github.com/Jeffail/benthos) is an excellent alternative.

## Project Goals

Go Fish is designed to provide a lower barrier of entry for Stream Processing than the Apache family of frameworks. The existing frameworks are predominantly written in Java and require at the very lease ZooKeeper to run, but usually Kafka and potentially Hadoop.

While projects like [Wallaroo](https://github.com/wallaroolabs/wallaroo) exist that provide native Go integration and are easier to get started it only supports Kafka and TCP as an input source. Go Fish aims to leverage existing cloud tooling like Kinesis and DynamoDB to provide easier setup and maintenance. Support for other cloud providers may be added in the future.

## Bare minimum

The bare minimum to have a working example requires you to create a Rule and an Event Type.
These must be compiled as plugins and the path to the compiled plugins supplied as an argument to go-fish.

To avoid duplication, the event struct can be extracted to a seperate file, as the struct is required by both the Rule and Event Type.
A repository of Rules, Event structs and Event Types may look like so:

```
.
├── build.sh
├── eventStructs
│   └── struct.go
├── eventTypes
│   ├── event_type.go
└── rules
    ├── rule.go

```

Where `build.sh` does the following:

```
for DIR in "eventTypes rules"; do
    for FILE in $(ls $DIR); do
        go build -buildmode=plugin -o ${DIR}/${FILE%.go}.so ${DIR}/${FILE}
    done
done
```

### Usage

A trivial pipeline can be defined in a json file and run like so:

```
go-fish -pipelineConfig pipeline.json
```

More realisticly you will want to run multiple pipelines. To do this create an API Configuration file and submit your pipelines via the API.

```json
{
  "listenAddress": "<IP_ADDRESS>:8000",
  "backendConfig": {
    "type": "boltdb|dynamodb",
    "boldDBConfig": {
      "bucketName": "go-fish",
      "databaseName": "go"
    },
    "dynamoDBConfig": {
      "region": "us-east-2",
      "tableName": "go-fish"
    }
  }
}
```

The DynamoDB Schema must be a Primary Key of "UUID" which is a binary blob and no Hash Key.

```
go-fish -apiConfig api.json
```

### Examples

See `examples/` for some implementations of go-fish. You can with the following command:
```
make certstream-example
```

### Defining a Pipeline

A Pipeline contains five top level keys, `eventFolder`, `rules`, `states`, `sources` and `sinks`. In a rule definition the source, state and sink should refer to a key in the respective top level.

```json
{
  "eventFolder": "testdata/eventTypes",
  "rules": {
    "searchRule": {
      "source": "fileInput",
      "state": "searchConversion",
      "plugin": "rules/searchRule.so",
      "sink": "conversionRule"
    },
    "conversionRule": {
      "source": "searchRule",
      "plugin": "rules/conversionRule.so",
      "sink": "fileOutput"
    }
  },
  "states": {
    "searchConversion": {
      "type": "KV"
    }
  },
  "sources": {
    "fileInput": {
      "type": "File",
      "file_config": {
        "path": "testdata/pipelines/input"
      }
    }
  },
  "sinks": {
    "fileOutput": {
      "type": "File",
      "file_config": {
        "path": "testdata/output"
      }
    }
  }
}
```

This Pipeline would create a Directed Acyclical Graph like so, although more complex topologies are possible:

```
fileInput ----> searchRule ----> conversionRule ----> fileOutput
```

#### Creating an Event Struct

The Event Struct simply defines the data structure for the event and implements the `event` interface. This is a trivial example where the event contains just a single string:

```
package eventStructs

const EventName = "exampleEventType"

type ExampleType struct {
	Str string
}

func (e ExampleType) TypeName() string {
	return EventName
}
```

In this example we export the `EventName` variable, which can be re-used in the Event Type to implement the `Name()` method.

#### Writing an Event Type

The Event Type is responsible for decoding the Event. It implements the `eventType` interface.

```
type eventType interface {
	Name() string
	Decode([]byte) (event.Event, error)
}
```

Where event is a minimal package that can be imported from `github.com/patrobinson/go-fish/event`.

#### Writing a Rule

The Rule must also include the Event struct, so that it can assert the Event it receives is the Event Struct it's expecting.
It implements the Rule interface.

```
type Rule interface {
	Init(state ...interface{}) error
	Process(interface{}) interface{}
	String() string
	WindowInterval() int
	Window() ([]output.OutputEvent, error)
	Close() error
}
```

Simple rules that do not require state or windowing can utilise the BasicRule helper. This implements everything except for the `Process` and `String` methods.

```
import "github.com/patrobinson/go-fish/ruleHelpers"

type simpleRule struct {
	rulehelpers.BasicRule
}
```

license: MIT

repository: patrobinson/go-fish

gha_language: Go

gha_license_id: MIT

repo_name: patrobinson/go-fish


text:
---
layout: default
---

# 1993-10-13 - Re: Spread-spectrum net (vulnerability of)

## Header Data

From<EMAIL_ADDRESS>(Doug Merritt)

To<EMAIL_ADDRESS>Message Hash: 19b4d0d63c15c4bb5ab75dbc2e0dc7c511f694921026dceb5096025b6baceb6d

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1993-10-13 03:36:35 UTC

Raw Date: Tue, 12 Oct 93 20:36:35 PDT

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Doug Merritt)
Date: Tue, 12 Oct 93 20:36:35 PDT
To<EMAIL_ADDRESS>Subject: Re: Spread-spectrum net (vulnerability of)
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Matthew J Ghio<EMAIL_ADDRESS>said:
>Depends how you place them. If you put them _on top_ of things, you'd need
>a helicopter to shoot 'em.

I used to work with the person who set up 80% of the West Coast ham
radio digital packet system repeaters. He had a bit of money to burn,
and he set up a *bunch* of these repeaters on various mountain tops up
and down the West Coast. This was circa '85 by the way.

Each repeater required:
	1) a source of power -- this is easy to forget, but it is
	   nontrivial. Batteries just don't cut it. Rather than explain
	   about batteries, I invite people to ponder power requirements
	   for given broadcast distances and guess battery lifetime.

	   Anyway, he had a PG&E (electric power) hookup and a monthly
	   bill for each of these. Good thing he had all that money...

	   He had trouble finding people who were willing to allow his
	   stations; people who live on mountain-tops tend to be
	   rugged individualists. :-) But he managed.

	2) Line of sight reception and transmission. The higher the
	   frequency used, the more that line of sight is required by
	   the physics of the situation. Even if line of sight isn't
	   strictly required, signal strength drops when you get out
	   of line of sight -- if you depend on radio wave diffraction
	   to get the signal to you and from you, you can count on heavy
	   signal loss in the process.

	3) Protection from the elements. He used rack-mounted 286 cards
	   in an industrial enclosure, but there was *no way* he could
	   leave the boxes exposed to the elements. A roof was required.

	   This is the kind of issue that is easy to neglect, but is
	   absolutely essential for real life conditions: weatherproofing
	   of the assembly. If you don't use as roof and walls, you'd
	   better have a genius mechanical engineer design the enclosure.
	   This is a lot harder than it sounds.

>Someone walking around a city shooting a rifle is likely to attract a
>lot more attention than a secret network would.

Your opponents wouldn't do that...the "enemy" here is simply the FCC,
count on it. They have field agents who triangulate illicit transmitters,
and once they find one, they simply get all the warrants and court orders
needed to deal with it.

>Secondly, the
>transmitter doesn't necessarily have to be exposed, it could be kept
>hidden and only the antenna would need to be exposed.    Plus the antenna would be easy
>to disguise or hide in many places.

Sure, this is possible. But it doesn't help *that* much. Triangulation
of signal spots any antenna quickly, and they cut that off immediately.
You can get increasingly elaborate about hiding the signal source, and
restoring antennae quickly as they're pinched, but it's sort of a losing
battle unless you assume real time response by the underground lead by
a brilliant EE type.

I am not saying that an underground wireless net is impossible. I *am*
saying that the difficulties are much higher than they may seem at first
blush.

I think doing something like this is possible, and it would have definite
benefits. But anyone moving on this would do well to get in touch with
the existing ham radio crowd who have dealt with the pragmatic issues
involved for the better part of a decade (or more).

>  Hence we have achieved
>our objective - you know how to contact site #3 in netspace - it has a
>cybernetic location relative to other sites, but that tells you nothing
>about it's actual physical location.

Right...in a sense, this is the easy part. Definitely one can hide
physical locations if everything is set up carefully.
	Doug

{% endraw %}
```

## Thread

+ Return to [October 1993](/archive/1993/10)

+ Return to<EMAIL_ADDRESS>(Doug Merritt)](/authors/doug_at_netcom_com_doug_merritt_)"
+ Return to "[Matthew J Ghio <mg5n+@andrew.cmu.edu>](/authors/matthew_j_ghio_mg5n_at_andrew_cmu_edu_)"

+ _Unknown thread root_
  + 1993-10-13 (Tue, 12 Oct 93 20:36:35 PDT) - Re: Spread-spectrum net (vulnerability of) -<EMAIL_ADDRESS>(Doug Merritt)_
    + 1993-10-13 (Wed, 13 Oct 93 15:19:59 PDT) - [Re: Spread-spectrum net (vulnerability of)](/archive/1993/10/7f34eb938f934f02dabc936b018a366eb88825563d0d37ac769b49e2607e0e28) - _Matthew J Ghio<EMAIL_ADDRESS>


text:
About
=====

This module provides the **mysqltotsv.Splitter** class and a CLI tool to allow conversion
of large MySQL dumps to the [TSV format](https://en.wikipedia.org/wiki/Tab-separated_values).

The module assumes the MySQL dump to be encoded in [UTF-8 format](https://en.wikipedia.org/wiki/UTF-8).

This is primarily intended for large MySQL dumps where opening them with a text editor is impractical
and handling them is easier through a tool.

Install
=======

To install from pypi:

    pip3 install --user mysqltotsv

Usage
=====

The following command takes a MySQL dump and creates a separate TSV file for each
table found inside (the output directory needs to be present):

    python3.7 mysql-to-tsv.py --file dump.sql --outdir out1

There is also a switch to only process certain tables and skip the rest. Example:

    python3.7 mysql-to-tsv.py --file dump.sql --outdir out1 --table-filter=updates,updates_edited

More details about the CLI switches:

```
usage: mysql-to-tsv.py [-h] --file FILE --outdir OUTDIR
                       [--table-filter TABLE_FILTER] [--only-schema]
                       [--strip-quotes]

Tool for conversion of large MySQL dumps to TSV format

optional arguments:
  -h, --help            show this help message and exit
  --file FILE           mysql dump file
  --outdir OUTDIR       output directory
  --table-filter TABLE_FILTER
                        filtered tables
  --only-schema         write the schema to the output directory
  --strip-quotes        strip quotes from values
```

Support
==================

For questions or requests for paid support, please send an e-mail to<EMAIL_ADDRESS>

repository: Laga/mysqltotsv

repo_name: Laga/mysqltotsv


text:
# Installation Guide for ReactJS-Apollo Simple Starter Kit.
For starting repo, follow these steps.

# Requirements
Install Node v6.3.0+

## Install Java JDK
[Download Java JDK Here.](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html)

# Install Webdriver Manager for E2E Tests

```js
npm install
./node_modules/protractor/bin/webdriver-manager update // Download the Selenium driver
```

# Set GraphQL Hub Endpoint

You can use https://graph.cool for setting GraphQL as a service for your project.

- Rename **./client/config/config.dev.json.example** removing the *example* extension.
- Set **GRAPHQL_ENDPOINT** const to your GraphQL Endpoint.

# Basic commands

```js
npm start // Start the application in dev mode. Open in http://localhost:3000
npm run dev:hot // Start the application in dev mode hit hot reload implementation. Open in http://localhost:8080
npm run test // Run respective Unit tests from app
npm run deploy // Run the unit tests and e2e test, after it build the bundle. Show in http://localhost:8000
```

# Enable Automated Documentation

For enabling automated documentation, please install esdoc via npm

```
npm install -g esdoc
```
It, enables the documentation generation running the stage or deploy commands.

For showing how to create a documentation for components or classes, please check the docs [Here.](https://esdoc.org/)

# Running the Project

Finally, run the project again

```js
npm run dev:hot
```

repository: Danjavia/ReatJS-Apollo-Starter-Kit

repo_name: Danjavia/ReatJS-Apollo-Starter-Kit


text:
# Adult family health plus levels

## Dataset

| Name | Value |
| :--- | :---- |
| Catalog | [Link](https://catalog.data.gov/dataset/adult-family-health-plus-levels-82909) |
| Metadata | [Link](https://data.cityofnewyork.us/api/views/2enn-s52j) |
| Data: JSON | [100 Rows](https://data.cityofnewyork.us/api/views/2enn-s52j/rows.json?max_rows=100) |
| Data: CSV | [100 Rows](https://data.cityofnewyork.us/api/views/2enn-s52j/rows.csv?max_rows=100) |
| Host | data.cityofnewyork.us |
| Id | 2enn-s52j |
| Name | Adult family health plus levels |
| Attribution | Human Resources Administration (HRA) |
| Category | City Government |
| Tags | jobs and economic mobility, human resources administration, hra, health, plus, adult |
| Created | 2013-03-19T18:07:23Z |
| Publication Date | 2013-06-21T20:09:58Z |

## Description

This table represents details of Family Health Plus insurance program. Family Health Plus is a public health insurance program for adults between the ages of 19 and 64 who do not have health insurance - either on their own or through their employers - but have incomes too high to qualify for Medicaid.  The Family Health Plus Premium Assistance Program is available to help eligible lower-income employees and their families participate in their employer?s health insurance plan.

## Columns

```ls
| Included | Schema Type    | Field Name     | Name               | Data Type | Render Type |
| ======== | ============== | ============== | ================== | ========= | =========== |
| No       | time           | :updated_at    | updated_at         | meta_data | meta_data   |
| Yes      | series tag     | family_size    | Family Size        | text      | text        |
| Yes      | numeric metric | yearly_income  | Yearly Income ($)  | number    | text        |
| Yes      | numeric metric | monthly_income | Monthly Income ($) | number    | text        |
| Yes      | numeric metric | weekly_income  | Weekly Income ($)  | number    | text        |
```

## Time Field

```ls
Value = updated_at
Format & Zone = seconds
```

## Data Commands

```ls
series e:2enn-s52j d:2013-03-19T11:07:30.000Z t:family_size="Single Adult and 19 - 20 Years Old
living alone" m:yearly_income=11172 m:weekly_income=215 m:monthly_income=931

series e:2enn-s52j d:2013-03-19T11:07:30.000Z t:family_size="Couples with No Childrens" m:yearly_income=15132 m:weekly_income=291 m:monthly_income=1261

series e:2enn-s52j d:2013-03-19T11:07:30.000Z t:family_size=1 m:yearly_income=16764 m:weekly_income=322 m:monthly_income=1397
```

## Meta Commands

```ls
metric m:yearly_income p:integer l:"Yearly Income ($)" t:dataTypeName=number

metric m:monthly_income p:integer l:"Monthly Income ($)" t:dataTypeName=number

metric m:weekly_income p:integer l:"Weekly Income ($)" t:dataTypeName=number

entity e:2enn-s52j l:"Adult family health plus levels" t:attribution="Human Resources Administration (HRA)" t:url=https://data.cityofnewyork.us/api/views/2enn-s52j

property e:2enn-s52j t:meta.view v:id=2enn-s52j v:category="City Government" v:attributionLink=http://www.nyc.gov/html/hia/html/public_insurance/adults.shtml v:averageRating=0 v:name="Adult family health plus levels" v:attribution="Human Resources Administration (HRA)"

property e:2enn-s52j t:meta.view.owner v:id=5fuc-pqz2 v:screenName="NYC OpenData" v:lastNotificationSeenAt=1491336998 v:displayName="NYC OpenData"

property e:2enn-s52j t:meta.view.tableauthor v:id=iacr-duv5 v:screenName=Tejas.Patel v:displayName=Tejas.Patel
```

## Top Records

```ls
| :updated_at | family_size                                     | yearly_income | monthly_income | weekly_income |
| =========== | =============================================== | ============= | ============== | ============= |
|<PHONE_NUMBER>  | Single Adult and 19 - 20 Years Old living alone | 11172         | 931            | 215           |
|<PHONE_NUMBER>  | Couples with No Childrens                       | 15132         | 1261           | 291           |
|<PHONE_NUMBER>  | 1                                               | 16764         | 1397           | 322           |
|<PHONE_NUMBER>  | 2                                               | 22704         | 1892           | 437           |
|<PHONE_NUMBER>  | 3                                               | 28644         | 2387           | 551           |
|<PHONE_NUMBER>  | 4                                               | 34584         | 2882           | 665           |
|<PHONE_NUMBER>  | 5                                               | 40524         | 3377           | 779           |
|<PHONE_NUMBER>  | For each additional person add                  | 5940          | 495            | 114           |
```

repository: SpellarBot/open-data-catalog

repo_name: SpellarBot/open-data-catalog


text:
# $ click_

[![Builf status](https://ci.appveyor.com/api/projects/status/pjxh5g91jpbhn7t84?
svg=true)](https://ci.appveyor.com/project/tygerbytes/resourcefitness)

Click is a Python package for creating beautiful command line interfaces
in a composable way with as little code as necessary.  It's the "Command
Line Interface Creation Kit".  It's highly configurable but comes with
sensible defaults out of the box.

It aims to make the process of writing command line tools quick and fun
while also preventing any frustration caused by the inability to implement
an intended CLI API.

Click in three points:
-   arbitrary nesting of commands
-   automatic help page generation
-   supports lazy loading of subcommands at runtime

## Example
What does it look like? Here is an example of a simple Click program:
```python
import click

@click.command()
@click.option('--count', default=1, help='Number of greetings.')
@click.option('--name', prompt='Your name',
              help='The person to greet.')
def hello(count, name):
    """Simple programe that greets NAME for a total of COUNT times."""
    for x in range(count):
        click.echo('Hello %s!' % name)

if __name__ == '__main__':
    hello()
```
And what it looks like when run:
```shell
$ python hello.py --count=3
Your nale: John
Hello John!
Hello John!
Hello John'
```

## Installation
The installation process is easy and contained.
```
$ pip install Click
```
Or you can build from source.
```
$ git clone https://github.com/Ublimjo/click.git
$ python setup.py install
```

## Documentation

Read the docs at `http://click.pocoo.org/`

## Status

This library is stable and active. Feedback is always welcome!

repository: Ublimjo/click

repo_name: Ublimjo/click


text:
# Android Application Samples by @lalongooo
===========================================

Contents

1. **FragmentFromXML**

	In this sample we create a fragment from an XML layout file.

2. **ShareIntent**

	We'll show different ways to share content.

	2.1 Text: Send and received text plain intents.
	`Important: The ACTION_SEND intent does not work with the Facebook Android App, as mentioned in this post: https://developers.facebook.com/bugs/332619626816423`

2. **SpinnerSample**

	Different examples using the Spinner view

3. **CircleProgressBar**

	Show a circular progress bar within the ObjectAnimator class and the AsyncTask.

	![Image](https://raw.githubusercontent.com/lalongooo/android-samples/master/CircleProgressBar/screenshots/device-2015-08-19-180340.png)

repository: lalongooo/android-samples

repo_name: lalongooo/android-samples


text:
## Just rewrite it!

![Just rewrute it](img/nike-logo.png) <!-- .element: width="512" height="267" -->

## Just rewrite it!

Netscape 6.0 is finally going into its first public beta. There never was a version 5.0. The last major release, version 4.0, was released almost three years ago. Three years is an awfully long time in the Internet world. During this time, Netscape sat by, helplessly, as their market share plummeted.

It’s a bit smarmy of me to criticize them for waiting so long between releases. They didn’t do it on purpose, now, did they?

Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

[Things You Should Never Do, Part I - Joel Spolsky](https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/)

# Just rewrite it!

The first rule of holes: If you are in one, stop digging.

Many software developers take this to mean that if you have a huge legacy mess in your software you should stop working on it and rewrite it from the ground up.

This is probably the worst thing you could do…

[The Big Redesign In The Sky - Uncle Bob Martin](http://www.luckymethod.com/2013/03/the-big-redesign-in-the-sky/)

## Just rewrite it!

- The app is already making (or saving) you $ <!-- .element: class="fragment" data-fragment-index="1" -->
- Let's face it you don't have tests for this <!-- .element: class="fragment" data-fragment-index="2" --> ![poop](img/poop-emoji.png) <!-- .element: class="fragment poop-emoji" width="60" height="60" data-fragment-index="2" -->


text:
---
title: 'How does a GitHub Forked Repository Sync with the Original Repository'
date: 2019-12-12T19:07:41+08:00
tags:
  - 'github'
  - 'git'
description: 'There are also a number of points to keep in mind when contributing code to the community.'
---

> This article is machine translated by DeepL from [amzrk2.cc](https://amzrk2.cc/post/2019/github-sync-forked-repo/) and has no real meaning.

It's been a long time since I've contributed code to a community project, and today I had a whim to contribute something more. It's definitely not possible to change it directly, and deleting the fork is not a good choice either, so how to synchronize your own fork project with the original repository?

<!--more-->

## Setting upstream warehouse

Nothing here...

## Get updates and merge

Nothing here...

repository: dsrkafuu/hugo-template-aofuji

branch_name: refs/heads/main

repo_name: dsrkafuu/hugo-template-aofuji


text:
# Lib JS  V1.5

Lib JS makes it easy to collect [payments in Brazil, Mexico, Chile, Colombia, Peru](https://www.ebanx.com/business/en/), using local currencies and payment methods in an easy and secure way. If you need help please reach us at <developer support channel here>.

Find more information in our [Dev Academy](https://www.ebanx.com/business/en/developers/integrations/lib-js).

## Start using EBANX Lib JS

## Pre-requisites
- [x] EBANX Merchant Credentials;
- [x] HTTPS Website: EBANX Checkout JS Only works on HTTPS;
- [x] Fingerprint providers registered on Merchant Configs (EBMNG)

## How to use EBANX Lib Js
### 1. Include Lib-Js in your webpage
```html
<script type="text/javascript" src="https://js.ebanx.com/assets/songbird/songbird-<dev|prod>.js"></script><!-- use dev or prod according to your needs -->
<script type="text/javascript" src="https://js.ebanx.com/ebanx-libjs-latest.min.js"></script>
```
### 2. Initialize EBANX Lib Js with your Merchant's Configurations
```javascript
EBANX.config.setMode('sandbox'); // Set mode. production/test
EBANX.config.setPublishableKey('put your key here'); // Set your Publishable key. To identify your site to EBANX API you must start by providing your [publishable key](https://developers.ebanx.com/merchant-area/merchant-options).
EBANX.config.setCountry('br'); // Set your checkout country (Alpha-2) (see: https://en.wikipedia.org/wiki/ISO_3166-1).
```
### 3. EBANX Lib Js will retrieve the fingerprint list from providers registered under Merchant's account
### 4. Create a Token callback method to manage the Tokenization response

```javascript
EBANX.config.setMode('test');
EBANX.config.setPublishableKey('YOUR KEY HERE');
EBANX.config.setCountry('br');

var createTokenCallback = function(ebanxResponse) {
  if (ebanxResponse.data.hasOwnProperty('status')) {
      document.getElementById('status').textContent = 'Success, the token is: ' + ebanxResponse.data.token;
  } else {
      var errorMessage = ebanxResponse.error.err.status_message || ebanxResponse.error.err.message;
      document.getElementById('status').textContent = 'Error ' + errorMessage;
  }
}
```
### 5. Tokenize the customer's card with
```javascript
// CUSTOMER CLICKS THE BUTTON
function createToken() {
  EBANX.card.createToken({
    card_number:<PHONE_NUMBER>111111,
    card_name: 'Teste',
    card_due_date: '02/2019',
    card_cvv: '123'
  }, createTokenCallback);
}
createToken();
```

### 6. Perform a 3DS authentication with
```javascript
const orderInformation = {
    "amountDetails": {
        "totalAmount": "10.04",
        "currency": "BRL"
    },
    "billTo": {
        "address1": "Rua Estanislau Szarek",
        "administrativeArea": "PR",
        "country": "BR",
        "email"<EMAIL_ADDRESS>        "homePhone": "41999999999",
        "locality": "Curitiba",
        "postalCode": "81315380",
        "mobilePhone": "41999999999"
    }
};

// success with challenge
const cardNumber = "4000000000001091";

// success without challenge
//const cardNumber = "4000000000001000";

// failt with challenge
//const cardNumber = "4000000000001109";

const paymentInformation = {
    "card": {
        "number": cardNumber,
        "expirationMonth": "12",
        "expirationYear": "34",
        "holderName": "JOAO DA SILVA"
    }
};

const personalIdentification = {
    "id": "97370192024",
    "type": "CPF"
};

const options = {
  orderInformation,
  paymentInformation,
  personalIdentification,
};

const shouldAuthenticate = await EBANX.threedsecure.checkIfShouldAuthenticate(options);

if (shouldAuthenticate) {
  const result = await EBANX.threedsecure.run(options);
}
```

![Usage flow Diagram](./usage-flow-diagram.png)

## Architecture
This library is composed by 8 main modules

### Card
* The main interface for Card Tokenization
### Device Fingerprint
* Retrieves the providers from Pay API
### Errors
* Translates tokenization errors to PT or ES
### HTTP
* Manages the HTTP client
### Tokenize
* Manages the token resource status and tokenization request callbacks
### Utils
* Provides resources for API Addressess and available countries
### Validation
* Validates API Key, countries and card data
### Threedsecure
* 3DS authentication

repository: ebanx/lib-js

gha_language: TypeScript

repo_name: ebanx/lib-js


text:
## Introduction

This is a Terraform 0.12 module to quickly setup an AWS VPC in 10.x.0.0/16. It uses the Terraform Registry AWS VPC module:

https://registry.terraform.io/modules/terraform-aws-modules/vpc/

Individual subnets use a /20 subnet mask and use the following tiers:

- Public: Resources that are available on the Internet. They will be avaible on the Internet if assigned an Elastic IP (EIP)
- Private: Resources that are available only within the VPC; however, require egress Internet access via NAT gateway(s).
- Local: Resources that are available within the VPC and do not have egress Internet access (Databases, caches, etc.)

Subnets are tagged so they can be used as a data resource for other modules:

```
data "aws_vpc" "vpc" {
  tags = {
    Name = var.vpc
  }

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.id

  tags = {
    Tier = "public"
  }
}
```

See https://www.terraform.io/docs/providers/aws/d/subnet_ids.html for more information.

## Example Usage

```
module "vpc" {
  source = "github.com/DennoVonDiesel/tf//module/aws/vpc?ref=v0.1.0"
  vpc    = "dev"
}
```

## Version History

v0.1.0: Initial release

## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| aws | n/a |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| azs | Availability zones of the VPC (max 4) | `list(string)` |
[
  "a",
  "b"
]
 | no |
| cidr | CIDR block of the VPC (10.NNN.0.0/16) | `number` | `10` | no |
| tags | Tags applied to all resources | `map` | `{}` | no |
| vpc | Name of the VPC | `string` | n/a | yes |

## Outputs

No output.

repository: DennoVonDiesel/tf

repo_name: DennoVonDiesel/tf


text:
# using-gatsby-script

Example project for the [Gatsby Script](https://www.gatsbyjs.com/docs/reference/built-in-components/gatsby-script/) component.

## Getting Started

Install dependencies:

```shell
npm install
```

Run the development server:

```shell
npm run develop
```

repository: gatsbyjs/gatsby

gha_language: JavaScript

repo_name: gatsbyjs/gatsby


text:
---
title: Deleting records
slug: deleting-records
description: Removing data from the database.
type: Article
---

# Deleting records

To delete rows in a database is fairly similar to updating records. You specify a table to look for rows in and a condition that determines if a row should be deleted.

This means that to delete a single record you should check on the primary key in the condition while deleting multiple records can be done with a more generic condition.

```
DELETE FROM users WHERE id = 2;
```

This specifies that we should look in the users table and delete the row with id 2.

```
DELETE FROM users WHERE activated = 0;
```

This query deletes all non-activated users in a database.

repository: BjornDCode/learndb

gha_language: PHP

repo_name: BjornDCode/learndb


text:
# Radiance
Radiance is a simple language that compiles to WebAssembly. It's syntax has a familiar and modern style created by blending choice parts of Swift, Rust and TypeScript. By utilising static type checking it allows you to create safer and more concise code than JavaScript, without hindering you and providing easy integration with your existing JavaScript code.

It's design intention is to be intuative and easy to write, while making the most of what the WebAssembly format has to offer.

```swift
export fn factorial (count: i32) -> i32 {
  let result: i32 = 1;
  let i: i32 = 0
  if count == 0 {
    1
  }
  else {
    while i < count {
      i = i + 1
      result = result * i
    }
  }
}
```

A more in depth look at the language can be found [here](Language%20Features.md).

## Status

You may have noticed it's been a little while since a commit! Well the project is not dead, although it has slowed somewhat. I started this while I was between jobs and had much more time. While working on the somewhat large addition of generics I realised I wanted to change how and when type inference occurs in relation to the parse and compile stages ( take at the struct generics branch if you are interested ). As this would be quite a substantial change I decided to attach some other pieces to it which made sense to resolve at the same time. Hence I am now in the process of rewriting the parser as it's on separate project. The typechecker and the AST definitions will also be separate to the compiler. My hope is that this will enable easier use and growth for the project as a whole, and gives me a chance to break things down into more testable chunks with better defined edges.

As I write this I have effectively finished the parser but am still in the process of writing the somewhat comprehensive tests. The compiler will be the last chunk to be reworked, but I'm hopeful that the time will be worth it. The parser rewrite has definitely resolved some small but long standing issues.

If you are interested in this project and want to help it develop please come say hi on the discussions page.

### Progress

- [ ] Rewrite the parser in it's own package.
- [ ] Define the AST as it's own package.
- [ ] Write an explicit type inference and checking stage, to be executed before compilation.
- [ ] Rewrite the compiler separate from the parser and type checker, integrating the new packages.
- [ ] Create a CLI package to wrap the compiler for AOT compilation.
- [ ] Publish the 5 main packages on npm: parser/ast/type-checker/compiler/cli.

## Installation

Radiance is available via NPM. It can either be used via the CLI or as a library.

```bash
npx radiance-lang hello.rad
```
The library exports the following methods:
```typescript
export declare function compile_string(source: string, filename: string): Uint8Array;

export declare function compile_file(filename: string): Promise<Uint8Array>;

export declare function execute_string(source: string, import_object?: {
  [key: string]: Function;
}): Promise<Record<string, WebAssembly.ExportValue>>;

export declare function execute_file(filename: string, import_object?: {
  [key: string]: Function;
}): Promise<Record<string, WebAssembly.ExportValue>>;
```
It also includes debug variants, these only differ in that they provide performance statistics on the compiler.

## Secondary goal
In part the syntax of this language was developed from the ideas of [Let's stop copying C](https://eev.ee/blog/2016/12/01/lets-stop-copying-c/). The article is well worth a read and discusses multiple evils that C has inspired in other languages, depending on your point of view some of these may be more controvertial than others. [Not based on C](not_based_on_c.md) discusses our take on the article, and what changes this has effected on Radiance.

## Learning the language
There's still a lot to be done on this language, such as the documentation! If you are reasonably technical then a good place to start at the moment is [src/parser/index.test.ts].

## Status
Development is still in a fairly early stage, but the language is usable for certain forms of program. Many language features have been implemented, but some higher level features are still missing requiring users to fall back on older more verbose styles of programming. Below is the implementation status of our planned feature list:

#### Basic features
- [x] Signed integer types ( i32, i64 )
- [x] Unsigned integers
- [x] Floating point types ( f32, f64 )
- [x] Functions
- [x] Logical operators ( not, and, or )
- [x] Bitshift
- [x] Bitwise operators ( and, or )
- [x] Basic math operators ( +, -, *, /, % )
- [x] Comparison operators ( ==, !=, <, >, <=, >= )
- [x] Type casting ( as )
- [x] While loop expressions
- [x] If/else expressions
- [x] Local variables
- [x] Import host functions
- [x] Export functions to host
- [ ] Break/Continue for loops
- [ ] Explicit function return
- [ ] Range literals
- [ ] For...of iterator loop
- [ ] Complex math operators ( sqrt, abs, ceil, floor, round, sign )
- [ ] Module system

#### Complex data types
- [x] Structure
- [x] Tuples
- [ ] Enumerations
- [x] Arrays
- [x] Strings
- [ ] Namespaces
- [ ] Methods
- [ ] Closures
- [ ] Generics
- [ ] Protocols

#### Compiler analysis
- [x] Type inference
- [ ] Escape analysis
- [ ] Object scalar replacement
- [ ] Unused function removal
- [ ] Unused variable removal

#### Syntactic sugar
- [ ] Destructing assignment

## Expression based
Inspired by Rust the majority of the language is expression based. There is no requirement for the ternary conditional expression, as the standard `if` syntax returns a value. Similarly `while` loops emit the result of their last iteration. A block will emit the value of the last expression within it. Functions behave much like blocks, meaning that no return statement is needed with fully described branches.

```swift
fn more_than_twelve (value: i32) -> i32 {
 if value > 12 {
  value
 }
 else {
  0
 }
}
```

## Memory management
Currently Radiance uses a simple but effective system for allocating memory. This is to have a fast bump allocator that is cleared automatically at set intervals. Bump allocators require no header information for allocations and are very fast to allocate/free memory. The downside being that you can effectively only free everything at once, so you can end up with a lot of zombie data.

An example use case for this sort of allocator would be in a game, temporary resources can be allocated quickly using the bump allocator and then cleared at the end of each frame. Games often have strict time budgets for each frame, so simplifying the memory management time is a nice boon for developers.

WebAssembly works by exposing functions to a host, so Radiance clears the allocator when the host calls one of those functions. This occurs at the start of a call. Why the start you might wonder? This means that if the host wishes to read a value back from memory after a call it can do.

The advantage of this system is that the user does not have to do any explicit memory management. Creating an instance of a struct allocates the memory for it, and then when the current callstack is finished the memory is freed automatically. The downside is that the WebAssembly module has no persistence between calls... The solution to this is to have some form of heap which objects can be moved to. How this occurs has yet to be decided, but there are some ideas in the works.

There is one addition to this behaviour which is not immediately obvious. WebAssembly modules can call imported host functions, which can in turn then call the module again. So to prevent memory being cleared in this situation a depth counter is used to decide if the module has been entered recursively.

Now it's important to note this isn't a flawless system, it's possible that an individual function call could allocate a very large amount of memory in a loop, leading to large amounts of zombie data that could be cleared. It's possible to reduce this through improved code analysis during compilation. But importantly this current system meets the initial use case for this language. Which is for set of medium sized functions exposed to the host, which the host calls into. This all may change in the future!

repository: shortercode/Radiance

repo_name: shortercode/Radiance


text:
<amplify-block-switcher>

<amplify-block name="Listener (iOS 11+)">

```swift
func resetPassword(username: String) {
    Amplify.Auth.resetPassword(for: username) { result in
        do {
            let resetResult = try result.get()
            switch resetResult.nextStep {
            case .confirmResetPasswordWithCode(let deliveryDetails, let info):
                print("Confirm reset password with code send to - \(deliveryDetails) \(info)")
            case .done:
                print("Reset completed")
            }
        } catch {
            print("Reset password failed with error \(error)")
        }
    }
}
```

</amplify-block>

<amplify-block name="Combine (iOS 13+)">

```swift
func resetPassword(username: String) -> AnyCancellable {
    Amplify.Auth.resetPassword(for: username)
        .resultPublisher
        .sink {
            if case let .failure(authError) = $0 {
                print("Reset password failed with error \(authError)")
            }
        }
        receiveValue: { resetResult in
            switch resetResult.nextStep {
            case .confirmResetPasswordWithCode(let deliveryDetails, let info):
                print("Confirm reset password with code send to - \(deliveryDetails) \(info)")
            case .done:
                print("Reset completed")
            }
        }
}
```

</amplify-block>

</amplify-block-switcher>

Usually resetting the password require you to verify that it is the actual user that tried to reset the password. So the next step above will be `.confirmResetPasswordWithCode`.

repository: SebastianKurp/docs

repo_name: SebastianKurp/docs


text:
**Contacts for Support**
- @rupertlssmith on https://elmlang.slack.com
- @rupert on https://discourse.elm-lang.org

# elm-auth

To learn how to use this the best place is to look at the demo code. The demo authenticates against Amazon Cognito.

[the-sett/elm-auth-demo](https://github.com/the-sett/elm-auth-demo) - Demo authenticating against AWS Cognito.

# A simplified API for authentication.

This is an API specification for authentication of Elm apps.

The idea here is to capture the interaction with a typical authentication API as an Elm package and to present that as a simpler API which applications needing authentication can make use of. The core of the API is the authentication commands, which are:

```
login : Credentials -> Cmd msg
refresh : Cmd msg
logout : Cmd msg
unauthed : Cmd msg
```

The purpose of these commands are:

* **login** - Supply a username and password as login credentials and attempt to authenticate.
* **refresh** - Attempt to refresh the authentication. Most authentication relies on tokens which expire after some time. Usually you do not have to call this, it will automatically be called prior to token expiry by the implementation using a timer.
* **logout** - Issue an explicit logout command, telling the back-end that this is happening if it supports that.
* **unauthed** - Resets the auth state to `LoggedOut` but without issuing an explicit logout command. This is usually used when some API returns a 401 or 403 response indicating that unauthorised access has been attempted.

The commands yield messages that must be given to an `update` function which has this signature and related types:

```
update : msg -> model -> ( model, Cmd msg, Maybe (Status auth chal fail))

type Status auth chal fail
    = Failed fail
    | Challenged chal
    | LoggedOut
    | LoggedIn
        { auth
            | scopes : List String
            , subject : String
        }
```

That is, each message will update the internal model, and may produce a change to the current authentication status.

The `update` function will only return a new `Status`, when the status changes, it will return `Nothing` if it processes commands but the status remains unchanged. Note that if a refresh happens and new authentication tokens are set, the `LoggedIn` state may be returned when already in this state - if the associated `AuthInfo` has changed.

# An extensible API with multiple implementations.

This package contains an API specification and some related utility code. The actual auth implementations that interact with various back-ends are:

[the-sett/elm-auth-aws](https://github.com/the-sett/elm-auth-aws) - Authenticate against AWS Cognito.

[the-sett/elm-auth-the-sett](https://github.com/the-sett/elm-auth-the-sett) - Authenticate against a custom auth server.

The API is presented as an extensible record of functions:

```
type alias AuthAPI config model msg auth chal ext fail =
    { ext
        | init : config -> Result String model
        , login : Credentials -> Cmd msg
        , logout : Cmd msg
        , unauthed : Cmd msg
        , refresh : Cmd msg
        , update : msg -> model -> ( model, Cmd msg, Maybe (Status auth chal fail) )
        , addAuthHeaders : model -> List Http.Header -> List Http.Header
    }
```

Each implementation fills in its own types for `config`, `model`, `msg`, `chal` , `ext` and `fail`.

The `config` type will contain whatever configuration settings an implementation needs.

The `model` and `msg` types for an implementation are likely to be opaque to hide the internal workings.

The `chal` type will be `Never` if the authentication API does not issue additional challenges, or some description of the possible extra challenges that may need to be answered in order to authenticate - for 2-factor authentication for example.

The `ext` type will contain whatever extra fields an implementation returns as part of its `AuthInfo`. There are standard fields defined in `AuthInfo` that all implementations must support, but as it is an extensible record, more fields can be added by implementations.

The `fail` type will describe all the reasons that an implementation can enter the `Failed` state. This can be useful in helping to decide how to respond to a failure, for example, by trying to log in again, by entering a password reset flow and so on.

Once authentication has been completed, further calls to some API often require that a 'bearer' token be added to HTTP headers to prove the callers granted authority to use that API obtained through the authentication. The `addAuthHeaders` function can ammend a list of HTTP headers on a request to include the necessary additions.

repository: the-sett/elm-auth

repo_name: the-sett/elm-auth


text:
# Monte_Carlo_Portfolio
A financial planner for retirement. This tool will forecast the performance of their retirement portfolio in 30 years. To do this, the tool will make an Alpaca API call via the Alpaca SDK to get historical price data for use in Monte Carlo simulations.

----------

## Technologies
This project leverages python 3.7 with the following packages:

**[Operating System Library](https://docs.python.org/3/library/os.html)** - For being able to work with our operating system.

**[Requests Library](https://pypi.org/project/requests/)** - For making requests to our APIs

**[JSON Library](https://www.json.org/json-en.html)** - For creating comprehensible data structures from our API responses.

**[Pandas Library](https://pandas.pydata.org/)** - For use of financial functions for calculations.

**[Dotenv Library](https://pypi.org/project/python-dotenv/)** - For being able to work with our .env files.

**[Alpaca Trade API](https://alpaca.markets/docs/)** - For retrieving historical pricing data for our stocks.

**[Monte Carlo Simulation Library](https://umn.bootcampcontent.com/SVivanco/valhalla/blob/491913853a10bb86a912f102ef3fb26746a2e6ec/code/MCForecastTools.py)** - For running Monte Carlo Simulations for our portfolio.

**[Matplot Library](https://matplotlib.org/)** - For being able to plot our calculated data.

----------

## Installation Guide

Before running the application first import the following dependencies:

```python
import os
import requests
import json
import pandas as pd
from dotenv import load_dotenv
import alpaca_trade_api as tradeapi
from MCForecastTools import MCSimulation

%matplotlib inline
```

Make sure the "requests library" & the "JSON library" are included in your conda dev environment.

```python
conda list requests

conda list json
```
The result on your screen should resemble the following image:

![Requests/JSON Verification](requests_json_image.png)

If your development environment is missing either package, you can directly install it.

To install the Requests library, check that your development environment is active, and then run the following command:

```python
conda install -c anaconda requests
```

To install the JSON library, check that your development environment is active, and then run the following command:

```python
conda install -c jmcmurray json
```

With the python-dotenv library, you can read key-value pairs from an environment file (.env) and add them as environment variables.

To install this library, run the following command in your terminal:

```python
pip install python-dotenv
```

**Install the Alpaca SDK**
Alpaca is an API for stock trading. With the Alpaca SDK, you can interact with the Alpaca API.

To install this SDK, run the following command in your terminal:

```python
pip install alpaca-trade-api
```

**Verify the Installations**
To verify that the library and SDK installations completed, call the pip list function together with the grep -E argument, which enables plain-text searches via the command line. The following code shows this function call:

```python
pip list | grep -E "python-dotenv|alpaca-trade-api"
```

The following image shows the results in your terminal:

![Alpaca Installation Verification](alpaca_verification.png)

Alternatively, you can call the conda list function twice, first with the library name and then with the SDK name, as the following code shows:

```python
conda list python-dotenv

conda list alpaca-trade-api
```

The following image shows the results in your terminal:

![Alpaca Installation Verification](alpaca_two_verification.png)

**Get the API Keys**
To use certain APIs, you need API keys. You use these unique identifiers to establish an authenticated, secure connection to an API. You'll get keys for both the Quandl and Alpaca APIs.

**Get the Quandl API Key**
To get your API key, you need to sign up for a Quandl account. Go to the Quandl homepage (Links to an external site.), and then click Sign Up (which appears on the main menu along the top of the page). The following image shows the Quandl homepage:

![Quandal Home Page](quandl_home_page.png)

On the “Step 1 of 3” sign-up page that appears, enter your first and last names, and then select Personal as your purpose for using Quandl. Click Next to move to the next page. The following image shows the “Step 1 of 3” page:

![Quandl Signup Page 1](quandl_page_one.png)

On the “Step 2 of 3” page that appears, enter your email address, and then on the “How will you be using this data” drop-down menu, select your reason for using Quandl data. We suggest selecting “For a personal software development project.” Then click Next. The following image shows the “Step 2 of 3” page:

![Quandl Signup Page 2](quandl_page_two.png)

Now, set a password—you need to enter it twice—and then check the “I’m not a robot” reCAPTCHA box. Accept the terms of service and privacy policy. Finally, click Create Account. The following image shows the “Step 3 of 3” sign-up page:

![Quandl Signup Page 3](quandl_page_three.png)

The page that appears displays a welcome message with your API key. Copy the key from the box, and then click Continue. Your new Quandl homepage appears, where you can exit the Quandl browser window. The following image shows the “Welcome to Quandl” page with your API key:

![Quandl Key](quandl_key.png)

Securely store your Quandl API key on your computer so that you can access it throughout the module. We recommend storing it in either a password keeper or document that resides on your local machine. The following image shows a suggestion for the format:

![Quandl Save Key](quandl_save_key.png)

Quandl sends you a confirmation email. Open the email, and then confirm your address to avoid any issues when using this API.

You need to generate your API credentials from the Alpaca API, so go to the **[Alpaca Markets sign-up page](https://alpaca.markets/docs/)**. To create an account, enter an email address, create a 12-character password, and then click “Sign up for free.” You’ll receive an email to verify the address before you can continue. The following image shows the Alpaca Markets sign-up page:

![Alpaca Signup Page](alpaca_signup_page.png)

Once you verify your email address, the "Welcome to Alpaca” page displays. Click the “Go to Paper Account” link (which appears at the far left of the page). The following image shows this page:

![Alpaca Welcome](alpaca_welcome.png)

The Paper Trading page displays. In the “Your API Keys” area (which exists on the right side of the page), click the View button. Then click “Generate API Keys.” This generates two keys: the API Key ID and a secret key. The following image shows the Paper Trading page with the View button.

![Alpaca Paper Trading](alpaca_paper_trading.png)

Copy both keys, and then securely store them on your computer so that you can access them throughout the module. I recommend storing them in either a password keeper or a document that you can secure—perhaps the same document where you stored the Quandl API key.

The following image shows a suggestion for the format:

![Save Apaca Keys](alpaca_key_saved.png)

**Display Your Hidden Files**
You’ll use environment (.env) files to protect your Quandl and Alpaca API keys. When you create an environment file, the system hides it in the folder structure of the project. Thus, if you navigate the folder structure of your computer via Finder (macOS) or File Explorer (Windows), you won’t find an environment file in the project folder.

To work with environment files, you need your folder structure to display them for you. To display the hidden files on your computer, complete the steps in the following subsections.

**Windows Users: Display Your Hidden Files in File Explorer**
To display the hidden files on a Windows computer, complete the following steps:

In the search bar, type “folder”.

Click “File Explorer Options”.

In the “File Explorer Options” dialog box that displays, click the View tab. Select “Show hidden files, folders, and drives,” and then click OK, as the following image shows:

![Display Hidden Files](display_hidden_files.png)

**macOS Users: Display Your Hidden Files in Finder**
To display the hidden files on a macOS computer, complete the following steps:

Open Finder.

Press the Command+Shift+Dot (.) key combination.

**Display Your Hidden Files in JupyterLab**
You can also observe your hidden files directly in JupyterLab. To do this, run the following command in the terminal when JupyterLab launches:

![Display Hidden Files in Jupyter](hidden_files_jupyter.png)

This command doesn’t permanently change the ability to observe hidden files in JupyterLab. You need to launch JupyterLab with this extended command every time you want to work with hidden files.

**Installation should now be complete**

## Usage

In order to use this program, in your terminal, make sure you are in your conda dev environment
and then from the directory you're working in launch jupyter lab. The following image shows this step:

![Launching Jupyter](jupyter_launch.png)

Thes result of the program should be to determine whether a person will be able to retire after only ten years.

First, we need to calculate their current retirement plan, which is set for 30 years. However, we want to determine if rebalancing their portfolio to include heavier stock weight, will allow the client to retire after only 10 years, instead of 30 years.

The application produces two histograms and summary statistics that we can use to make our detemination on this client's retirement.

Following, are the results of the 30-year retirement calculation for the client:

![30-year Summary Statistics](thirty_year_summary.png)

Next, the histogram showing the range that we have a 95% certainty of happening:

![30-year Histogram](thirty_year_hist.png)

From the summary statistics and the histogram, we can tell that the average return this client will get in 30 years is right at 20 times the initial investment (current portfolio value).

Now the calculations for the 10-year retirement plan, starting with the summary statistics:

![10-year Summary Statistics](ten_year_summary.png)

We will now plot the data in a histogram:

![10-year Histogram](ten_year_hist.png)

From the summary statistics and the histogram, we can tell that the average return this client will get in 30 years is right at 3.7 times the initial investment (current portfolio value).

Given only this information, we can determine that the portfolio value in 10 years will **not** be sufficient for early retirement.

---

## Contributors

**Created by:** Tony Landero

**Email Adress:**<EMAIL_ADDRESS>
---

## License

MIT


text:
# How to contribute

If you are looking to contribute, please take a look at the current open issues in Github: chances are these are the most urgent and pressing issues where your impact will be the most valuable!

Of course feel free to make your own issues if you think something needs to added or fixed.

# Basic quality checks

Please ensure that all tests pass before submitting changes. Most projects have a full test suite that are automatically run as part of the default phase of the npm lifecycle (`npm install`).

They can also be run individually: unit tests with `npm run test`.

We follow the default Airbnb code formatting style, please make sure to follow and adhere to these conventions when submitting code.

# Submitting Changes

Please sign the [Contributor License Agreement](https://docs.google.com/forms/d/e/1FAIpQLSc6O3CudbzEvSEos_VbXAf6bah05dvx_OZRXDHBx7gFLe-uJA/viewform).

This form is also available [here](https://s3.us-east-2.amazonaws.com/amberdata-public/docs/icla.pdf) in PDF format.

Submit a pull request rebased on top of master
 * Include a descriptive commit message.
 * Changes contributed via pull request should focus on a single issue at a time.

Please allow a few business days for us to comment on your PR, depending on the number of open PRs.


text:
# Nemo

Nemo is a cross-platform TUI application for learning foreign languages.

## Building

``` shell
cmake . -Bbuild
cmake --build build
```

## Usage

After running Nemo, you will see several options. To choose one of them, type the digit preceding the given option. Then (if you have not chosen the `Exit` option), you will be prompted to enter the path to the word list. The path can be either relative or absolute. Below is a detailed description of each option.

### Test

Use this option to test your knowledge. You will be asked to choose ordering. The possibilities are:

1. In sequence
2. Randomly

The first option causes the data to appear in the order in which they appear in the file. The second is self-explanatory. Then, you will be prompted to translate the text to the left of the dash character. After providing an answer, you will be informed if it was correct or not. If not, the correct translation will be displayed additionally. You can stop the test at any time by pressing Ctrl+D on Unix or Ctrl+Z followed by Enter on Windows. After the test, the percentage of correct answers will be displayed.

### Create new word list

To create new word list choose this option, specify the path to it, and enter the data line by line. After completing the data entry, press Ctrl+D on Unix or Ctrl+Z followed by Enter on Windows to save the word list and back to menu. Each line should have the following format:

```
text-translation1;translation2;...;translationN
```

You can also use an external text editor for this task.

### Modify word list

If you select this option and enter the correct path to the file, the word list with line numbering followed by a few options will be displayed. The options are:

1. Modify lines
2. Erase lines
3. Add lines
4. Return

After choosing one of the first two, you will be prompted to enter the range in the following form: `begin-end`. The first option replaces the lines in the given range to those given by the user. The third one appends lines at the end of the word list.

### Show word list

This option shows the word list with line numbering.

### Delete word list

After specifying the path, confirm that you want to delete the file by entering `y` or cancel by entering `n`. In the first case, the given file will be deleted.

### Exit

Select this option to exit the program.

## License

[MIT](https://github.com/wadiim/nemo/blob/master/LICENSE)

repository: wadiim/nemo

repo_name: wadiim/nemo


text:
# M3

**Please note:** This documentation is a work in progress and more detail is required.

## About

M3 is a metrics platform that is built on M3DB, a distributed timeseries database. The top level project repository can be found [here][m3db].

[m3db]: https://github.com/m3db/m3db


text:
The Schema Prelude
==================

The IPLD Schema "prelude" is a series of type declarations that are implicitly present at the top of all schemas.
It introduces all the types that are seen used in nearly every schema,
which makes them available consistently under consistent names, and saves a little redundancy for users.
However, beyond that, there's nothing magical about the Prelude (as you'll see!).

Here is the Prelude, in IPLD Schema DSL form:

```ipldsch
type Bool bool
type Int int
type Float float
type String string
type Bytes bytes
type Any any
type Map {String:Any}
type List [Any]
type Link &Any
type Null unit representation null
```

### Use of the Prelude is optional

Note that not all schemas technically need the prelude;
it's perfectly possible to author a schema which has a name for every single one of its types,
and doesn't use any of the names provided by the prelude.
It's relatively rare to see this in practice, however.

For example, if we examine the schema-schema, we can see it often defines its own names for most of its types.
However, we also see it use `String` and `Int` and `Bool` in several places;
these type names come from the prelude.

repository: ipld/ipld

gha_language: Nunjucks

gha_license_id: NOASSERTION

repo_name: ipld/ipld


text:
[![Alt text](./images/original.png "Optional title")](http://kenji-special.info/)
http://kenji-special.info/

#portfolio-2017

source codes for kenji saito portfolio site for 2017(http://kenji-special.info/). it is pure webgl site. I used three.js for building webgl.

#sketches
sketches were made based on portfolio codes. the basic function and data are same, but the effects to the por are different.

####sketch00

[![Alt text](./images/sketch00.png "Optional title")](http://kenji-special.info/sketches/00)

http://kenji-special.info/sketches/00

codes: https://github.com/kenjiSpecial/portfolio-2017/tree/sketch00

####sketch01

[![Alt text](./images/sketch01.png "Optional title")](http://kenji-special.info/sketches/01)

http://kenji-special.info/sketches/01

codes: https://github.com/kenjiSpecial/portfolio-2017/tree/sketch01

## Usage

the dev environment is based on ["budo-gulp-starter"](https://github.com/mattdesl/budo-gulp-starter) by [Matt](https://github.com/mattdesl)

```sh
git clone https://github.com/mattdesl/budo-gulp-starter.git
cd budo-gulp-starter

# install dependencies
npm install

# start development server & open browser
npm run open
```

## tasks

```
npm run
  start  - start dev server
  open   - start dev server and open the browser to localhost
  build  - the compressed production build
```

## License

MIT, see [LICENSE.md](http://github.com/mattdesl/budo-gulp-starter/blob/master/LICENSE.md) for details.


text:
credit-scoring-exercise
===========================

# Exercise 1

Once the model is activated, Portfolio A’s owner will evaluate its performance using only loans repayment information collected AFTER the model was activated. I am assuming that this is how the TEST set is collected.

The above evaluation criteria is not reliable because it introduces selection bias, as we would only know the repayment results of those clients whom the model has accepted. There could be credit-worthy clients that the model has rejected but there will be no way of knowing this result. This problem is also known as partial feedback.

I will assess the models based on validation set only, using Area Under the ROC curve as the evaluation metric for binary classification. Model 1 is the best model.

validation auc: model1=0.647, model2=0.638

Code: [ex1.ipynb](ex1.ipynb)

# Exercise 2

Code: [ex2.ipynb](ex2.ipynb)

## Models
Tree-based methods; LightGBM is a boosting ensemble designed to reduce bias, whereas Random Forest is a bagging ensemble designed to reduce variance.

Advantages of tree-based methods:
- Cheap to build and make inference (cost grows with tree depth).
- Non-parametric: does not assume any probability distribution type.
- Robust to noise and outliers because outliers are unlikely to be the split point. Unlike linear models and neural nets, we do not need to perform standardisation or attempt to map the features to a normal distribution.

Tune the following hyperparameters to avoid overfitting: minimum number of examples in leaf, number of features to consider for splitting, number of examples available to grow each tree.

## Feature Engineering
Encode categorical variables with frequency encoding which is better for decision trees as compared to one-hot encoding. It is difficult for decision trees to look for a good split point where there are very few split points available (such as binary 0/1 in the case of one-hot encoding).

To ensure no leaks from the test set, frequencies are obtained only from the training set.

Feature crosses to create interaction terms for important features, which are obtained from the model (goodness of a split point, how much impurity is reduced). This helps to model non-linearities in feature space by combining two or more features together.

## Feature Selection
KS Test to test whether two samples are drawn from the distribution. Both samples are drawn from the training set (ensure no data leaks from the test set).

Drop features where p value is less than or equals level of significance alpha (10%). These features are dropped because they exhibit too much variance across different samples.

Drop 7 features={'Var77', 'Var82_Var83', 'Var130', 'Var9', 'Var83_Var7', 'Var29', 'Var30'}

## Evaluation
Stratified 5-fold cross validation is important because we have a class imbalance problem (positive class is present at 2% of training set, 1% of test set). K-folds allows all examples to be used for validation once and for training (k - 1) times. We reduce the risk of the result having a heavy dependency on the membership of the training and validation sets.

CV results
- LGBM best score=0.710, std=0.087
- Random Forest best score=0.715, std=0.072

(Important to indicate the standard deviation so that we can assess how the models will fare on unseen data)

Based on the validation results, RF seems to be the better model because it has a slightly higher score and smaller standard deviation.

The test set shows that the LGBM Model is able to generalize slightly better than random forest.
auc_lgbm=0.635, auc_rf=0.615

## Notes
I tried a simple blend of the two models but the ensemble did not outperform the best single model which is LGBM. Adding more models (different machine learning methods) to the blend may improve its performance by reducing variance. All models learn from noise but they may partially cancel out each other’s errors.

The evaluation metric did not have good results for both validation and test, so we have an underfitting problem. To reduce model bias, I would first invest my time in feature engineering as this has the most significant effect on improving the model. After getting good enough result from a single model, I will then look into ensembles.

repository: seahrh/credit-scoring-exercise

repo_name: seahrh/credit-scoring-exercise


text:
---
title: Harbor 2.9 Documentation
---

Welcome to the Harbor 2.9.x documentation. This documentation includes all of the information that you need to install, configure, and use Harbor.

## Harbor Installation and Configuration

This section describes how to install Harbor and perform the required initial configuration. These day 1 operations are performed by the Harbor Administrator. [Read more](install-config/_index.md)

## Harbor Administration

This section describes how to use and maintain your Harbor registry instance after deployment. These day 2 operations are performed by the Harbor Administrator. [Read more](administration/_index.md)

## Working with Harbor Projects

This section describes how users with the developer, maintainer, and project administrator roles manage users, and create, configure, and participate in Harbor projects. [Read more](working-with-projects/_index.md)

## Building, Customizing, and Contributing to Harbor

This section describes how developers can build from Harbor source code, customize their deployments, and contribute to the open-source Harbor project. [Read more](build-customize-contribute/_index.md)

## Access the Documentation Source Files

The source files for this documentation set are located in the [Harbor repository on Github](https://github.com/goharbor/harbor/tree/release-2.8.0/docs).

For the previous versions of the docs, go to the [`docs` folder in the Github repository](https://github.com/goharbor/harbor/tree/main/docs) and select the appropriate `release-X.Y.Z` branch.

repository: goharbor/website

gha_language: HTML

repo_name: goharbor/website


text:
# JKDrawer

A Maps like drawer for iOS.

Features:

- control drawer size
- snap to preferred sizes
- multiple stacked drawers
- dragging and closing using gestures
- handling nested scroll views
- subtle animations
- no need to subclass view controllers

[![CocoaPods Compatible](https://img.shields.io/badge/CocoaPods-compatible-brightgreen?style=flat)](https://cocoapods.org/pods/JKDrawer)
[![Carthage Compatible](https://img.shields.io/badge/Carthage-compatible-brightgreen?style=flat)](https://github.com/Carthage/Carthage)
[![Swift Package Manager Compatible](https://img.shields.io/badge/Swift_Package_Manager-compatible-brightgreen?style=flat)](https://github.com/Carthage/Carthage)

## Requirements

- iOS 10.0+
- Xcode 10.2+
- Swift 4+

## Installation

### CocoaPods

[CocoaPods](https://cocoapods.org) is a dependency manager for Cocoa projects. For usage and installation instructions, visit their website. To integrate JKDrawer into your Xcode project using CocoaPods, specify it in your `Podfile`:

```ruby
pod 'JKDrawer', '~> 0.6.0'
```

### Carthage

[Carthage](https://github.com/Carthage/Carthage) is a decentralized dependency manager that builds your dependencies and provides you with binary frameworks. To integrate JKDrawer into your Xcode project using Carthage, specify it in your `Cartfile`:

```ogdl
github "johankool/Drawer" ~> 0.6.0
```

### Swift Package Manager

The [Swift Package Manager](https://swift.org/package-manager/) is a tool for automating the distribution of Swift code and is included with Xcode 11+. To integrate JKDrawer into your Xcode project using Swift Package Manager, paste this URL into Xcode via File > Swift Packages > Add Package Dependency:

```
https://github.com/johankool/Drawer.git
```

## Usage

To be able to present a view controller as a drawer it must conform to the `DrawerPresentable` protocol. This protocol requires that configuration variable is present. This is a `DrawerConfiguration` struct specifying certain aspects like initial offset and allowed range.

```swift
class DrawerViewController: UIViewController, DrawerPresentable {

    var configuration = DrawerConfiguration(offset: 300, isDraggable: true, isClosable: false)

}
```

The presenting view controller must conform to the `DrawerPresenting` protocol. Some methods have default implementations, others are callbacks for your convenience.

```swift
class HostViewController: UIViewController, DrawerPresenting {

    func someAction() {
        let drawerViewController = DrawerViewController()
        openDrawer(drawerViewController, animated: true)
    }

    func willOpenDrawer(_ drawer: DrawerPresentable) {

    }

    func didOpenDrawer(_ drawer: DrawerPresentable) {

    }

    func willCloseDrawer(_ drawer: DrawerPresentable) {

    }

    func didCloseDrawer(_ drawer: DrawerPresentable) {

    }

    func didChangeSizeOfDrawer(_ drawer: DrawerPresentable, to size: CGFloat) {

    }

}
```

### Navigation Controllers

Since you can't open a drawer over a `UINavigationController`, consider using a custom view controller wrapping the `UINavigationController` instead or use `DrawerNavigationController`.

## License

Copyright (c) 2018-2021 Johan Kool

Licensed under the [MIT license](LICENSE.md)

repository: johankool/Drawer

gha_language: Swift

repo_name: johankool/Drawer


text:
# FingerTap music play from Live Video Feed
|| Finger Tapped Music || Koi.. Mil Gaya 😃 ||
This is an attempt to mimic the computer sounds showed in one of my favorite childhood movies "Koi.. Mil Gaya".
It is a hobby project. It utilizes Tensorflow JS and Tensorflow Handpose Js to initiate the processings.
You can see the  Demo here
## Usage
1. Clone the repository and run through a server.
2. To start a python server, open terminal/cmd in the repository location and type
```python
python -m http.server
```
3. Now visit localhost:8000 and wait for the js modues to load. (takes 2-3 minutes to download the essentials)
4. Enjoy
Give this repo a star if you like it.


text:
Video is still in early stages of development. The basic API is set, although a
lot of the details are subject to change. Once Video reaches v1.0, this API
will be fixed.

Currently, the `master` git branch is used for development. However, it is
important to try to keep it relatively stable. For example, try not to push
code that breaks the tests.

Releases get there own branches/tags, which match their release name (v0.0,
v0.1, etc.). These branches should remain stable. Additionally, tags should be
fixed. So if a patch needs to be applied to one of these branches, it is
important to increment the version number (v0.1.1, v0.1.2, etc). Finally, make
sure to update the LOG file whenever a new release is made.

New releases happen when a major change happens to the language. Currently this
happens 2-3 times a year.

Pre-Releases get their type following the version number with a dash.
Subsequent versions of the same type get a version number attached to the end.
For example, the following versions indicate pre-release versions: v0.2-alpha,
v0.2-beta.3, v0.2-rc.3, etc.

Once the project becomes more stable, we will create a more structured
contributing guide. For the moment, however, feel free to open PRs and push
directly to the master branch.

However, please make sure that you do NOT push broken builds to the `master`
branch.  Additionally, and changes that break backwards compatibility MUST be
noted in the `LOG` file. It would also be nice if your changes keep a linear
history, but this is not strictly necessary.

Any code in the `video/private` folder is not part of the public API, and can
change at any time without notice. Code in `video/tests` are no implementation
code should go there. Documentation for your contributions belongs in
`video/scribblings/`. The identifiers exported by the remaining paths are
either public or protected. If they are documented they are public, and can
only be changed between MAJOR releases. Protected identifiers are not
documented, but still exported by a module (or submodule) outside of these
designated directories. These can change, but should be noted in LOG.

Some experimental code is allowed in this repo.
However, if it is still extremely experimental please put it in the
http://github.com/videolang/render-prototype git repo.

repository: LiberalArtist/video

repo_name: LiberalArtist/video


text:
ChavBenchmarkPHP
================

PHP benchmark tool to compare PHP code performance, under Apache 2.0 License.

![Screenshot](/screenshot/ChavBenchmarkPHP.png "Screenshot")

## Requirements

* PHP 5.3

## How to use

Copy the repository to your server and launch **index.php**.

If you want to make your own benchmarks, edit **benchmark.xml**. As you can see in this file, benchmarks are cut in different section and each section contains commands. Each of these is benchmarked between them and results are shown.

A section is composed as follows :
```
<section name="Your section name" description="Your section description">
  <command>
    <![CDATA[
    Your script (commands) to compare
    ]]>
  </command>
  <command>
    <![CDATA[
    Your other script (commands) to compare
    ]]>
  </command>
  ...
</section>
```

In addition of the command list, you can add hooks.

## Hooks

There's two types of hooks :
* Hooks executed before benchmark (```type="before"```)
* Hooks executed after each loop of the benchmark (```type="loop_end"```)

Like you see, the hooks are scripts that are executed at one point of the benchmark. You can for example declare or increment variables in hooks.

To add a hook, create a ```<hook type="...">...</hook>``` tag in the concerned section, like :
```
<section name="Your section name" description="Your section description">
  <hook type="before">
  	<![CDATA[
  	$answer = 42;
  	]]>
  </hook>
  <command>
  ...
  </command>
  ...
</section>
```

See current **benchmark.xml** for more details and examples.

## Feedback

Don't hesitate to fork this project, improve it and make a pull request.

repository: Chavjoh/ChavBenchmarkPHP

repo_name: Chavjoh/ChavBenchmarkPHP


text:
# LINKED LIST

## Data Struct
```
struct ListNode {
  int data;
  struct ListNode *next;
}
```

## Code
```
/********** Given a reference (pointer to pointer) to the head
  of a list and an int, appends a new node at the end  */
void append(struct Node** head_ref, int new_data)
{
    /* 1. allocate node */
    struct Node* new_node = (struct Node*) malloc(sizeof(struct Node));
    struct Node *last = *head_ref;  /* used in step 5*/

    /* 2. put in the data  */
    new_node->data  = new_data;

    /* 3. This new node is going to be the last node, so make next
          of it as NULL*/
    new_node->next = NULL;

    /* 4. If the Linked List is empty, then make the new node as head */
    if (*head_ref == NULL)
    {
      *head_ref = new_node;
      return;
    }

    /* 5. Else traverse till the last node */
    while (last->next != NULL)
        last = last->next;

    /* 6. Change the next of last node */
    last->next = new_node;
    return;
}
```

repository: evanharmon/eph-headspace

repo_name: evanharmon/eph-headspace


text:
# Add Marketplace Image to lab

This script provides a way to add a new allowed Marketplace Image to a DevTest Lab via Azure PowerShell. Before running the script you need to log in to your azure account and select the subscription which contains the lab.

The script takes two parameters - the first specifies the name of the DevTestLab. The second is an array containing the display names of the marketplace image(s) to be added. The list of images specified will be added to the list of allowed images in the lab, so any previously allowed marketplace images will continue to be allowed. If the lab currently allows all marketplace images then no action will be taken.

Examples:

```powershell
Login-AzureRmAccount
Select-AzureRmSubscription -SubscriptionId ...

# Add one additional image to the lab
.\AddMarketplaceImages.ps1 -DevTestLabName 'MyDevTestLab' -ImagesToAdd 'Windows Server 2016 Datacenter'

# Add multiple additional images to the lab
.\AddMarketplaceImages.ps1 -DevTestLabName 'MyDevTestLab' -ImagesToAdd 'Windows Server 2016 Datacenter', 'Red Hat Enterprise Linux 7.3'
```

repository: Scapenighter/Azure-Test-Lab

gha_language: PowerShell

repo_name: Scapenighter/Azure-Test-Lab


text:
---
layout: post
title: Generics to the Rescue
date: 2022-08-11 00:00:00
image: '/assets/img/owen.png'
description: A tale of performance improvements using Generics
tags: [programming, go]
categories: [Programming]
twitter_text: A tale of performance improvements using Generics
---
![Generics To the Rescue](../images/generics-to-the-rescue.png)

### A bit of background

I'm working on a tool that uses the [AWS Go SDK](https://aws.github.io/aws-sdk-go-v2/docs/){:target="_blank"} to pull back a lot of information about an account using the API.

Some services require several secondary API calls - for example to get a full picture about an S3 bucket with details about encryption, policies, ACL, logging, versioning and Public Access Block configuration requires six more calls.

When dealing with a medium to large number of buckets, making these calls sequentially is time consuming; extrapolating that over many services and you have a processing time that is not insignificant.

### A bit more detail

Lets put a little bit more detail around this and take SQS for example.

The first thing that we need to do is get all of the SQS queues. The API returns them in pages, so that needs to be handled too...

```go
client = sqs.NewFromConfig(ctx.SessionConfig()) // import "github.com/aws/aws-sdk-go-v2/service/sqs/types"

var apiQueueURLS []string
var input types.ListQueuesInput // import "github.com/aws/aws-sdk-go-v2/service/sqs/types"
for {
	output, err := a.client.ListQueues(a.Context(), &input)
	if err != nil {
		return nil, err
	}
	apiQueueURLs = append(apiQueueURLs, output.QueueUrls...)
	if output.NextToken == nil {
		break
	}
	input.NextToken = output.NextToken
}

```

We now have a list of all the `QueueURLs` for a given region, but we likely need more information than that so we're going to have to make some more calls.

We want to populate the data in a our `Queue` object
```go
type Queue struct {
	QueueURL   string
	Encryption Encryption
	Policies   []iam.Policy
}

type Encryption struct {
	KMSKeyID          string
	ManagedEncryption bool
}
```

If we create a function to get the details - with some of the logic removed for brevity

```go
func adaptQueue(queueUrl string) (*Queue, error) {

	// make another call to get the attributes for the Queue
	queueAttributes, err := a.client.GetQueueAttributes(a.Context(), &sqs.GetQueueAttributesInput{
		QueueUrl: aws.String(queueUrl),
		AttributeNames: []types.QueueAttributeName{
			types.QueueAttributeNameSqsManagedSseEnabled,
			types.QueueAttributeNameKmsMasterKeyId,
			types.QueueAttributeNamePolicy,
			types.QueueAttributeNameQueueArn,
		},
	})
	if err != nil {
		return nil, err
	}

	queueARN := queueAttributes.Attributes[string(types.QueueAttributeNameQueueArn)]
	queue := &sqs.Queue{
		QueueURL: queueUrl,
		Policies: []iam.Policy{},
		Encryption: sqs.Encryption{
			KMSKeyID:          "",
			ManagedEncryption: false,
		},
	}

	sseEncrypted := queueAttributes.Attributes[string(types.QueueAttributeNameSqsManagedSseEnabled)]
	kmsEncryption := queueAttributes.Attributes[string(types.QueueAttributeNameKmsMasterKeyId)]
	queuePolicy := queueAttributes.Attributes[string(types.QueueAttributeNamePolicy)]

	if sseEncrypted == "SSE-SQS" || sseEncrypted == "SSE-KMS" {
		queue.Encryption.ManagedEncryption = true
	}

	if kmsEncryption != "" {
		queue.Encryption.KMSKeyID = kmsEncryption
	}

	if queuePolicy != "" {
		policy, err := iamgo.ParseString(queuePolicy) // import "github.com/liamg/iamgo"
		if err == nil {
			queue.Policies = append(queue.Policies, iam.Policy{
				Document: iam.Document{
					Parsed:   *policy,
				},
				Builtin: false,
			})
		}

	}
	return queue, nil
}
```

In this case, we only had to make a single additional request to the API, but the argument is clear for trying to parallelise this process a little.

### What do we want to do?

Ideally, we would get the list of queues so we know how many we're talking about and this can drive our progress mechanism we might have in out UX.

Once we have the list of URLs, we want to parallelise the adaption process. But wait!! the `adaptQueue` function in this case takes a `[]string`, but for buckets it might be a `[]s3.Bucket` or with IAM an `[]iam.Role`.

We need a `generic` function that will take the slice of inputs and a function to call for each item.

# Generics to the rescue!!

Lets say out input (in this case a `[]string`) is type `T` and the output we want (in this case a `[]Queue`) is type `S`, we can create a generic function to handle this

```go
func ParallelAdapt[T any, S any](items []T adapt func(T) (*S, error)) []S {
	processes := getProcessCount(DefaultStrategy)

    mu := sync.Mutex{}
    var results []S

    var ch = make(chan T, 50)
	wg := sync.WaitGroup{}
	wg.Add(processes)

	for i := 0; i < processes; i++ {
		go func() {
			for {
				in, ok := <-ch
				if !ok {
					wg.Done()
					return
				}
				out, err := adapt(in)
				if err != nil {
					log.Debug("Error while adapting resource %v: %w", in, err)
					continue
				}

				if out != nil {
					mu.Lock()
					results = append(results, *out)
					mu.Unlock()
				}
			}
		}()
	}

	for _, item := range items {
		ch <- item
	}

	close(ch)
	wg.Wait()

	return results
}
```

Let's break down this function - first, we use a strategy that we've used to get the number of processes to run.

We need to keep in mind the rate limiting of AWS API so there is no point using unbounded number of go routines.

```go
type Strategy int

const (
	DefaultStrategy Strategy = iota
	CPUCountStrategy
	OneAtATimeStrategy
)

func getProcessCount(strategy Strategy) int {
	switch strategy {
	case OneAtATimeStrategy:
		return 1
	case CPUCountStrategy, DefaultStrategy:
		return runtime.NumCPU()
	default:
		// this shouldn't be reached but at least we don't crash
		return 1
	}
}
```

In this example, we're running as many Go routines as we have CPUs.

The next thing to create a slice of results of type `S` (in this case `[]Queue`) where our results are going to go. We need a `sync.Mutex` here to safely add items to the slice.

Now for the concurrency bit...

We create a `sync.WaitGroup` that has space for the number of process we're going to use.

We also create a channel with an arbitrary number of slots where we're going to push the inputs of type `T` for processing.

Next, we create as many Go routines as we have processes that takes a value off the channel `ch` and if its ok, it will run the `adapt` function provided against the value of type `T`. We get an error and an output result of type `S` from the function which we can safely add to the `results` then the go routine can go back to the channel for another item of work.

The last step is to add everything to the channel - we loop over the items (in this case `[]string`) sending them to the channel. When this is done, we `close` the channel to tell it nothing else is coming then `Wait` for the `WaitGroup` to be done (this is done when the channel has nothing else on it, and each process will complete).

Finally, we return the results back to the caller.

### Using the function

We can easily use this function, sticking with the same example we would call

```go
results := ParallelAdapt(apiQueueURLS, adaptQueue) // we have a slice of *Queue
```

The number of concurrent calls is always something that needs to be analysed and a suitable trade off come to. You don't want to create IO issues in the name of concurrency.


text:
# syntec theme

I actually managed to import my current theme style from phpStorm to Atom.
The theme I'm using is based and inspired from

* https://github.com/abhimanyusharma003/phpstorm-spacegray

and therefore by
* https://kkga.github.io/spacegray

Many thanks to those guys.

> Below screenshots are taken from several project files (symfony,html boilerplate,bootstrap)

## PHP Syntax
![tec_php](https://cloud.githubusercontent.com/assets/2684102/8796795/d330fbde-2f97-11e5-94ac-1b7ca1d82019.png)

## HTML Syntax
![tec_html](https://cloud.githubusercontent.com/assets/2684102/8796793/d32eb04a-2f97-11e5-8455-7ebe0e4ee618.png)

## JavaScript Syntax
![tec_js](https://cloud.githubusercontent.com/assets/2684102/8796796/d3328558-2f97-11e5-96a8-020d0cf796f8.png)

## CSS Syntax
![tec_css](https://cloud.githubusercontent.com/assets/2684102/8796794/d32ee538-2f97-11e5-9307-d556115e7415.png)

repository: rogatec/atom-syntec-syntax

repo_name: rogatec/atom-syntec-syntax


text:
---
author: sivel
categories:
- Plugins
- WordPress
date: '2007-09-18'
description: Polaroid on the Fly WordPress Plugin
slug: polaroid-on-the-fly
title: Polaroid on the Fly WordPress Plugin
type: post
---

  Update:As of February 2, 2009 I will no longer be updating the Polaroid on the Fly WordPress plugin; this plugin is now end of life.

I am releasing version 0.7 of the Polaroid on the Fly plugin.

Creates polaroids of images on the fly for thumbnails in posts. Support for lightbox included. Built on modified [Polaroid-o-nizer][1] v0.7.2 sources.

I wanted to mimic the Polaroid look that you can achieve with [Google Picasa][2] to add thumbnails to my posts. I found a plugin titled [WP-Polaroidonizer][3] that did what I wanted to some extent but it didn't have the right feel to it.

The other problem I have with Polaroid-o-nizer and WP-Polaroidonizer is the lack of security. When I mention lack of security there is by default no way to restrict external sites and users from generating Polaroid pictures using your installation of Polaroid-o-nizer.

The goal of this plugin was to use rel= style html img tags to transform a simple URL into a Lightbox URL in which there is a thumbnail image in your post which is linked to an image that can be loaded using [Lightbox][4] or any of the Lightbox clones/alternatives.

**<big>Screenshots</big>**

**<big>Requirements</big>**

1.  WordPress 2.0
2.  PHP GD Module
3.  Lightbox - Not really a requirement but a recommendation if you want to have a nice ajax/javascript image overlay on your page

**<big>Installation</big>**

1.  Upload the \`polaroid-on-the-fly\` folder to the \`/wp-content/plugins/\` directory
2.  Activate the plugin through the 'Plugins' menu in WordPress

**<big>Upgrade</big>**

1.  Deactivate the plugin through the 'Plugins' menu in WordPress
2.  Delete the previous \`polaroid-on-the-fly\` folder from the \`/wp-content/plugins/\` directory
3.  Upload the new \`polaroid-on-the-fly\` folder to the \`/wp-content/plugins/\` directory
4.  Activate the plugin through the 'Plugins' menu in WordPress

**<big>Usage</big>**

1.  Create an image link in your post in the following format:
    `<img src="http://domain.tld/directory/to/image.jpg" rel="polaroid" alt="Picture[album]" />`

2.  Be sure to include rel="polaroid" as this activates the plugin.
3.  If alt="Picture[album]" is included the portion listed here as Picture will be displayed as the image title in Lightbox and the portion listed as [album] will group multiple pictures into an album called album. The alt tag is not required and if used the [album] portion is not required. Read the Lightbox usage for more details on Titles and albums.
4.  Be sure to place each html img tag on a separate line.
5.  This plugin supports the following image formats: jpg, png, gif

> NOTE: Do not use the visual editor for doing the above use the code editor.

**<big>FAQ</big>**

> Q: What Do The Error Messages Mean:
>
> A:
> **Source image resolution below min (200x200px)**
> The image referenced in the image tag has a resolution less than 200x200 pixels.
>
> **Source image resolution above max (2000x2000px)**
> The image referenced in the image tag has a resolution greater than 2000x2000 pixels.
>
> **Source URL Incorrect of does not Exist**
> The URL of the image referenced in the image tag does not exist.
>
> **Background color is invalid use RGB only**
> The background color specified on the options page is not a valid 24 bit RGB color. When written, RGB values in 24 bits per pixel (bpp), also known as Truecolor, are commonly specified using three integers between 0 and 255, each representing red, green and blue intensities, in that order. For example:
>
> > white is 255,255,255
> > black is 0,0,0
> > red is 255,0,0
> > green is 0,255,0
> > blue is 0,0,255
>
> **Source image unknown file type (jpg,gif,png only)**
> You should never see this message. The regex used to find the img tag with rel="polaroid" will only select img tags including jpg,gif and png. I figured hey why not create a useless image it may come in handy some day.
>
> **PHP GD Module Not Found. Polaroid on the Fly will not work. See http://us.php.net/gd for installation information.**
> You need to have the PHP GD Module Installed. This plugin requires the GD module to make the polaroid image. This plugin will not function without this module.
>
> **PHP GD Module Found. However, it does not include support for GIF. You will not be able to use a GIF for the source image. See http://us.php.net/gd for more information.**
> You have the PHP GD Module installed but the GD version does not support GIF images. This plugin will still work but you will not be able to use a GIF for the source image.

**<big>Change Log</big>**

> **0.7 (2008-07-31):**
>
> *   Updated for compatibility with WordPress 2.6 (wp-content and wp-config.php changes).
> *   Changes to gen-polaroid.php to accommodate Ubuntu PHP GD Packages.
> *   Updated path to the font ttf file in gen-polaroid.php.
> *   Output image now in png format instead of jpg.
>
> **0.6 (2008-01-23):**
>
> *   Added ability to opt in to using Lightbox.
> *   Added option to select how the image will open. (ie. new window, self)
> *   Added option to specify the hright and width of the thumbnail
> *   Added option to reset all options to their defaults
> *   Updated descriptions on the Options page.
>
> **0.5 (2007-10-24):**
>
> *   Added error reporting. Errors will be displayed as a replacement to the image in the post.
> *   Added images for use when displaying the above errors.
> *   Added error reporting for GD Module. Errors will be displayed on the options page.
> *   Added functionality to opt in to using encoded URLs.
> *   Fixed case sensitvity in regex used to match the img tag (now case insensitive).
> *   Removed large amounts of uneeded code in the Polaroid-o-nizer script.
> *   Converted all files to UNIX format.
>
> **0.4 (2007-10-03):**
>
> *   Added options page (Options->Polaroid on the Fly)
> *   Added functionality to opt in to security restrictions
> *   Added functionality to not require the use of tinyurl.com and fopen
> *   Added functionality to display the photo caption using the alt text
> *   Added support to change the background color from the options page
> *   Removed requirement for .htaccess
> *   Renamed the Polaroid-o-nizer files
> *   Changed references to Polaroid-o-nizer files
>
> **0.3 (2007-09-18):**
>
> *   Initial Public Release

**<big>To Do</big>**

1.  Add features for x and y offset
2.  Add features for rotation angle
3.  Add additional error reporting for the offset and angle
4.  Add functionality to options page to generate a URL for placement elsewhere on the site.
5.  Add functionality to cache the polaroid images.

**<big>Download</big>**
[Polaroid on the Fly version 0.7][5]
[Archived Versions][6]

 [1]: http://polaroidonizer.b3ta.org/
 [2]: http://picasa.google.com/
 [3]: http://wordpress.org/extend/plugins/wp-polaroidonizer/
 [4]: http://wordpress.org/extend/plugins/lightbox-2-wordpress-plugin/
 [5]: http://downloads.wordpress.org/plugin/polaroid-on-the-fly.0.7.zip
 [6]: http://wordpress.org/extend/plugins/polaroid-on-the-fly/download/