text:
# moduleSwift

![Swift](https://img.shields.io/badge/Swift-4.1-brightgreen.svg)

Generamba template with use Swinject and SwinjectStoryboard for DI, LightRoute for routing, Quick and Nimble for testing:

* [Swinject](https://github.com/Swinject/Swinject) (Dependency injection framework for Swift)
* [SwinjectStoryboard](https://github.com/Swinject/SwinjectStoryboard) (Swinject extension for automatic dependency injection via Storyboard)
* [LightRoute](https://github.com/SpectralDragon/LightRoute) (Helpers for creation proper module connections and routing)
* [Quick](https://github.com/Quick/Quick) (The Swift (and Objective-C) testing framework)
* [Nimble](https://github.com/Quick/Nimble) (A Matcher Framework for Swift and Objective-C)

It's a shared template for [Generamba](https://github.com/rambler-digital-solutions/Generamba) code generator.

## Configure

add template to Rambafile:
```
...
- {name: viperSwift, git: 'https://github.com/petroladkin/viperSwift'}
...
```

run in root project folder:
```
> generamba template install
```

## Usage

Run command to create a new module
```
> generamba gen <module_name> viperSwift
```

by default, generamba will generate view class with a parent of UIViewController, but to customize you can add a view_type parameter.
```
> generamba gen <module_name> viperSwift [--custom_parameters view_type:<view_type>]
```

view_type:
- [x] 'view' - view class has parent UIViewController class
- [ ] 'table' - view class has parent UITableViewController class
- [ ] 'tabbar' - view class has parent UITabBarController class
- [ ] 'nav' - view class has parent UINavigationController class

The detailed information about a template structure is available in [Generamba Wiki](https://github.com/rambler-digital-solutions/Generamba/wiki/Template-Structure).

repository: petroladkin/generamba-catalog

repo_name: petroladkin/generamba-catalog


text:
## Manage My Budget

Managing budget for vulnerable people.

### Requirements
The following version of the tools are required for this project:
```
Lerna v3.18.4
Terraform v0.12.13
Typescript Version 3.7.2
```
### Installation
To install, run:
```
lerna bootstrap
```

### Contributing

### License
[MIT](https://choosealicense.com/licenses/mit/)

license: MIT

repository: zuhlke/ManageMyBudget

branch_name: refs/heads/master

extension: md

gha_language: TypeScript

license_type: permissive

repo_name: zuhlke/ManageMyBudget


text:
# ircb

[![Build Status](https://travis-ci.org/aerth/ircb.svg?branch=master)](https://travis-ci.org/aerth/ircb)

friendly channel bot

  * karma system
  * http title parser
  * bot master must be identified with services (uses NickServ ACC)

## help

See [docs](https://aerth.github.io/ircb/) and [GoDoc](https://godoc.org/github.com/aerth/ircb/lib/ircb) and visit `##ircb` on Freenode for help

### usage

for upgrade command to function as expected:

  * 'makefile' should exist next to 'ircb' executable.
  * running 'make' should output new executable 'ircb' or return non-zero error

quick setup:

```

go get -d -v github.com/aerth/ircb
cd $(go env GOPATH)/src/github.com/aerth/ircb
make
cp default.json config.json
vim config.json
./ircb

```

#### authentication

send two private messages to bot: `/msg bot !help` and `/msg bot $upgrade`

the first message from master, ircb will check if user is identified with services
if so, you will be 'authenticated' for 5 minutes

the second command, it will try to fetch newest source code and rebuild and redeploy itself
if fails, should private message master reason

#### commands

See p.go in ./plugins/ directory for an example plugin

Build the plugin.so, move next to ircb executable, and send `$update-plugins` command to ircb

repository: aerth/ircb

repo_name: aerth/ircb


text:
Synata's SparkGraphAnalysis class
=========================

Tutorial
=========================
The easiest way to get going is to follow our [Tutorial](Tutorial.md)

Useage
=========================
```
sbt "run load <Gmail Access Token>"
```

This will dump your gmail and load metadata into Mongo.
The easiest way to get your Gmail Access Token is with the Gmail API Playground. Go to:
https://developers.google.com/gmail/api/v1/reference/users/messages/list
and under "Try It" click the "Authorize Oauth" button. Open the developer console, and find the request that ends with /me - Look for the header:

```
"Authorization: Bearer <TOKEN>"
```

Copy and paste <TOKEN> into the command.

```
sbt "run analyze"
```

This will run the actual graph analysis (assuming you have data in Mongo)

TODO
=========================
1. Add more configuration options
2. Add retries for usage throttling to the Gmail Loader

repository: uberbrodt/SparkGraphAnalysis

gha_language: Scala

repo_name: uberbrodt/SparkGraphAnalysis


text:
# frost-link <br />
* [API](#api)
* [Examples](#examples)

## API

| Attribute | Type | Value | Description |
| --------- | ---- | ----- | -----------
| `priority` | `string` | `primary` | primary link - opens content in a new tab |
|  |  | `secondary` | secondary link - opens content in the same tab |
| `design` | `string` | `inline` | custom link styling for in-line text, opens content in the same tab.  Should not be used with `priority` and `size`. |
|  | | `info-bar` | custom link styling for the info-bar context, opens content in the same tab.  Requires `icon` to be specified.  Should not be used with `priority` and `size`. |
| `size` | `string` | `small` | small size link |
|  |  | `medium` | medium size link |
|  |  | `large` | large size link |
| `disabled` | `boolean` | `false` | **default** - basic link |
|  |  | `true` | disabled link |
| `autofocus` | `boolean` | `false` | **default** - basic link |
|  |  | `true` | link in focus |
| `icon` | `string` | `<icon-name>` | the name of a frost-icon |
| `hook` | `string` | `<unique-name>` | name used for testing with ember-hook |
| `onClick` |`string` | `<action-name>` | triggers associated action when the link is clicked prior to transition |

## Examples

### Primary - small
```handlebars
{{#frost-link 'route name'
  priority='primary'
  size='small'
}}
  Primary
{{/frost-link}}
```

### Primary - medium
```handlebars
{{#frost-link 'route name'
  priority='primary'
  size='medium'
}}
  Primary
{{/frost-link}}
```

### Secondary - large
```handlebars
{{#frost-link 'route name'
  priority='secondary'
  size='large'
}}
  Secondary
{{/frost-link}}
```

### Disabled
```handlebars
{{#frost-link 'route name'
  priority='primary'
  size='small'
  disabled=true
}}
  Primary
{{/frost-link}}
```

### Design - in-line (font based on size)
```handlebars
{{#frost-link 'link.min'
  design='inline'
}}
  link
{{/frost-link}}
```

### Design - info-bar
```handlebars
{{#frost-link 'route name'
  design='info-bar'
  icon='icon'
}}
  Action
{{/frost-link}}
```

repository: sophypal/ember-frost-core

repo_name: sophypal/ember-frost-core


text:
# fa_dict_info

**dict_info** works on fanuc robot controllers. Sometimes I have the problem, that I do not know the IDs of an dictionary element. So I wrote me a program to look up the ids for me.

## Install

The KAREL program creates a CGI style webservice. Some settings has to be done bevor the nodejs client can connect to the robot controller.

1. create a .env file:
```
URL=<robot url>
DICT=<directory name>
```
2. install all nodejs packages
```
npm install
```
3. create the .PC file (in this eample the pc file is dictinfo.pc named)
```
ktrans.exe dictinfo.kl
```
4. copy the .PC file to the robot contoller
```
ftp <robot_url>
ftp> get dictinfo.pc
ftp> quit
```
5. open the KAREL Browser execution on the controller in the HostComm Settings
6. start the application
```
npm start
```

This application jjust runs until a -1 id is returned from the robot controller. This means, that there is an error or no data is found. This is just a very small example so you can try to make a better use.


text:
# Learning Journal: Code 301, Day 6
###### Mon, Oct 30, 2017

I think today is the first lecture where I tried to pay attention the whole time, and I still didn't get the majority of the concepts. Usually, I just do the readings and then I can multitask during lecture, and I still feel like I have a good grasp of what we're learning.  This must be what people felt like in the middle of 201.

Things I only have a nebulous concept of:
* Servers
* MVC
* API
* Ajax
* Ajax with JSON
* Ajax with HTML vs Ajax with JSON
* HTTP
* REST

Amazingly, we covered almost all of these things this afternoon, haha. Anyway, before my head explodes, I'm going to go focus on really important things like what I'm going to wear for Halloween tomorrow.

repository: strawbee/daily-learning-journals

repo_name: strawbee/daily-learning-journals


text:
# ed25519-dalek-rustgo

[![Documentation](https://godoc.org/github.com/FiloSottile/ed25519-dalek-rustgo/edwards25519?status.svg)](http://godoc.org/github.com/FiloSottile/ed25519-dalek-rustgo/edwards25519)

> This repository is a demo for [rustgo](https://blog.filippo.io/rustgo), a technique to directly call Rust code from Go programs with near-zero overhead, meant to replace manually written assembly.
>
> You'd probably enjoy [the article](https://blog.filippo.io/rustgo) more.

edwards25519 implements operations on an Edwards curve that is isomorphic to curve25519 by wrapping the excellent pure-Rust [curve25519-dalek](https://github.com/isislovecruft/curve25519-dalek) library.

 Benchmarks show this package to be more than 3 times faster than [a pure-Go alternative](https://godoc.org/github.com/agl/ed25519/edwards25519#GeScalarMultBase).

## Installation

To install using pre-built artifacts, download a tarball from [the Releases page](https://github.com/FiloSottile/ed25519-dalek-rustgo/releases) matching your Go version and untar it into `$GOPATH`. (The `_haswell` version is about 10% faster, but will only run on Haswell CPUs and newer.)

```
tar -C$GOPATH -xvf ed25519-dalek-rustgo_go1.8.3.tar.gz
```

That's it, you're ready to use the `github.com/FiloSottile/ed25519-dalek-rustgo/edwards25519` package.

### Manual installation

To build from source, you'll need Go, Rust, make and a linker.

Simply run `make install` from the root of this repository, which must be checked out at the right place in `$GOPATH`.

Cross-compilation is supported and based on `GOOS`/`GOARCH`. You'll need Rust setup and a linker for your target, specified with the `LD` variable. Set the `MUSL` environment variable to target `x86_64-unknown-linux-musl` for `linux_amd64`.

```
MUSL=1 GOOS=linux make clean install LD=x86_64-linux-musl-ld
```

The default `RUSTFLAGS` will target your native CPU.

## Usage

Just what you're used to. Import it, call its functions, `go build` your program.

Cross-compilation works normally, as long as a `.a` was installed for the target. Binary distributions currently include darwin_amd64 and linux_amd64 pre-built archives.

The only function currently exposed is multiplication of a scalar by the Curve25519 basepoint into an Edwards compressed point. PRs to add more APIs are welcome.


text:
---
title: "1950s-Inspired Dress"
date: 2020-10-31
author: Creative
author_profile: true
layout: archive
excerpt: "1950s-inspired dress with removeable belt and petticoat."
header:
  teaser: https://emhastings.github.io/images/50s-dress/front.jpg
flexgallery:
  - aspect: "0.7467"
    url: https://emhastings.github.io/images/50s-dress/front.jpg
    image_path: https://emhastings.github.io/images/50s-dress/front-th.jpg
    alt: "Front of the dress"
  - aspect: "0.7467"
    url: https://emhastings.github.io/images/50s-dress/back.jpg
    image_path: https://emhastings.github.io/images/50s-dress/back-th.jpg
    alt: "Back of the dress"
  - aspect: "0.7467"
    url: https://emhastings.github.io/images/50s-dress/petti.jpg
    image_path: https://emhastings.github.io/images/50s-dress/petti-th.jpg
    alt: "Petticoat"
    end_row: "true"
  - aspect: "0.5617"
    url: https://emhastings.github.io/images/50s-dress/model.jpg
    image_path: https://emhastings.github.io/images/50s-dress/model-th.jpg
    alt: "Me in the dress"
  - aspect: "0.7505"
    url: https://emhastings.github.io/images/50s-dress/makeup.jpg
    image_path: https://emhastings.github.io/images/50s-dress/makeup-th.jpg
    alt: "Hair and makeup"
  - aspect: "0.6481"
    url: https://emhastings.github.io/images/50s-dress/inspo.jpg
    image_path: https://emhastings.github.io/images/50s-dress/inspo.jpg
    alt: "The extant dress"
    end_row: "true"
---

I made this dress and removeable belt based on McCall's pattern M6557 to emulate an extant dress (see last image). The petticoat is based on [this tutorial by The Sewing Lab](http://www.thesewinglab.net/2012/08/tutorial-petticoat.html). The outfit was started in the costume shop in roughly 2014-2015, and finished in 2020. I attempted a 50s-inspired makeup look to wear with the dress.

{% include flexgallery caption="Last image from https://i.pinimg.com/564x/29/8e/10/298e10b37a72225be50ef27ebd81dfc4.jpg?fbclid=IwAR2Aj5d9H4Wa_vOtqRc24NDilCfT3LiFgx0Sht_BVjljAQuFRTtPP5TaZek" %}


text:
# Panta Rhei - InfluxDB - Adapter

#### This is the Adapter between [Panta Rhei](https://github.com/iot-salzburg/panta_rhei) messaging system and this [DataStack](https://github.com/iot-salzburg/datastack).

The adapter is based on the Panta Rhei (or Digital Twin) client which allows the multi-tenant streaming of data easily.

```python3
from client.digital_twin_client import DigitalTwinClient
config = {"client_name": "demo_app1", "system_name": "demo-system",
          "kafka_bootstrap_servers": "localhost:9092", "gost_servers": "localhost:8082"}
client = DigitalTwinClient(**config)
client.register(instance_file="digital_twin_mapping/instances")
client.send(quantity="demo_temperature", result=23.4)
```

## Contents

1. [Requirements](#requirements)
2. [Quickstart](#quickstart)
3. [Deploy on a Cluster](#deployment)

## Requirements

* A running instance of InfluxDB, set it up is described [here](../InfluxDB_Grafana/README.md).
* A running [Panta Rhei](https://github.com/iot-salzburg/panta_rhei) messaging system with data.

## Quickstart

This is an instruction on how to set up a demo scenario on your own hardware.
Here, we use Ubuntu 18.04.

```bash
cd influxdb-adapter
git clone https://github.com/iot-salzburg/panta_rhei.git adapter/panta_rhei
```

Configure the adapter in the `docker-compose.yml`:

```yaml
version: '3.4'
services:
  datastore-adapter:
    image: <IP_ADDRESS>:5001/datastore-adapter
    build: .
    network_mode: host
    environment:
      # InfluxDB configuration
      INFLUXDB_HOST: "localhost"
      # Panta Rhei configuration
      CLIENT_NAME: "influxdb-adapter"
      SYSTEM_NAME: "at.srfg.iot.dtz"
      SENSORTHINGS_HOST: "<IP_ADDRESS>:8082"
      BOOTSTRAP_SERVERS: "<IP_ADDRESS>:9092,<IP_ADDRESS>:9093,<IP_ADDRESS>:9094"
```

To start the adapter, run the following command:

```bash
python3 influxdb-adapter/adapter/influxdb_adapter.py
```
This step requires a reachable InfluxDB endpoint on [localhost:8086](localhost:8086).
In this connector, the package [influxdb-python](https://influxdb-python.readthedocs.io/en/latest/include-readme.html)
was used. The used InfluxDB table is called `at.srfg.iot.dtz`.

Check if data is stored in `InfluxDB` using this script
 (it counts the number of rows in a set database):

 ```bash
# using curl
curl -G 'http://localhost:8086/query?db=at.srfg.iot.dtz' --data-urlencode 'q=SELECT * FROM "at.srfg.iot.dtz"'
#> {"results":[{"statement_id":1234}]}

# using a python script:
python3 InfluxDB_Grafana/db_interface.py
#> {'time': '1970-01-01T00:00:00Z', 'count_result': 321825}
```

## Deployment

Make sure you have a running [Docker Swarm](https://www.youtube.com/watch?v=x843GyFRIIY).
Add the service `datastore-adapter` with:

```bash
docker-compose up --build -d
```

Now, data from the **Panta Rhei system** will be streamed into the **DataStack**.

repository: iot-salzburg/datastack

repo_name: iot-salzburg/datastack


text:
# Running API tests for Loginbuddy

Loginbuddy uses SOAPUI for testing APIs. I got stuck with SOAPUI for many years and, although the name may not indicate it, it is good at testing REST APIs, too.

SOAPUI (Open Source) can be downloaded [here](https://www.soapui.org/downloads/soapui.html).

To run the tests a few pre-requisites are required:

- Update the hosts file and add these entries (you may already have some of them configured)
  - *<IP_ADDRESS> loginbuddy-sidecar loginbuddy-oidcdr local.loginbuddy.net democlient.loginbuddy.net demoserver.loginbuddy.net soapui.loginbuddy.net*
- Launch SOAPUI and update these preferences:
  - check *Preferences - HTTP Settings - Pre-Encoded Endpoints*

## Run tests

Next to this file there are three directories:

- **docker:** this contains a docker-compose file and configuration, used with the tests
- **scripts:** this was only used when adding support for *response_type=id_token*
- **soapui:** this contains SOAPUI project and properties files

To run tests follow these steps:

- launch the Loginbuddy test setup
  - ```$ cd ./docker```
  - ```$ docker-compose -f docker-compose-test.yml up```
- Launch SOAPUI
- Import the project files found at *./soapui/project*
- Double-click each project, select *TestSuites* and hit the green arrow
- Verify that everything is green
- You can also open the browser at *https://democlient.loginbuddy.net* for manual verifications. Not everything is automated yet!
- Done!

## Configure SOAPUI

Both SOAPUI projects are using properties instead of hard coded values. These can be found here:

```$ ./properties/template.properties```

If you want to use your own properties, simply copy that file and load them info SOAPUI for each project.

## Known issues

- The SOAPUI project *Loginbuddy-Flows* has many duplicated test steps. This requires some effort to keep them in sync. When I get to it, I will do some refactoring
- Not everything is automated, but running the test indicates a high chance that most features are working as expected
- Manual verifications are required via the browser UI

## Tips and tricks for using SOAPUI

Here are a few notes for using SOAPUI.

### Extract header from previous response and stick into variable

    //Find the 'Location' header of the response
    def location = testRunner.testCase.testSteps["test-step-name"].testRequest.response.responseHeaders["Location"][0]

    // Write the 'location' into the 'endpoint' of the testStep 'test-step-target'
    def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
    groovyUtils.setPropertyValue("test-step-target", "Endpoint", location.toString())

### Extract header from previous response and assert value

    //Find the 'Location' header of the response
    def location = testRunner.testCase.testSteps["test-step-name"].testRequest.response.responseHeaders["Location"][0]
    assert location.startsWith('value-to-assert');

### Extract header from current response

    // Find the 'Location' header of the response
    assert messageExchange.responseHeaders["Location"] != null

    def location = messageExchange.responseHeaders["Location"][0]

    // Check if all expected parameters are included in the redirect_uri
    assert(location.contains("client_id"));

### Access message body of current test step

    def bodyAsString = new String().valueOf(messageExchange.responseContent)

repository: Pasteis/loginbuddy

repo_name: Pasteis/loginbuddy


text:
---
layout: post
title:  "Big Data Processing with Apache Spark - Part 1"
date:   2020-09-25
excerpt: "How Spark came to my rescue"
tag:
- sample post
- post
- images
comments: true
---

# Take your data processing to the next level

## What led me to discover Apache Spark

I am an experienced Data Scientist and I have been at the forefront of managing the entire data science project life cycle including defining project scope, identifying data requirements and acquisition, data cleaning, feature engineering, building, evaluating and deploying machine learning models.

This time around in one of the projects, I had to analyze the high frequency data and build a machine learning model pipeline. I started with a sample to understand the data structure.

But soon I realized it was not possible to scale what I was doing to the entire dataset. Talking to my peers I came across 1. Dask, an open source library for parallel computing written in Python and 2. Koalas, which is the pandas DataFrame API for handling Big Data. Very recently Koalas has been highlighted as a good alternative to PySpark as there is no learning curve for the user if you already know pandas. Check: [Koalas - pandas API on Apache Spark](https://github.com/databricks/koalas).

In this blogpost I will take you through the Spark architecture to understand how is it able to handle big data processing and we will go through a used case together in the next blogpost. I cannot discuss about the project that I was working on because of the client confidentiality but we can definitely play around with open source datasets :)
That is the power of being a Data Science practitioner, nothing can stop you from learning if you want to.

Let's start

![](../imgs/apache_spark.png)

## How is Spark handling big data processing

Traditionally when we were working only on our laptops we were limited to the storage capacity and the processing power as per the configuration of the laptop.

Since the inception of cloud we are able to scale up our storage and processing and having second thoughts about upper limit on both is almost out of scope now. Be it Azure or AWS you just pay for as much as you want.

But what is it that Spark is doing to get complex processing run on such high amount of datasets. The answer lies in parallelization.

Let us understand with an example, let's say if I am given a task to count the M&Ms in a gift box which I got on my birthday, I can do it pretty easily, right.
This is same as when you have a usual CSV file such as the infamous Titanic dataset which we can easily handle with the Pandas and do the manipulations.
On the other hand if now I am given a task to count the number of M&Ms in the store next to my house, I will hesitate a bit but after spending a day or two dedicatedly, I can do it. It is slow but still something I can handle.
Now if I am asked to count the number of M&Ms in every store in Vancouver, I give up right away. I cannot handle this task myself, what I can do instead is call all my UBC friends to go to the stores near their house and give me a count of M&Ms. In the end what I will do is collect all the counts from all my UBC friends and sum them all.

**M&Ms in my gift box**

![](../imgs/mnm_image.jpeg)

**M&Ms in whole of Vancouver**

| Orphelia | Aditya | Sayanti | Akansha |
| ------ | ------ | ------ | ------ |
| ![](../imgs/mnm_image.jpeg) | ![](../imgs/mnm_image.jpeg) |![](../imgs/mnm_image.jpeg)| ![](../imgs/mnm_image.jpeg)|

What happened here, I split the task into the hands of my friends(equivalent to the Workers in Spark) and then aggregated the final results. Note one more thing, we encourage our friends to not interact till they reach the final count of their share of M&Ms, that means no information is exchanged and no gossip about who is having higher count is happening here. Any gossip amongst them has the potential of slowing down the process.
This is what is happening in Spark. It splits the data in partitions which lie on different cluster nodes you can simple say different laptops and when you execute some operations they are executed in parallel fashion and we aggregate the results in the end.

Now some one may ask what is so different from MapReduce, it was doing the same parallelization into Map and Reduce steps. That is a valid question and I will answer that in the upcoming section.

Let us understand the architecture of the Spark and some essential components.

## Apache Spark architecture

Apache Spark is an open-source distributed general-purpose cluster-computing framework.  Instead of trying to process computationally-expensive programs on a single computer, these tasks can be divided between multiple computers that communicate with each other.

This is how the Spark Eco System looks like.

![](../imgs/spark_ecosystem.png)

### Essential components of the ecosystem are:

**1. Spark Core**

Spark Core is the heart of the Apache Spark framework. It holds the components for task scheduling, fault recovery, interacting with storage systems and memory management. Spark Core is also home to the API that defines resilient distributed datasets (RDDs), which are Spark’s main programming abstraction. RDDs represent a collection of items distributed across many compute nodes that can be manipulated in parallel. Spark Core provides many APIs for building and manipulating these collections.

**2. Spark SQL**

Spark SQL is Spark’s package for working with structured data.It allows querying data via SQL as well as the Apache Hive variant of SQL—called the Hive Query Language (HQL)—and it supports many sources of data, including Hive tables, Parquet, and JSON.

**3. Spark Streaming**

Spark Streaming is a Spark component that supports scalable and fault-tolerant processing of streaming data. It uses Spark Core's fast scheduling capability to perform streaming analytics. A common example is processing log data from a website or server.

**4. Spark GraphX**

GraphX is the Spark API for graphs and graph-parallel computation. To manipulate graph, it supports various fundamental operators like subgraph, join Vertices, and aggregate Messages.

**5. Spark Machine Learning(MLlib)**

Spark comes with a library containing common machine learning (ML) functionality, called MLlib. MLlib provides multiple types of machine learning algorithms, including classification, regression, clustering, and collaborative filtering, as well as supporting functionality such as model evaluation and data import. All of these methods are designed to scale out across a cluster.

**6. Spark R**

It is an R package that provides a distributed data frame implementation. It also supports operations like selection, filtering, aggregation but on large data-sets.

Under the hood another very important component is the **Cluster manager**.
Spark is designed to efficiently scale up from one to many thousands of compute nodes. To achieve this while maximizing flexibility, Spark can run over a variety of cluster managers, including Hadoop YARN, Apache Mesos, and a simple cluster manager included in Spark itself called the Standalone Scheduler.

### Data Representation

**Spark Resilient Distributed Dataset (RDD)**

RDDs are the fundamental data structures of Spark or we can say the building blocks of any Spark application. RDDs Stands for:

*Resilient*: Fault tolerant and capable of rebuilding data on failure
*Distributed*: Distributed data among the multiple nodes in a cluster
*Dataset*: Collection of partitioned data with values

There are two ways to create RDDs − **parallelizing** an existing collection in your driver program, or **referencing a dataset** in an external storage system, such as a shared file system, HDFS, HBase, or any data source offering a Hadoop Input Format.
Spark makes use of RDDs to achieve the same or similar processing results as MapReduce but at a much faster speed usually of the order by 10 to 100x.

**MapReduce Vs Spark**

MapReduce has established itself as one of the best technologies to process and generate huge datasets parallelly using distributed algorithms in distributed environments. It helps users and developers to do parallel computations using high-level APIs and the users do not have to get involved and take care of the intricacies of work distribution, concurrency issues and making the processing systems fault-tolerant.

When we need to reuse data between different computation stages say in MapReduce processing, the only way to do this is to write the output of the first stage to physical storage e.g. HDFS. Both the iterative and interactive applications need data sharing to be very fast across parallel jobs. But the data sharing is not that fast in MapReduce due to disk IO, serialization, deserialization, and replication involved in writing to stable storage for intermediate results. It is found that Hadoop applications spend almost 90% of their time in reading and writing to storage systems.

Iterative Operations on MapReduce

![](../imgs/mapreduce_memory_storage.jpg)

In interactive operations, the user runs an ad-hoc query on the same subset of data and each time the query will go to the disk and perform IO to fetch the data and return back to the user. This increases the interactive query time for the user and hampers the user experience.

To overcome this slowness Spark was developed which works on the concept of Resilient Distributed Datasets which essentially are in-memory objects but are partitioned and distributed on a cluster and also fault-tolerant. So the intermediate stage outputs do not require to be written to stable storage and can be accessed from memory avoiding the biggest bottleneck of the MapReduce systems. This in-memory sharing makes Spark 10 to 100 times faster than MapReduce.

Let us now see how iterative and interactive operations take place in Spark’s Resilient Distributed Datasets.

Iterative Operations on Spark RDD

![](../imgs/rdd_storage_inmemory.jpg)

The diagram above shows how Spark’s RDD work in iterative applications. The intermediate results are written to memory instead of the stable disk storage and the subsequent steps can read the same memory RDD objects. Only when the memory(RAM) is insufficient to store the whole RDD, the results are spilled to the disk. But overall the whole system is way faster than the MapReduce application due to this optimization

When to use RDDs:

- Low-level transformation and actions and control on dataset
- Data is unstructured, such as media streams or streams of text
- No requirement of imposing a schema, such as columnar format, while processing or accessing data attributes by name or column
- Can afford to forgo some optimization and performance benefits available with DataFrames and Datasets for structured and semi-structured data.

**Spark DataFrames**

Spark DataFrames came into focus in the recent releases of Spark, DataFrame is an immutable distributed collection of data. Unlike an RDD, data is organized into named columns, like a table in a relational database. Designed to make large data sets processing even easier, DataFrame allows developers to impose a structure onto a distributed collection of data, allowing higher-level abstraction

**Spark Datasets**

Datasets in Apache Spark are an extension of DataFrame API which provides type-safe, object-oriented programming interface.

When to use DataFrames or Datasets?

- For rich semantics, high-level abstractions, and domain specific APIs, use DataFrame or Dataset.
- For high-level expressions, filters, maps, aggregation, averages, sum, SQL queries, columnar access and use of lambda functions on semi-structured data, use DataFrame or Dataset.
- Higher degree of type-safety at compile time, want typed JVM objects, take advantage of Catalyst optimization, and benefit from Tungsten’s efficient code generation, use Dataset.
- Unification and simplification of APIs across Spark Libraries, use DataFrame or Dataset.
- If you are a R user, use DataFrames.
- If you are a Python user, use DataFrames and resort back to RDDs if you need more control.

### Working of Spark Architecture

 Spark has a concept of a master and slave(worker) nodes. The master node has a driver program which drives the application.
The first thing Driver node does is to create a Spark Context which is an entry point to any Spark functionality just like the Database connection. Spark Context works with the Cluster Manager to manage various jobs. The Driver and the Spark Context take care of the jobs on the clusters. Any job is split into multiple tasks which are split across the worker nodes. The worker nodes are responsible for executing the task assigned to them. With more workers you can split the data into more partitions.

![](../imgs/spark_architecture1.png)

The driver talks to the cluster manager and negotiates the resources. Cluster manager launches executors in worker nodes on behalf of the driver. At this point, the driver will send the tasks to the executors based on data placement. When executors start, they register themselves with drivers. So, the driver will have a complete view of executors that are executing the task.
During the course of execution of tasks, driver program will monitor the set of executors that runs. Driver node also schedules future tasks based on data placement.

### RDD Operations

The Spark operations are combination of **Transformation** and **Actions**. Transformations create new RDDs from existing dataset and Actions return a value after running a computation on the dataset. For instance *MAP* is a transformation that passes each element of the dataset into the function but *REDUCE* is the action that aggregates all the elements of the RDD and passes through a function and returns the final result to the driver program.

All the transformations in Spark are lazy, which means that they do not compute the results right away. Instead they just memorize the series of transformations that are acting on a dataset. These transformations are computed only when a action requires a result to be returned to the driver program. When we run the MAP transformation followed by REDUCE action the large mapped dataset is not returned to the Driver rather only the final computation result from the REDUCE is sent. This makes execution through Spark much more efficient.

![](../imgs/transformations_actions.png)

I hope this overview gave you better understanding of the working of Spark. In the next part we will work on a used case for better understanding of how a large dataset is partitioned and processed with Spark.

## References
1. [Spark_Architecture](https://www.edureka.co/blog/spark-architecture/)
2. [Getting_started_with_Spark](https://pages.databricks.com/rs/094-YMS-629/images/Databricks-on-AWS-01-Getting-Started-Apache-Spark-Slides.pdf)
3. [Resilient Distributed Dataset (RDDs)](https://www.knowledgehut.com/tutorials/apache-spark-tutorial/resilient-distributed-datasets)
4. [RDDs Vs DataFrames Vs Datasets](https://databricks.com/blog/2016/07/14/a-tale-of-three-apache-spark-apis-rdds-dataframes-and-datasets.html)
5. [Spark Operations](https://training.databricks.com/visualapi.pdf)


text:
# Single page application

The app is built with the Django framework based on course materials from zenva.com course "The Complete Responsive Web Design Course" (marked as obsolete in Jan 2021). After this course, I cannot recommend this portal as a good place for learning. But the intellectual property, of course, is coded in this site.

The general purpose of this file is to demonstrate the whole Web Development eco system needed to develop and test any product.

For demonstration this Django project has been kept as simple as possible but with usage of most common development tasks.
 - File upload
 - DataBase migration
 - Manage of static files from proxy

# Config

List of Environemnt variables used to configure the app.

Some may be provided in `.env` files loaded in section `env_file`  inside docker-compose-\*.yml file.
ENV file is a list of environemnt variables that you don't want to repeat and store it once. Structure of the files looks liek this
```.env
SECRET_KEY = 'ChangeME!'
ALLOWED_HOSTS = '<IP_ADDRESS>, localhost`
DEBUG = False
DB_NAME = 'zspa'
DB_USER = 'user'
DB_PASS = 'password'
DB_HOST = '<IP_ADDRESS>'
DB_PORT = '3306'
googleForm = 'https://docs.google.com/forms/d/e/1FAI....
googleMaps = 'https://www.google.com/maps/embed?pb=!1m18...
```

Variable | Description | Deafult
------------ | ------------- | -------------
DEBUG | SECURITY WARNING: don't run with debug turned on in production! | False
SECRET_KEY | *SECURITY WARNING:* keep the secret key used in production secret! | SECURITY WARNING: No Default Value
ALLOWED_HOSTS | Ip or/and DNS name of the server | *SECURITY WARNING:* No Default Value
DB_NAME | Name of databese in database server | zspa
DB_USER | Name of user that have access to database | user
DB_PASS | Password for the database user | password
DB_HOST | IP or/and DNS name of the database server | localhost
DB_PORT | Port to connect to the database | 3306
MIGRATE | Execute migration on startup. Default docker command applied in `scripts\entrypoint.sh` | false
googleForm | Link to google form for contact purpose | Empty
googleMaps | Link to google maps of your company place | Empty

## NGINX

Used as proxy to serve website and static files from the same host.
Aplied configuration allow to upload images up to 5M. Can be changed in proxy\default.conf `client_max_body_size` variable. \
Default port to listen is 8080.

## Database

Database MariaDB or MySQL can be hosted locally on docker or remote server. \
Migration to database can be done with usage of predefined default values in `settings.py` or to server defined in `.env`.
With spread out environment this should be triggered only on demand.
Default for execution data base migration is set to false.

*SECURITY WARNING:* Super User for django admin portal must be done manually via localhost manage.py file commands configured to local or remote databese.

## Docker

Configuration files provided in docker directory are mean to be used for development.
Docker files can be used to test application, deploy it on remote test environment and self
host necessary tools that were used for development.

Default `docker-compose.yml` contains configuration in debug mode for web app development.
Second configuration file `docker-compose-deploy.yml` use NGINX as proxy server for testing in Debug mode on or off.
Third file `docker-compose-remote.yml` is provided to host the app on different environment than localhost.
It can be called alfa, beta, integration or simple develop environment.
Many companies name it accordingly for their development process.

Some environment variables can be hold in dot env files.

There isn't `docker-compose-prodcution.yml` as this app is for learning purpose and this part
you can do on your own if you see value in this code to push it beyond.

# Development

It should be noticed that this repository is for learning purpose and the app is far from ready to go live production. This should demonstrate complex aspect of DevOps work needed to self host web application.

## Localhost

Development on local machine support a few solutions which differentiate with DataBase server.
As Django require some database to run and host necessary data in case of insufficient resources,
for local development may be used SQLite.

### Solution no.1
On local you can use simple \
`python manage.py runserver`
And work with DEBUG set to True in `.env` file. \
DataBase may be hosted on Docker if you not have other resources for remote one. \
In this type of work you have full Django featured debugging app to work with.
### Solution no.2
You can use `docker-compose.yml` file that binds to your local resources and listen on files changes.
It is also bundled with MySQL server. The preferred way as this will encourage usage of docker.

### Solution no.3
If you prefer to run python manage commands in your local terminal and
do not have MySQL DB server you can start `docker-compose.yml` and stop the web app.
DB will run without app. When you use all services you will have the same data inside.

## SonarQube

Static analysis tool that will keep guard on basic code style and catch issues.
In enterprise environment such tools hosted on company networks help with team work and collaboration.
Once defined rule set will guide all developers with the same standards and principles.
For stand alone development it will provide some feedback even if you code alone.
As SonarQube provide good dashboard to display and browse code we can use it along PR to check
if new code does not introduce any issues to code base.

Beside analyzing code it also measure code coverage.
For purpose of analysing and making valid file for sonarqube project use most popular module [coveragepy](https://github.com/nedbat/coveragepy).
Result of coverage script execution is a compressed xml file which is send to SonarQube Server.
Default Quality Gate require 80% code coverage.

To self host SonarQube service repository hold `docker-compose-sonarqube.yml` recipe.
Be advise that SQ server require 2gb of memory but 3gb is recommended.
If its applicable SQ may be hosted on different device inside your network.

File `sonarqube.sh` shell script can be executed before code is merged to Lead branch `master`..
To use the file a few settings must be provided as dotfiles.
- `.sonarHostUrl` provide IP/DNS address to SonarQube service.
- `.sonarLogin` contains private access token generated on new project registration. For development purpose
community edition is sufficient.

Each developer has different login token and may have different sonar address.

When those tools are used from beginning many issues are neglect or barely noticeable.

# Deployment

## Docker

The remote docker compose does not contains service for data base as on this level
 Data Base should be hosted on Robust and dedicated server.
In case of deploying the app on multiple nodes media static files are copied via rsync network of client-server relation.

Data base migration.\
To Migrate data in remote ENV the MIGRATION env variable may be used.
Connection from local machine to remote data base may be better way to handle any error
that may came along.
For this purpose on root level of project modify .env file with connection for remote DB
server and execute commands.

## Database

Should be used DataBase installed on dedicated machine that is secured to protect data.
For this reason `docker-compose-remote.yml` does not contain db service.
ENV file or inline environment variables should be provided instead.

## Rsync

For media files backup there is client server [Rsync](https://linux.die.net/man/1/rsync)
set with docker image of [eeacms/rsync](https://hub.docker.com/r/eeacms/rsync).
Basic usage you can find and learn [here](https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories).

### Client
Client that is configured in `docker-compose-remote.yml` will use ssh keys aut generated on start up.
The first handshake with server have to be done manually within container after you provide public key it the server.
This have to be done once after you add new machine to development environment.

Other solution is to mount volume to host keys like:
```yaml
   volumes:
      - static_data:/data
      - /etc/ssh/:/etc/ssh/
      - /root/.ssh/:/root/.ssh/
```
With this mounted keys form the host you inherit all known hosts and the handshake with new server will be valid inside docker container.

This can be achieved with simple dry run of Rsync command that provides address to your server.
```bash
rsync -e 'ssh -p 2222 -i ~/.ssh/erysnc' -anvx --numeric-ids root@<IP_ADDRESS>:/data/ /docker/ersync/data
```

Cron task will periodically push and pull images form the server. Maintaining backup for data.
In this example app we were practicing sync of images but this solution will backup any data for given directories.
In `docker-compose-remote.yml` you have to provide IP/DNS to your rsync Server.

### Server
Server `docker-compose-rsync-server.com` contains pre configured server that can be deployed on remote.
To connect client you have to add new environment key with name of `SSH_AUTH_KEY_{{nameOfTheClient}}` where
nameOfTheClient is for maintain purpose to easily identify keys in env variable list.

If server keys are refreshed you can purge keys on client machines with command
```bash
ssh-keygen -f "/root/.ssh/known_hosts" -R "[<IP_ADDRESS>]:2222"
```
Then you have to perform handshake and download latest public key of the server.

gha_language: Python

gha_license_id: MIT


text:
---
title: 'Does Cranberry Really Cure Urinary Tract Infections?'
date: 2021-05-02T15:04:10.000Z
description: >-
 Cranberries and cranberry-derived products are commonly used as a remedy for urinary tract infections, especially among women.
---

Cranberries contain proanthocyanidins (PACs), specifically A type (PAC-A). It is argued that this component of cranberries fights and prevents UTIs, just as an antibiotic would. However, there is much controversy on this topic.

## Clinical Trials and Studies on Cranberry for UTIs

Freire Gde C. completed a large review of 24 studies totaling 4,473 participants. In the research, there was minimal evidence suggesting that cranberry products prevented UTIs. Within those studies, cranberry juice decreased the number of UTIs in women who had recurring UTIs; however, the studies that showed a significant efficacy were small trials. When studies grew in size, the difference between the placebo groups and the cranberry groups shrank.

Furthermore, a Stapleton AE et al study completed a randomized control trial of 176 participants, of whom 120 were taking cranberry juice and 56 were given the placebo over the course of 168 days. The adherence to instruction was similar between both groups (91.8% and 90.3%). However, cranberry juice did not significantly reduce UTIs.

## Drug Interactions, Additive Effects and Side Effects

Lexicomp, a drug database used by medical professionals, still advises people to monitor cranberry intake while taking warfarin; however, there is little research to suggest the two adversely interact. Since cranberry juice is acidic, it is important to avoid drinking it within 2 hours of taking erythromycin, a common antibiotic, because it will decrease metabolism of the drug.

Common side effects of cranberry are nausea, vomiting, tightness in the chest, itching, cough, swelling of the face or lips, seizures, GI upset, and loose stools.

>The argument here is that for cranberry products to work, the cranberry product must be carefully authenticated to ensure that it indeed contains PACs-A, which is a component typically lost in processing.

repository: Marienoir/one-click-hugo-cms

gha_language: CSS

repo_name: Marienoir/one-click-hugo-cms


text:
# 49 - [GHOST](GHOST.md)
**E**thereum’s
**G**reedy
**H**eaviest
**O**bserved
**S**ubtree

Blocks take time to propagate through the network and multiple miners are simultaneously producing valid blocks.

This leads to the blockchain considering multiple blocks at the same level but ultimately choosing only one block at any level that creates the canonical blockchain.

This choice is dictated by **E**thereum’s **G**reedy **H**eaviest **O**bserved **S**ubtree (GHOST) protocol which includes stale blocks up to seven levels in the calculation of the longest chain.

Stale blocks are called uncles or ommers.

___
## Slide Screenshot
![049.png](../images/ethereum101/049.png)
___
## Slide Text
- Miners -> Blocks
- Network -> Propagation
- Multiple Valid Blocks
- Must Choose one block
- Greedy Heaviest Observed Subtree Protocol (GHOST)
- Canonical Blockchain
- Stale Blocks -> Ommers (Uncles)


text:
# Google Tag Manager Bundle

[![Package version](https://img.shields.io/packagist/v/norsys/google-tag-manager-bundle.svg?style=flat-square)](https://packagist.org/packages/norsys/google-tag-manager-bundle)
[![Total Downloads](https://img.shields.io/packagist/dt/norsys/google-tag-manager-bundle.svg?style=flat-square)](https://packagist.org/packages/norsys/google-tag-manager-bundle)
[![Build Status](https://img.shields.io/travis/M6Web/ApiExceptionBundle/master.svg?style=flat-square)](https://travis-ci.org/M6Web/ApiExceptionBundle)
[![Scrutinizer Coverage](https://img.shields.io/scrutinizer/coverage/g/norsys/google-tag-manager-bundle.svg?style=flat-square)](https://scrutinizer-ci.com/g/norsys/google-tag-manager-bundle/?branch=master)
[![Scrutinizer Code Quality](https://img.shields.io/scrutinizer/g/norsys/google-tag-manager-bundle.svg?style=flat-square)](https://scrutinizer-ci.com/g/norsys/google-tag-manager-bundle/?branch=master)
[![License](https://img.shields.io/github/license/norsys/google-tag-manager-bundle.svg?style=flat-square)](https://github.com/norsys/google-tag-manager-bundle/blob/master/LICENSE)

[![SensioLabsInsight](https://insight.sensiolabs.com/projects/eb4b735d-4782-4f2e-933f-916e20a2fb71/big.png)](https://insight.sensiolabs.com/projects/eb4b735d-4782-4f2e-933f-916e20a2fb71)

This project is a bundle which eases the configuration and handling of Google Tag Manager.

It provides a bunch of base classes to extend from.

Installation
============

Step 1: Add the repository & download the Bundle
------------------------------------------------

Open a command console, enter your project directory and execute the
following commands:

```bash
# download the latest stable version of this bundle:
$ composer require norsys/google-tag-manager-bundle
```

This command requires you to have `composer` installed globally, as explained
in the [installation chapter](https://getcomposer.org/doc/00-intro.md)
of the Composer documentation.

Step 2: Register the Bundle
---------------------------

Then, enable the bundle by adding it to the list of registered bundles
in the `app/AppKernel.php` file of your project:

```php
<?php
// app/AppKernel.php

// ...
class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            // ...
            new Norsys\GoogleTagManagerBundle\NorsysGoogleTagManagerBundle(),
            // ...
        );
        // ...
    }
    // ...
}
```

Step 3: Configuration & Implementation
--------------------------------------

The GTM bundle allows you to define google tag manager parameters, providing a default values management.

### Basic integration

```yaml
# app/config/config.yml
#...
norsys_google_tag_manager:
    id: 'GTM-XXXXXX'
```

#### Template integration

To ease quick integration, the bundle provides a `twig` partial and a javascript file to include in your template:

Example:

```twig
{# Google Tag Manager #}
{% block google_tag_manager %}
    {% include<EMAIL_ADDRESS>%}
    {{ asset('web/bundles/norsysgoogletagmanager/js/google-tag-manager.js') }}
{% endblock google_tag_manager %}
```

#### Controlling GTM loading

By default, GoogleTagManager is initialized at the point the javascript file is included.

You can also tied it to a specific DOM event:

```yaml
# app/config/config.yml
#...
norsys_google_tag_manager:
    id: 'GTM-XXXXXX'
    on_event:
        enabled: true
        name: load
        container: body
```

### DataLayer configuration

The GoogleTagManager Bundle handles 2 distinct types of parameters:
- [Static parameters](#usage:static)
- [Dynamic parameters](#usage:dynamic)

 > :warning: As a rule of thumb,  `static` and `dynamic` parameters must have different names.

#### Static parameters

Here is a very basic config example:

```yaml
# app/config/config.yml
#...
norsys_google_tag_manager:
    data_layer:
        default:
            static:
                virtualPageURL: '/home'

        pages:
            faq:
                static:
                    virtualPageURL: '/otherLink/faq'
```

The bundle will merge defaults and per-route static parameters, and issue the resulting data layer.

The dataLayer sent for the page responding to the `faq` route will be:

```javascript
    dataLayer = [ { "virtualPageURL": "/otherLink/faq" } ];
```

#### Dynamic parameters

On some situations, it can be useful to have some parameters value changing dynamically upon the context.

For this purpose, the bundle provides support for dynamic parameters.

Each dynamic parameter consists of a service implementing [NorsysGoogleTagManagerBundle\Dynamic\ParameterInterface](src/Dynamic/ParameterInterface.php).

The custom dynamic parameter class will have to implement the 2 following methods:
- `getValue()` The method responsible for calculating the parameters value. It receives the whole merged and resolved config as unique argument
- `getName()`  This method must return the unique parameter name, the one that represents it in the config

```php
# src/AppBundle/DynamicParameter/Acme.php
<?php

namespace AppBundle\DynamicParameter;

use Norsys\GoogleTagManagerBundle\Config\ParameterBag;

use Norsys\GoogleTagManagerBundle\Dynamic\ParameterInterface;

class Acme implements ParameterInterface
{
    public function getValue(ParameterBag $configPage): string
    {
        // do something to calculate value
        // ...

        return $calculatedValue;
    }

    public function getName(): string
    {
        return 'acmeDynamicParam';
    }
}
```

Each dynamic parameter class refered to in the bundle config must be registered, hence declared using the appropriate tag:

```yaml
# app/config/services.yml
#...
    app.google_tag_manager.config.dynamic_parameter.acme:
        class: AppBundle\DynamicParameter\Acme
        tags:
            - { name: 'norsys_google_tag_manager.dynamic_parameter' }

#...
```
Now we can use the parameter in the bundle config, referencing to it via the alias defined by its `getName()` method.

```yaml
# app/config/config.yml
#...
norsys_google_tag_manager:
    data_layer:
        default:
            # ...
            dynamic:
                acmeDynamicParam: ~
            # ...
```

Even if rarely ever needed for most cases, it is possible to pass the parameter an initialization string, and then fetch it in the associated service.

As an example, let's say we want to pass a C-like template string:

```yaml
# app/config/config.yml
#...
norsys_google_tag_manager:
    data_layer:
        default:
            # ...
            dynamic:
                acmeDynamicParam: '%s:%s:%s'
            # ...
```

We can then easily fetch this init value for further processing inside the `getValue()` method:

```php
# src/AppBundle/DynamicParameter/Acme.php
<?php

namespace AppBundle\DynamicParameter;

use Norsys\GoogleTagManagerBundle\Config\ParameterBag;

use Norsys\GoogleTagManagerBundle\Dynamic\ParameterInterface;

class Acme implements ParameterInterface
{
    // ...

    public function getValue(ParameterBag $configPage): string
    {
        // The value passed in the parameter config can be retreived easily
        $initValue = $configPage->get($this->getName());

        // Fetch $locale, $route, $land, for example from an object injected in the service's constructor
        // ...

        $calculatedValue = sprintf($initValue, $locale, $route, $land);

        return $calculatedValue;
    }

    // ...

}
```

Dynamic parameters can also be overriden, on a per-route basis:

```yaml
norsys_google_tag_manager:
    data_layer:
        default:
            # ...
            dynamic:
                acmeDynamicParam: 'whatever-init-value'

        pages:
            faq:
                dynamic:
                    acmeDynamicParam: 'another-init-value'
```

#### Dynamic parameters aliases

Additionnally this bundle also supports aliases for dynamic parameters.

Let's consider using an alias of the previously created `acmeDynamicParam`:

```yaml
norsys_google_tag_manager:
    data_layer:
        aliases:
            myDummyParam: acmeDynamicParam

        default:
            # ...
            dynamic:
                acmeDynamicParam: 'whatever-init-value'
```

Now we can use the aliased name to refer directly to its target:

```yaml
norsys_google_tag_manager:
    data_layer:
        aliases:
            myDummyParam: acmeDynamicParam

        default:
            # ...
            dynamic:
                # The following is equivalent to:
                #
                # acmeDynamicParam: 'whatever-init-value'
                #
                myDummyParam: 'whatever-init-value'
```

### Conclusion

Now, taking for granted the 2 above `static` and `dynamic` configured parameters, the resulting generated dataLayer for the `faq` route would be something like:

```javascript
    dataLayer = [ { "virtualPageURL": "/otherLink/faq", "acmeDynamicParam": "constructed-value" } ];
```

## Credits

Developped by :heart: with [Norsys](https://www.norsys.fr/)

## License

This project is licensed under the [MIT license](LICENSE).

repository: norsys/google-tag-manager-bundle

gha_language: PHP

repo_name: norsys/google-tag-manager-bundle


text:
# terraform-aws-organizations

A terraform module to AWS Organizations and child resources, like organizational units, accounts, tag policies and SCP Service Control Policies. Inspired by and adapted from [this doc](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_introduction.html).

Read the [AWS Organizations docs Best Practices ](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_best-practices_mgmt-acct.html).

WARNING! This Repo is mirrored from [Gitlab](https://gitlab.com/arkhoss/terraform-aws-organizations)

## Assumptions

* You have an AWS Account with AWS Organizations enable
* You have full access to AWS Organizations

## Usage example

A full example is contained in the [examples/basic directory](https://gitlab.com/arkhoss/terraform-aws-organizations/-/tree/master/examples).

```hcl

    module "my-org" {
        source             = "./terraform-aws-organizations"

        my_org = "something here"

        ous = {
          "name" = "some org unit"
        }

        accounts = {
          "name"  = "another account"
        }

        tags     = var.tags
    }

```

## Conditional creation

```hcl

```

### Resources Names

Name of the resources in AWS will follow a predefined pattern, like:

```
prefix + resource service + resource name
```

## Other documentation

* [Quotas for AWS Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_reference_limits.html): specifies quotas that affect AWS Organizations..
* [AWS Organizations FAQ](https://aws.amazon.com/organizations/faqs/): Frequently Asked Questions

## Doc generation

Code formatting and documentation for variables and outputs is generated using [pre-commit-terraform hooks](https://github.com/antonbabenko/pre-commit-terraform) which uses [terraform-docs](https://github.com/segmentio/terraform-docs).

Follow [these instructions](https://github.com/antonbabenko/pre-commit-terraform#how-to-install) to install pre-commit locally.

And install `terraform-docs` with `go get github.com/segmentio/terraform-docs` or `brew install terraform-docs`.

## Contributing

Report issues/questions/feature requests on in the [issues](https://gitlab.com/arkhoss/terraform-aws-organizations/-/issues) section.

Full contributing [guidelines are covered here](https://gitlab.com/arkhoss/terraform-aws-organizations/-/blob/master/CONTRIBUTING.md).

## Change log

The [changelog](https://gitlab.com/arkhoss/terraform-aws-organizations/-/blob/master/CHANGELOG.md) captures all important release notes from v1.0.0

## Authors

Created by:

- David Caballero [Gitlab](https://gitlab.com/arkhoss) | [Github](https://github.com/arkhoss) |<EMAIL_ADDRESS>
## License

MIT License

Copyright (c) 2020 The terraform-aws-organizations module Authors.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| aws | n/a |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| aws\_service\_access\_principals | description | `list` | `[]` | no |
| create\_org | This variable controls if Organization will be created or not | `bool` | `false` | no |
| create\_organizational\_units | This variable controls if Organizations Organizational unit will be created or not | `bool` | `false` | no |
| create\_organizations\_accounts | This variable controls if Organizations Accounts will be created or not | `bool` | `false` | no |
| create\_organizations\_policies | This variable controls if Organizations Policies will be created or not | `bool` | `false` | no |
| enabled\_policy\_types | List of Organizations Policy Types to enable in the Organization Root | `list` | `[]` | no |
| feature\_set | n/a | `string` | `"ALL"` | no |
| organizational\_units | Organizational Units |
list(object({
    name      = string
    parent_id = string
  }))
 |
[
  {
    "name": "my_ou",
    "parent_id": "parent_id"
  }
]
 | no |
| organizations\_accounts | Organizations Accounts |
list(object({
    name      = string
    email     = string
    parent_id = string
    role_name = string
  }))
 |
[
  {
    "email": "my@myorg.com",
    "name": "my_ou",
    "parent_id": "parent_id",
    "role_name": ""
  }
]
 | no |
| organizations\_policy | Organizations Policies |
list(object({
    name        = string
    description = string
    type        = string
    content     = string
  }))
 |
[
  {
    "content": "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": {\n    \"Effect\": \"Allow\",\n    \"Action\": \"*\",\n    \"Resource\": \"*\"\n  }\n}\n",
    "description": "this is my policy",
    "name": "my_policy",
    "type": "SERVICE_CONTROL_POLICY"
  }
]
 | no |
| policy\_id | The unique identifier (ID) of the policy that you want to attach to the target | `string` | `""` | no |
| tags | Tags | `map(string)` |
{
  "Name": ""
}
 | no |
| target\_id | The unique identifier (ID) of the root, organizational unit, or account number that you want to attach the policy to | `any` | `[]` | no |

## Outputs

No output.

repository: soorepalli/terraform-aws-organizations

repo_name: soorepalli/terraform-aws-organizations


text:
# Sample-efficient Adversarial Imitation Learning from Observation
Faraz Torabi, Sean Geiger, Garrett Warnell, Peter Stone

## Abstract
Imitation from observation is the framework of learning tasks by observing demonstrated state-only trajectories.

Recently, adversarial approaches have achieved significant performance improvements over other methods for imitating complex behaviors.

However, these adversarial imitation algorithms often require many demonstration examples and learning iterations to produce a policy that is successful at imitating a demonstrator’s behavior.

This high sample complexity often prohibits these algorithms from being deployed on physical robots.

In this paper, we propose an algorithm that addresses the sample inefficiency problem by utilizing ideas from trajectory centric reinforcement learning algorithms.

We test our algorithm and conduct experiments using an imitation task on a physical robot arm and its simulated version in Gazebo and will show the improvement in learning rate and efficiency

repository: tigerneil/awesome-deep-rl

gha_language: HTML

repo_name: tigerneil/awesome-deep-rl


text:
# Publications

Since DFTK is mostly developed as part of academic research,
we would greatly appreciate if you cite our research papers as appropriate.
See the [CITATION.bib](https://github.com/JuliaMolSim/DFTK.jl/blob/master/CITATION.bib)
in the root of the DFTK repo and the publication list
on this page for relevant citations.
The current DFTK reference paper to cite is
```bibtex
@article{DFTKjcon,
  author = {Michael F. Herbst and Antoine Levitt and Eric Cancès},
  doi = {10.21105/jcon.00069},
  journal = {Proc. JuliaCon Conf.},
  title = {DFTK: A Julian approach for simulating electrons in solids},
  volume = {3},
  pages = {69},
  year = {2021},
}
```

Additionally the following publications describe DFTK or one of its algorithms:

- M. F. Herbst and A. Levitt.
  [*A robust and efficient line search for self-consistent field iterations*](https://arxiv.org/abs/2109.14018)
  (Submitted).
  [ArXiv:2109.14018](https://arxiv.org/abs/2109.14018).
  ([Supplementary material and computational scripts](https://github.com/mfherbst/supporting-adaptive-damping/)).

- M. F. Herbst, A. Levitt and E. Cancès.
  [*DFTK: A Julian approach for simulating electrons in solids.*](https://doi.org/10.21105/jcon.00069)
  JuliaCon Proceedings, **3**, 69 (2021).

- M. F. Herbst and A. Levitt.
  [*Black-box inhomogeneous preconditioning for self-consistent field iterations in density functional theory*](https://doi.org/10.1088/1361-648X/abcbdb).
  Journal of Physics: Condensed Matter, **33**, 085503 (2021).
  [ArXiv:2009.01665](https://arxiv.org/abs/2009.01665).
  ([Supplementary material and computational scripts](https://github.com/mfherbst/supporting-ldos-preconditioning/)).

## Research conducted with DFTK
The following publications report research employing DFTK as a core component.
Feel free to drop us a line if you want your work to be added here.

- E. Cancès, G. Dusson, G. Kemlin and A. Levitt.
  [*Practical error bounds for properties in plane-wave electronic structure calculations*](https://arxiv.org/abs/2111.01470)
  (Submitted).
  [ArXiv:2111.01470](https://arxiv.org/abs/2111.01470).
  ([Supplementary material and computational scripts](https://github.com/gkemlin/paper-forces-estimator)).

- E. Cancès, G. Kemlin and A. Levitt.
  [*Convergence analysis of direct minimization and self-consistent iterations*](https://doi.org/10.1137/20M1332864)
  SIAM Journal on Matrix Analysis and Applications, **42**, 243 (2021).
  [ArXiv:2004.09088](https://arxiv.org/abs/2004.09088).
  ([Computational script](https://github.com/JuliaMolSim/DFTK.jl/blob/80c7452ef728f5e9f413f70e6d5eb4f8357075bc/examples/silicon_scf_convergence.jl)).

- M. F. Herbst, A. Levitt and E. Cancès.
  [*A posteriori error estimation for the non-self-consistent Kohn-Sham equations.*](https://doi.org/10.1039/D0FD00048E)
  Faraday Discussions, **224**, 227 (2020).
  [ArXiv:2004.13549](https://arxiv.org/abs/2004.13549).
  ([Reference implementation](https://github.com/mfherbst/error-estimates-nonscf-kohn-sham)).


text:
---
layout: post
title: "VoicePing"
description:
date: 2015-07-14 01:59:22 +0800
categories: real-time
keywords: "voiceping, real-time"
---
Lesson learned while building [VoicePing](https://www.linkedin.com/pulse/my-new-startup-voiceping-available-google-play-store-zhou-wenhan).

It’s neither about pursuing the big ideas nor technical challenges associated to building a highly mutable state machine with multitude of side effects.

Not even about achieving the elegant architecture of cooperating microservices which would scale to billion clients.

Yes, at some point the above big goals will seem to be the overall purpose or the end result of a real-time system. But right now:

80% are about making sure things are working as expected at the right time for:

1. Consumers of the services (mobile clients).
2. End users of the mobile clients.

20% are about making sure every little part of the software should be written and configured in the most understandable way, with as little overhead as possible in terms of memory or processing resources.

branch_name: refs/heads/source

gha_language: Stylus


text:
# Todist

> A simple todo application inspired by [Microsoft Todo](https://todo.microsoft.com/en-us)

#### The API is avaliable at: https://asp-net-todo.azurewebsites.net/api
#### The application is avaliable at: https://todist-app.firebaseapp.com/
#### Initial login could take some time as Azure shuts down free servers after a few minutes.
#### This is a work in progress.

### TODO:
- [ ] Fix Search / Sidebar Bug on mobile devices
- [ ] Improve code quality
- [ ] Add tests

### Screenshots:
![Home](./ScreenShots/Home.png)
![Dashboard](./ScreenShots/Dashboard.png)

### Running it locally:
#### Pre Requisite:
* MongoDB needs to be up a running.
* A DB named `asp_net_todo` needs to be created
* Collections `todo` and `user` also needs to be created

##### Open a terminal:
```bash
    cd TodoAspNetCore
    dotnet run
```
> Server will be running on http://localhost:5000
##### Open another Terminal:
```bash
    cd client
    yarn start
```
> Client will be running on http://localhost:3000

### Api Endpoints:
* All endpoints start with: http://asp-net-todo.azurewebsites.net/api or http://localhost:59821 when running locally
* All `todo` routes require a token which is supplied during login
```
    /auth/register
    POST
    username: Username,
    password: Password
```

```
    /auth/login
    POST
    username: Username,
    password: Password
```

```
    /todo/add_category?category=<name>
    POST
    name: New category name
```

```
    /todo/add_todo
    POST
    title: Todo Title,
    category: Todo Category
```

```
    /todo/all
    GET
```

```
    /todo/category?category=<name>
    GET
    name: Category Name
```

```
    /todo/starting?datetime=<date>
    GET
    date: Unix timestamp in milliseconds
```

```
    /todo/due?datetime=<date>
    GET
    date: Unix timestamp in milliseconds
```

```
    /todo?id=<id>
    GET
    id: Todo Id
```

```
    /todo/completed
    GET
```

```
    /todo/incomplete
    GET
```

```
    /todo/priority?priority=<priority>
    GET
    priority: Number for priority
```

```
    /todo/update_todo
    PUT
    Id: Todo Id,
    DueDate: ISO Date string,
    Note: Extra Content,
    Priority: Priority
```

```
    /todo/complete_todo?id=<id>
    PATCH
    id: Todo Id
```

```
    /todo/mark_incomplete_todo?id=<id>
    PATCH
    id: Todo Id
```

```
    /todo/update_category?category=<name>&newcategoryname=<newName>
    PATCH
    name: Old Category Name,
    newName: New Name for Category
```

```
    /todo/change_priority?priority=<priority>&id=<id>
    PATCH
    priority: Number for priority,
    id: Todo Id
```

```
    /todo?id=<id>
    DELETE
    id: Todo Id
```

```
    /todo/completed?category=<name>
    DELETE
    name: Category Name
```

```
    /todo/category?category=<name>
    DELETE
    name: Category Name
```


text:
# MutableBillingEntity

## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**created** | **datetime** | { \"description\" : \"The UTC DateTime when the object was created.\", \"verbs\":[] } | [optional]
**changed_by** | **str** | { \"description\" : \"ID of the user who last updated the entity.\", \"verbs\":[] } | [optional]
**updated** | **datetime** | { \"description\" : \"The UTC DateTime when the object was last updated.\", \"verbs\":[] } | [optional]
**id** | **str** | ID of the Synchronization Job. | [optional]
**config_id** | **str** | This is config id which links to this sync job. |
**organization_id** | **str** | Organization associated with the refund. |
**state** | **str** | This is the state of job. Pending jobs have not run. Complete jobs have run without error. Failed jobs have one of more errors. Cancelled jobs did not run. |
**type** | **str** | This is the type of job. Incremental jobs just sync changes, fully jobs sync all data. |
**scope** | **str** | The scope of the data synch'd. |
**target** | **str** | This is the target of the job. |
**name** | **str** | Name of the Job. |
**description** | **str** | Description of the Job. |
**data_from** | **datetime** | UTC DateTime of the start of the data to sync. | [optional]
**data_till** | **datetime** | UTC DateTime of the start of the data to sync. | [optional]
**started** | **datetime** | UTC DateTime of the end of the data to sync. | [optional]
**stopped** | **datetime** | UTC DateTime of the start of the data to sync. | [optional]
**created_by** | **str** | The account who created the synch job. | [optional]
**max_retry_times** | **int** | The max number of times the records will try to resync. |
**retry_attempted** | **int** | The number of times the records tried to resync. |
**deleted** | **bool** | Is the sync job deleted. | [default to False]

[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

repository: billforward/bf-python

repo_name: billforward/bf-python


text:
phonepoll
=========

This is a Drupal module that provides phone access to Drupal polls. Using this module and [Tropo](http://tropo.com/) people can vote in polls via calls or SMS.

This is an early release. Setup is a little manual right now, but will improve shortly.

Setup
-----

Copy this directory to your prefered module directory. If you don't have a prefered location, sites/all/modules is always a good choice. Enable this module in admin/build/modules

If there's no tropo-webapi-php directory inside this module directory, download it from http://github.com/tropo/tropo-webapi-php and place it inside this directory.

Enable the poll module if you haven't already. Create a poll. Note the node ID of the poll. You'll need that for the next step.

Create a Tropo account at http://tropo.com/ and create a WebAPI application. For the URL of your application, enter http://yoursite.example.com/phonepoll/poll/NODE-ID (replacing NODE-ID with the node ID of your poll)

Add a phone number -- Tropo has numbers throughout the US and in 41 countries.

Call or SMS your new number and vote.

Planned enhancements
--------------------

Panned enhancements include...

* more customization of the dialogs
* Allow phone calls out
* Improve Job queue support to better handle SMS throttles
* One-click setup of the Tropo app
* Multi-tenancy. Allow token and number to be set on per-poll basis

extension: markdown


text:
# Podunk

A tiny web framework on top of Rack

## Installation

Add this line to your application's Gemfile:

    gem 'podunk', github: 'bchase/podunk'

And then execute:

    $ bundle

## Limitations

You'll be interested to learn that, at the moment, Podunk is almost entirely comprised of limitations.

Inside, you'll find:

1. rudimentary query string parameter parsing
2. simple routing for `GET` requests (all right!)
3. thrilling plaintext body

That's about it!

## Usage

```ruby
### app.rb ###

class Application < Podunk::App
  route do
    get '/' => 'home'
  end

  def home
    "Welcome home, #{params['name']}."
  end
end
```

```ruby
### config.ru ###

require './app.rb'

run Application.new
```

```bash
$ rackup
```

```bash
$ curl "localhost:9292?name=Bud"
Welcome home, Bud.%
```

repository: bchase/podunk

repo_name: bchase/podunk


text:
# Unofficial TradingView Desktop Version

Unofficial TradingView Desktop Version built with [Electron](https://www.electronjs.org)

![](assets/images/screenshot.png)

[简体中文](README.zh.md)

# Features
* Block ads
* Block trial notification
* Refresh(F5 or Ctrl+R)
* Check for updates

# Solve the login failure problem
If the following situation occurs when you login

> You have been locked out due to too many login attempts. Please try again later.

you need to try the following steps:

* Click the menu on the top left
* Choose Home
* Choose your language
* Re-login

# How to build

* Install [Node.js](https://nodejs.org)

* Open Terminal / PowerShell / cmd

* Clone this repo

  ```bash
  git clone https://github.com/unknown-marketwizards/tradingview-desktop.git
  ```

* Enter folder

  ```bash
  cd tradingview-desktop
  ```

* Download electron

  ```bash
  npm install
  ```

* Build

  ```bash
  npm run build
  ```

---
欢迎加入`交易奇才 Slack 交流群`，让我们一起成为更好的交易者！

![](assets/images/qrcode.jpeg)

> 可扫码关注公众号，并回复：**加群**

branch_name: refs/heads/main


text:
---
layout: post
title:      "API Driven Development"
date:       2020-04-20 15:15:31 +0000
permalink:  api_driven_development
---

Hi – This was a good week coding.  I hit my milestones and had time to study algos / work on HackerRank problems.  The algo studying was not as bad as I expected it to be.  Way back when, I took a course on computer algorithms.  I probably understood 40% of it at the time.  Now those memories are coming in handy – providing a base to learn the meat of algorithm speed estimation.

Why was I so productive this week?  I think it comes down to discipline.  I worked on something important every day and stayed focused.  As a result, I brought the week in on time and on budget.

Please check my sprint cards below [here](https://docs.google.com/spreadsheets/d/1dT6-Y_PDX4rD-E3hAKP-anQ_oFOjTwil4rX66Uk9DqQ/edit#gid=0).

NOTE:  I used an API for my COVID-19 report [here](https://www.cpdundon.com/COVID-19).  If you pound on it, it locks out all report users.  The API will only serve an application a fixed rate of data.  Once we hit our quota, they cut us off for a while.  When I last checked, the site was receiving data.  Please do not keep pressing the get data button for no good reason.

NOTE-NOTE:  I am going to fix the problem above by caching the API data in database under my control.  It is going to take a few weeks as I need to set up the right technology.  First and foremost, I need a DB on Lightsail and that is my next programming task.

I am headed back to AWS set-up land.  AWS documentation is strong and I am looking forward to pressing it into service.  Working solo freelance, requires a level of self reliance.  Right now, I do not have the budget for a support contract and expert advice.  I will be resourceful over the next couple of years and that is great.

Goals for this week are similar to the ones for last week:
* 4 HackerRank problems
* 8 Hours of algorithm study
* Nail the sprint card for the week
* One blog post

See you next week.  Please Stay Safe!


text:
# Create, read, update, delete

## Tree CRUD APIs

A tree can be created via `Tree::new()` which constructs the root node only with
associated data.

```rust,no_run
use trees::Tree;
let mut tree = Tree::new(9);
```

This will construct a tree composed of only one root node with data 9.

```text
.............
.     9     .
.............
```

The root node can be accessed via `Tree::root()`/`Tree::root_mut()`.

```rust,no_run
let root = tree.root();
assert!( root.has_no_child() );
```

The associated data can be read/updated via `Tree::data()`/`Tree::data_mut()`.

```rust,no_run
assert_eq!( root.data(), &9 );

let root = tree.root_mut();
*root.data_mut() = 0;
```

The root data has been modified:

```text
.............
.     0     .
.............
```

A tree can adopt other trees as children via `Tree::push_back()`.

```rust,no_run
tree.push_back( Tree::new(1) );
tree.push_back( Tree::new(2) );
```

This will add two child nodes.

```text
.............
.     0     .
.   /   \   .
.  1     2  .
.............
```

The child nodes can be accessed via `iter()`.

```rust,no_run
let iter = tree.iter();
assert_eq!( iter.next().unwrap().data(), &1 );
assert_eq!( iter.next().unwrap().data(), &2 );
```

Specially, the first/last child can be accessed via `front()`/`back()`.

```rust,no_run
assert_eq!( tree.front().unwrap().data(), &1 );
assert_eq!( tree.back() .unwrap().data(), &2 );
```

A node can adopt other trees as children via `Node::push_back()`.

```rust,no_run
let node_1 = tree.front_mut().unwrap();
node_1.push_back( Tree::new(3) );
node_1.push_back( Tree::new(4) );
node_1.push_back( Tree::new(5) );
```

The tree will be:

```text
.............
.     0     .
.   /   \   .
.  1     2  .
. /|\       .
.3 4 5      .
.............
```

Nodes can be removed via `Node::detach()`.

```rust,no_run
let tree_4 = node_1.iter_mut().nth(1).unwrap().detach();
```

The tree will be:

```text
.............
.     0     .
.   /   \   .
.  1     2  .
. / \       .
.3   5      .
.............
```

Specially, the first/last child can be removed via `pop_front()`/`pop_back()`.

```rust,no_run
node_1.pop_front();
```

The tree will be:

```text
.............
.     0     .
.   /   \   .
.  1     2  .
.  |        .
.  5        .
.............
```

## Forest CRUD APIs

The `Forest`'s APIs are similar with `Tree`'s. The main difference is that a
`Forest` does not have root data, and may be empty.

## Node CRUD APIs

The `Node`'s APIs are similar with `Tree`'s. The main difference is that `Node`s
provided to library users are always in the form of `&Node<_>` or
`Pin<&mut Node<_>>`, which do not have ownership.

repository: oooutlk/trees

gha_language: Rust

gha_license_id: NOASSERTION

repo_name: oooutlk/trees


text:
# ud851-Exercises
I have combined all the Exercises here and added all my solutions. Practice makes perfect. 🚀

# Lesson10-Hydration-Reminder - ChargingBroadcastReceiver

## Getting the Current Battery State

Getting the Current Battery State
As mentioned, our code currently contains a bug. Our app adds and removes the dynamic broadcast receiver in onResume and onPause. When the app is not visible, the plug's image will not update. This can lead to the plug sometimes having the incorrect image when the app starts.

![lifecycle](https://user-images.githubusercontent.com/11560987/45581678-18623900-b868-11e8-861b-852d0a2986e3.png)

Now we could move the code to dynamically add and remove the broadcast receiver in different lifecycle methods, for example onCreate and onDestroy, but this would cause us to waste cycles swapping around an image which isn't even on screen. A better approach is to check what the current battery state is when the app resumes and update the image accordingly.

There are two ways to do this, depending on whether you're on API level 23+ or before.

Getting Charging State on API level 23+
To get the current state of the battery on API level 23+, simply use the battery manager system service:

BatteryManager batteryManager = (BatteryManager) getSystemService(BATTERY_SERVICE);
boolean isCharging = batteryManager.isCharging();
Getting Charging State with a Sticky Intent
Prior to Android 23+ you needed to use a sticky intent to get battery state. As we've seen, a normal, broadcasted intent will be broadcasted, possibly caught by an intent filter, and then disspear after it is processed. A sticky intent is a broadcast intent that sticks around, allowing your app to access it at any point and get information from the broadcasted intent. In Android, a sticky intent is where the current battery state is saved.

You don't need a broadcast receiver for a sticky intent, but you use similar looking code to registering a receiver:

```IntentFilter ifilter = new IntentFilter(Intent.ACTION_BATTERY_CHANGED);
   Intent batteryStatus = context.registerReceiver(null, ifilter);

Notice how registerReceiver is used, but instead of passing in a broadcast receiver, null is passed.  The registerReceiver method will return an intent, and it is that intent which has all of the battery information, which you can use:

boolean isCharging = status == BatteryManager.BATTERY_STATUS_CHARGING || status == BatteryManager.BATTERY_STATUS_FULL;

For more information on how to getting information about the battery, check out the Monitoring the Battery Level and Charging State documentation.

Now that you know how to get battery state, you should be able to complete the following exercise and fix the bug. The code is below.

Note: If you need to check whether the user is on API 23+, you can use the following code:

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M)

branch_name: refs/heads/student

gha_language: Java


text:
---
layout: post
title: The Wolf Report - July 22nd, 2022
---

- [SPA Alternative](https://htmx.org/essays/spa-alternative/)
- [I Regret my $46k Website Redesign ·mtlynch.io](https://mtlynch.io/tinypilot-redesign/)
- [Event Carried State Transfer: Keep a local cache! - CodeOpinion](https://codeopinion.com/event-carried-state-transfer-keep-a-local-cache/)
- [MonoLambdas, Nano Functions, and Goldilocks - The Symphonium](https://blog.symphonia.io/posts/2022-07-20_lambda_event_routing)
- [Write your own Javascript/Typescript tests runner in 80 lines of code </> - by Liron Hazan - Jun, 2022 - ITNEXT](https://itnext.io/write-your-own-javascript-typescript-tests-runner-in-80-lines-of-code-8eb00eeb2e63)
- [RedwoodJS vs. BlitzJS: The Future of Fullstack JavaScript Meta-Frameworks - RisingStack Engineering](https://blog.risingstack.com/redwoodjs-vs-blitzjs-comparison/)
- [Supercharging A/B Testing at Uber](https://eng.uber.com/supercharging-a-b-testing-at-uber/)
- [How Stripe Builds APIs - Postman Blog](https://blog.postman.com/how-stripe-builds-apis/)
- [The Universe of Discourse : Things I wish everyone knew about Git (Part II)](https://blog.plover.com/prog/git/tips-2.html)
- [AWS LAMBDA Proxy Integration - AMAZON API GATEWAY REST APIs - .NET ON AWS - AWS Serverless - YouTube](https://www.youtube.com/watch?v=13Urq-AqH8M)
- [Getting started with document databases in .NET using AWS DynamoDB - YouTube](https://www.youtube.com/watch?v=GzyMqh3BBzk)
- [GitHub - xt0rted/dotnet-run-script: A dotnet tool to run arbitrary commands from a project's "scripts" object](https://github.com/xt0rted/dotnet-run-script)

gha_language: SCSS


text:
# Validation

## Accept attribute

If you're only using `<FileUpload>` you may use the `@accept` argument to suggest which file types could be selected. This maps directly to the `accept` attribute on an input element.

Unforunately this is unreliable as it is not handled the same by all browsers.

For more details see the [MDN accept attribute reference](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/accept).

## Custom validation

Both components provide callback arguments with which you may implement custom validation of chosen files.

`<FileUpload>` may be passed `@onSelect` and `<FileDropzone>` may be passed `@onDrop`.

These are called *after* files have been chosen and *before* `@onFileAdd` is called by the queue.

To implement validation, you may (optionally) return a subset of files from these callbacks to restrict which files are queued for upload.

See the example below where the same validation callback is used for both selection methods.

Commonly validated file properties are `type`, `name` and `size`. For more details see the [MDN File reference](https://developer.mozilla.org/en-US/docs/Web/API/File).

```hbs
<FileDropzone
  @name="photos"
  @onDrop={{this.validateFiles}}
  as |dropzone queue|
>
  ...
  <FileUpload
    @name="photos"
    @onSelect={{this.validateFiles}}
    @onFileAdd={{perform this.uploadImage}}
  />
  ...
</FileDropzone>
```

```js
import Component from '@glimmer/component';

const allowedTypes = [
  "image/gif",
  "image/jpeg",
  "image/png",
  "image/webp",
];

export default class ExampleComponent extends Component {
  ...
  validateFiles(files) {
    return files.filter((file) => allowedTypes.includes(file.type));
  }
}
```

repository: mnutt/ember-file-upload

gha_language: JavaScript

repo_name: mnutt/ember-file-upload


text:
# SimplePipeline
[![Build Status](https://travis-ci.org/JensYvanDeCraecker/SimplePipeline.svg?branch=master)](https://travis-ci.org/JensYvanDeCraecker/SimplePipeline)
[![Build status](https://ci.appveyor.com/api/projects/status/075htx0xlbjaraue?svg=true)](https://ci.appveyor.com/project/JensYvanDeCraecker/simplepipeline)

SimplePipeline is an easy to use pipeline system. Our code uses custom interfaces making it highly extensible.

## Getting Started

### Download & Install

Please go to the [releases](https://github.com/JensYvanDeCraecker/SimplePipeline/releases) section to download the latest release of SimplePipeline or install the library directly from [NuGet](https://www.nuget.org/packages/SimplePipeline/).

```
PM> Install-Package SimplePipeline
```

### Documentation

Documentation and examples for this code can be found on the [Wiki page](https://github.com/JensYvanDeCraecker/SimplePipeline/wiki) (coming soon).

## Versioning

We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/JensYvanDeCraecker/SimplePipeline/tags).

## Authors

* **Jens Yvan De Craecker** - *Initial development* - [GitHub](https://github.com/JensYvanDeCraecker/)

## License

This project is licensed under the MIT [License](LICENSE.txt).

repository: hendeavors/SimplePipeline

repo_name: hendeavors/SimplePipeline


text:
---
title: An article on Cuban rafters
tags:
  - Jan 1987
---

An article on Cuban rafters lists all crossings made in 1986. [Photos]

Newspapers: **Miami Morning News or The Miami Herald**

Page: **1**, Section: **B**


text:
---
layout: post
title: Can an indie scientist get funding to run a real lab?
date: 2016-04-18 19:38
# Should be longer than 150 words, so it will appear as post summary
description: Even if you bootstrap your way into a genuine scientific education and a minimal lab, can you make a full-time living as an independent researcher?
# tags will also be used as html meta keywords.
categories: ["science", "funding", "blockchain"]

show_meta: true
comments: true
mathjax: true
gistembed: true
published: true
noindex: false
nofollow: false
---

### Indie Scientists?
In my last post I looked at a full-stack approach to open science - getting up and running
as that rarest of birds, the self-educated indie scientist. Long thought to have gone extinct in the Victorian
era, some believe the "gentleman scientist" - gentleperson, surely? - may be due for a comeback.

Is this actually a viable approach for someone who wants to do more than vanity research?
I'll assume for the moment that we're talking about someone who has the emotional support, the motivation,
the intellectual firepower, and the good sense to do quality work, bearing in mind that many practicing
scientists lack one or more of these qualities.

In that case, it comes down to lab equipment, space, and living resources. Or, in other words, money, money, and money.

What is the state of the field for getting funding? Well, there's bad news, and then there's more bad news.

### Big money and an independent lab
The National Institutes of Health makes government science grants, and if you're looking for millions they're the place
to get it. However, NIH grants require applicants to [have a Ph.D.](https://www.niaid.nih.gov/researchfunding/grant/strategy/pages/1qualify.aspx) as well as substantial experience in administering a
lab and publishing research. In fact, government funding is working on reducing the time it
takes for a postdoc to receive their first major grant. That's their major innovation. If a postdoctoral researcher has
a tough time getting funding to open her own lab, an independent researcher without a Ph.D. will find it even more difficult.

If you want to do top-tier research, then, you'd better go where the smart people, and big money is: academia.
Right? Probably, but here's more bad news. As [How to succeed in science: a concise guide for young biomedical scientists](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2685175/) points
out, the average age at which a postdoc receives a major grant is 43.

That's for someone who went straight through grad school, I presume. Now you're two decades away from retirement, and you're just
getting in a position to do the research you're really interested in.The "concise guide" above
notes that it really did use to be easier thirty or forty years ago.
For more, [Science](http://www.sciencemag.org/careers/2009/07/independent-postdocs-resources) and [Nature](http://blogs.nature.com/naturejobs/2015/03/30/the-postdoc-series-finding-funding/) discuss some of the problems a postdoc faces in moving out on her own.

Even under the relatively pro-science - in economic terms - Obama administration, the funding for science has remained flat.
Because this flatlining followed a small funding boom, the field is now glutted with postdoctoral researchers: the ["Postdocalypse"](http://www.motherjones.com/environment/2014/03/inquiring-minds-ethan-perlstein-postdocalypse).

Now, if you do get a lab up and running, unaffiliated with a university or hospital, you have the advantage of
nimbleness, which may help you to generate results faster, and result in a better track record of getting
funding. Or that was the [case 30 years ago, anyway.](http://www.the-scientist.com/?articles.view/articleNo/10178/title/NIH-Funding--Independent-Labs-Have-Best-Success-Rate/)

### Funding the unaffiliated researcher
Okay, if the big money goes to established researchers, and it's very hard to become one, how about the prospects
as an indie researcher?

Let's start by pointing out the obvious, as this [Quora post](https://www.quora.com/What-is-it-like-to-be-an-indie-scientist) does well. Although an independent research theoretically has more
research freedom by not worrying about grant proposals and restrictions, about bureaucratic administration,
lack of time, money, and equipment make going indie very hard.

For those reasons, I think, very few people become indie researchers, and "citizen science" doesn't do hard, cutting-edge science.
I'm sure training and funding are the major reasons for this, but again let's stick to funding and assume
you can self-train ([like this](https://www.quora.com/How-does-one-get-started-as-an-independent-researcher)).

### It's the end of the world as we know it, and I feel fine
[Some](http://brightworkcoresearch.com/postdocalypse-opportunity-disguise/) hope the postdocalypse could be a biotech boom opportunity in disguise. [Ethan Perlstein](http://www.sciencemag.org/careers/2013/05/going-rogue) couldn't find an acadmic
position despite sterling Ivy-league credentials, and decided to crowdfund his work before forming
a [public benefit corporation](http://www.plab.co/) to continue researching "orphan diseases".

(A public benefit corporation is a very interesting profit-nonprofit hybrid, in which the corporate officers balance duties to mission and to profit)

More broadly, others argue the dual rise of crowdfunding and cheaper tools will pave the way for [an alternative to the university system](http://www.theguardian.com/higher-education-network/blog/2014/jun/16/diy-labs-exciting-alternative-university-science-research)
of doing science, and allow more people [to get started as independent researchers](http://brightworkcoresearch.com/how-does-one-get-started-as-an-independent-researcher/).

### Diversifying funding sources?
In the same vein, Rhiannon argues that science funding may be diversifying. Private grant-giving institutions and patrons
such as Peter Thiel, Mark Zuckerberg, Elon Musk, and Y Combinator prefer to give to non-government labs -
a good sign for independent researchers. If government grantors follow suit, then both funding, and active scientists, will become more diverse.
However, that's stil an if.

Are these resources sufficient for a full-time researcher to live on and build a private lab?
That's a tough sell, but maybe.

-   If you identify as a [citizen scientist](https://www.zooniverse.org/), you have a [couple](http://www.citizensciencecenter.com/citizen-science-funding/) of [resources](http://www.grants.gov/web/grants/search-grants.html?keywords=citizen%20science) to investigate.

-   Entrepreneurship is always an option. Nobelist Peter Mitchell, Craig Venter, Stephen Wolfram, Ray Kurzweil, and
Ethan Perlstein all became independent researchers, funding their work through entrepreneurship or other means
(Venter combined entrepreneurship with staggeringly large NIH grants).

-   Much of the question may come down to whether you can get access to slighly larger amounts of money via grant-giving institutions.

I don't really know whether, with the assistance of [Instrumentl](https://www.instrumentl.com/), or quasi-grant institutions
like [BreakoutLabs](http://www.breakoutlabs.org/), or [other grantors](http://oedb.org/ilibrarian/100_places_to_find_funding_your_research/), one can move beyond [RocketHub](https://www.rockethub.com/) or Experiment. (I also don't know whether
the NSF's standards are as restrictive as the NIH's).

Crowdfunding may not (or, less likely, may) provide sufficient funding for a few. Entrepreneurship will work for a few,
but biotech is perhaps harder than academia. Big grants from the NIH are out. For synthetic biology, funding [may increase](http://www.synbioproject.org/publications/u.s-trends-in-synthetic-biology-research-funding/) in the future.
Smaller grants are a possibility, but I don't know, and I suspect few others do either.

The path forward remains uncertain for someone who doesn't want to suffer through the academic meat grinder - but
it remains uncertain as well for those who do.


text:
# videojs-constant-timeupdate

Adds a new event `constant-timeupdate` to player - an alternative to `timeupdate` event which will be triggered on a constant interval unlike `timeupdate`

## Installation

```sh
npm install --save videojs-constant-timeupdate
```

## Usage

To include videojs-constant-timeupdate on your website or web application, use any of the following methods.

### `<script>` Tag

This is the simplest case. Get the script in whatever way you prefer and include the plugin _after_ you include [video.js][videojs], so that the `videojs` global is available.

```html
<script src="//path/to/video.min.js"></script>
<script src="//path/to/videojs-constant-timeupdate.min.js"></script>
<script>
  var player = videojs('my-video');
  player.constantTimeupdate({
      interval: 1000,
      roundFn: Math.round
  });
</script>
```

## License

MIT. Copyright (c) Arjun Ganesan<EMAIL_ADDRESS>

[videojs]: http://videojs.com/