text:
---
title: "Configuration Reference"
date:
draft: false
weight: 40
---

# PostgreSQL Operator Installer Configuration

When installing the PostgreSQL Operator you have many configuration options, these
options are listed in this section.

## General Configuration

These variables affect the general configuration of the PostgreSQL
Operator.

| Name | Default | Required | Description |
|------|---------|----------|-------------|
| `archive_mode` | true | **Required** | Set to true enable archive logging on all newly created clusters. |
| `archive_timeout` | 60 | **Required** | Set to a value in seconds to configure the timeout threshold for archiving. |
| `backrest_aws_s3_bucket` |  |  | Set to configure the *bucket* used by pgBackRest with Amazon Web Service S3 for backups and restoration in S3. |
| `backrest_aws_s3_endpoint` |  |  | Set to configure the *endpoint* used by pgBackRest with Amazon Web Service S3 for backups and restoration in S3. |
| `backrest_aws_s3_key` |  |  | Set to configure the *key* used by pgBackRest with Amazon Web Service S3 for backups and restoration in S3. |
| `backrest_aws_s3_region` |  |  | Set to configure the *region* used by pgBackRest with Amazon Web Service S3 for backups and restoration in S3. |
| `backrest_aws_s3_secret` |  |  | Set to configure the *secret* used by pgBackRest with Amazon Web Service S3 for backups and restoration in S3. |
| `backrest_aws_s3_uri_style` |  |  | Set to configure whether “host” or “path” style URIs will be used when connecting to S3. |
| `backrest_aws_s3_verify_tls` |  |  | Set this value to true to enable TLS verification when making a pgBackRest connection to S3.f |
| `backrest_port` | 2022 | **Required** | Defines the port where pgBackRest will run. |
| `badger` | false | **Required** | Set to true enable pgBadger capabilities on all newly created clusters. This can be disabled by the client. |
| `ccp_image_prefix` | registry.developers.crunchydata.com/crunchydata | **Required** | Configures the image prefix used when creating containers from Crunchy Container Suite. |
| `ccp_image_pull_secret` |  |  | Name of a Secret containing credentials for container image registries. |
| `ccp_image_pull_secret_manifest` |  |  | Provide a path to the Secret manifest to be installed in each namespace. (optional) |
| `ccp_image_tag` | {{< param centosBase >}}-{{< param postgresVersion >}}-{{< param operatorVersion >}} | **Required** | Configures the image tag (version) used when creating containers from Crunchy Container Suite. |
| `create_rbac` | true | **Required** | Set to true if the installer should create the RBAC resources required to run the PostgreSQL Operator. |
| `crunchy_debug` | false |  | Set to configure Operator to use debugging mode. Note: this can cause sensitive data such as passwords to appear in Operator logs. |
| `db_name` |  |  | Set to a value to configure the default database name on all newly created clusters. By default, the PostgreSQL Operator will set it to the name of the cluster that is being created. |
| `db_password_age_days` | 0 |  | Set to a value in days to configure the expiration age on PostgreSQL role passwords on all newly created clusters. If set to “0”, this is the same as saying the password never expires |
| `db_password_length` | 24 |  | Set to configure the size of passwords generated by the operator on all newly created roles. |
| `db_port` | 5432 | **Required** | Set to configure the default port used on all newly created clusters. |
| `db_replicas` | 0 | **Required** | Set to configure the amount of replicas provisioned on all newly created clusters. |
| `db_user` | testuser | **Required** | Set to configure the username of the dedicated user account on all newly created clusters. |
| `default_instance_memory` | 128Mi |  | Represents the memory request for a PostgreSQL instance. |
| `default_pgbackrest_memory` | 48Mi |  | Represents the memory request for a pgBackRest repository. |
| `default_pgbouncer_memory` | 24Mi |  | Represents the memory request for a pgBouncer instance. |
| `delete_metrics_namespace` | false |  | Set to configure whether or not the metrics namespace (defined using variable `metrics_namespace`) is deleted when uninstalling the metrics infrastructure. |
| `delete_operator_namespace` | false |  | Set to configure whether or not the PGO operator namespace (defined using variable `pgo_operator_namespace`) is deleted when uninstalling the PGO. |
| `delete_watched_namespaces` | false |  | Set to configure whether or not the PGO watched namespaces (defined using variable `namespace`) are deleted when uninstalling the PGO. |
| `disable_auto_failover` | false |  | If set, will disable autofail capabilities by default in any newly created cluster |
| `disable_fsgroup` | false |  | Set to `true` for deployments where you do not want to have the default PostgreSQL fsGroup (26) set. The typical usage is in OpenShift environments that have a `restricted` Security Context Constraints. |
| `exporterport` | 9187 | **Required** | Set to configure the default port used to connect to postgres exporter. |
| `grafana_admin_password` |  |  | Set to configure the login password for the Grafana administrator. |
| `grafana_admin_username` | admin |  | Set to configure the login username for the Grafana administrator. |
| `grafana_install` | false |  | Set to true to install Crunchy Grafana to visualize metrics. |
| `grafana_storage_access_mode` | ReadWriteOnce |  | Set to the access mode used by the configured storage class for Grafana persistent volumes. |
| `grafana_storage_class_name` | fast |  | Set to the name of the storage class used when creating Grafana persistent volumes. |
| `grafana_supplemental_groups` | 65534 |  | Set to configure any supplemental groups that should be added to security contexts for Grafana. |
| `grafana_volume_size` | 1G |  | Set to the size of persistent volume to create for Grafana. |
| `metrics` | false | **Required** | Set to true enable performance metrics on all newly created clusters. This can be disabled by the client. |
| `metrics_namespace` | pgo |  | Namespace in which the `metrics` deployments with be run. |
| `namespace` | pgo |  | Set to a comma delimited string of all the namespaces Operator will manage. |
| `namespace_mode` | dynamic |  | Determines which namespace permissions are assigned to the PostgreSQL Operator using a ClusterRole. Options: `dynamic`, `readonly`, and `disabled` |
| `pgbadgerport` | 10000 | **Required** | Set to configure the default port used to connect to pgbadger. |
| `pgo_add_os_ca_store` | false | **Required** | When true, includes system default certificate authorities. |
| `pgo_admin_password` | examplepassword | **Required** | Configures the pgo administrator password. |
| `pgo_admin_perms` | * | **Required** | Sets the access control rules provided by the PostgreSQL Operator RBAC resources for the PostgreSQL Operator administrative account that is created by this installer. Defaults to allowing all of the permissions, which is represented with the * |
| `pgo_admin_role_name` | pgoadmin | **Required** | Sets the name of the PostgreSQL Operator role that is utilized for administrative operations performed by the PostgreSQL Operator. |
| `pgo_admin_username` | admin | **Required** | Configures the pgo administrator username. |
| `pgo_apiserver_port` | 8443 |  | Set to configure the port used by the Crunchy PostgreSQL Operator apiserver. |
| `pgo_apiserver_url` | https://postgres-operator |  | Sets the `pgo_apiserver_url` for the `pgo-client` deployment. |
| `pgo_client_cert_secret` | pgo.tls |  | Sets the secret that the `pgo-client` will use when connecting to the PostgreSQL Operator. |
| `pgo_client_container_install` | false |  | Run the `pgo-client` deployment with the PostgreSQL Operator. |
| `pgo_client_install` | true |  | Enable to download the `pgo` client binary as part of the Ansible install |
| `pgo_client_version` | {{< param operatorVersion >}} | **Required** |  |
| `pgo_cluster_admin` | false | **Required** | Determines whether or not the cluster-admin role is assigned to the PGO service account. Must be true to enable PGO namespace & role creation when installing in OpenShift. |
| `pgo_disable_eventing` | false |  | Set to configure whether or not eventing should be enabled for the Crunchy PostgreSQL Operator installation. |
| `pgo_disable_tls` | false |  | Set to configure whether or not TLS should be enabled for the Crunchy PostgreSQL Operator apiserver. |
| `pgo_image_prefix` | registry.developers.crunchydata.com/crunchydata | **Required** | Configures the image prefix used when creating containers for the Crunchy PostgreSQL Operator (apiserver, operator, scheduler..etc). |
| `pgo_image_pull_secret` |  |  | Name of a Secret containing credentials for container image registries. |
| `pgo_image_pull_secret_manifest` |  |  | Provide a path to the Secret manifest to be installed in each namespace. (optional) |
| `pgo_image_tag` | {{< param centosBase >}}-{{< param operatorVersion >}} | **Required** | Configures the image tag used when creating containers for the Crunchy PostgreSQL Operator (apiserver, operator, scheduler..etc) |
| `pgo_installation_name` | devtest | **Required** | The name of the PGO installation. |
| `pgo_noauth_routes` |  |  | Configures URL routes with mTLS and HTTP BasicAuth disabled. |
| `pgo_operator_namespace` | pgo | **Required** | Set to configure the namespace where Operator will be deployed. |
| `pgo_tls_ca_store` |  |  | Set to add additional Certificate Authorities for Operator to trust (PEM-encoded file). |
| `pgo_tls_no_verify` | false |  | Set to configure Operator to verify TLS certificates. |
| `prometheus_install` | false |  | Set to true to install Crunchy Grafana to visualize metrics. |
| `prometheus_storage_access_mode` | ReadWriteOnce |  | Set to the access mode used by the configured storage class for Prometheus persistent volumes. |
| `prometheus_storage_class_name` | fast |  | Set to the name of the storage class used when creating Prometheus persistent volumes. |
| `prometheus_supplemental_groups` | 65534 |  | Set to configure any supplemental groups that should be added to security contexts for Prometheus. |
| `prometheus_volume_size` | 1G |  | Set to the size of persistent volume to create for Prometheus. |
| `reconcile_rbac` | true |  | Determines whether or not the PostgreSQL Operator will granted the permissions needed to reconcile RBAC within targeted namespaces. |
| `scheduler_timeout` | 3600 | **Required** | Set to a value in seconds to configure the `pgo-scheduler` timeout threshold when waiting for schedules to complete. |
| `service_type` | ClusterIP |  | Set to configure the type of Kubernetes service provisioned on all newly created clusters. |
| `sync_replication` | false |  | If set to `true` will automatically enable synchronous replication in new PostgreSQL clusters. |

## Storage Settings

The store configuration options defined in this section can be used to specify
the storage configurations that are used by the PostgreSQL Operator.

## Storage Configuration Options

Kubernetes and OpenShift offer support for a wide variety of different storage
types and we provide suggested configurations for different environments. These
storage types can be modified or removed as needed, while additional storage
configurations can also be added to meet the specific storage requirements for
your PostgreSQL clusters.

The following storage variables are utilized to add or modify operator storage
configurations in the with the installer:

| Name | Required | Description |
|------|----------|-------------|
| `storage<ID>_name` | Yes | Set to specify a name for the storage configuration. |
| `storage<ID>_access_mode` | Yes | Set to configure the access mode of the volumes created when using this storage definition. |
| `storage<ID>_size` | Yes | Set to configure the size of the volumes created when using this storage definition. |
| `storage<ID>_class` | Required when using the `dynamic` storage type | Set to configure the storage class name used when creating dynamic volumes. |
| `storage<ID>_supplemental_groups` | Required when using NFS storage | Set to configure any supplemental groups that should be added to security contexts on newly created clusters. |
| `storage<ID>_type` | Yes  | Set to either `create` or `dynamic` to configure the operator to create persistent volumes or have them created dynamically by a storage class. |

The ID portion of storage prefix for each variable name above should be an
integer that is used to group the various storage variables into a single
storage configuration.

### Example Storage Configuration

```yaml
storage3_name: 'nfsstorage'
storage3_access_mode: 'ReadWriteMany'
storage3_size: '1G'
storage3_type: 'create'
storage3_supplemental_groups: 65534
```

As this example storage configuration shows, integer `3` is used as the ID for
each of the `storage` variables, which together form a single storage
configuration called `nfsstorage`. This approach allows different storage
configurations to be created by defining the proper `storage` variables with a
unique ID for each required storage configuration.

### PostgreSQL Cluster Storage Defaults

You can specify the default storage to use for PostgreSQL, pgBackRest, and other
elements that require storage that can outlast the lifetime of a Pod. While the
PostgreSQL Operator defaults to using `hostpathstorage` to work with
environments that are typically used to test, we recommend using one of the
other storage classes in production deployments.

| Name | Default | Required | Description |
|------|---------|----------|-------------|
| `backrest_storage` | hostpathstorage | **Required** | Set the value of the storage configuration to use for the pgbackrest shared repository deployment created when a user specifies pgbackrest to be enabled on a cluster. |
| `backup_storage` | hostpathstorage | **Required** | Set the value of the storage configuration to use for backups, including the storage for pgbackrest repo volumes. |
| `primary_storage` | hostpathstorage | **Required** | Set to configure which storage definition to use when creating volumes used by PostgreSQL primaries on all newly created clusters. |
| `replica_storage` | hostpathstorage | **Required** | Set to configure which storage definition to use when creating volumes used by PostgreSQL replicas on all newly created clusters. |
| `wal_storage` |  |  | Set to configure which storage definition to use when creating volumes used for PostgreSQL Write-Ahead Log. |

#### Example Defaults

```yaml
backrest_storage: 'nfsstorage'
backup_storage: 'nfsstorage'
primary_storage: 'nfsstorage'
replica_storage: 'nfsstorage'
```

With the configuration shown above, the `nfsstorage` storage configuration would
be used by default for the various containers created for a PG cluster
(i.e. containers for the primary DB, replica DB's, backups and/or `pgBackRest`).

### Considerations for Multi-Zone Cloud Environments

When using the Operator in a Kubernetes cluster consisting of nodes that span
multiple zones, special consideration must be taken to ensure all pods and the
volumes they require are scheduled and provisioned within the same zone.  Specifically,
being that a pod is unable mount a volume that is located in another zone, any
volumes that are dynamically provisioned must be provisioned in a topology-aware
manner according to the specific scheduling requirements for the pod. For instance,
this means ensuring that the volume containing the database files for the primary
database in a new PostgreSQL cluster is provisioned in the same zone as the node
containing the PostgreSQL primary pod that will be using it.

### Default Storage Configuration Types

#### Host Path Storage

| Name | Value |
|------|-------|
| storage1_name | hostpathstorage |
| storage1_access_mode | ReadWriteMany |
| storage1_size | 1G |
| storage1_type | create |

#### Replica Storage

| Name | Value |
|------|-------|
| storage2_name | replicastorage |
| storage2_access_mode | ReadWriteMany |
| storage2_size | 1G |
| storage2_type | create |

#### NFS Storage

| Name | Value |
|------|-------|
| storage3_name | nfsstorage |
| storage3_access_mode | ReadWriteMany |
| storage3_size | 1G |
| storage3_type | create |
| storage3_supplemental_groups | 65534 |

#### NFS Storage Red

| Name | Value |
|------|-------|
| storage4_name | nfsstoragered |
| storage4_access_mode | ReadWriteMany |
| storage4_size | 1G |
| storage4_match_labels | crunchyzone=red |
| storage4_type | create |
| storage4_supplemental_groups | 65534 |

#### StorageOS

| Name | Value |
|------|-------|
| storage5_name | storageos |
| storage5_access_mode | ReadWriteOnce |
| storage5_size | 5Gi |
| storage5_type | dynamic |
| storage5_class | fast |

#### Primary Site

| Name | Value |
|------|-------|
| storage6_name | primarysite |
| storage6_access_mode | ReadWriteOnce |
| storage6_size | 4G |
| storage6_type | dynamic |
| storage6_class | primarysite |

#### Alternate Site

| Name | Value |
|------|-------|
| storage7_name | alternatesite |
| storage7_access_mode | ReadWriteOnce |
| storage7_size | 4G |
| storage7_type | dynamic |
| storage7_class | alternatesite |

#### GCE

| Name | Value |
|------|-------|
| storage8_name | gce |
| storage8_access_mode | ReadWriteOnce |
| storage8_size | 300M |
| storage8_type | dynamic |
| storage8_class | standard |

#### Rook

| Name | Value |
|------|-------|
| storage9_name | rook |
| storage9_access_mode | ReadWriteOnce |
| storage9_size | 1Gi |
| storage9_type | dynamic |
| storage9_class | rook-ceph-block |

## Pod Anti-affinity Settings
This will set the default pod anti-affinity for the deployed PostgreSQL
clusters. Pod Anti-Affinity is set to determine where the PostgreSQL Pods are
deployed relative to each other There are three levels:

- required: Pods *must* be scheduled to different Nodes. If a Pod cannot be
  scheduled to a different Node from the other Pods in the anti-affinity
  group, then it will not be scheduled.
- preferred (default): Pods *should* be scheduled to different Nodes. There is
  a chance that two Pods in the same anti-affinity group could be scheduled to
  the same node
- disabled: Pods do not have any anti-affinity rules

The `POD_ANTI_AFFINITY` label sets the Pod anti-affinity for all of the Pods
that are managed by the Operator in a PostgreSQL cluster. In addition to the
PostgreSQL Pods, this also includes the pgBackRest repository and any
pgBouncer pods. By default, the pgBackRest and pgBouncer pods inherit the
value of `POD_ANTI_AFFINITY`, but one can override the default by setting
the `POD_ANTI_AFFINITY_PGBACKREST` and `POD_ANTI_AFFINITY_PGBOUNCER` variables
for pgBackRest and pgBouncer respectively

| Name | Default | Required | Description |
|------|---------|----------|-------------|
| `pod_anti_affinity` | preferred |  | This will set the default pod anti-affinity for the deployed PostgreSQL clusters. |
| `pod_anti_affinity_pgbackrest` |  |  | This will set the default pod anti-affinity for the pgBackRest pods. |
| `pod_anti_affinity_pgbouncer` |  |  | This will set the default pod anti-affinity for the pgBouncer pods. |

## Understanding `pgo_operator_namespace` & `namespace`

The Crunchy PostgreSQL Operator can be configured to be deployed and manage a single
namespace or manage several namespaces.  The following are examples of different types
of deployment models:

### Single Namespace

To deploy the Crunchy PostgreSQL Operator to work with a single namespace (in this example
our namespace is named `pgo`), configure the following settings:

```yaml
pgo_operator_namespace: 'pgo'
namespace: 'pgo'
```

### Multiple Namespaces

To deploy the Crunchy PostgreSQL Operator to work with multiple namespaces (in this example
our namespaces are named `pgo`, `pgouser1` and `pgouser2`), configure the following settings:

```yaml
pgo_operator_namespace: 'pgo'
namespace: 'pgouser1,pgouser2'
```

## Deploying Multiple Operators

The 4.0 release of the Crunchy PostgreSQL Operator allows for multiple operator deployments in the same cluster.
To install the Crunchy PostgreSQL Operator to multiple namespaces, it's recommended to have an configuration file
for each deployment of the operator.

For each operator deployment the following variables should be configured uniquely for each install.

For example, operator could be deployed twice by changing the `pgo_operator_namespace` and `namespace` for those
deployments:

Install A would deploy operator to the `pgo` namespace and it would manage the `pgo` target namespace.

```yaml
# Config A
pgo_operator_namespace: 'pgo'
namespace: 'pgo'
...
```

Install B would deploy operator to the `pgo2` namespace and it would manage the `pgo2` and `pgo3` target namespaces.
```yaml
# Config B
pgo_operator_namespace: 'pgo2'
namespace: 'pgo2,pgo3'
...
```

Each install of the operator will create a corresponding directory in `$HOME/.pgo/<PGO NAMESPACE>` which will contain
the TLS and `pgouser` client credentials.

repository: jufi-git/postgres-operator

repo_name: jufi-git/postgres-operator


text:
---
layout: post
title: Buyers Profile 1
---

George

Background

* Web designer student at the Art Institute of Miami

Demographics

* Russian
* Age 22
* Lives in Urban area of Miami

Identifiers

* Russian accents
* Very calm
* Easy going
* Knowledgable

Goals

* To Graduate from Ai Institute with a BS in Graphic and Web design
* Expand knowledge on various areas of business

Challenges

* Solving problems in the web design classes that I am faced with everyday
* Not being distracted
* Time

How we help

* Making sure the information given is concise and clear
* Assisting with the best approache when challenges arise

Real Quotes

* "What ever is best for the solution to the problem, that is the option I go with."

Common Objections

* The project might tke to long
* The budget is not enough to cover the expenses

Marketing Messaging

* The process might take long but with the right tool time can be in our favor if we are digilent.

Elevator Pitch

* We offer a more apprehensive and hands-on approach to any design project, compared to other competitors in our industry we make customer service and quality a priority in our company.

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# formatter

This is a simple library designed to do one thing and one thing only -
replace variables in strings with variable values.  It is built in such a
way that the formatter strings are parsed and you are provided with a
function than can efficiently be called to provide the custom output.

[![NPM](https://nodei.co/npm/formatter.png)](https://nodei.co/npm/formatter/)

[![Build Status](https://travis-ci.org/DamonOehlman/formatter.png?branch=master)](https://travis-ci.org/DamonOehlman/formatter)
[![stable](http://hughsk.github.io/stability-badges/dist/stable.svg)](http://github.com/hughsk/stability-badges)

[![browser support](https://ci.testling.com/DamonOehlman/formatter.png)](https://ci.testling.com/DamonOehlman/formatter)

## Example Usage

```js
var formatter = require('formatter');
var likefood = formatter('I like {{ 0 }}, {{ 0 }} is excellent and kicks the pants off {{ 1 }}.');

// I can then log out how much I like bacon
console.log(likefood('bacon', 'bread'));
// <-- I like bacon, bacon is excellent and kicks the pants off bread.
```

__NOTE__: Formatter is not designed to be a templating library and if
you are already using something like Handlebars or
[hogan](https://github.com/twitter/hogan.js) in your library or application
stack consider using them instead.

## Using named variables

In the examples above we saw how the formatter can be used to replace
function arguments in a formatter string.  We can also set up a formatter
to use particular key values from an input string instead if that is more
suitable:

```js
var formatter = require('formatter');
var likefood = formatter('I like {{ great }}, {{ great }} is excellent and kicks the pants off {{ poor }}.');

console.log(likefood({ great: 'bacon', poor: 'bread' }));
// <-- I like bacon, bacon is excellent and kicks the pants off bread.
```

## Nested Property Values

Since version `0.1.0` you can also access nested property values, as you
can with templates like handlebars.

## Partial Execution

Since version `0.3.x` formatter also supports partial execution when using
indexed arguments (e.g. `{{ 0 }}`, `{{ 1 }}`, etc).  For example:

```js
var formatter = require('formatter');
var likefood = formatter('I like {{ 0 }}, {{ 0 }} is excellent and kicks the pants off {{ 1 }}.');
var partial;

// get a partial
console.log(partial = likefood('bacon'));
// <-- [Function]

// pass the remaining argument it's waiting for
console.log(partial('bread'));
// <-- I like bacon, bacon is excellent and kicks the pants off bread.
```

In the case above, the original formatter function returned by `formatter`
did not receive enough values to resolve all the required variables.  As
such it returned a function ready to accept the remaining values.

Once all values have been received the output will be generated.

## Performance

I've done some
[performance benchmarks](http://jsperf.com/formatter-performance) and
formatter is faster than handlebars, but that isn't surprising as it is far
simpler and doesn't have the smarts of HBS.  The test is really there to
ensure that I didn't do anything too silly...

Additionally, it should be noted that using formatter is 100% slower than
concatenating strings, so don't use it where performance is critical.
Do use it where not repeating yourself is.

## Modifiers

### Length Modifier (len)

The length modifier is used to ensure that a string is exactly the length specified.  The string is sliced to the required max length, and then padded out with spaces (or a specified character) to meet the required length.

```js
// pad the string test to 10 characters
formatter('{{ 0|len:10 }}')('test');   // 'test      '

// pad the string test to 10 characters, using a as the padding character
formatter('{{ 0|len:10:a }}')('test'); // 'testaaaaaa'
```

gha_language: JavaScript

gha_license_id: NOASSERTION


text:
# Multilayerd-General-Neural-Network
This is my personal Multilayered General Neural Network project built from scratch. Currently, the only file needed is *NN.py*. The implementation utilizes Python with NumPy and Matplotlib (only needed if you want to plot the error) as dependencies. These can be installed with the following commands:
```
pip install numpy
pip install matplotlib
```
For great slides on how neural networks do their thing, check out Keith Downing's (my professor) [slides](http://www.idi.ntnu.no/emner/it3105/tdt76/) from the Deep Learning Theory Module held the autumn of 2017 at NTNU (especially lecture 2). My implementation does not currently utilize Jacobian matrices for the backpropagation.

## Status
The current code base should support any basic neural network configuration. By basic, I mean the following:
* Multilayered networks
* 1 to *n* nodes per layer
* *n* fully-connected layers
* Static learning rate
* Runs for *n* epochs, regardless of error convergence

### Other features
* Perform validation testing every *n* epoch
* Dropout

### Activations functions
Currently, the following activation functions are supported:
* Logistic

### Loss functions
Currently, the following loss functions are supported:
* Squared error

## Usage
```python
nn = NN.NeuralNetwork(layer_sizes=[8, 32, 16, 4], learning_rate=0.1)
nn.Train(20, training_cases, validation_interval=100)
nn.Test(test_cases)
nn.PlotError()
```
The *case_base* must have the following structure:
```python
case_0 =
[
  [[input_0, input_1, ...]], #Input
  [target_0, target_1, ...] #Expected output/target
]
case_1 = [...]
case_base = [case_0, case_1, ...]
```
No parameters are needed when running the script.

**Warning:** I've tested the implementation, and as far as I can tell it works as intented. However, unknown errors might be present.


text:
# email-parser

The fastest and lightest email parsing Rust library!\
This library has no dependency by default (and only a small optional one).

## Goal

The goal of this library is to be fully compliant with RFC 5322. However, this library does not intend to support the obsolete syntax because it has been obsolete for 12 years, and it would slow down everything.\
This library supports MIME and will support PGP in the future.

## Example

```rust
let email = Email::parse(
    b"\
    From: Mubelotix<EMAIL_ADDRESS>    Subject:Example Email\r\n\
    To: Someone<EMAIL_ADDRESS>    Message-id<EMAIL_ADDRESS>    Date: 5 May 2003 18:58:34 +0000\r\n\
    \r\n\
    Hey!\r\n",
)
.unwrap();

assert_eq!(email.subject.unwrap(), "Example Email");
assert_eq!(email.sender.name.unwrap(), vec!["Mubelotix"]);
assert_eq!(email.sender.address.local_part, "mubelotix");
assert_eq!(email.sender.address.domain, "mubelotix.dev");
```

## Pay for what you use

Mails can be elaborated. No matter what you are building, you are certainly not using all of its features.\
So why would you pay the parsing cost of header fields you are not using? This library allows you to enable headers you need so that other header values will be parsed as an unstructured header, which is much faster.\
By disabling all header value parsing, this library can parse an entire mail twice faster! But don't worry if you need everything enabled; this library is blazing fast anyway!

## Zero-Copy (almost)

This library tries to avoid usage of owned `String`s as much as possible and is using `Cow<str>` instead.\
Thanks to this method, around 90% of the strings are references.

## Benchmarks

This chart shows the time took to parse a single email.

![Benchmark](https://cdn.discordapp.com/attachments/694923348844609597/789162705494868020/unknown.png)

Run these benchmarks by yourself with `rustup run nightly cargo bench` and `rustup run nightly cargo bench --no-default-features`.\
Tests require a `mail.txt` file containing a raw mail next to the `Cargo.toml`.\
Some libraries suffer from huge performance variations depending on the content of the mail, so this library is not **always** the fastest.

License: MIT

repository: harisraharjo/email-parser

repo_name: harisraharjo/email-parser


text:
---
layout: post
title: "Django useful info"
subtitle: "Tips and tricks for Django"
categories: coding
tags: django
comments: true
---
* [Django status](https://pypi.org/project/django-status/):
 Application that provides an API to check the status of some parts and some utilities like ping.

gha_language: Ruby


text:
redux-todo
============================

A basic todo app using redux and material-ui covering the concept of `reducers`, `actions`, `stores`, `container components, presentation components` and others in redux.

### Usage

```
npm install
npm start
open http://localhost:3000
```

repository: addityasingh/redux-todo

repo_name: addityasingh/redux-todo


text:
# Introduction
_This page explains dynamic insert statements. Before studying this page, you should read about [static insert statements](Insert.md)._

If you know the exact structure of your inserts at compile time, statically constructed insert statements are perfect. But if the structure depends on runtime information like user input, you will need dynamic insert statements. For those only provide part fields of table schema, leave others to default value or NULL, dynamic insert statement is a must. Depending on your needs, you can choose the required dosage.

# A Basic Example
So let us construct insert statement with a dynamic part
```C++
auto s = dynamic_insert_into(db, foo).dynamic_set(
    foo.name = name,
);
s.insert_list.add(foo.id = runtimeStatement.id);
s.insert_list.add(foo.hasFun = runtimeStatement.hasFun);
const int64_t result = db(s);
```
Admittedly, a rather simplistic example (please suggest nicer ones), but anyway, this is what's happening:
```C++
dynamic_insert_into(db, table)
```
This initializes a dynamic insert. One major difference between `dynamic_insert_into` and `insert_into` is the first argument of `dynamic_insert_into`: a database connection. This is used to evaluate the dynamic parts of the statement as they are added.

```C++
.dynamic_set(...);
...
s.insert_list.add(foo.id = runtimeStatement.id);
s.insert_list.add(foo.hasFun = runtimeStatement.hasFun);
```
The first part creates a new insert object that accepts a dynamically constructed set expression. In this case the user can determine whether to search for a certain id or a certain hasFun, or neither or both.

```C++
const int64_t result = db(s);
```
Finally use `db(s)` to execute the insert statement, and returns a `int64_t` value. If `foo.id` column is a `AUTO INCREMENT` (auto generated) id field, this will returns the auto generated id, for others this will returns 0.


text:
[ESA](https://www.esa.int), [JAXA](https://global.jaxa.jp/) and [NASA](https://www.nasa.gov/) have combined their resources, technical knowledge, and expertise to produce this Earth Observing Dashboard to strengthen our global understanding of the changing environment with human activies.

### EXPLORE DATASETS

Here you can browse the Earth Observation datasets and use the interactive features, including maps that compute simple analytics by drawing an area of interest. Other datasets are tabular data, presented as charts. All charts can be downloaded as CSV.

The open data and project source code can be accessed in [Github](https://github.com/eurodatacube/eodash)

repository: eurodatacube/eodash

branch_name: refs/heads/staging

gha_license_id: MIT

repo_name: eurodatacube/eodash


text:
---
layout: post
date:   2021-01-02
title:  "I Am Online"
permalink: ./blog/index.php/2021/01/i-am-online/
categories: blog
published: true
tags: [Uncategorized]
comments: false
---
It's been a while since my last post. Eight years actually. Unfortunately when the blog went offline back in January 2013 I _lost my muse_ for a while. Like all IT professionals working in Ops, I did still write the occasional "TL;DR" email or publish an article in my workplaces' internal Wiki. I was initially put off by the fact that the blog had to be set up from scratch.

When I look back at what happened in the last eight years I realise that I have been far from idle. I recently also started getting photo reminders from my Cloud provider "AI" of events that happened a number of years ago. I will not get into whether photos should or should not be stored in Cloud storage - opinions vary, and that is a matter for another discussion.

So, what's been going on?

### 2013

I worked with various teams (Enterprise Architects, Server Admins, Storage Admins, Network Admins, DBAs for other DBMS', Management, and a variety of business users) to design a new consolidated environment for the Government of Malta (my country of birth).

This same year saw me experiencing career growth, becoming the "unofficial" Lead DBA (SQL Server) in a team of three. Our collaboration and camaraderie resulted in successful projects.

A holiday to Skerries, Ireland was a welcome break from work.

### 2014

Once the hardware was procured, delivered and installed, we started the installation and configuration of the new consoldated environment. This came hand-in-hand with provisioning development and other non-production environments, as well as planning the upgrades and migrations of a considerable number of database systems to the new version. This was a long-term project spilling over to 2015 and, if I recall correctly, we migrated/upgraded (or worked with Developers to upgrade) more than 250 applications, a lot of which were still on SQL Server 2000, and some even on SQL Server 7.

This year my "disconnect time" was a holiday to Donegal, Ireland.

My workplace also gave me the opportunity to attend the Microsoft TechEd, held in Barcelona, Spain in October/November.

### 2015

The migration, consolidation and upgrade of various application databases continued almost non-stop.

My wife and I were also considering our move to Ireland more seriously. A dream that had been postponed for various reasons, for a number of years.

I completed the EDX course "DAT203X: Data Science and Machine Learning Essentials", passing the exam. That was a tough nut to crack, especially since I did not have any background in Mathematics or Statistics.

### 2016

My wife and I decided to take the plunge and move to Ireland. Leaving family and friends behind, and my job of sixteen years, was a tough decision but we felt it was for the best. We spent the next three years in a rented apartment in Howth, Co. Dublin.

Moving to another country also gave me the opportunity to attend my first SQL Saturday Ireland conference at the National College Of Ireland in Dublin.

After that conference I joined the SQL Ireland User Group, and attend monthly meetings organised through MeetUp.

### 2017

The original version of my SQL Monitor was based on a number of SSIS packages, however I wasn't too happy with the dependency on the SSIS service nor the fact that the packages required a specific IDE version to apply changes; so I decided to write a new version of SQL Monitor using PowerShell and publish it to GitHub. More about this in a future article.

I attended the SQL Saturday Ireland 2017 conference at Trinity College in Dublin.

### 2018

My employer provided me with the opportunity to attend my first major SQL Server conference - the "SQLBits 2018 - The Magic of Data" held at the Olympia in London, England.

Later that year I attended SQL Saturday Ireland 2018 conference at the University College Cork (UCC), in Cork.

### 2019

We finally got to buy our own piece of Irish soil and moved into our new home.

I attended the SQL Saturday Ireland 2019 conference which was once again held at the University College Cork (UCC), in Cork.

Sadly, I had to sell my Honda Shadow VT-600C motorcycle after 18 years of great adventures. I miss that bike some days.

I decided to try something new, and work on a topic that wasn't within my comfort zone, and took up the CompTIA CySA+ Cybersecurity Analyst course.

I began to look into GutHub Pages and Jekyll as a platform to host the new version of this blog.

### 2020

Passed my CompTIA CySA+ Cybersecurity Analyst exam.

I changed jobs in March. On the sixth day of my new employment, the country went on lockdown and I relocated to a desk at home. That has been my workplace ever since.

This was the year inspriation hit; I decided it was time to bring my blog back to life. I converted the original SQL Server Diaries blog content from an eight year old Wordpress backup (one huge XML file) to individual Markdown files. I will be sharing the experience at a later date.

I have a lot of new ideas and can't wait to share them with you over the course of 2021.

Thank you for visiting. Stay safe and have a great day!

gha_language: HTML


text:
---
rank: 7
related_endpoints:
  - post_metadata_templates_schema#classifications
  - put_metadata_templates_enterprise_securityClassification-6VMVochwUWo_schema#add
  - put_metadata_templates_enterprise_securityClassification-6VMVochwUWo_schema#update
  - put_metadata_templates_enterprise_securityClassification-6VMVochwUWo_schema#delete
  - post_files_id_metadata_enterprise_securityClassification-6VMVochwUWo
  - post_folders_id_metadata_enterprise_securityClassification-6VMVochwUWo
related_resources:
  - classification
---

# Classifications

Box enables users and applications to apply a security classification label
to files, and cascade a classification label to folders and their contents.
Classifications help protect shared sensitive content from negligent access
through Box's Governance and Shield products.

The Classification API can be used to create new classification labels
and to assign classifications to files and folders.

<ImageFrame border center>

  ![String field](./classification-example.png)

</ImageFrame>

Classifications use the [Metadata APIs](g://metadata) to create classification
labels, and to assign classifications to files and folders. For more details on
metadata templates and instances, please explore our guides
on [Metadata](g://metadata).

## Classifications & Metadata

Working with classifications requires a developer to work with metadata
templates and instances.

* **The Classification Template:**  to work with classifications, an enterprise
 needs to have a classification metadata template with at least one
 classification. This template needs to have a `scope`/`templateKey` of
 `enterprise.securityClassification-6VMVochwUWo`. This template will hold the
 possible classification levels, their label names, descriptions, and `colorID`
 values.
* **Template Instances**: to apply a classification to a file or folder, a
  developer applies an instance of the
  `enterprise.securityClassification-6VMVochwUWo` template to the item. When the
  template is applied, one of the classifications is selected from the list of
  classifications on the template.

branch_name: refs/heads/main


text:
---
layout: post
title: Library Quantification - M.magister MBD BSseq Libraries with Qubit
date: '2020-12-02 08:58'
tags:
  - qubit
  - Dungeness crab
  - Metacarcinus magister
  - MBD
categories:
  - Miscellaneous
---
After reviewing the [Bionalyzer assays for the MBD BSseq libraries](https://robertslab.github.io/sams-notebook/2020/11/25/Bioanalyzer-M.magister-MBD-BSseq-Libraries.html) Mac indicated she'd like to have the libraries quantified using the Qubit.

I used 1uL of all samples with the 1x DNA HS Qubit Assay (Invitrogen) on the Roberts Lab Qubit 3.0.

---

#### RESULTS

Qubit data (Google Sheet):

- [20201202_qubit_DNA_mmag_MBD-libraries](https://docs.google.com/spreadsheets/d/1og3084vUozB8cQEpja3nVl3G7_reLGVq5RlvaVCkNWw/edit?usp=sharing)

Will use this data, in conjunction with the Bioanalyzer average fragment size to determine molarity of each library, in preparation for sample pooling.

All data has been added to the principal Google Sheet for tracking this project:

- [OA Crab Sample Collection 071119](https://docs.google.com/spreadsheets/d/1ym0XnYVts98tIUCn0kIaU6VuvqxzV7LoSx9RHwLdiIs/edit#gid=1430155532)

Additional details are available in this GitHub repo:

- [project-dungeness-crab](https://github.com/RobertsLab/project-dungeness-crab)

repository: RobertsLab/sams-notebook

gha_language: Shell

repo_name: RobertsLab/sams-notebook


text:
# Changelog
All notable changes to this project will be documented in this file.

## 2.0.0 (2020-09-06)

* Upgrade adapter to support extent version 5 [Issue 11](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/11)]

## 1.5.1 (2020-08-11)

* Refactor exception to fine grained in URLOutputSteam [Issue 14](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/14)

## 1.5.0 (2020-08-10)

* Add date and time to report folders [Issue 12](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/12)
* Fix report folder name issue [Issue 13](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/13)

## 1.4.0 (2020-07-21)

* Support for Json formatter [Issue 7](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/7)
* Fixed docstring display [Issue 8](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/8)

## 1.3.2 (2020-06-03)

* Bug fix for missing screenshot

## 1.3.1 (2020-05-14)

* Minor bug fix used String.value

## 1.3.0 (2020-05-12)

* Support for reading system info from extent.properties & maven cmd [Issue 4](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/4)

## 1.2.0 (2020-04-22)

* Added support for strict=true option [Issue 3](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/3)

## 1.1.0 (2020-04-21)

* Updated extentreports version to 4.1.5
* Fixed issue with undefined steps displayed as passed [Issue 1](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/1)
* Partly fixed issue with hooks getting displayed [Issue 2](https://github.com/grasshopper7/extentreports-cucumber5-adapter/issues/2)

## 1.0.0 (2020-03-22)

* Released based on extentreports-cucumber4-adapter


text:
# Backup.sh

[![Build Status](https://travis-ci.org/hmerritt/backup-script.svg?branch=master)](https://travis-ci.org/hmerritt/backup-script)

A simple script that compresses files/folders and moves them to a backup location

## Getting Started

### Install Dependencies
Install script dependencies via `install` argument.

```bash
$ sudo ./backup.sh install
```

#### Bash on Windows
Thanks to `WSL`, you can run bash scripts on windows!

- [Enable WSL (Windows Subsystem for Linux)](https://docs.microsoft.com/en-us/windows/wsl/install-win10)

### Config file
#### Create config
In order to add backups, you need to create a separate config file.
```bash
$ ./backup.sh setup
```

#### Load config
You can load the config file by adding its path after `backup`: `backup <path-to-config>`
```bash
$ ./backup.sh <path-to-config>
```
> You can have as many config files as you like (this could be used to run different backups)

## Usage

```bash
$ ./backup.sh <path-to-config>
```

### Add files/folders to backup

- Open the config file (`backup-config.sh`) and find the *`ENTER FOLDERS TO BACKUP HERE`* text
- Add a file/folder by following the syntax guide below

```bash
# ENTER FOLDERS TO BACKUP HERE
##############################
# backup "name-of-folder" "/directory-of-parent-folder/" "/directory-of-parent-backup-folder/"
# backup "profile-images" "/my/images/" "/my/backup/google-drive/images/"

backup "name of file/folder" "full path to folder" "full path to backup location"

# Lets try a more real example

# We are backing up a folder called "music" that exists in our "documents" folder
# This is then packaged up and moved to the "backup" directory

backup "music" "/home/user/documents/" "/home/user/backup/documents/"

# Original     "/home/user/documents/music/"
# Backed-up                            "/home/user/backup/documents/music.tar.gz"
```


text:
# `BCB420.2019.HPO`

###### [Rachel Silverstein](https://orcid.org/0000-0001-5724-2252<EMAIL_ADDRESS>

This project uses the Human Phenotype Ontology (Dec 21, 2018 11:56:54 AM)

### About this Package

----

This package provides instructions to create a data frame to map HGNC symbols to Human Phenotype Ontology terms associated with those genes.

Additionally, steps to annotate an example gene set with HPO terms is provided and some characteristics of this gene set are analyzed.

#### Contents

```text
--BCB420.2019.HPO/
   |__.gitignore
   |__.Rbuildignore
   |__BCB420.2019.HPO.Rproj
   |__DESCRIPTION
   |__dev/
      |__rptTwee.R      # creates this tree
      |__toBrowser.R    # display contents of README to browser
   |__inst/
      |__extdata/
         |__HGNCtoHPO.RData   # mapping data frame created below
         |__test_lseq.dat
         |__xAnnotations.txt  # annotations of example gene set
      |__img/                 # Images for the .md file
         |__fig_1.png
         |__fig_2.png
         |__fig_3.png
         |__fig_4.png
      |__scripts/
         |__fullScript.R  # contains all R code from README.md
   |__LICENSE
   |__man/
      |__lseq.Rd
   |__NAMESPACE
   |__R/
      |__lseq.R
      |__zzz.R
   |__README.md
   |__tests/
      |__testthat.R
      |__testthat/
         |__helper-functions.R
         |__test_lseq.R
```

### About the Data Source

----

The [Human Phenotype Ontology](https://hpo.jax.org/app/) is a controlled vocabulary that describes abnormal phenotypes associated with human diseases. Phenotypes are curated from the literature as well as from Orphanet, DECIPHER, and OMIM. In addition to providing the ontology, HPO provides files which annotate human genes with HPO terms associated with disruptions in that gene. It is the annotation data that is used in this project.

The version of HPO used for this project is Build #153 (Dec 21, 2018 11:56:54 AM). A 'build history' is available at [Project hpo.annotations.monthly](http://compbio.charite.de/jenkins/job/hpo.annotations.monthly/), where you can select Build #153. Note that using the [downloads tab](https://hpo.jax.org/app/download/annotation) of the [HPO page](https://hpo.jax.org/app/) may not provide the same results as this page is updated monthly with new versions of the annotations file.

### Data Download and Import

----

Navigate to [Build #153](http://compbio.charite.de/jenkins/job/hpo.annotations.monthly/153/), select 'Expand all' to see a list of files available for download. Before downloading data, note the [license restrictions](https://hpo.jax.org/app/license) that apply to all files provided by the HPO project.

Download the file entitled:
```text
ALL_SOURCES_ALL_FREQUENCIES_genes_to_phenotype.txt
```
Save this file in a sister directory of your working directory entitled `data`. `file.exists("../data/ALL_SOURCES_ALL_FREQUENCIES_genes_to_phenotype.txt")` should return `TRUE`.

Install the `readr` package in order to read the data into a data frame in R.

```R
if (!require("readr")) {
  install.packages("readr")
}
library(readr)
```

Now we will look at the first line of the text file to determine the structure of the data frame that we want to create.

```R
header <- read_lines("../data/ALL_SOURCES_ALL_FREQUENCIES_genes_to_phenotype.txt", n_max=1)
cat(header)

# Format: entrez-gene-id<tab>entrez-gene-symbol<tab>HPO-Term-Name<tab>HPO-Term-ID
```

We can see that the data file includes 4 columns, the entrez gene ID, entrez gene symbol, HPO term, and the HPO term symbol. Now that we know the contents of the columns, read in the data using `read_tsv()` from `readr`.

```R
HPOdata <- read_tsv("../data/ALL_SOURCES_ALL_FREQUENCIES_genes_to_phenotype.txt",
  comment = "#",
  col_names = c(
    'entrezID',
    'entrezSymb',
    'HPOterm',
    'HPOtermID'),
  col_types = cols(
    entrezID = col_character(),
    entrezSymb = col_character(),
    HPOterm = col_character(),
    HPOtermID = col_character()
))

head(HPOdata)

# A tibble: 6 x 4
#   entrezID entrezSymb HPOterm                         HPOtermID
#      <dbl> <chr>      <chr>                           <chr>
# 1     8192 CLPP       Seizures                        HP:0001250
# 2     8192 CLPP       Short stature                   HP:0004322
# 3     8192 CLPP       Primary amenorrhea              HP:0000786
# 4     8192 CLPP       Autosomal recessive inheritance HP:0000007
# 5     8192 CLPP       Microcephaly                    HP:0000252
# 6     8192 CLPP       Hypoplasia of the uterus        HP:0000013
```

### Structure of the Data

----

The data set retrieved from HPO contains multiple rows with the same gene since each phenotype is listed in a separate row. Since we want to make a data frame with one row for each HGNC symbol, we must condense all the different phenotypes for each gene into the same row. We will do this by creating a single string to represent all the phenotypes and separate them by the comma `,` character so that they can be easily split later on.

```R
# See how many unique genes there are in the HPO data
unique_genes <- unique(HPOdata$entrezSymb)
length(unique_genes) # 3924
# Initialize a data frame of the correct dimensions
HPO <- data.frame(matrix(nrow = 3924, ncol = 3))
colnames(HPO) <- c("entrezSymb", "entrezID", "HPOterms")

# collapse the phenotypes into one string... this may take a few minutes to run
for (i in seq_along(unique_genes)) {
  HPO$entrezSymb[i] <- unique_genes[i]
  sel <- HPOdata$entrezSymb == unique_genes[i]
  rows <- HPOdata[sel,]
  phenotypes <- rows$HPOterm
  phenoStr <- paste(phenotypes, collapse = ",")
  HPO$HPOterms[i] <- phenoStr
  ID <- unique(rows$entrezID) # makes sure they are all the same or else you will get an error since you cannot add a multi elemnt vecctor to another vector
  HPO$entrezID[i] <- ID
}
head(HPO)
```

The HPO data includes entrez ID and entrez gene symbol. NCBI reports that gene symbols are provided by HGNC so most of these symbols are likely already what we need. However, it is possible that some of the symbols are out of date since we do not know how often NCBI updates their symbols and we do not know how long ago HPO created this list of NCBI symbols.

Thus, we will create our own mapping of HGNC symbols to entrez gene IDs to make sure that we are able to map all of the genes in HPO to the most current HGNC symbol.

### Download HGNC Data

----

First, we must retrieve the relevant data from the HGNC website. Navigate to the [custom downloads section of the HGNC website](https://www.genenames.org/download/custom/). Select only "Approved symbol,	Previous symbols, and	Synonyms" under the data provided by HGNC. Select "NCBI Gene ID(supplied by NCBI)" under data downloaded from external sources. These are the entrez IDs which we will need to use to map the data. Then download the resulting data file and save it as `"HGNCdata.txt"` to a sister directory called "data" of you working directory. This should be saved in the same place as `ALL_SOURCES_ALL_FREQUENCIES_genes_to_phenotype.txt` which was saved previously.

Now that the data is downloaded, proceed to load this data into R.

```R
HGNC <- HPOdata <- read_tsv("../data/HGNCdata.txt",
                            skip = 1,
                            col_names = c(
                              'sym',
                              'prevSym',
                              'synonyms',
                              'entrezID'),
                            col_types = cols(
                              sym = col_character(),
                              prevSym = col_character(),
                              synonyms = col_character(),
                              entrezID = col_character()
                            ))

```

### Mapping HGNC symbols to HPO terms

----

Now let's clean up the data frame so that all of the entrez IDs are unique and can be used for mapping...

```R
# We want this data frame to be searchable by entrezID so they should be unique
sel <- duplicated(HGNC$entrezID)
((HGNC$entrezID)[sel]) # all the duplicated ones are just missing values. Remove these.
sel <- is.na(HGNC$entrezID)
# keep the HGNC symbols with missing IDs in a data frame for later
missingIDs <- HGNC[sel,]
# remove them from HGNC
HGNC <- HGNC[!sel,]
# now there should be no duplicate entrezIDs so we can set them as the rownames
rownames(HGNC) <- HGNC$entrezID
```

Now that all the entrez IDs in the HGNC data frame are unique, we can use them to map the entrez symbols in the HPO data to their respective HGNC symbols. First, though, we will see which of the HPO data are already mapped to appropriate HGNC symbols.

```R
# check that all the HGNC symbols in HPO data are correct
HGNC_syms <- HGNC$sym
incorrect_sym <- !(HPO$entrezSymb %in% HGNC_syms)
sum(incorrect_sym) # 38 of the gene symbols in HPO are not current HGNC symbols

# get the rows with incorrect symbols
incorrect <- HPO[incorrect_sym,]
# and remove them from the HPO data frame
nrow(HPO) # 3924
HPO <- HPO[!incorrect_sym,]
nrow(HPO) # 3924 - 38 = 3886... correct

# Map the incorrect symbols to the correct ones using the entrezID
for (i in seq_along(incorrect$entrezID)) {
  entrezID <- incorrect$entrezID[i]
  HGNCrow <- HGNC[entrezID,]
  correctSym <- HGNCrow$sym # if the entrezID is not in the HGNC data frame this will be NA
  if (!is.na(correctSym)) {
    incorrect$entrezSymb[i] <- correctSym
  }
}

# check whether there are still any entrez symbols that are wrong
incorrect_sym <- !(incorrect$entrezSymb %in% HGNC_syms)
sum(incorrect_sym) # 2
# This step corrected 36 of the 38 wrong HGNC symbols. Let's add the corrected ones to the HPO data frame.
corrected <- incorrect[!incorrect_sym,]
incorrect <- incorrect[incorrect_sym,]
HPO <- rbind(HPO, corrected)

# Now let's look at the ones that are still incorrect:
(incorrect)

# entrezSymb  entrezID
# 551     H19-ICR 105259599
# 1481    HBB-LCR 109580095

sel <- grepl('H19-ICR', HGNC$synonyms)
sum(sel) # 0. H19-ICR is not listed as a synonym
sel <- grepl('H19-ICR', HGNC$prevSym)
sum(sel) # 0. H19-ICR is not listed as a previous symbol
('H19-ICR' %in% missingIDs$sym) # FALSE, it is not one of the genes that had a missing entrezID and was removed from the HGNC data frame.

sel <- grepl('HBB-LCR', HGNC$synonyms)
sum(sel) # 0. HBB-LCR is not listed as a synonym
sel <- grepl('HBB-LCR', HGNC$prevSym)
sum(sel) # 0. HBB-LCR is not listed as a previous symbol
('HBB-LCR' %in% missingIDs$sym) # FALSE, it is not one of the genes that had a missing entrezID and was removed from the HGNC data frame.

# We will omit these 2 genes from the final data frame as we were unable to map them to an HGNC symbol.

# Check that all the HGNC symbols in our final mapping are valid:
sum(!(HPO$entrezSymb %in% HGNC$sym)) # 0... correct

# Now we can remove the entrez IDs from the HPO data frame to make the mapping tool
HGNCtoHPO <- HPO[, c("entrezSymb", 'HPOterms')]
head(HGNCtoHPO)

#And name the entrez gene column "HGNCsym" since they are now all HGNC symbols
colnames(HGNCtoHPO) <- c("HGNCsym", "HPOterms")
rownames(HGNCtoHPO) <- HGNCtoHPO$HGNCsym

# save the mapping tool
save(HGNCtoHPO, file = file.path("inst", "extdata", "HGNCtoHPO.RData"))
```

### Analysis of the Entire Gene Set Provided by HPO

----

Now we have created a data frame that maps all of the HPO phenotype annotations to current HPNC symbols. First, let's look at what fraction of all HGNC symbols have HPO annotations.

```R
# Coverage
nrow(HGNCtoHPO)/nrow(HGNC)*100 # 9.441729
# the HPO database only lists phenotypes for about 10% of human genes.
```

The HPO has annotations for about 10% of human genes. This makes sense since we do not know how and if most genes contribute to phenotype in humans.

Now let's see how many phenotypes are associated with each gene in the HPO data set.

```R
number_of_phenotypes <- 0
for (i in seq_along(HGNCtoHPO$HGNCsym)) {
  phenoVec <- unlist(strsplit(HGNCtoHPO$HPOterms[i], split = ","))
  phenoNum <- length(phenoVec)
  number_of_phenotypes[i] <- phenoNum
}

hist(number_of_phenotypes, breaks = 100, xlab = "Number of Phenotypes",
     ylab = "Number of Genes", main = "Number of phenotypes associated \n with all HPO annotated genes")
```

![](./inst/img/fig_1.png?sanitize=true "Phenotype distribution")

The number of phenotypes associated with each gene seems to decay exponentially. A few genes are associated with many phenotypes while many genes are associated with few phenotypes. It is not clear whether this reflects the biology of the genes or whether the genes with many phenotype annotations are simply better-studied than other genes.

```R
mean(number_of_phenotypes) # 35

```

The mean number of phenotypes for each gene is 35.

Now let's see how many genes are associated with each phenotype.
```R
# How many genes associated with each phenotype?
phenoLst <- strsplit(HGNCtoHPO$HPOterms, split = ",") # list of vectors
phenoVec <- unlist(phenoLst, recursive = FALSE)
phenoTab <- table(phenoVec)
phenoTab <- data.frame(phenoTab)
barplot(height = sort(phenoTab$Freq, decreasing = TRUE), xlab = 'Phenotype', ylab = 'Number of Genes',
        main = 'Number of genes associated \n with each phenotype')
```

![](./inst/img/fig_2.png?sanitize=true "gene distribution")

What is the mean number of genes for which a phenotype is associated?
```R
mean(phenoTab$Freq) # average phenotype associated with about 20 genes
(20/nrow(HGNCtoHPO) * 100) # average phenotype associated with about 0.5% of the genes in HPO data set
```

The average phenotype is associated with abot 20 genes which corresponds to about 0.5% of the total genes that have HPO annotations.

What phenotypes are the most common?

```R
# list the top 10 most common phenotypes
sel <- order(phenoTab$Freq, decreasing = TRUE)
ordered <- phenoTab[sel,]
(ordered[1:10,])

# phenoVec Freq
# 1296 Autosomal recessive inheritance 2187
# 3933         Intellectual disability 1780
# 1294  Autosomal dominant inheritance 1373
# 3155      Global developmental delay 1203
# 6112                        Seizures 1196
# 6239                   Short stature  895
# 4906                       Nystagmus  723
# 4517                    Microcephaly  713
# 3102           Generalized hypotonia  706
# 4694              Muscular hypotonia  682
```

### Annotations of the Example Gene Set

----

The example gene set (xSet) was copy and pasted from the BCB420 resources repository on GitHub.

These genes are functionally related in that they are associated with the phagosome / lysosome fusion system.

```
# ANNOTATION OF EXAMPLE GENE SET
xSet <- c("AMBRA1", "ATG14", "ATP2A1", "ATP2A2", "ATP2A3", "BECN1", "BECN2",
          "BIRC6", "BLOC1S1", "BLOC1S2", "BORCS5", "BORCS6", "BORCS7",
          "BORCS8", "CACNA1A", "CALCOCO2", "CTTN", "DCTN1", "EPG5", "GABARAP",
          "GABARAPL1", "GABARAPL2", "HDAC6", "HSPB8", "INPP5E", "IRGM",
          "KXD1", "LAMP1", "LAMP2", "LAMP3", "LAMP5", "MAP1LC3A", "MAP1LC3B",
          "MAP1LC3C", "MGRN1", "MYO1C", "MYO6", "NAPA", "NSF", "OPTN",
          "OSBPL1A", "PI4K2A", "PIK3C3", "PLEKHM1", "PSEN1", "RAB20", "RAB21",
          "RAB29", "RAB34", "RAB39A", "RAB7A", "RAB7B", "RPTOR", "RUBCN",
          "RUBCNL", "SNAP29", "SNAP47", "SNAPIN", "SPG11", "STX17", "STX6",
          "SYT7", "TARDBP", "TFEB", "TGM2", "TIFA", "TMEM175", "TOM1",
          "TPCN1", "TPCN2", "TPPP", "TXNIP", "UVRAG", "VAMP3", "VAMP7",
          "VAMP8", "VAPA", "VPS11", "VPS16", "VPS18", "VPS33A", "VPS39",
          "VPS41", "VTI1B", "YKT6")
```

Let's determine how many of the genes in the example gene set have HPO annotated phenotypes.

```R
# Determine which of these genes have HPO annotations
sum(xSet %in% HGNCtoHPO$HGNCsym) # 21
sum(xSet %in% HGNCtoHPO$HGNCsym) / length(xSet) * 100 # 24.70588
# About 25% of the genes in the example gene set have annotated phenotypes
# this is enriched over 2x compared to all genes
```

About 25% of the genes in the example set have HPO phenotypes which is enriched over 2 times compared to the total gene set. Perhaps this is because they are important genes since they are involved in an important biological process and thus more likely to have phenotypes. Alternatively, they may be more well-studied genes than the average gene.

Finally, let's create a data frame of the annotations for the example gene set and save it.

```R
# Create a data frame of these annotations
# list phenotypes as the empty string if there are not HPO annotations
xAnnotations <- data.frame(matrix(ncol = 2, nrow = length(xSet)))
colnames(xAnnotations) <- c("HGNCsym", "HPOterms")
for (i in seq_along(xSet)) {
  sym <- xSet[i]
  xAnnotations$HGNCsym[i] <- sym
  phenotypes <- HGNCtoHPO[sym, "HPOterms"]
  if (!is.na(phenotypes)) {
    xAnnotations$HPOterms[i] <- phenotypes
  } else {
    xAnnotations$HPOterms[i] <- ""
  }
}

# save this annotated example set
write_tsv(xAnnotations, "./inst/extdata/xAnnotations.txt")
```

The annotated gene set is saved as a tsv as `"./inst/extdata/xAnnotations.txt"`.

### Analysis of the Example Gene Set

----

How many phenotypes are associated with each gene?

```R
number_of_phenotypes2 <- 0
nonZero <- xAnnotations[xAnnotations$HPOterms != "",]
for (i in seq_along(nonZero$HGNCsym)) {
  phenoVec <- unlist(strsplit(nonZero$HPOterms[i], split = ","))
  phenoNum <- length(phenoVec)
  number_of_phenotypes2[i] <- phenoNum
}

hist(number_of_phenotypes2, breaks = 6, xlab = "Number of Phenotypes",
     ylab = "Number of Genes", main = "Number of phenotypes associated \n with Example Gene Set")

```
![](./inst/img/fig_3.png?sanitize=true "pheno distribution example")

The number of phenotypes per gene also decays (exponentially?) as in the larger HPO data set.

```R
mean(number_of_phenotypes2) # 41
```

The mean number of phenotypes per gene is larger than for the entire HPO data set. Once again, perhaps this is because they are well studied or biologically important genes (or both).

```R
# How many genes per phenotype?
phenoLst <- strsplit(nonZero$HPOterms, split = ",") # list of vectors
phenoVec <- unlist(phenoLst, recursive = FALSE)
phenoTab <- table(phenoVec)
phenoTab <- data.frame(phenoTab)

barplot(height = sort(phenoTab$Freq, decreasing = TRUE), space = 0.5, xlab = 'Phenotype', ylab = 'Number of Genes',
        main = 'Number of genes associated \n with each phenotype in Example Set')
```

![](./inst/img/fig_4.png?sanitize=true "gene distribution example")

```R
mean(phenoTab$Freq) # average phenotype associated with about 1.5 genes
(1.5/85 * 100) # average phenotype associated with 1.8 % of genes in list (enriched from 0.5% for all HPO data)
```

The average phenotype is associated with 1.5 genes which corresponds to 1.8 percent of genes in the example data set. This is more than the 0.5 percent of genes that are associated with the average phenotype in the entire HPO dataset. This is as expected since all of the genes in the example set are involved in the same process. Thus, we would expect them to have more phenoypes in common than a group of unrelated genes.

What are the most common phenotypes associated with the example gene set?

```R
# list the top 10 most common phenotypes
sel <- order(phenoTab$Freq, decreasing = TRUE)
ordered <- phenoTab[sel,]
(ordered[1:10,])

# phenoVec Freq
# 323         Intellectual disability   14
# 88  Autosomal recessive inheritance   10
# 87   Autosomal dominant inheritance    9
# 492                        Seizures    8
# 192                      Dysarthria    7
# 263      Global developmental delay    7
# 83                           Ataxia    6
# 59                        Agitation    5
# 65    Amyotrophic lateral sclerosis    5
# 72                          Anxiety    5
```

We see some phenotypes that are the same as the most common phenotypes the entire HPO data set but also some that are unique to this group.

### References and Acknowledgements

----

Sebastian Köhler, Leigh Carmody, Nicole Vasilevsky, Julius O B Jacobsen, et al. Expansion of the Human Phenotype Ontology (HPO) knowledge base and resources. Nucleic Acids Research. (2018) doi: 10.1093/nar/gky1105

This package was based off of the [RPT package by Boris Steipe](https://github.com/hyginn/rpt).


text:
# MDSmonkey
MDSmonkey helps the user explore a new (to the user) MDSplus database and read
the array-type data into xarray DataArray objects. See use example below to get
started. This assumes that you are set up to use the thick-client Python
interface to the MDSplus tree you want to access. MDSmonkey was designed and
tested to work on the NSTX / NSTX-U MDSplus installation.

Dependencies:
    django
    MDSplus
    xarray (optional but recommended)

Usage example to get started:

    >from MDSmonkey import MDSmonkey as mm
    >tree = mm.get_tree('efit01', 134020) #mdsplus tree type returned
    >tree  #hit <Return> to print attribute list, or tab-complete in ipython
     name: number of sub-branches
     analysis: 3
     raw_data: 2
     tags: 103 #special branch to separate out the tags so you aren't overwhelmed
     help        #no sub-branches
    >tree.analysis
     flux: (time: 20, z: 50, r: 30)
     current: (time: 20, z: 50, r: 30)
     q_on_axis: (time: 20)
    >flux = tree.analysis.flux.data #Now you will receive an xarray data-array
    >flux #return to print, will show array dimensions & names & units.
          #If xarray is not installed, these will instead be branches

repository: zack-vii/MDSmonkey

repo_name: zack-vii/MDSmonkey


text:
---
redirect_to: https://danburzo.ro/react-recipes/setting-up-shop/
---

# Setting up shop: Start a React project from scratch

Before we can actually start writing React code, we need to set a few things up. We can't just include our code and libraries in `<script>` tags and call it a day1, since we'll be relying on JSX syntax and more recent additions to JavaScript (such as modules and classes).

We need to transform our code before it works in browsers. In particular, a React project needs:

__A tool to _transpile_ your code__. A transpiler transforms your code from JSX / fancy JavaScript to normal JavaScript that works in browsers; [Babel](https://babeljs.io) and [Bublé](https://buble.surge.sh/guide/) are some popular choices.

__A tool to _bundle_ your JavaScript modules__, and any code you import from npm modules, into a single JavaScript file. The bundler usually has a way to communicate with the _transpiler_ so that your JavaScript is transformed to browser-compatible code in the process. It will also handle other types of files, such as HTML, CSS, JSON, or SVG, so you can import them in your project as you would normal JS modules. [Webpack](https://webpack.js.org/), [Rollup](https://rollupjs.org/), and [Parcel](https://parceljs.org/) are examples of bundlers.

Parcel, a bundler that works for most things we need without requiring any configuration, is a good choice for quickly setting up a React playground.

__Note:__ I'm on macOS, but the steps should be similar enough on other operating systems.

## Setting up

### Prerequisites

Like many other JavaScript libraries, React is distributed through [npm packages](https://docs.npmjs.com/getting-started/what-is-npm); You add it to your project with the command line.

To start a React project we need:

* A command-line such as the Terminal in macOS
* [Node.js](https://nodejs.org/en/) and npm (if you installed Node, it comes with the `npm` command-line tool)

### Create, and then initialize, your project

Make the `my-project` folder and navigate to it using these commands:

```bash
mdkir my-project
cd my-project
```

Before we can install React, we need a `package.json` file that lists all the dependencies in our project. We create it with:

```bash
npm init --yes
```

The `--yes` flag instructs npm to skip all the questions and just initialize a run-of-the-mill `package.json` that looks like this:

```json

  "name": "my-project",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}
```

### Install React and ReactDOM

We install the `react` and `react-dom` packages as dependencies to our project:

```bash
npm install react react-dom
```

### Install the Parcel bundler

We install the Parcel bundler as a _development_ depenency to our project — our app only depends on Parcel while we're building it2.

```bash
npm install --save-dev parcel-bundler
```

### Create a project structure

While you can set up your project's structure in many ways, let's keep it simple and just add a couple of folders:

```
my-project
  dist/
  src/
```

__src/__ will hold our application's source code: HTML, JavaScript, CSS, SVG, images and all we want to include. The __dist/__ folder is where our app will get built, so we can deploy it to a server.

To make these folders, we can run:

```bash
mkdir src dist
```

Let's make an _index.js_ file inside our `src` folder:

__src/index.js__

```js
console.log('Hello world');
```

And a _index.html_ file as well, where we fetch our `index.js` script:

__src/index.html__

```html
<!DOCTYPE html>
  <html>
    <head>
      <meta charset='utf-8'/>
      <title>My project</title>
    </head>
    <body>
      <div id='app'>
        <!-- Our React App goes here -->
      </div>

      <script src='./index.js'></script>
    </body>
</html>
```

If you open `index.html` in a browser and open the Developer Tools, you should see the `Hello World` message in the console. This means we've properly linked the script inside the HTML file.

Let's now configure Parcel to process our files so we can start writing some modules.

### Add scripts to `package.json`

In `package.json` we add two scripts:

* _start_ will make Parcel watch for changes in our source files and continously build our bundle
* _build_ will build an optimized version of our code that we can then deploy to a server.

__package.json__

```json
{
  "name": "my-project",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "parcel src/index.html",
    "build": "parcel build src/index.html"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "react": "^16.13.1",
    "react-dom": "^16.13.1"
  },
  "devDependencies": {
    "parcel-bundler": "^1.12.4"
  }
}
```

Let's run the scripts! To start the development server:

```bash
npm run-script start
```

The command outputs:

```bash
><EMAIL_ADDRESS>start /Users/danburzo/projects/my-project
> parcel src/index.html

Server running at http://localhost:1234
✨  Built in 1.20s.
```

Opening [http://localhost:1234](http://localhost:1234) in your browser lets you access your app. When you make changes to your source files, the app automatically reloads with the changes. To close the server, press Ctrl + C in the Terminal window.

To build an optimized bundle suitable for putting it in front of users:

```bash
npm run-script build
```

The command outputs:

```bash
><EMAIL_ADDRESS>build /Users/danburzo/projects/my-project
> parcel build src/index.html

✨  Built in 1.80s.

dist/src.67e05423.js        1.13 KB    110ms
dist/src.67e05423.js.map      189 B      3ms
dist/index.html               166 B    1.63s
```

This builds your whole app in the `dist/` folder, the content of which you can then take and put up on a server.

Now that we've set up Parcel, we can finally start writing Fancy JavaScript™.

### Create your React App

Under `src/`, we create a `components/` folder and write a simple _Hello world_ React component:

__src/components/App.js__

```jsx
import React from 'react';

class App extends React.Component {
  render() {
    return "Hello world!";
  }
}

export default App;
```

And then, in our _index.js_ file, we import the module containing our React component and [add it to our page](./inserting-components.md).

__src/index.js__

```jsx
import React from 'react';
import ReactDOM from 'react-dom';

import App from './components/App';

ReactDOM.render(<App/>, document.getElementById('app'));
```

If we head back to the browser, the app should now display the message _Hello world!_ on-screen, which means we've successfully set up a React project. 🙌

### A dash of CSS

To demonstrate how to load other types of files, such as CSS, let's make a minimal stylesheet for our app:

__src/style.css__

```css
body {
  font-family: sans-serif;
}
```

In our main JS file, _index.js_, we import the CSS much like we would a normal JS module:

__src/index.js__

```jsx
import React from 'react';
import ReactDOM from 'react-dom';

// importing CSS files in your app will create a CSS bundle
// that gets automatically included in your index.html file.
import './style.css';

import App from './components/App';

ReactDOM.render(<App/>, document.getElementById('app'));
```

The browser should now show us the same _Hello World!_ message, but now with a different font.

And that, friends, is a wrap! 🙌

## Alternatives

I hope you found this article, at the end of which we have pretty much all we need to start playing with React, not too long, nor hard to follow. There are other ways to get a similar result:

* Use the [`create-react-app`](https://github.com/facebook/create-react-app) command-line tool to set up a React environment even faster; it's frankly a great experience, with the caveat that it comes with a bunch of extras that make it harder to discern what's going on when you're first starting out.
* Take the scenic route and walk through this [excellent article](https://blog.usejournal.com/creating-a-react-app-from-scratch-f3c693b84658) on setting up React with Babel and Webpack, if you prefer to expose yourself to some of the unsightly configuration that Parcel sweeps under the rug for you.

## A couple of notes on Git / GitHub

Not to stretch this article further, but a few quick notes if you put your project on GitHub.

First off, you should create a `.gitignore` file in your root folder to which you add the following lines:

```
node_modules/
dist/
```

This instructs Git not to include the two folders in version control. We don't want these hanging around on Github.

And if you're curious on how to publish the content of the `dist/` folder to GitHub Pages, check out the [`gh-pages`](https://npmjs.org/package/gh-pages) package.

---

1 [Technically you can](https://reactjs.org/docs/getting-started.html#online-playgrounds), for small experiments on your machine. You'll probably want to avoid publishing them as-is on the web, though.

2 When bundling JavaScript packages, all dependencies are technically _development dependencies_, because they're only ever needed as you build the project. However, it's useful to make a distinction between things that end up in the bundle and those who don't.

repository: olmesm/react-recipes

repo_name: olmesm/react-recipes


text:
# FKFirebaseKit - a Firebase iOS SDK Wrapper

**FKFirebaseKit** is an opensource framework simplifying **Firebase iOS SDK**'s boilerplate code usage and its documentation. It's totally built on the Firebase iOS SDK and wraps it for you by aiming to **boost your productivity and  improve code quality** to let you focus on the real valuable part of your iOS app.

  - Simple methods with internal Error Handling.
  - Grouping methods for redability/usability as GET, SET, UPDATE, DELETE and LIST.
  - Internal logger to track request and response easily in hard-to-read Xcode log trace window.
  - Easy-to-use Authentication with Phone feature provided by Firebase.

# Why to use FKFirebaseKit instead Firebase iOS SDK?

FKFirebaseKit's API is redesigned for developer friendly usage. If you want to create a page listening and listing the user list, it's super easy with FKFirebaseKit.

Do you want to make an app having phone authentication with OTP? Lets look at that Interaction layer of an example VIPER module for both FKFirebaseKit and Firebase iOS SDK code comparison.

FKFirebaseKit handles the filtering as human-readable. Just look at that code block comparison filtering phones with a match case. You can replace .equal with .prefix to get all phones starting with the prefix wanted.

# Installation

FKFirebaseKit SDK **requires iOS 11 and newer** to run.

```
pod 'FKFirebaseKit'
```

...or for spesific version

```
pod 'FKFirebaseKit',  '~> 0.1.0'
```

# Usage

### Endpoint

You can see an endpoint argument in that all types of request methods given below. Endpoint is the exact child path that we want to send request. It's an String of Array. You can add all children to the String of Array with the same order in the Firebase Realtime Database structure.

Assume you have a database scheme and want to update the user's informations whose key is "1dsa-1dsa-adas".

```
{
  "Users" : {
    "1dsa-1dsa-adas" : {
      "createdAt" : "2020-06-18T09:55:39Z",
      "password" : "aaa",
      "phone" : "+445121231122"
    },
    "oajsd-ofc-2131" : {
      "createdAt" : "2020-06-18T10:00:45Z",
      "password" : "112233",
      "phone" : "+91000000000"
    },
    "os4bBojJPogdSvtQerydnBN6Leg2" : {
      "createdAt" : "2020-06-18T08:25:51Z",
      "password" : "1234",
      "phone" : "\"+1444444444"
    }
  }
}
```

In that case, your endpoint must be ["Users", "1dsa-1dsa-adas"]

### Response

In FKFirebaseKit, there is a constant response model which is Array of ResponseModel. ResponseModel is a generic Struct including T which is your exact Codable model to decode.

For example, consider your database scheme is like above. Your response must be ```[ResponseModel<UserModel>]``` for that request getting all users.

ResponseModel is so simple and consists of just two parameter.

```
public struct ResponseModel<T: Codable>: Codable {
    public var key: String?
    public var result: T?
}
```

**key:** It's the parent key of your inner object. In this example, it's childByAutoId like 1dsa-1dsa-adas, oajsd-ofc-2131, os4bBojJPogdSvtQerydnBN6Leg2.

**result:** It's the inner and exact object. All result models must be implemented **Codable and Initializable protocols** with **FKFirebaseKit import**. Becasuse Initializable protocol is a part of FKFirebaseKit and required while converting Dictionary which is response type of Firebase to custom Codable model.

```
import Foundation
import FKFirebaseKit

struct UserModel: Codable, Initializable {
    var phone: String?
    var password: String?
    var createdAt: Date?
}
```

### Set Request
You can save any data which must implement Codable easily. As a default, childByAutoId is false and it's required if you save your objects under a unique key created by Firebase. If childByAutoId is not need, just ignore and don't pass to the method.

```
let modelForSaving = UserModel(phone: "+4400000000", password: "G0asxıf1", createdAt: Date())
FKFirebaseKitManager.shared.request(set: modelForSaving, endpoint: ["Users"], childByAutoId: true) {
    print("Success")
} onError: { (message: String) in
    print(message)
}
```

### Get Request
There are two types of get request. They are listen and once. Listen type of get requests return the response to user and continue to listen the remote database if data being listened is changed. If any change occurs in realtime database, the method is triggered.

```
var observer = FKFirebaseKitManager.shared.request(get: .listen, endpoint: ["Users"]) { (response: [ResponseModel<UserModel>]) in
    for item in response {
        print(item.key)
        print(item.result)
    }
} onError: { (message: String) in
    print(message)
}
```
Get method with listen type has a return value as UInt. This UInt is created by Firebase to point observer. So If the method above is not required anymore, for example you push another ViewController, or you don't care if remote data is changed or not, just remove the observer reference. You must pass the same endpoint used in the above method to remove observer correctly.

```
FKFirebaseKitManager.shared.remove(observer: observer, for: ["Users"])
```

Another get request type is the method only called once. Here an example. Default get request is type of .once. So don't need to specify get: parametere as .once and you can ignore it.
```
FKFirebaseKitManager.shared.request(endpoint: ["Users"]) { (response: [ResponseModel<UserModel>]) in
    for item in response {
        print(item.key)
        print(item.result)
    }
} onError: { (message: String) in
    print(message)
}
```

### Update Request
Assume you have a profile page for an app and just want to update an information of a user. Just jump the related user in the realtime database structure with Users > AUTO_CHILD_KEY_OF_THE_USER path and update the object.

```
let modelForUpdating = UserModel(phone: "+4400000000", email<EMAIL_ADDRESS>updatedAt: Date())
FKFirebaseKitManager.shared.request(update: modelForUpdating, endpoint: ["Users", AUTO_CHILD_KEY_OF_THE_USER]) {
    print("Updated user data.")
} onError: { (message: String) in
    print(message)
}
```

### Delete Request
It is enough to just jump the related child to remove data.

```
FKFirebaseKitManager.shared.request(delete: ["Users", AUTO_CHILD_KEY_OF_THE_USER]) {
    print("Account of the user removed.")
} onError: { (message: String) in
    print(message)
}
```

### Listen Request
There are 4 types of listenChild request. They are .added, .changed, .removed, .moved. Both of them listes the childs with event type. So the request type returns a observer key as UInt. The management of the observer key must be tracked in application lifecycle like described in Get Request.

Just get each user records newly added by listening the remote database.
```
FKFirebaseKitManager.shared.listenChild(forChild: .changed, endpoint: ["Users"]) { (response: [ResponseModel<UserModel>]) in
    // Do whatever you want with up-to-date data.
    // For example, update the table view.
} onError: { (message: String) in
    print(message)
}
```

### List Request
There are 5 types of filtering queries that .prefix, .starting, .ending, .equal, .between. You can use that filter methods by specifying the key that will be filtered.
```
FKFirebaseKitManager.shared.list(key: "age", filterBy: .between(22, 25), endpoint: ["Users"]) { (response: [ResponseModel<UserModel>]) in
    // Parse user datas whose ages between 22 and 25
} onError: { (message: String) in
    print(message)
}
 ```
 .. or
 ```
FKFirebaseKitManager.shared.list(key: "creditCard", filterBy: .prefix("3512"), endpoint: ["Users"]) { (response: [ResponseModel<UserModel>]) in
    // Get all users whose credit card numbers starts with 3512.
} onError: { (message: String) in
    print(message)
}
```

### Authentication with Phone
FKFirebaseKit is only supporting authentication with phone option by v0.1.0. Other options like email, sign in with Apple and other social login authorizations will be supported in the v1.0. You can review Backlog chapter.

Sending OTP with phone number.
```
FKAuthenticationManager.shared.languageCode = "tr"
FKAuthenticationManager.shared.phoneCode = "+90"

FKAuthenticationManager.shared.verify(phoneNumber: phone) {
    self.presenter?.verifyOTP()
} onError: { (message :String) in
    print("Error: \(message)")
}
```

Verifying OTP sent to user. Just input the OTP code from UITextField and pass to the verify method.

```
FKAuthenticationManager.shared.verify(otp: code) {
    print("Registered successfully")
    self.presenter?.otpVerified()
} onError: { (message: String) in
    print("Error: \(message)")
}
```

Here another authentication method check and get current user informations.

```
print("isLoggedIn: \(FKAuthenticationManager.shared.isLoggedIn)")
print("currentUser: \(FKAuthenticationManager.shared.currentUser?.phoneNumber)")
print("logout: \(FKAuthenticationManager.shared.logout())")
print("isLoggedIn: \(FKAuthenticationManager.shared.isLoggedIn)")
```

Possible output of the print group given above.

```
isLoggedIn: true
currentUser: Optional("+4400000000")
isLoggedIn: false
```

# Backlog

  - Implementation of Cloude Firestore.
  - Expanding authentication options.
  - Better log trace.
  - Detail the documentation with Firebase iOS SDK setup.
  - Limit queries with a number in order to last or first.
  - Your valuable issues and pull-requests

# Development

Want to contribute? Great! 🍺

Any issue, fork, star, pull-request is appreciated. 🚀

# Dependency
  - Firebase iOS SDK with v7.1.0

# Resource

If you want to view official annoucement about the Firebase iOS SDK and read the documentation just briefed above,

| Topic | Link |
| ------ | ------ |
| Setup | https://firebase.google.com/docs/database/ios/start |
| Release Notes | https://firebase.google.com/support/release-notes/ios |
| Recommended Data Structure | https://firebase.google.com/docs/database/ios/structure-data |
| CRUD | https://firebase.google.com/docs/database/ios/read-and-write |
| List Processes | https://firebase.google.com/docs/database/ios/lists-of-data |

# Author
  - **Furkan Kaplan** https://github.com/furkankaplan
  - Twitter : [@kaplanfurkan07](https://twitter.com/kaplanfurkan07)
  - LinkedIn : [@furkankaplan07](https://www.linkedin.com/in/furkankaplan07/)
  - Email<EMAIL_ADDRESS>

repository: furkankaplan/fk-firebase-kit-ios

gha_language: Swift

repo_name: furkankaplan/fk-firebase-kit-ios


text:
# lazy-worksheets

This library is designed for language teachers to produce small worksheets for students using text/articles from websites.

## Installation instuctions
To import all the python dependencies in the terminal run:
`pip install -r requirements.txt`

In order to import the (English) language model go to the terminal and run:
`python -m spacy download en_core_web_sm `

To sort out the (potential) issues with nltk
see here:
https://stackoverflow.com/questions/54876404/unable-to-import-sqlite3-using-anaconda-python

Basically you need to find the sqlite3.dll which on ly computer was
in C:\Users\USER\anaconda3\Library\bin
and paste it somewhere your virtual environment can find
I put put here:
C:\Users\USER\.conda\envs\language-education\Scripts

repository: aulaLanguageEducation/lazy-worksheets

gha_language: Python

repo_name: aulaLanguageEducation/lazy-worksheets


text:
# Visual Transfer Tool
Transfers NPC appearence from specified mods even if overriden by higher prioority onces

You're going to need the latest [zEdit](https://github.com/z-edit/zedit) release

repository: mangomango/zedit-visual-transfer

repo_name: mangomango/zedit-visual-transfer


text:
[![Code Climate](https://codeclimate.com/repos/53ca49ba69568036df008861/badges/92a10b8438ef2bed5e79/gpa.png)](https://codeclimate.com/repos/53ca49ba69568036df008861/feed)
[![Test Coverage](https://codeclimate.com/repos/53ca49ba69568036df008861/badges/92a10b8438ef2bed5e79/coverage.png)](https://codeclimate.com/repos/53ca49ba69568036df008861/feed)

## Training application

I'm just developing my RoR skills thanks to Monterail

### Features

1.  DONE ! Users need to create profiles to add questions and answers.
2.  DONE ! Question has title and contents. Title should be required.
3.  DONE ! Answer has only contents. It should be required.
4.  DONE ! Users can update their questions, but not answers.
5.  DONE ! Users can like answers added by users. It should be visible how many likes each answer has received.
6.  DONE ! Question's author can accept one answer to the question. This answer should be marked as "Accepted".
7.  DONE ! No one can add new answer to a question that already has an accepted answer.
8.  DONE ! Users collect points. New users get 100 points for free.
9.  DONE ! When user's answer is accepted she receives 25 points, when liked she receives 5 points.
10. DONE ! Creating a question costs 10 points.
11. DONE ! Users can set their names and it should be displayed everywhere instead of e-mail.

  Hint: User profiles are implemented using [devise](https://github.com/plataformatec/devise) gem. Check its documentation to find out how to handle additional attributes on registration and edit profile pages.

12. DONE ! Users can upload avatars which should be automatically scaled to 100x100px size.

  Hint: Use a gem that handles file uploads and image manipulation.

13. Once a user reaches 1000 points, she receives Superstar badge that is visible on his profile page.
14. Question's author receives e-mail notification when someone answers his question.
15. User receives e-mail notification when his answer is accepted.
16. DONE ! There is a leaderboard page where users are sorted by points.

17. E-mails are sent via background jobs.

  Often in production environments, time consuming tasks are performed in background, not during user's request. Sending e-mails is such a task because it involves communication with remote servers which may be slow or may not respond. We do not want to make our user wait that long for the response, so we need to respond immediately as if the e-mail has been succesfully sent, but send it in background.

18. Users can login using their GitHub accounts.

  People don't like passwords. To make it easy for them many sites allow logging in with Facebook, Twitter or GitHub accounts in a secure way. We're going to use just GitHub.

19. DONE ! Liking answers should not reload the page.

  Let's improve user experience a bit. When user likes an answer he should immediately see the result without reloading the whole page. Hint: Try using jQuery javascript library to perform AJAX requests.

20. Questions and aswers can be written in Markdown format.

  This `README` file is in Markdown. GitHub automatically formats it and displays a nice readable HTML, we want the same with question and answer contents.

### Development Guidelines

1. Create small, atomic commits.
2. Try to use English for everything from variable names to commit messages.
3. We're using [slim](http://slim-lang.com/) templating engine for views, not ERB which is a default in Ruby world. It's quite easy, but if you don't feel comfortable, feel free to use ERB. The same thing stands for other gems. It's your code, use whatever suits you.
4. Some features are defined in files in `features/features` directory. If you're curious, they're written in Cucumber and this is executable code. You can run it with `bundle exec cucumber` and check if your code satisfies these definitions.
5. If you have any problems ask for help in our public [Chat Room](https://www.hipchat.com/gVsjIkRpD). We really mean it! There's nothing wrong in seeking advice. (Please pay attention to the topic message)

## License

See attached LICENSE.txt file.

Copyright (c) 2014 Monterail.com LLC


text:
# `src/cold`

This directory holds partial HTML files that will be surrounded with a header
and footer at build-time and rendered into pages.

For the most part, things here originated as Markdown from [`src/warm`](../warm)
but got too complex to keep in Markdown, or got stable enough that I felt better
"freezing" them into HTML one last time.

gha_language: Go


text:
# React Global State

`yarn add @ejp/react-global-state`

Share state around an app with familiar React setState like syntax.

_Or yet another globel state manager for React._

---

Full description coming...

Project was bootstraped with the excellent [TSDX](https://github.com/jaredpalmer/tsdx) framework.

repository: edwardpayton/react-global-state

gha_language: TypeScript

repo_name: edwardpayton/react-global-state


text:
# redux-axios-middleware

Redux middleware for fetching data with axios HTTP client

## Installation

```bash
npm i -S redux-axios-middleware
```

## How to use?

### Use middleware

By default you only need to import middleware from package and add it to redux middlewares
and execute it with first aargument being with axios instance. second optional argument are middleware
options for customizing

```js
import {createStore, applyMiddleware} from 'redux';
import axios from 'axios';
import axiosMiddleware from 'redux-axios-middleware';

const client = axios.create({ //all axios can be used, shown in axios documentation
  baseURL:'http://localhost:8080/api',
  responseType: 'json'
});

let store = createStore(
  reducers, //custom reducers
  applyMiddleware(
    //all middlewares
    ...
    axiosMiddleware(client), //second parameter options can optionally contain onSuccess, onError, onComplete, successSuffix, errorSuffix
    ...
  )
)
```

### Dispatch action

Every action which have `payload.request` defined will be handled by middleware. There are two possible type
definitions.

- use `action.type` with string name
action with type will be dispatched on start, and then followed by type suffixed with underscore and
success suffix on success, or error suffix on error

defaults: success suffix = "_SUCCESS" error suffix = "_FAIL"

```javascript
export function loadCategories() {
  return {
    type: 'LOAD',
    payload: {
      request:{
        url:'/categories'
      }
    }
  }
}
```

- use `action.types` with array of types `[REQUEST,SUCCESS,FAILURE]`
`REQUEST` will be dispatched on start, then `SUCCESS` or `FAILURE` after request result

```javascript
export function loadCategories() {
  return {
    types: ['LOAD','AWESOME','OH_NO'],
    payload: {
      request:{
        url:'/categories'
      }
    }
  }
}
```

Actions that are handled by this middleware return a promise.  This gives you the ability to chain actions.  A good example of this might be a form.  In the form you might dispatch an actions to store the form values.  The normal flow of the action into the reducers would not be altered but you can chain a then/catch onto the initial dispatch.

```javascript
this.props.saveForm(formData)
  .then(() => {
    // router the user away
    this.context.router.push("/my/home/page")
  })
  .catch((response) => {
    //handle form errors
  })
```

Another example might be a set of actions that you want dispatched together.

```javascript
Promise.all([
  dispatch(foo()),
  dispatch(bar()),
  dispatch(bam()),
  dispatch(boom())

]).then(() => {
  dispatch(
    loginSuccess(
      token
    )
  )
})
```

### Next middleware

By default next middleware will receive new action object:

#### on success

```javascript
{
  type: 'AWESOME', //success type
  payload: [1,2,3] //data from response
  meta: {
    request:{
      url:'/categories' //whole reqquest from payload
    },
    response: {
      // ... axios response object without data
      status:200,
      statusTest:'OK',
      headers: {},
      config: {}
    },
    // all meta keys
  }
}
```

#### on error

Error action is same as success action with one difference, there's no key `payload`, but now there's `error`;

```js
{
    type:'OH_NO',
    error:['Error1', 'Error2'] //data from axios response error
    // ... same as success
}
```

#### if axios failed fataly

```js
{
   type:'redux-axios-middleware/FATAL_ERROR',
   error: Error, // instance of error,
   meta: ActionObject // action which fataled
}
```

### Middleware options

When adding middleware to redux you can specify 3 new keys

1. `onSuccess` can change default on success handling
```javascript
export const onSuccess = ({action, next, response, getState, dispatch}) => {
  ... //handle success
}
```

2. `onError` can change default on error handling
```javascript
export const onError = ({action, next, error, getState, dispatch}) => {
  ... //handle error
}
```

3. `onComplete` can trigger new on complete action
NOTE: action in `onComplete` handler is the new one returned by `onSuccess` or `onError` handler
```javascript
export const onComplete = ({action, next, getState, dispatch}) => {
  ... //handle complete
}
```

4. `errorSuffix` and `successSuffix` for changing keys suffixed to action type

## License

MIT

## Acknowledgements

[Dan Abramov](https://github.com/gaearon) for Redux
[Matt Zabriskie](https://github.com/mzabriskie) for [Axios](https://github.com/mzabriskie/axios). A Promise based HTTP client for the browser and node.js

repository: hotay/redux-axios-middleware

repo_name: hotay/redux-axios-middleware


text:
# Changelog

All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

## 1.0.0 (2020-01-06)

### New features

* add first draft of standard version for expo ([7f821d2](https://github.com/bycedric/standard-version-expo/commit/7f821d2458f8115b28a19495b163bbc85089f9c6))

### Pipeline changes

* add simple workflow to test changes ([bf64ce0](https://github.com/bycedric/standard-version-expo/commit/bf64ce034558780f89aca7d11cbc8f2d1a9b9806))

### Other chores

* add standard version as release script ([535a7a4](https://github.com/bycedric/standard-version-expo/commit/535a7a45e9275c558b937caf57a901e86136c853))

### Documentation changes

* add example to run standard version ([82598b1](https://github.com/bycedric/standard-version-expo/commit/82598b129e6f71b2e3e4bd6283df2cceb1fad846))
* fix typo and add enhanced example script ([13f3bab](https://github.com/bycedric/standard-version-expo/commit/13f3bab0128f171d7bbab72e6b55dd04210608cb))
* update example config with require resolve ([dcb2c8b](https://github.com/bycedric/standard-version-expo/commit/dcb2c8b41426d8d8a01fe48d739c7fb8ac92aaab))
* use proper badge references in readme ([c85797e](https://github.com/bycedric/standard-version-expo/commit/c85797e6993df1cd36faef437255fb576b2f1253))

repository: awinograd/standard-version-expo

repo_name: awinograd/standard-version-expo