text:
## Description
Repository for versioning my linux environment (packages and configuration files).

To deploy the environment locally for your current user, use:
```bash
make install
```
*Warning: deploying the environment overrides your existing configuration files*

By default, symbolic links pointing to the configuration files in the `config_files` directory will be created at the relevant location
in the home folder of the current user. This allows any modification to the configuration files to be easily saved
in the git repository.

If you prefer to _copy_ the configuration files to their relevant location instead of creating symlinks to them, use:
```bash
make install SYMLINKS=false
```

*Note: your user must be able to use `sudo`.
Your user password will be asked to perform operations that require privilege escalation.*

## Requirements
[Ansible](https://docs.ansible.com/ansible/latest/index.html) is used to deploy the environment. `python` and `ansible` must be installed on your system.
If you have [`pip`](https://pip.pypa.io/en/stable/), you can install ansible with:
```bash
sudo python -m pip install ansible
```

repository: matletix/env

repo_name: matletix/env


text:
---
layout: post
title: Segment-Based URLs with Query Strings in CodeIgniter 2
date: 2010-11-08 22:31:43.000000000 +00:00
categories:
- CodeIgniter
- PHP
- Web Development
tags: []
status: publish
type: post
published: true
author: Joseph Wynn
extra_head:
- <link rel="stylesheet" href="/css/highlight.css">
---

My latest CodeIgniter 2 project requires that I use query strings in some of my URLs. CodeIgniter 1 was notoriously difficult to work with when you enabled query strings, and unfortunately CodeIgniter 2 is no different. Whereas in CodeIgniter 1 you could change two configuration options to enable a combination of segment-based URLs and query strings, this same approach only makes matters worse in CodeIgniter 2.

<!--more-->

```php
$config['uri_protocol'] = "PATH_INFO";
$config['enable_query_strings'] = TRUE;
```

You would expect that using this configuration would allow you to use query strings, while still using the `PATH_INFO` URI protocol for your segment-based URLs. In CodeIgniter 1 this would produce URLs like `example.com/users/profile?user=Joseph&edit=true`. CodeIgniter 2 however would produce `example.com/?users/profile?user=Joseph&edit=true` - and the reason for this becomes apparent when you compare CodeIgniter 2's `CI_Config::site_url()` method with the same method in CodeIgniter 1.

I'm sure their intentions were good, but it appears that the CodeIgniter team assumes that if you enable query strings, you will not use segment-based URLs. Based on this assumption, the `site_url()` method appends a question mark (?) to the end of your `base_url`.

Getting around this is a very simple task; only one line needs to be modified. In `system/core/Config.php`, change line 221 from:

```
if ($this->item('enable_query_strings') == FALSE)
```

To:

```
if ($this->item('enable_query_strings') == FALSE OR $this->item('uri_protocol') == 'PATH_INFO')
```

And there you have it. Setting `uri_protocol` to `PATH_INFO` and enabling query strings will allow you to have  attractive segment-based URLs and still make use of query strings.

Note: It is considered good practice to extend core classes rather than modify them. I highly recommend doing this, as it will prevent your changes from being overwritten when you update CodeIgniter. To extend the `CI_Config` class, create the file `application/core/MY_Config.php` and paste the following code into it:

```php
class MY_Config extends CI_Config
{
    /**
     * Site URL
     *
     * Extended to allow a combination segment-based URLs and query strings when using the
     * uri_protocol = PATH_INFO / enable_query_strings = TRUE configuration setting.
     *
     * @access    public
     * @param    string    the URI string
     * @return    string
     */
    function site_url($uri = '')
    {
        if ($uri == '')
        {
            if ($this->item('base_url') == '')
            {
                return $this->item('index_page');
            }
            else
            {
                return $this->slash_item('base_url').$this->item('index_page');
            }
        }

        if ($this->item('enable_query_strings') == FALSE OR $this->item('uri_protocol') == 'PATH_INFO')
        {
            if (is_array($uri))
            {
                $uri = implode('/', $uri);
            }

            $suffix = ($this->item('url_suffix') == FALSE) ? '' : $this->item('url_suffix');
            return $this->slash_item('base_url').$this->slash_item('index_page').trim($uri, '/').$suffix;
        }
        else
        {
            if (is_array($uri))
            {
                $i = 0;
                $str = '';
                foreach ($uri as $key => $val)
                {
                    $prefix = ($i == 0) ? '' : '&';
                    $str .= $prefix.$key.'='.$val;
                    $i++;
                }

                $uri = $str;
            }

            if ($this->item('base_url') == '')
            {
                return $this->item('index_page').'?'.$uri;
            }
            else
            {
                return $this->slash_item('base_url').$this->item('index_page').'?'.$uri;
            }
        }
    }
}
```

license: MIT

branch_name: refs/heads/jekyll-src

extension: md

gha_language: HTML

license_type: permissive


text:
# Better Brakes - List of Manufacturers Reporting

## Dataset

| Name | Value |
| :--- | :---- |
| Catalog | [Link](https://catalog.data.gov/dataset/better-brakes-list-of-manufacturers-reporting-4ebfd) |
| Metadata | [Link](https://data.wa.gov/api/views/2dwu-pywu) |
| Data: JSON | [100 Rows](https://data.wa.gov/api/views/2dwu-pywu/rows.json?max_rows=100) |
| Data: CSV | [100 Rows](https://data.wa.gov/api/views/2dwu-pywu/rows.csv?max_rows=100) |
| Host | data.wa.gov |
| Id | 2dwu-pywu |
| Name | Better Brakes - List of Manufacturers Reporting |
| Attribution | The Washington State Department of Ecology |
| Category | Natural Resources & Environment |
| Tags | better brakes, baseline report |
| Created | 2013-06-24T17:02:11Z |
| Publication Date | 2013-07-23T22:01:00Z |

## Description

Manufacturers are required to report certain information to Ecology by Chapter 70.285 RCW and Chapter 173-901 WAC. This dataset contains a list of manufacturers that have meet these requirements.

## Columns

```ls
| Included | Schema Type | Field Name       | Name             | Data Type | Render Type |
| ======== | =========== | ================ | ================ | ========= | =========== |
| No       | time        | :updated_at      | updated_at       | meta_data | meta_data   |
| Yes      | series tag  | manufacturername | ManufacturerName | text      | text        |
```

## Time Field

```ls
Value = updated_at
Format & Zone = seconds
```

## Data Commands

```ls
series e:2dwu-pywu d:2016-07-22T13:49:18.000Z t:manufacturername="OE Quality Friction" m:row_number.2dwu-pywu=1

series e:2dwu-pywu d:2016-07-22T13:49:18.000Z t:manufacturername="Danaher Auto Spare Parts Co., Ltd." m:row_number.2dwu-pywu=2

series e:2dwu-pywu d:2016-07-22T13:49:18.000Z t:manufacturername="ABS Friction Inc." m:row_number.2dwu-pywu=3
```

## Meta Commands

```ls
metric m:row_number.2dwu-pywu p:long l:"Row Number"

entity e:2dwu-pywu l:"Better Brakes - List of Manufacturers Reporting" t:attribution="The Washington State Department of Ecology" t:url=https://data.wa.gov/api/views/2dwu-pywu

property e:2dwu-pywu t:meta.view v:id=2dwu-pywu v:category="Natural Resources & Environment" v:attributionLink=http://www.ecy.wa.gov/programs/hwtr/betterbrakes.html v:averageRating=0 v:name="Better Brakes - List of Manufacturers Reporting" v:attribution="The Washington State Department of Ecology"

property e:2dwu-pywu t:meta.view.license v:name="Public Domain"

property e:2dwu-pywu t:meta.view.owner v:id=a9ty-dhvu v:screenName="ECY-Wesley, Ian" v:displayName="ECY-Wesley, Ian"

property e:2dwu-pywu t:meta.view.tableauthor v:id=a9ty-dhvu v:screenName="ECY-Wesley, Ian" v:roleName=publisher v:displayName="ECY-Wesley, Ian"
```

## Top Records

```ls
| :updated_at | manufacturername                           |
| =========== | ========================================== |
|<PHONE_NUMBER>  | OE Quality Friction                        |
|<PHONE_NUMBER>  | Danaher Auto Spare Parts Co., Ltd.         |
|<PHONE_NUMBER>  | ABS Friction Inc.                          |
|<PHONE_NUMBER>  | REMSA of America                           |
|<PHONE_NUMBER>  | Shandong Gold Phoenix Co., Ltd.            |
|<PHONE_NUMBER>  | FRAS-LE North America, INC.                |
|<PHONE_NUMBER>  | Advics North America, Inc.                 |
|<PHONE_NUMBER>  | Pujiang Winsafe Friction Material Co., Ltd |
|<PHONE_NUMBER>  | Brake Parts Inc., LLC                      |
|<PHONE_NUMBER>  | Nisshinbo Automotive Manufacturing Inc.    |
```

repository: SpellarBot/open-data-catalog

branch_name: refs/heads/master

repo_name: SpellarBot/open-data-catalog


text:
# JSONLoggerExamples

[![Build Status](https://travis-ci.org/seniverse/ex_json_logger_examples.svg?branch=master)](https://travis-ci.org/seniverse/ex_json_logger_examples)

Usage examples centered on how to log messages with JSON format in Elixir based on [ex_json_logger](https://github.com/rentpath/ex_json_logger).

See [test/json_logger_examples_test.exs](test/json_logger_examples_test.exs) for usage cases and [config/config.exs](config/config.exs) for how to do configuration for logger to set a custom formatter.

See [ex_json_logger - GitHub](https://github.com/rentpath/ex_json_logger) for more instructions about usage.

See [Logger - Elixir](https://hexdocs.pm/logger/Logger.html) for official doc about `Logger`.

## Try it out

```sh
$ iex -S mix
iex> JSONLoggerExamples.log "abc"
:ok
iex> {"time":"2019-05-29 17:31:31.424","pid":"#PID<0.197.0>","msg":"abc","module":"Elixir.JSONLoggerExamples","line":6,"level":"info","function":"log/1","file":"lib/json_logger_examples.ex","application":"ex_json_logger_examples"}
```
It shows how actual messages printed in an Elixir module looks like in practice.

## Explanations

1. JSON is friendly for fields extension. We can add any fields we want in practice to a JSON via [Logger.Metadata](https://hexdocs.pm/logger/Logger.html#module-metadata) or other ways.

2. Just printing log to `stdout`. See [Logs - 12factor](https://12factor.net/logs) for its reason. Let's develop [Cloud-Native](https://pivotal.io/cloud-native) Apps!


text:
---
layout: default
---

# 1996-12-27 - Re: cryptoanarchy

## Header Data

From: "Omegaman"<EMAIL_ADDRESS>To: "Vladimir Z. Nuri"<EMAIL_ADDRESS>Message Hash: de76521169f03383f437de31eaa7759f1f2b507e750dc84e012f373bdb553dfa

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1996-12-27 16:41:34 UTC

Raw Date: Fri, 27 Dec 1996 08:41:34 -0800 (PST)

## Raw message

```
{% raw  %}From: "Omegaman"<EMAIL_ADDRESS>Date: Fri, 27 Dec 1996 08:41:34 -0800 (PST)
To: "Vladimir Z. Nuri"<EMAIL_ADDRESS>Subject: Re: cryptoanarchy
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

> >Brutality amongst human beings has little to do with what type of
> >government (or lack thereof) we have established.
>
> bzzzzzzt, history readily denies this.
>
>   Nor is brutality
> >inevitable amongst human beings; governments have little or no
> >affect on how individuals think and behave.
>
> bzzzzzt, history readily denies this. but again it is amusing to see
> the patently incorrect assertions that cryptoanarchists embrace and
> flout.
>
> >1) Do you agree that these things are an inevitable consequence of
> >anonymous untraceable payment systems?
>
> murder, assassination, kidnapping, they all already exist.

Exactly my point.  I believe interactions between individuals have
far more to do with our (dis)inclinations towards brutal behavior
that governments do.  Governments have little influence over human
attitudes.

> I am
> dubious that the existence of anonymous payments will change much in
> this area. I don't think it will become any more prevalent. what TCM

I agree, actually.  I don't see an increase in these types of crimes
because of the existence of anonymous payment methods.  What TCM and
others have argued is that getting away with these crimes will be
much easier due to anonymous payment schemes.

Only if untraceable anonymous digital cash becomes a ubiquitous
(and easy-to-utilize) standard will such crimes be more difficult to
catch.  (TCMay says only one such system is necessary.  I disagree
and will get back to this point in a later message)

> seems to imply in much of his writing, but fails to outrightly
> assert because he's such a weasel, is that the world would be a
> *better*place* with all these things, which I vehemently reject.

I think you're reading a little more into it than is there, but
that's your perogative.

> I'm in favor of anonymous cash, but
> I am also in favor of social/legal mechanisms to minimize its
> subversive impact. note that "not dealing with kidnappers or
> terrorists" is one such approach that does not involve police.

I'm not in favor of legal mechanisms.  I think social mechanisms are
all but inevitable. (more on this later) I don't think it's necessary
or even desirable to build in orwellian schemes.

The government's desire to limit untracability has far more to do
with taxation than the four horsemen scenario.  Yes, I've heard of
memes.

me
--------------------------------------------------------------
 Omegaman<EMAIL_ADDRESS> PGP Key fingerprint =  6D 31 C3 00 77 8C D1 C2
                        59 0A 01 E3 AF 81 94 63
Send a message with the text "get key" in the
"Subject:" field to get a copy of my public key.
--------------------------------------------------------------

{% endraw %}
```

## Thread

+ Return to [December 1996](/archive/1996/12)

+ Return to<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)](/authors/dlv_at_bwalk_dm_com_dr_dimitri_vulis_kotm_)"
+ Return to "["Mark M." <markm@voicenet.com>](/authors/mark_m__markm_at_voicenet_com_)"
+ Return to "["Omegaman" <omega@bigeasy.com>](/authors/omegaman_omega_at_bigeasy_com_)"
+ Return to "[Omegaman <omega@bigeasy.com>](/authors/omegaman_omega_at_bigeasy_com_)"
+ Return to "["Timothy C. May" <tcmay@got.net>](/authors/timothy_c_may_tcmay_at_got_net_)"
+ Return to "["Vladimir Z. Nuri" <vznuri@netcom.com>](/authors/vladimir_z_nuri_vznuri_at_netcom_com_)"

+ 1996-12-22 (Sat, 21 Dec 1996 20:25:46 -0800 (PST)) - [Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/2e81aaa022cec788e8db7d0f4d1c66300a1be0435cb35847fc9c308ef009da78) - _"Timothy C. May"<EMAIL_ADDRESS>  + 1996-12-23 (Sun, 22 Dec 1996 18:21:03 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/cb8b9ac56d1d62931b74f513a82ed3ec6c9f89c0db743414d6d7a03583bbf696) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
  + 1996-12-23 (Sun, 22 Dec 1996 22:17:36 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/c821ac7fbfdc13924506d0c9b6780a9a7f47f6942936c238aa1a25418cb48923) - _"Vladimir Z. Nuri"<EMAIL_ADDRESS>    + 1996-12-23 (Mon, 23 Dec 1996 07:01:44 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/99fef50a975de5efbb79d94e8477d36c33d9858ca967d5568f1a63acf79748ba) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
  + 1996-12-23 (Mon, 23 Dec 1996 07:30:35 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/2fc2ac895780b8e24513a19b34b89ee959ce6589c96deca1abc1ead89f310a0f) - _"Omegaman"<EMAIL_ADDRESS>    + 1996-12-23 (Mon, 23 Dec 1996 11:11:56 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/a8a5c655c0ee195f20bf390e0571b055fc9b5f66d4fcd4da09f6d5de5c92c771) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
    + 1996-12-24 (Tue, 24 Dec 1996 12:38:25 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/e21fc737ba48831ac78fc08d47ab1de7ab167c1dde4c3963ab9e841fe93423d0) - _"Vladimir Z. Nuri"<EMAIL_ADDRESS>      + 1996-12-24 (Tue, 24 Dec 1996 15:39:18 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/5a58b5a7694bf972cca620ad11a4afcb3c1828d3c675823e6c8edbce31db6a91) - _Omegaman<EMAIL_ADDRESS>        + 1996-12-27 (Thu, 26 Dec 1996 21:19:23 -0800 (PST)) - [cryptoanarchy](/archive/1996/12/70489a23e86f88211a690b027933617c42c6c37286776d0fd411869ecafd1a4e) - _"Vladimir Z. Nuri"<EMAIL_ADDRESS>        + 1996-12-27 (Fri, 27 Dec 1996 08:41:34 -0800 (PST)) - Re: cryptoanarchy - _"Omegaman"<EMAIL_ADDRESS>  + 1996-12-29 (Sun, 29 Dec 1996 15:12:45 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/417f20a2a93ae1bcc38ee99ebf2140cc47d73899f20cef789f1329e8d18988c9) - _Omegaman<EMAIL_ADDRESS>  + 1996-12-29 (Sun, 29 Dec 1996 15:31:52 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/ae10844d3925579cac039aa20a8b131726567dd14008660a90e4390c15782216) - _"Timothy C. May"<EMAIL_ADDRESS>    + 1996-12-30 (Sun, 29 Dec 1996 22:00:21 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/d2f8c562c28430f10a7a0db04c6fea4c39cb14a3000e83d340c7470146ae796a) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
    + 1996-12-30 (Mon, 30 Dec 1996 08:05:20 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/66aaf1baeb153d3ec1963f1817d51ab0184a5350d62c5e98649ead8e3de4fcb9) - _Omegaman<EMAIL_ADDRESS>      + 1996-12-30 (Mon, 30 Dec 1996 12:55:21 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/949611c4f35f4c4d1823e08f7d700ff7fbaa02e7e1832250f90f664231b0d15c) - _"Mark M."<EMAIL_ADDRESS>      + 1996-12-30 (Mon, 30 Dec 1996 14:31:36 -0800 (PST)) - [Re: Untraceable Payments, Extortion, and Other Bad Things](/archive/1996/12/d31f40af21c904501d06abbe82c5c5d499e9453167b5a8026bc529b5746ead6a) - _"Timothy C. May"<EMAIL_ADDRESS>

gha_license_id: MIT


text:
# Security infrastructure

We maintain multiple flavors of infrastructure with various degrees of management.

## GCP infrastructure basics

GCP infrastructure is configured via [terraform](https://www.terraform.io/) in the [infrastructure repository](https://github.com/sourcegraph/infrastructure/). All configuration for security projects should be stored in the [security subdirectory](https://github.com/sourcegraph/infrastructure/tree/main/security). Please adhere to this [terraform style guide](https://docs.sourcegraph.com/dev/background-information/languages/terraform) when working here.

For instructions on how to deploy this infrastructure, see [GCP Deployment Playbooks](./playbooks.md#gcp-deployment-playbooks).

#### Logging

To deploy logging infrastructure, see the [Deploying logging infrastructure](./playbooks.md#deploying-logging-infrastructure) playbook.

Logging configuration exists in many different places at present, which makes it complex.

- `pubsub.tf` in the [cloud](https://github.com/sourcegraph/infrastructure/blob/main/cloud/pubsub.tf) and [dogfood](https://github.com/sourcegraph/infrastructure/blob/main/dogfood/pubsub.tf) directories of the infrastrucutre repository.

  - This should remain mostly static, but the filter may change as filtering rules are refined, and additional logging sinks may be added for the [staging environment](#logging-stage).

  - Note that not all cloud pubsub configuration belongs to security.
  - This creates a logging sink for cloud and dogfood which sends logs via pub/sub to the security project.

- `gke-logging.tf` in the [cloud](https://github.com/sourcegraph/infrastructure/blob/main/cloud/pubsub.tf) and [dogfood](https://github.com/sourcegraph/infrastructure/blob/main/dogfood/pubsub.tf) directories of the infrastrucutre repository.

  - This should remain static.
  - This deploys the gke-logging module to the k8s cluster.

- The `gke-logging` module in the [modules folder](https://github.com/sourcegraph/infrastructure/tree/main/modules/gke-logging).

  - This should remain static.
  - This module pushes GKE node audit logs to stackdriver.
  - [Reference configuration](https://github.com/GoogleCloudPlatform/k8s-node-tools/blob/master/os-audit/cos-auditd-logging.yaml)

- The [logging folder](https://github.com/sourcegraph/infrastructure/tree/main/security/logging) in the security project.

  - This contains the GCP configuration for the logging projects owned by security.

- The [helm directory](https://github.com/sourcegraph/infrastructure/tree/main/security/logging/helm/) in the security project.

  - This contains the configuration for all helm deployments for security.
  - Currently, this is only pubsubbeats.

- Elastic cloud's [production logging deployment](#elastic-logging)

  - Elastic manages our logs, as well as our retention policy on our log data.
  - Expected to be re-configured whenever new sources of logs are added, as well as monitored to ensure it doesn't run out of disk space.

#### Logging stage

To implement in [#17281](https://github.com/sourcegraph/sourcegraph/issues/17281).

Will likely be similar to the above [logging infrastructure](#logging).

## GKE deployment basics

This section explains how to use various tools used to generate and deploy kubernetes configuration to GKE.

### Helmfile

Instead of using helm, we use [helmfile](https://github.com/roboll/helmfile). The reasoning for doing this is that helmfile allows basic script execution as part of the templating process, which is used to decrypt the secrets used for pubsubbeats. Additionally, it supports conditional configuration based on the deployment environment, which makes it harder to accidentally desynchronize the staging and production configurations.

### Kubectl

Kubectl is used to interact with kubernetes clusters. For basic information see the existing [kubectl tips and tricks document](../../../deployments/kubernetes.md). See the linked documents for examples of how kubectl is used to [configure](./playbooks.md#gke-deployment-playbooks) or [debug](./playbooks.md#debugging-logging).

## Projects

These are security's current GCP projects, and what they do.

For instructions on how to deploy these projects, see [GKE Deployment Playbooks](./playbooks.md#gke-deployment-playbooks).

### sourcegraph-security-logging

Currently ingests all stackdriver logs from the projects `sourcegraph-dev`(cloud) and `sourcegraph-dogfood`(dogfood). Will later ingest logs from other sources using additional deployments within the cluster.

### sourcegraph-security-logging-stage

To implement in [#17281](https://github.com/sourcegraph/sourcegraph/issues/17281).

This is a testbed to allow us to test changes to logs without risking production logs. This pushes logs to the [stage logging environment](#elastic-logging-stage), so that they don't pollute production logs in [elastic](#elastic-cloud).

## Managed services

### Elastic cloud

We currently use elastic cloud to store centralized security logs. This allows us to avoid the overhead of managing it ourselves, while getting something that's reasonably performant and stable.

Elastic cloud web portal is [here](https://cloud.elastic.co/home). Credentials are stored in 1Password.

#### Elastic logging

Currently contains all stackdriver logs from the GCP projects `sourcegraph-dogfood` and `sourcegraph-dev`. Note that stackdriver also contains OS audit logs from GKE nodes on the primary GKE clusters for those projects. This is due to the afforementioned `gke-logging` module being deployed in them as part of our [logging infrastructure](#logging).

Note that the pubsubbeat index lifecycle policy is set to a maximum index size is 50GB, and rollover is enabled.

Note that the index refresh interval is 30 seconds.

#### Elastic logging stage

To implement in [#17281](https://github.com/sourcegraph/sourcegraph/issues/17281).

repository: zag/about

branch_name: refs/heads/main

repo_name: zag/about


text:
---
layout: default
---

# 1995-11-13 - Re: DejaNews all over again

## Header Data

From<EMAIL_ADDRESS>(Robert Hettinga)

To<EMAIL_ADDRESS>Message Hash: 96253a9b13c65381da2c1f4f6a21b71a39956421a6a0b1a1d99024c93e30c6ed

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1995-11-13 02:26:48 UTC

Raw Date: Mon, 13 Nov 1995 10:26:48 +0800

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Robert Hettinga)
Date: Mon, 13 Nov 1995 10:26:48 +0800
To<EMAIL_ADDRESS>Subject: Re: DejaNews all over again
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

>  Jeff, I think it's in your best interest to not waste your time
>responding to this moron (who bears many features in common with
>Detweiler)

Except for his esteemed "credentials", of course...

Cheers,
Bob Hettinga

-----------------
Robert Hettinga<EMAIL_ADDRESS>e$, 44 Farquhar Street, Boston, MA 02131 USA<PHONE_NUMBER>
"Reality is not optional." --Thomas Sowell
The e$ Home Page: http://www.webstuff.apple.com/~vinnie/Rah
>>>>Phree Phil: Email<EMAIL_ADDRESS>http://www.netresponse.com/zldf <<<<<

{% endraw %}
```

## Thread

+ Return to [November 1995](/archive/1995/11)

+ Return to<EMAIL_ADDRESS>(Robert Hettinga)](/authors/rah_at_shipwright_com_robert_hettinga_)"

+ 1995-11-13 (Mon, 13 Nov 1995 10:26:48 +0800) - Re: DejaNews all over again -<EMAIL_ADDRESS>(Robert Hettinga)_


text:
## Introduction

We will go into detail on our tech stack, breaking down components according to their type and responsibilities, answering the question if a fixed-schedule release applies or not, and if not, what are the best strategies for rollout and deliver value minimizing operational risk at the same time.

All of the time-frame release process share a same goal. They deliver a certain number of features (plus patches and security updates) in a fixed period of time (it can be bi-weekly, monthly, quarterly, and so on).

# Component breakdown

![component breakdown](./images/release_components.png)

## **Libraries**

- 3box-js
- 3box-activity

Our core features for the people that are integrating with us, are delivered through the libraries. So, it makes sense to have a release schedule for this. Since we are on 2-week long sprints, it will make sense to leverage this cadence and make the library releases a ritual at the end of a sprint, for example.

**Proposal:** Schedule updates with a cadence that the team agrees on (at the end of every release or every other release). These releases will only include features, not bugfixes or security updates.

**Release process:** At the end of every release (after the sprint review and before the nexr sprint planning), follow the procedure:

- Create a release branch (release/x.y.z)
- Create release notes
- Merge those changes a master using `git-flow`
- Publish to npm

Ideally those steps (except for writing release notes could be automated in the near future)

create a release branch fr PR to master, tag accordingly and publish to npm automatically leveraging our CI integration.

**Benefits of this approach:**

- Delivering value periodically.
- Avoiding long-running features (white elephants).
- Minimize risk of releasing breaking-change features.
- Developers relying on us will know beforehand they may have to update their libraries

**Possible-side effects of this approach:**

- Feature creep.
- It may require code freeze so it can slow down development a bit.
- It will require dedication and commitment of the team to quality overall.

## **Servers:**

- address-server [to be deprecated]
- pinning-server

## caching-service

These services are the backbone in which our platform is built and scaled. The servers receive infrequent updates and those aren't usually tied with the product development per se (they bring stability, scalability, security). Forcing them to release on schedule doesn't make sense since the idea is to affect them as least as possible unless they improve some of the characteristics.

**Proposal:** Don't set a schedule to release server updates, unless if they are tied (or blocking) a given functionality implemented or required by the libraries.

**Release process:**  "1-Line" deploy. Make the deployment script available to the whole engineering team. The script should implement a rollback feature too. Test the changes in a small subset of servers, then apply it to all the server we have running.

**Benefits of this approach:** Releases are painless and can be executed at any time without downtime using an advance deployment configuration such as A/B deployment [3]

**Dapps (frontend):**

- 3box-dapp

The 3Box dapp delivers features to the end users (view/edit profile, activity and so on). The way the dapp interact with users is similar to a SaaS product. In those kinds of products, there is not a fixed schedule and the features are shipped to the users as soon as they are ready.

**Proposal**: Don't fix schedule dapp updates (unless a given feature is tied to any of the libraries). Feature-flag features and A/B test them instead, so it makes sense to release features one-by-one.

**Release process:** Release full features (ideally sequenced) using gitflow.

**Benefits of this approach:** Tag releases by the feature name, so it's easy to keep track of features at the dapp level. Also a reliable source to find regressions or other types of bugs.

# References

[1] Gitlab release process - [https://about.gitlab.com/2015/12/17/gitlab-release-process/](https://about.gitlab.com/2015/12/17/gitlab-release-process/)

[2] Ubuntu release process -  [https://wiki.ubuntu.com/TimeBasedReleases](https://wiki.ubuntu.com/TimeBasedReleases)

[3] Advanced deployment strategies - [https://docs.openshift.com/container-platform/3.6/dev_guide/deployments/advanced_deployment_strategies.html](https://docs.openshift.com/container-platform/3.6/dev_guide/deployments/advanced_deployment_strategies.html)


text:
# Change Log

All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

## [0.1.1](https://github.com/vivaxy/babel-folder/compare/v0.1.0...v0.1.1) (2017-05-27)

### Bug Fixes

* **git:** :bug:Commit missing files ([9bced61](https://github.com/vivaxy/babel-folder/commit/9bced61))

# 0.1.0 (2017-04-11)

### Features

* **First Commit:** :tada: ([b8155d6](https://github.com/vivaxy/babel-folder/commit/b8155d6))

repository: vivaxy/babel-folder

gha_language: JavaScript

repo_name: vivaxy/babel-folder


text:
# Instructions

Your friend Kojo is a big fan of numbers.
He has a small website for playing around with numbers: www.fun-with-numbers.com.
Kojo is not that good at programming so he asked you for help.

You will build two helper functions for new number games on Kojos' website and a third one to validate some input the user can enter.

## 1. Calculate the sum for the numbers on the slot machine

One of the games on Kojos' website looks like a slot machine that shows two groups of wheels with digits on them.
Each group of digits represents a number.
For the game mechanics, Kojo needs to know the sum of those two numbers.

Write a function `twoSum` that accepts two arrays as parameters.
Each array consists of one or more numbers between 0 and 9.
The function should interpret each array as a number and return the sum of those two numbers.

```javascript
twoSum([1, 2, 3], [0, 7]);
//=> 130

// [1, 2, 3] represents 123 and [0, 7] represents 7.
// 123 + 7 = 130
```

## 2. Determine if a number is a palindrome

Another game on the website is a little quiz called "Lucky Numbers".
A user can enter a number and then sees whether the number belongs to some secret sequence or pattern.
The sequence or pattern of the "lucky numbers" changes each month and each user only has a limited number of tries to guess it.

This months' lucky numbers should be numbers that are palindromes.
Palindromic numbers remain the same when the digits are reversed.

Implement the new `luckyNumber` function that accepts a number as parameter.
The function should return `true` if the number is a palindrome and `false` otherwise.
The input number will always be a positive integer.

```javascript
luckyNumber(1441);
//=>  true

luckyNumber(123);
//=> false
```

## 3. Generate an error message for invalid user input

In various places on the website, there are input fields where the users can enter numbers and click a button to trigger some action.
Kojo wants to improve the user experience of his site.
He wants to show an error message in case the user clicks the button but the field contains an invalid input value.

Here is some more information on how the value of an input field is provided.

- If the user types something into a field, the associated value is always a string even if the user only typed in numbers.
- If the user types something but deletes it again, the variable will be an empty string.
- Before the user even started typing, the variable can be `undefined` or `null`.

Write a function `errorMessage` that accepts the user input as parameter.
If the user did not provide any input, `errorMessage` should return `'Required field'`.
If the input does not represent a non-zero number (according to the JavaScript conversion rules), `'Must be a number besides 0'` should be returned.
In all other cases you can assume the input is valid, the return value should be an empty string.

```javascript
errorMessage('123');
// => ''

errorMessage('');
// => 'Required field'

errorMessage('abc');
// => 'Must be a number besides 0'
```

repository: rvsp/javascript

repo_name: rvsp/javascript


text:
IRC Modes
=========

This chapter explains how various IRC channel modes are interpteted by the bridge, and how you can best configure your
channels to work well with the bridge.

Note: While the bridge tries to follow the RFCs where possible, due to the somewhat fragmented nature of IRC the bridge is
designed for maximum compatibility with [Freenode's IRCd](https://github.com/freenode/ircd-seven).

## User modes to power levels

User modes are mapped to certain power levels on Matrix. For those unaware, Matrix power levels typically range between
0-100 where zero represents a user that may only speak and 100 is an admin of the room (although these are modifiable).

By default the bridge maps users like so

```yaml
# taken from config.sample.yaml
modePowerMap:
    o: 50
    v: 1
```

An operator on the IRC side of the bridge will be given moderator privileges (PL50) and a voiced user will be given
PL1. In `+m` (moderated) rooms, users who are not voiced cannot speak and so the Matrix room will not allow users with
PL0 (that is, without +v) to speak.

The IRC bridge admin can customise this power level mapping to however they see fit.

## Bans, Invite only and Registered only (+b / +i / +r)

The IRC bridge will kick a Matrix user on join if they are not able to join the IRC channel. This means if the IRC
user is banned, the room is invite only (and the user has not been invited) or the channel is registered only then
the user will not be able to join it. A reason is given in the Matrix ban, and a more verbose error is given in the
user's admin room.

### Invite exemption (+I)

Some communties prefer to keep their channels closed but allow the bridge to access the channel. It is trivial to do
this by using the `+I` channel mode. The format is the same as an [extban](https://freenode.net/kb/answer/extbans).

For example to allow matrix.org bridge users to access your invite-only channel, you would do:

```
/mode #the-secret-cookies +I<EMAIL_ADDRESS>```

## +s (Secret)

By default the IRC bridge will automatically insert any newly joined rooms into the homeserver's room directory.
The +s mode will mark the channel as secret and the bridge will not show it in the
directory. The room will still be joinable however.

## Key protected (+k)

If a channel is protected by a key, it cannot be entered by joining via an alias. Instead you may join by using
the [`!join`](admin_room.md#join) command. Be aware that the bridge purposefully does not store channel keys in
the database as a security precaution so you should be expected to do this again on bridge restart.

## Channel Forwarding (+f)

Channel forwarding is presently unsupported, see https://github.com/matrix-org/matrix-appservice-irc/issues/214
for information and updates.


text:
## Olympics.
> The modern Olympic Games or Olympics are leading international sporting events featuring summer and winter sports competitions in which thousands of athletes from around the world participate in a variety of competitions. The Olympic Games are considered the world's foremost sports competition with more than 200 nations participating. The Olympic Games are held every four years, with the Summer and Winter Games alternating by occurring every four years but two years apart.

This repository contains code to analyze the dataset published on Kaggle at [this link](https://www.kaggle.com/heesoo37/120-years-of-olympic-history-athletes-and-results)

---
#### Repository structure
The structure of the repository is following.
- **README.md**  describes the repository and its contents.
- **visualization/** has code required to index data in an [ElasticSearch](https://www.elastic.co/) index and later use [Kibana](https://www.elastic.co/webinars/getting-started-kibana?elektra=home&storm=sub2) as a visualization tool.
- **scripts/** has code that is required to analyze the data in simple python scripts.
- **notebooks/** has Jupyter notebooks used for analysis of the data.
- **results/** has the result images, summaries and visuals from the analysis.
- **data/** has datasets that is used for analysis. The data files are tracked using `git lfs`. The description of dataset can be found in the README inside this folder.

---
#### Environment setup
- **OS**
Windows 10/x64
- **Python installation**
Miniconda installation with conda 4.5.1 + Python 3.6.4
- **ElasticSearch**
Installation and initialization of ElasticSearch is quite simple.
  + [Download](https://www.elastic.co/start) and unzip Elasticsearch to a folder of your choice.
  + Run bin/elasticsearch (or bin\elasticsearch.bat on Windows).

  > Elasticsearch instance should be running at http://localhost:9200 in your browser if you run with default configuration.

  Keep the terminal open where elastic search is running to be able to keep the instance running.
  You could also use `nohup` mode to run the instance in the background.
``` json
"version" : {
    "number" : "6.3.1",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "eb782d0",
    "build_date" : "2018-06-29T21:59:26.107521Z",
    "build_snapshot" : false,
    "lucene_version" : "7.3.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  }
```
- **Kibana**
Installation and initialization is similar to that of Elasticsearch:

  + Download and unzip Kibana.
  + Open `config / Kibana.yml` in an editor.
  + Set elasticsearch.url to point at your Elasticsearch instance.
  + Change the directory to Kibana folder.
  + Run bin/Kibana (or bin\Kibana.bat on Windows)

  > Kibana instance should be running at http://localhost:5601 in your browser if you run with default configuration.

  Keep the terminal open where Kibana was run to be able to keep the instance running.
  You could also use `nohup` mode to run the instance in the background.

---
#### Summary of discoveries from dataset
Summary of discoveries can be found in the [README.md](./results/README.md) file in ` results/` folder.


text:
# Scala Faker

Scala Fake Data Generator. This repository includes Scala implementation for
[Golang Faker](https://github.com/bxcodec/faker).

[![Build Status](https://travis-ci.org/stevenchen3/scala-faker.svg?branch=master)](https://travis-ci.org/stevenchen3/scala-faker)
[![Coverage Statue](https://img.shields.io/codecov/c/github/stevenchen3/scala-faker/master.svg)](https://codecov.io/gh/stevenchen3/scala-faker)
[![License](https://img.shields.io/github/license/stevenchen3/scala-faker.svg)](https://github.com/stevenchen3/faker/blob/master/LICENSE)
[![Maven Central](https://img.shields.io/maven-central/v/com.github.stevenchen3/scala-faker_2.12.svg)](https://maven-badges.herokuapp.com/maven-central/com.github.stevenchen3/scala-faker_2.12)

# Get Started

**Supported Scala versions**
* Scala 2.11
* Scala 2.12

This library is available at Maven Central Repository. To use the latest stable version, append the
following to your `build.sbt` and replace `LATEST_VERSION` with the latest version published, which
should be [![Maven Central](https://img.shields.io/maven-central/v/com.github.stevenchen3/scala-faker_2.12.svg)](
https://maven-badges.herokuapp.com/maven-central/com.github.stevenchen3/scala-faker_2.12)

```scala
libraryDependencies += "com.github.stevenchen3" %% "scala-faker" % "LATEST_VERSION"
```

## Build Current `master`

Check scalastyle, run tests and generate coverage report across all supported Scala versions:

```bash
sbt +scalastyle +scalafixcheck +coverage +test +coverageReport
```

## Publish Locally

Make sure `export GPG_TTY=$(tty)` (this fixes `gpg: signing failed: Inappropriate ioctl for device`
error, checkout [here](https://github.com/keybase/keybase-issues/issues/2798) to see why) is
appended to `~/.bashrc` or has been loaded to current shell.

```bash
source ~/.bashrc
sbt +publishLocalSigned
```

## Examples

```scala

import io.alphash.faker._

val number = Phone().phoneNumber()
println(s"$number") // output some phone number like, "+1<PHONE_NUMBER>"

val name = Person().name
println(s"$name")  // e.g., "Blanca Spencer"

val price = Price().amountWithCurrency
println(s"$price") // e.g., "CAD 42545.1"

val ip = Internet().ipv4
println(s"$ip")    // e.g., "<IP_ADDRESS>"

...

```

# Available Fakers

It currently supports the following fakers:

**Datetime :**
* Datetime
* Time

**Geolocation :**
* Latitude and Longitude

**Internet :**
* Email
* Mac address
* Domain name
* URL
* Username
* IP Address (IPv4, IPv6 )
* Password

**Lorem :**
* Word
* Sentence
* Paragraph

**Payment :**
* Credit Card Type (VISA, MASTERCARD, AMERICAN EXPRESS, DISCOVER, JCB, DINERS CLUB)
* Credit Card Number

**Person :**
* Title male
* Title female
* First name
* First name male
* First name female
* Last name
* Name with title
* Name (first last)

**Phone :**
* Phone number
* Country code (US and Singapore)
* Area code
* Toll free phone number
* E164PhoneNumber

**Price :**
* Currency
* Amount


text:
<!---
This is an autogenerated file!

Please do not edit this file directly, but instead make changes to the
sigs.yaml file in the project root.

To understand how this file is generated, see https://git.k8s.io/community/generator/README.md
--->
# Naming Working Group

## Stakeholder SIGs
* SIG Architecture
* SIG Contributor Experience
* SIG Docs

## Meetings
* Regular WG Meeting: [Mondays at 10:30 PT (Pacific Time)](https://zoom.us/j/91522666403?pwd=WnRSNlNhNXhDWkR2ZU9ydGpsNWxtZz09) (monthly - second Monday of month). [Convert to your timezone](http://www.thetimezoneconverter.com/?t=10:30&tz=PT%20%28Pacific%20Time%29).
  * [Meeting notes and Agenda](https://bit.ly/k8s-wg-naming-agenda).
  * [Meeting recordings](https://www.youtube.com/playlist?list=PL69nYSiGNLP3BrAtDHyr8KTUfhBhCG7CD).

## Organizers

* Celeste Horgan (**[@celestehorgan](https://github.com/celestehorgan)**), CNCF
* Jaice Singer DuMars (**[@jdumars](https://github.com/jdumars)**), Apple
* Stephen Augustus (**[@justaugustus](https://github.com/justaugustus)**), Cisco

## Emeritus Organizers

* Zach Corleissen (**[@zacharysarah](https://github.com/zacharysarah)**)

## Contact
- Slack: [#wg-naming](https://kubernetes.slack.com/messages/wg-naming)
- [Mailing list](https://groups.google.com/forum/#!forum/kubernetes-wg-naming)
- [Open Community Issues/PRs](https://github.com/kubernetes/community/labels/wg%2Fnaming)
- Steering Committee Liaison: Bob Killen (**[@mrbobbytables](https://github.com/mrbobbytables)**)
<!-- BEGIN CUSTOM CONTENT -->

**The following section will be reworked and formalized as a charter once the
Working Group has been approved by the Steering Committee.**

## Goals

- Evaluate language and naming choices within the Kubernetes project, with
  a specific initial focus of:
  - Removing barriers to contribution and adoption by replacing harmful language with neutral terms whenever possible, including but not limited to language linked to racism, sexism, homophobia, transphobia, ableism, or discrimination against any protected or historically underrepresented group.
  - Improving clarity of codebases and documentation by replacing idioms,
    metaphors and slang specific to the English language
- Create a list of harmful terms with proposed replacements
- Define how any member of the Kubernetes project can
  recommend language, how others can evaluate that proposal, and how to
  implement replacements across all codebases.
  - Provide an easily findable location for language recommendations and
    follow-up issues, similar to an architectural decision record
  - Define long-term ownership of this process
- Work with stakeholder SIGs to implement the changes recommended. We
  anticipate the following:
  - Provide stakeholder SIGs with guidance on naming, language conventions, and
    processes
  - Collaborate with SIG Architecture and other stakeholders on an
    implementation timeline and strategies for dealing with follow-up issues
    from renaming, like deprecations
  - Collaborate with SIG Docs and SIG Contributor Experience on documenting
    language recommendations and processes
  - Work with the Code of Conduct Committee to add code architecture to the COC

## Dissolution Criteria

Once the Kubernetes community has:

- A process in place to evaluate language changes on an ongoing basis
- A binding list of terms to avoid in codebases across the project
- A timeline on which to replace component names in the kubernetes/kubernetes
  codebase
- Defined long-term ownership of the policies and processes this WG creates

this WG will dissolve.

## Post-formation Discussion Points

- Discuss appropriate process for branch renaming with GitHub Administration
  subproject (SIG ContribEx)
- Work with the Code of Conduct Committee to add code architecture to the COC
- A timeline on which to replace component names in the kubernetes/kubernetes
- Clarify that WG should not dissolve until after changes have been made

<!-- END CUSTOM CONTENT -->

repository: kubernetes/community

gha_language: Jupyter Notebook

repo_name: kubernetes/community


text:
# fusion-solr-plugins
Plugins for Solr to communicate with Fusion query and index pipelines. Solr plugins approach should be used when direct
integration of the application with Fusion query and index pipelines is not possible (WebSphere, Hybris).

The jar file is compiled with Oracle JDK 1.6 and can be downloaded from [releases](https://github.com/LucidWorks/fusion-solr-plugins/releases).

Downloadable Jar
=================

Download the latest jar file from [releases](https://github.com/LucidWorks/fusion-solr-plugins/releases)

Build jar files locally
=============

  * Dependencies:
    * JDK 1.6+
  * `./gradlew fatJar`

Create the search cluster and import the Solr collection to Fusion
==================================================================

* Create a search cluster for the Solr instance inside Fusion
* Then import the collection using the searchCluster name inside the Fusion Admin collections API

Configuring Fusion query-pipelines for the plugins
==================================================

* The query-pipelines should be modified so that they can work with the Solr searchComponents below.
    * Add the conditional `!request.hasParam("isFusionPluginQuery")` to the solr-query stage in the pipeline
    * Add the Return query-params stage at the end of your pipeline.

      **Example**:

            {
              "id" : "MC_10001_CatalogEntry_en_US-default",
              "stages" : [ {
                "type" : "search-fields",
                "id" : "1138f220-4c55-4314-ae2c-ea480c95b077",
                "rows" : 10,
                "start" : 0,
                "skip" : false,
                "label" : "search-fields",
                "type" : "search-fields"
              }, {
                "type" : "facet",
                "id" : "e371277a-2d2a-4730-89c5-0725b66da656",
                "skip" : false,
                "label" : "facet",
                "type" : "facet"
              }, {
                "type" : "solr-query",
                "id" : "10c39b09-a28e-4b1b-9d93-9136d6c1bb16",
                "skip" : false,
                "label" : "solr-query",
                "condition" : "!request.hasParam(\"isFusionPluginQuery\")",
                "type" : "solr-query"
              }, {
                "type" : "return-queryparams",
                "id" : "er4l5wmi",
                "skip" : false,
                "label" : "return-queryparams",
                "type" : "return-queryparams"
              } ]
            }

Configure solrconfig.xml to use Fusion Query Pipelines
===========================================================

* Add the jar file location to the `solrconfig.xml` config file for the collection that you would like to use.

        <lib dir="{plugins_dir}" regex="solr-{version_number}-plugins-for-fusion.*\.jar" />

* Add the searchComponents `FusionQPSearchComponent` and `LogToFusionComponent` to the solrconfig.xml file. The two parameters that the component requires are 'fusion_base_url' and 'collection_name'.

   * FusionQPSearchComponent:
        This component queries Fusion query-profiles to get the updated list of query-params. This search component queries the default
        query-profile and due to this, the pipeline used by the query-profile can be changed anytime through the Fusion Admin UI. Read the section `Configuring Fusion query-pipelines for the plugins` on how to configure query-pipelines for this plugin

            <searchComponent name="fusionQuery" class="com.lucidworks.solr.fusion.FusionQPSearchComponent" >
               <str name="fusion_base_url">http://admin:password123@localhost:8764/api/apollo</str>
               <str name="collection_name">{collection_name}</str>
            </searchComponent>

   * LogToFusionComponent:
        This component logs all the queries for the Solr collection through the Fusion searchLogs feature. All the logged queries can be visualized through the Banana dashboards.

            <searchComponent name="logToFusion" class="com.lucidworks.solr.fusion.LogToFusionComponent">
               <str name="fusion_base_url">http://admin:password123@localhost:8764/api/apollo</str>
               <str name="collection_name">{collection_name}</str>
            </searchComponent>

* Configure the search request handler to use the search components. `FusionQPSearchComponent` should always be first
 and `LogToFusionComponent` should be last.

          <requestHandler name="search" class="solr.SearchHandler" default="true">
            <!-- default values for query parameters can be specified, these
                 will be overridden by parameters in the request
              -->

            <arr name="first-components">
                <str>fusionQuery</str>
            </arr>

            <arr name="last-components">
                <str>logToFusion</str>
            </arr>
          </requestHandler>

* Reload your collection and all the queries to the configured collection should go through Fusion query-pipelines to
get query params, be logged. If the component failed to connect to Fusion or if there are some
processing errors, then the main query is executed as normal.

Configure Solr DIH to use Fusion Index Pipelines
================================================

1. Add the jar file location to the `solrconfig.xml` config file for the collection that you would like to use.

        <lib dir="$FUSION_HOME/solr-plugins/fusion"/>

2. Add the update processor `FusionUpdateProcessorFactory` to the solrconfig.xml

         <updateRequestProcessorChain name="sendDocsToFusion">
            <processor class="com.lucidworks.solr.fusion.FusionUpdateProcessorFactory">
                 <bool name="enabled">true</bool>
                 <str name="fusion_base_url">http://admin:password123@localhost:8764/api/apollo</str>
                 <str name="collection_name">{collection_name}</str>
            </processor>
            <processor class="solr.LogUpdateProcessorFactory" />
            <processor class="solr.RunUpdateProcessorFactory" />
          </updateRequestProcessorChain>

3. Configure the DIH request handler with the update chain

          <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
            <lst name="defaults">
                <str name="config">wc-data-config.xml</str>
              <str name="update.chain">sendDocsToFusion</str>
                <!-- Enable if using dynamic spellCheck field.
                <str name="update.chain">wc-conditionalCopyFieldChain</str>
                -->
            </lst>
          </requestHandler>

4. **Note**: This is a very slow process since each doc is sent to Fusion one by one.

repository: manisnesan/fusion-solr-plugins

repo_name: manisnesan/fusion-solr-plugins


text:
# silverstripe-product-catalog

simple, extendable product catalog

## Requirements

- SilverStripe 3.2

## Installation

This is how you install silverstripe-product-catalog.

## Example usage

You use silverstripe-product-catalog like this.

## Documentation

See the [docs/en](docs/en/index.md) folder.

repository: jsirish/silverstripe-product-catalog

repo_name: jsirish/silverstripe-product-catalog


text:
# Real-World Machine Learning Projects Using TensorFlow [Video]
This is the code repository for [Real-World Machine Learning Projects Using TensorFlow [Video]](https://www.packtpub.com/big-data-and-business-intelligence/real-world-machine-learning-projects-using-tensorflow-video?utm_source=github&utm_medium=repository&utm_campaign=9781789340174), published by [Packt](https://www.packtpub.com/?utm_source=github). It contains all the supporting project files necessary to work through the video course from start to finish.
## About the Video Course
Machine learning algorithms and research are mushrooming due to their accuracy at solving problems. This course walks you through developing real-world projects using TensorFlow in your ML projects.

The initial project will deal with assessing the viability of expanding your Restaurant business using a single variable linear regression. You will use Linear Regression with multiple variables with an example involving buying and selling a property at the best prices and use a dataset containing 11 features to deal with it. Next, you will create an algorithm to detect anomalous behavior in server computers using Gaussian methods. Finally, you'll design and build a convolutional Neural Networks model on a Traffic Signal Classifier from scratch.

By the end of this course you will be using TensorFlow in real-world scenarios, and you'll be confident enough to use ML Algorithms to build your own projects.

What You Will Learn

Explore topics such as classification, clustering, regression, and anomaly detection to build efficient ML models using TensorFlow

Use multiple ML algorithms and explore how algorithms are used to solve problems by using them effectively

Implement the most widely used machine learning algorithms and learn to design and build a convolutional neural network from scratch

Build real-world projects with predictive models, classification, anomaly detection algorithms, and Support Vector Machines.

Create data models and understand how they work by using different types of dataset.

Compare ML algorithms, and pick the best one for specific tasks

## Instructions and Navigation
### Assumed Knowledge
To fully benefit from the coverage included in this course, you will need:

Prior familiarity with TensorFlow and Machine Learning algorithms
### Technical Requirements
This course has the following software requirements:

### the Data_set's used
https://drive.google.com/drive/folders/13ivg5X5kSbIJM10qDAZ2mpHDG9PA7Sxv?usp=sharing
### Minimum Hardware Requirements

For successful completion of this course, students will require the computer systems with at least the following:

● OS: Windows

● Processor: intel

● Memory: 4 GRAM

● Storage: 30 GB

### Recommended Hardware Requirements

For an optimal experience with hands-on labs and other practical activities, we recommend the following configuration:

● OS: Windows

● Processor: intel CPU + advanced GPU

● Memory: 4 GRAM

● Storage: 60 GB

### Software Requirements

● Operating system: Windows

● Browser: Firefox

● Anaconda , Pycharm

## Related Products
* [Real-World Machine Learning Projects with Scikit-Learn [Video]](https://www.packtpub.com/big-data-and-business-intelligence/real-world-machine-learning-projects-scikit-learn-video?utm_source=github&utm_medium=repository&utm_campaign=9781789131222)

* [Real-World Python Deep Learning Projects [Video]](https://www.packtpub.com/big-data-and-business-intelligence/real-world-python-deep-learning-projects-video?utm_source=github&utm_medium=repository&utm_campaign=9781788620161)

* [Learning Python Artificial Intelligence by Example [Video]](https://www.packtpub.com/big-data-and-business-intelligence/learning-python-artificial-intelligence-example-video?utm_source=github&utm_medium=repository&utm_campaign=9781788839532)


text:
# aws-terraform-efs

This module sets up a basic Elastic File System on AWS for an account in a specific region.

## Basic Usage

```HCL
module "efs" {
  source = "git@github.com:rackspace-infrastructure-automation/aws-terraform-efs//?ref=v0.0.7"

  encrypted       = "false"
  name            = "EFSTest-minimal-options-unencrypted"
  security_groups = ["${aws_security_group.efs.id}"]
  vpc_id          = "${module.vpc.vpc_id}"
}
```

Full working references are available at [examples](examples)
## Other TF Modules Used
Using [aws-terraform-cloudwatch_alarm](https://github.com/rackspace-infrastructure-automation/aws-terraform-cloudwatch_alarm) to create the following CloudWatch Alarms:
	- efs_burst_credits

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| create\_internal\_dns\_record | Whether or not to create a custom, internal DNS record for the EFS endpoint's generated DNS name. If \"true\", the `internal_zone_id` MUST be provided, and a specific `internal_record_name` MAY be provided. Default is \"false\". | string | `"false"` | no |
| create\_parameter\_store\_entries | Whether or not to create EC2 Parameter Store entries to expose the EFS DNS name and Filesystem ID. | string | `"true"` | no |
| custom\_tags | Optional tags to be applied on top of the base tags on all resources | map | `<map>` | no |
| cw\_burst\_credit\_period | The number of periods over which the EFS Burst Credit level is compared to the specified threshold. | string | `"12"` | no |
| cw\_burst\_credit\_threshold | The minimum EFS Burst Credit level before generating an alarm. | string | `"1000000000000"` | no |
| encrypted | Whether or not the disk should be encrypted. | string | `"true"` | no |
| environment | Application environment for which this network is being created. e.g. Development/Production | string | `"Development"` | no |
| internal\_record\_name | If `internal_zone_id` is provided, Terraform will create a DNS record using the provided `internal_record_name` as the subdomain. If no `internal_record_name` is provided, the convention \"efs-<name>-<environment>\" will be used. | string | `""` | no |
| internal\_zone\_id | A Route 53 Internal Hosted Zone ID. If provided, a DNS record will be created for the EFS endpoint's DNS name, which can be used to reference the mount target. | string | `""` | no |
| kms\_key\_arn | The ARN for the KMS key to use for encrypting the disk. If specified, `encrypted` must be set to \"true\"`. If left blank and `encrypted` is set to \"true\", Terraform will use the default `aws/elasticfilesystem` KMS key. | string | `""` | no |
| mount\_target\_subnets | Subnets in which the EFS mount target will be created. | list | `<list>` | no |
| mount\_target\_subnets\_count | Number of `mount_target_subnets` (workaround for `count` not working fully within modules) | string | `"0"` | no |
| name | A unique name (a maximum of 64 characters are allowed) used as reference when creating the Elastic File System to ensure idempotent file system creation. | string | n/a | yes |
| notification\_topic | List of SNS Topic ARNs to use for customer notifications. | list | `<list>` | no |
| performance\_mode | The file system performance mode. Can be either "generalPurpose" or "maxIO". | string | `"generalPurpose"` | no |
| provisioned\_throughput\_in\_mibps | The throughput, measured in MiB/s, that you want to provision for the file system. **NOTE**: Setting a non-zero value will automatically enable \"provisioned\" throughput mode. To use \"bursting\" `throughput mode, leave this value set to \"0\". | string | `"0"` | no |
| rackspace\_alarms\_enabled | Specifies whether alarms will create a Rackspace ticket.  Ignored if rackspace_managed is set to false. | string | `"false"` | no |
| rackspace\_managed | Boolean parameter controlling if instance will be fully managed by Rackspace support teams, created CloudWatch alarms that generate tickets, and utilize Rackspace managed SSM documents. | string | `"true"` | no |
| security\_groups | A list of EC2 security groups to assign to this resource | list | n/a | yes |
| vpc\_id | The VPC ID. | string | n/a | yes |

## Outputs

| Name | Description |
|------|-------------|
| filesystem\_arn | The ARN for the filesystem |
| filesystem\_dns\_name | The DNS name for the filesystem |
| filesystem\_dns\_name\_ssm\_parameter | Name of the SSM parameter containing the captured filesystem DNS name |
| filesystem\_id | The ID that identifies the file system |
| filesystem\_id\_ssm\_parameter | Name of the SSM parameter containing the captured filesystem ID |
| mount\_target\_dns\_name | The DNS name for the mount target in a given subnet/AZ |
| mount\_target\_id | The ID of the mount target |
| mount\_target\_internal\_r53\_record | Internal Route 53 record FQDN for the EFS mount target |
| mount\_target\_network\_interface\_id | The ID of the network interface automatically created for the mount target |

repository: rmsrohan/aws-terraform-efs

repo_name: rmsrohan/aws-terraform-efs


text:
# Contributing

> Thank you for contributing. Contributions are always welcome, no matter how large or small.

## Table of Contents

- [Guidelines](#guidelines)
- [Pull Requests](#pull-requests)
- [Clone the Repository](#clone-repo)
- [Install Dependencies](#install-dependencies)
- [File Structure](#file-structure)

---

## Guidelines

As a contributor, here are the guidelines we would like you to follow:

- [Code of conduct](https://github.com/seantrane/serverless/blob/master/CODE_OF_CONDUCT.md)
- [How can I contribute?](#how-can-i-contribute)
- [Using the issue tracker](#using-the-issue-tracker)
- [Submitting a Pull Request](#submitting-a-pull-request)
- [Coding rules](#coding-rules)
- [Working with code](#working-with-code)

We also recommend to read [How to Contribute to Open Source](https://opensource.guide/how-to-contribute).

---

## Pull Requests

Thank you for contributing.

- Create your branch from `master`.
- Ensure your [git commit messages follow the required format](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#git-commit-messages).
- Ensure your scripts are well-formed, well-documented and object-oriented.
- Ensure your scripts are stateless and can be reused by all.
- Update your branch, and resolve any conflicts, before making pull request.
- Fill in [the required template](https://github.com/seantrane/serverless/blob/master/.github/PULL_REQUEST_TEMPLATE.md).
- Do not include issue numbers in the PR title.
- Include screenshots and animated GIFs in your pull request whenever possible.
- Follow the [style guide](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md) [applicable to the language](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#languages) or task.
- Include thoughtfully-worded, well-structured tests/specs. See the [Tests/Specs Style Guide](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#tests).
- Document new code based on the [Documentation Style Guide](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#documentation).
- End all files with a newline.

---

## Clone the Repository

```sh
git clone git@github.com:seantrane/serverless.git serverless && cd serverless
```

## Install Dependencies

```sh
# Install lerna globally (to use lerna binary at command line)
npm install -g lerna

# Install repo dependencies, link local packages together and install remaining package dependencies
npm install

# List local packages
npm run list

# Run tests across all packages
npm run test

# Run tests across changed packages
npm run test:changed

# Run linting across all packages
npm run lint

# Run builds across all packages
npm run build

# Run builds across changed packages
npm run build:changed

# Run linting, tests and builds across all packages
npm run ci

# Clean all auto-generated files
npm run clean

# Clean all auto-generated files within packages
npm run clean:packages

# Uninstall all packages
npm run uninstall

# Add dependency to a sub-package
npm run add -- <package>[@version] [--dev] [--exact] [packages/...]

npm run add -- pkg packages/hello-world
# "dependencies": {
#   "pkg": "^1.1.0"
# }

npm run add --<EMAIL_ADDRESS>packages/hello-world
# "dependencies": {
#   "pkg": "^1.0.1"
# }

npm run add --<EMAIL_ADDRESS>--exact packages/hello-world
# "dependencies": {
#   "pkg": "1.0.1"
# }

npm run add -- pkg --dev packages/hello-world
# "devDependencies": {
#   "pkg": "^1.1.0"
# }
```

## Important Links

- [Learn how to use `lerna`](https://github.com/lerna/lerna#readme)
- [Add a dependency to matched packages](https://github.com/lerna/lerna/tree/master/commands/add#readme) `lerna add <package>[@version] [--dev] [--exact]`
- [Run an npm script in each package that contains that script](https://github.com/lerna/lerna/tree/master/commands/run#readme) `lerna run <script> -- [..args]`
- [Run an arbitrary command in each package](https://github.com/lerna/lerna/tree/master/commands/exec#readme) `lerna exec -- <command> [..args]`
- [Remove the node_modules directory from all packages](https://github.com/lerna/lerna/tree/master/commands/clean#readme) `lerna clean`

---

## File Structure

```text
seantrane/serverless/
 ├─ services/                    * directory containing all Serverless services
 │   ├─ <service>/               * <service> directory
 │   :   ├─ __tests__/           * tests directory
 │       │   └─ handler.test.ts  * unit tests for function code
 │       │
 │       ├─ src/                 * source code directory
 │       │   └─ handler.ts       * function code, referenced by definitions in serverless.yml
 │       │
 │       ├─ event.json           * invoke function-data; `serverless invoke -p event.json`
 │       ├─ handler.js           * generated during deploy by `serverless-plugin-typescript`
 │       ├─ package.json         * <service> config, required for `npm run <script>`
 │       ├─ README.md            * each <service> should have a README with details therein
 │       ├─ serverless.yml       * Serverless functions and infrastructure resources
 │       └─ tsconfig.json        * TypeScript config
 │
 ├─ .markdownlint.yaml           * Markdown-lint rules and config
 ├─ .npmignore                   * npm publish ignore rules
 ├─ .releaserc.js                * semantic-release config
 ├─ .travis.yml                  * Travis CI config
 ├─ CODEOWNERS                   * default PR reviewers based on code-ownership
 ├─ CONTRIBUTING.md              * contribution guidelines
 ├─ CHANGLOG.md                  * changelog autogenerated by `@semantic-release/changelog`
 ├─ lerna.json                   * Lerna config
 ├─ mocha.opts                   * Mocha config
 ├─ package-lock.json            * npm package dependency lock file
 ├─ package.json                 * monorepo package config
 ├─ ROADMAP.md                   * repo and packages roadmap
 ├─ sonar-project.properties     * SonarCloud config, where each <service> must be declared
 ├─ STYLE_GUIDES.md              * Style guides, standards and practices
 ├─ tsconfig.json                * shared/base TypeScript config
 ├─ tslint.json                  * TSLint config
 └─ typedoc.json                 * TypeDoc config
```

---

## How can I contribute?

### Improve documentation

Consider helping to improve the documentation by finding _documentation issues_ that need help, and by submitting pull requests for corrections, clarifications, more examples, new features, etc.

Please follow the [Documentation guidelines](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#documentation).

### Give feedback on issues

Some issues are created without information requested in the [Bug report guideline](#bug-report). Help making them easier to resolve by adding any relevant informations.

Issues with the [design label](https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aseantrane+archived%3Afalse+label%3Adesign) are meant to discuss the implementation of new features. Participating in the discussion is a good opportunity to get involved and influence the future.

### Fix bugs and implement features

Confirmed bug and ready to implement features are marked with the [help wanted label](https://github.com/issues?utf8=%E2%9C%93&q=is%3Aopen+is%3Aissue+user%3Aseantrane+archived%3Afalse+label%3A%22help+wanted%22). Post a comment on an issue to indicate you would like to work on it and to request help from the [@seantrane/maintainers](https://github.com/orgs/seantrane/teams/contributors) and the community.

---

## Using the issue tracker

The issue tracker is the channel for [bug reports](#bug-report), [features requests](#feature-request) and [submitting pull requests](#submitting-a-pull-request) only. Please use the [Support](https://github.com/seantrane/serverless/blob/master/README.md#support) section for help, troubleshooting and questions.

Before opening an Issue or a Pull Request, please use the [GitHub issue search](https://github.com/issues?utf8=%E2%9C%93&q=user%3Aseantrane) to make the bug or feature request hasn't been already reported or fixed.

### Bug report

A good bug report shouldn't leave others needing to chase you up for more information. Please try to be as detailed as possible in your report and fill the information requested in the _[Bug/Issue Template](https://github.com/seantrane/serverless/blob/master/.github/ISSUE_TEMPLATE.md)_.

### Feature request

Feature requests are welcome. But take a moment to find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Please provide as much detail and context as possible and fill the information requested in the _[Agile Story Template](https://github.com/seantrane/serverless/blob/master/.github/STORY_TEMPLATE.md)_.

---

## Submitting a Pull Request

Good pull requests whether patches, improvements or new features are a fantastic help. They should remain focused in scope and avoid containing unrelated commits.

**Please ask first** before embarking on any significant pull request (e.g. implementing features, refactoring code), otherwise you risk spending a lot of time working on something that the project's developers might not want to merge into the project.

If you never created a pull request before, then [learn how to submit a pull request (great tutorial)](https://opensource.guide/how-to-contribute/#opening-a-pull-request).

#### Here is a summary of the steps to follow:

##### 1. [Set up the workspace](#set-up-the-workspace)

##### 2. If you cloned a while ago, get the latest changes from upstream and update dependencies:

```sh
git checkout master
git pull upstream master
rm -rf node_modules
npm install
```

##### 3. Create a new topic branch (off the main project development branch) to contain your feature, change, or fix:

```sh
git checkout -b <topic-branch-name>
```

##### 4. Make your code changes, following the [Coding rules](#coding-rules)

##### 5. Push your topic branch up to your fork:

```sh
git push origin <topic-branch-name>
```

##### 6. [Open a Pull Request](https://help.github.com/articles/creating-a-pull-request/#creating-the-pull-request) with a clear title and description.

##### Tips:

- For ambitious tasks, open a Pull Request as soon as possible with the `[WIP]` prefix in the title, in order to get feedback and help from the community.
- [Allow semantic-release maintainers to make changes to you Pull Request branch](https://help.github.com/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) this way we can rebase it and make some minor changes if necessary. All changes we make will be done in new commit and we'll ask for your approval before merging them.

---

## Coding rules

- [Commit message guidelines](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#git-commit-messages)
- [Documentation](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#documentation)
- [Lint](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#lint)
- [Source Code](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#source-code)
- [Tests/Specs](https://github.com/seantrane/serverless/blob/master/STYLE_GUIDES.md#tests)

---

## Working with code

### Set up the workspace

[Fork](https://guides.github.com/activities/forking/#fork) the project, [clone](https://guides.github.com/activities/forking/#clone) your fork, configure the remotes and install the dependencies:

```bash
# Clone your fork of the repo into the current directory
$ git clone git@github.com:seantrane/serverless.git
# Navigate to the newly cloned directory
$ cd monorepo
# Assign the original repo to a remote called "upstream"
$ git remote add upstream git@github.com:seantrane/serverless.git
# Install the dependencies
$ npm install
```

---

#### Happy coding!

license: ISC

repository: seantrane/serverless

repo_name: seantrane/serverless


text:
---
title: "The complete mitochondrial genome of the White-Nose Syndrome pathogen, Pseudogymnoascus destructans"
collection: publications
permalink: /publication/2017-01-01-mito
excerpt: 'A little side project on annotating the mitocondrial genome of P. destructans'
date: 2017-01-01
venue: 'Mitochondrial DNA Part B'
paperurl: 'AdrianForsythe.github.io/files/Forsythe2017-mito.pdf'
citation: 'Forsythe, A., & Xu, J. (2017). The complete mitochondrial genome of the White-Nose Syndrome pathogen, Pseudogymnoascus destructans. Mitochondrial DNA Part B, 2(1), 48–49.'
---

Download paper here

A little side project on annotating the mitocondrial genome of P. destructans

Recommended citation: Forsythe, A., & Xu, J. (2017). The complete mitochondrial genome of the White-Nose Syndrome pathogen, Pseudogymnoascus destructans. Mitochondrial DNA Part B, 2(1), 48–49.


text:
[SDS-2.2, Scalable Data Science](https://lamastex.github.io/scalable-data-science/sds/2/2/)
===========================================================================================

Archived YouTube video of this live unedited lab-lecture:

[![Archived YouTube video of this live unedited lab-lecture](http://img.youtube.com/vi/4tZ0-d_0yZA/0.jpg)](https://www.youtube.com/embed/4tZ0-d_0yZA?start=156&end=2823&autoplay=1)

Twitter Hashtag Count
=====================

Using Twitter Streaming is a great way to learn Spark Streaming if you don't have your streaming datasource and want a great rich input dataset to try Spark Streaming transformations on.

In this example, we show how to calculate the top hashtags seen in the last X window of time every Y time unit.

Extracting knowledge from tweets is "easy" using techniques shown here, but one has to take responsibility for the use of this knowledge and conform to the rules and policies linked below.

Remeber that the use of twitter itself comes with various strings attached. Read:

-   [Twitter Rules](https://twitter.com/rules)

Crucially, the use of the content from twitter by you (as done in this worksheet) comes with some strings. Read:
- [Developer Agreement & Policy Twitter Developer Agreement](https://dev.twitter.com/overview/terms/agreement-and-policy)

``` run
"scalable-data-science/sds-2-2/025_a_extendedTwitterUtils2run"
```

``` scala
import org.apache.spark._
import org.apache.spark.storage._
import org.apache.spark.streaming._

import twitter4j.auth.OAuthAuthorization
import twitter4j.conf.ConfigurationBuilder
```

>     import org.apache.spark._
>     import org.apache.spark.storage._
>     import org.apache.spark.streaming._
>     import org.apache.spark.streaming.twitter.TwitterUtils
>     import scala.math.Ordering
>     import twitter4j.auth.OAuthAuthorization
>     import twitter4j.conf.ConfigurationBuilder

### Step 1: Enter your Twitter API Credentials.

-   Go to https://apps.twitter.com and look up your Twitter API Credentials, or create an app to create them.
-   Run the code in a cell to Enter your own credentials.

``` %scala
// put your own twitter developer credentials below instead of xxx
// instead of the '%run "scalable-data-science/secrets/026_secret_MyTwitterOAuthCredentials"' above
// this notebook we just ran contains the following commented code block

/*
import twitter4j.auth.OAuthAuthorization
import twitter4j.conf.ConfigurationBuilder

def MyconsumerKey       = "xxx"
def MyconsumerSecret    = "xxx"
def Mytoken             = "xxx"
def MytokenSecret       = "xxx"

System.setProperty("twitter4j.oauth.consumerKey", MyconsumerKey)
System.setProperty("twitter4j.oauth.consumerSecret", MyconsumerSecret)
System.setProperty("twitter4j.oauth.accessToken", Mytoken)
System.setProperty("twitter4j.oauth.accessTokenSecret", MytokenSecret)
*/
```

The cell-below is hidden to not expose my Twitter API Credentials: `consumerKey`, `consumerSecret`, `accessToken` and `accessTokenSecret`. Use the code above to enter your own credentials!

``` run
"scalable-data-science/secrets/026_secret_MyTwitterOAuthCredentials"
```

If you see warnings then ignore for now:
<https://forums.databricks.com/questions/6941/change-in-getargument-for-notebook-input.html>.

### Step 2: Configure where to output the top hashtags and how often to compute them.

-   Run this cell for the input cells to appear.
-   Enter your credentials.
-   Run the cell again to pick up your defaults.

``` scala
val outputDirectory = "/datasets/tweetsStreamTmp" // output directory

//Recompute the top hashtags every N seconds. N=1
val slideInterval = new Duration(10 * 1000) // 1000 milliseconds is 1 second!

//Compute the top hashtags for the last M seconds. M=5
val windowLength = new Duration(30 * 1000)

// Wait W seconds before stopping the streaming job. W=100
val timeoutJobLength = 20 * 1000
```

>     outputDirectory: String = /datasets/tweetsStreamTmp
>     slideInterval: org.apache.spark.streaming.Duration = 10000 ms
>     windowLength: org.apache.spark.streaming.Duration = 30000 ms
>     timeoutJobLength: Int = 20000

### Step 3: Run the Twitter Streaming job.

Go to SparkUI and see if a streaming job is already running. If so you need to terminate it before starting a new streaming job. Only one streaming job can be run on the DB CE.

``` scala
// this will make sure all streaming job in the cluster are stopped
StreamingContext.getActive.foreach{ _.stop(stopSparkContext = false) }
```

Clean up any old files.

``` scala
dbutils.fs.rm(outputDirectory, true)
```

>     res17: Boolean = true

Let us write the function that creates the Streaming Context and sets up the streaming job.

``` scala
var newContextCreated = false
var num = 0

// This is a helper class used for ordering by the second value in a (String, Int) tuple
import scala.math.Ordering
object SecondValueOrdering extends Ordering[(String, Int)] {
  def compare(a: (String, Int), b: (String, Int)) = {
    a._2 compare b._2
  }
}

// This is the function that creates the SteamingContext and sets up the Spark Streaming job.
def creatingFunc(): StreamingContext = {
  // Create a Spark Streaming Context.
  val ssc = new StreamingContext(sc, slideInterval)
  // Create a Twitter Stream for the input source.
  val auth = Some(new OAuthAuthorization(new ConfigurationBuilder().build()))
  val twitterStream = ExtendedTwitterUtils.createStream(ssc, auth)

  // Parse the tweets and gather the hashTags.
  val hashTagStream = twitterStream.map(_.getText).flatMap(_.split(" ")).filter(_.startsWith("#"))

  // Compute the counts of each hashtag by window.
  // reduceByKey on a window of length windowLength
  // Once this is computed, slide the window by slideInterval and calculate reduceByKey again for the second window
  val windowedhashTagCountStream = hashTagStream.map((_, 1)).reduceByKeyAndWindow((x: Int, y: Int) => x + y, windowLength, slideInterval)

  // For each window, calculate the top hashtags for that time period.
  windowedhashTagCountStream.foreachRDD(hashTagCountRDD => {
    val topEndpoints = hashTagCountRDD.top(20)(SecondValueOrdering)
    dbutils.fs.put(s"${outputDirectory}/top_hashtags_${num}", topEndpoints.mkString("\n"), true)
    println(s"------ TOP HASHTAGS For window ${num}")
    println(topEndpoints.mkString("\n"))
    num = num + 1
  })

  newContextCreated = true
  ssc
}
```

>     newContextCreated: Boolean = false
>     num: Int = 0
>     import scala.math.Ordering
>     defined object SecondValueOrdering
>     creatingFunc: ()org.apache.spark.streaming.StreamingContext

Create the StreamingContext using getActiveOrCreate, as required when starting a streaming job in Databricks.

``` scala
val ssc = StreamingContext.getActiveOrCreate(creatingFunc)
```

>     ssc: org.apache.spark.streaming.StreamingContext = org.apache.spark.streaming.StreamingContext@3f9f7eb1

Start the Spark Streaming Context and return when the Streaming job exits or return with the specified timeout.

``` scala
ssc.start()
ssc.awaitTerminationOrTimeout(timeoutJobLength)
ssc.stop(stopSparkContext = false)
```

>     Wrote 477 bytes.
>     ------ TOP HASHTAGS For window 0
>     (#izmirescort,2)
>     (#YouAre,2)
>     (#갓세븐_YouAre_정오공개,1)
>     (#TFB,1)
>     (#Doctors,1)
>     (#CCTVDiskominfoMKS,1)
>     (#longsor,1)
>     (#MakeLifeBetter,1)
>     (#갓세븐
>     #7for7,1)
>     (#ㅅㅏ설토ㅌㅗ추천사ㅇㅣ트
>     #ㅅㅏ설ㅌㅗ토추천사ㅇㅣ트
>
>     ❕☽❇M☣⛹
>     A
>     ☄
>
>     ➡들어가기➡,1)
>     (#ターナーアワード,1)
>     (#Zaragoza,1)
>     (#박우진,1)
>     (#DiaInternacionaldelaNena,1)
>     (#윤지성,1)
>     (#방탄소년단,1)
>     (#정국,1)
>     (#워너원…,1)
>     (#ﷺ,1)
>     (#JeuConcours,1)
>     Wrote 511 bytes.
>     ------ TOP HASHTAGS For window 1
>     (#YouAre,4)
>     (#izmirescort,4)
>     (#MPN,3)
>     (#シナモンメルツ,3)
>     (#방탄소년단,3)
>     (#LittleMix,2)
>     (#KCAArgentina,2)
>     (#恋愛,1)
>     (#안전공원
>     #ㅅㅏ설토토ㅅㅏㅇㅣ트추천
>
>     안-전-놀-이-터
>
>     ✩안전-놀-이-터✩
>
>     ㊙,1)
>     (#tspo,1)
>     (#BackTheBlue,1)
>     (#odaibako
>     帰りたいですね,1)
>     (#amwriting,1)
>     (#Resistencia,1)
>     (#拡散希望,1)
>     (#セクシー,1)
>     (#Passion
>
>     https://t.co/CBKI0z2rSR,1)
>     (#ゴーストホスピタル　療養病棟担当　オーレリー,1)
>     (#Dosogas,1)
>     (#BiggBoss11…,1)

Check out the Clusters 'Streaming\` UI as the job is running.

It should automatically stop the streaming job after `timeoutJobLength`.

If not, then stop any active Streaming Contexts, but don't stop the spark contexts they are attached to using the following command.

``` scala
StreamingContext.getActive.foreach { _.stop(stopSparkContext = false) }
```

### Step 4: View the Results.

``` scala
display(dbutils.fs.ls(outputDirectory))
```

| path                                             | name              | size  |
|--------------------------------------------------|-------------------|-------|
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_0  | top\_hashtags\_0  | 0.0   |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_1  | top\_hashtags\_1  | 30.0  |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_10 | top\_hashtags\_10 | 163.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_11 | top\_hashtags\_11 | 277.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_12 | top\_hashtags\_12 | 317.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_13 | top\_hashtags\_13 | 345.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_14 | top\_hashtags\_14 | 290.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_15 | top\_hashtags\_15 | 256.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_16 | top\_hashtags\_16 | 227.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_17 | top\_hashtags\_17 | 184.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_18 | top\_hashtags\_18 | 141.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_19 | top\_hashtags\_19 | 170.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_2  | top\_hashtags\_2  | 106.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_20 | top\_hashtags\_20 | 183.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_21 | top\_hashtags\_21 | 165.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_22 | top\_hashtags\_22 | 203.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_23 | top\_hashtags\_23 | 239.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_24 | top\_hashtags\_24 | 200.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_25 | top\_hashtags\_25 | 175.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_26 | top\_hashtags\_26 | 211.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_27 | top\_hashtags\_27 | 192.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_28 | top\_hashtags\_28 | 217.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_29 | top\_hashtags\_29 | 216.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_3  | top\_hashtags\_3  | 256.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_30 | top\_hashtags\_30 | 349.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_31 | top\_hashtags\_31 | 291.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_32 | top\_hashtags\_32 | 168.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_33 | top\_hashtags\_33 | 205.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_34 | top\_hashtags\_34 | 191.0 |
| dbfs:/datasets/tweetsStreamTmp/top\_hashtags\_35 | top\_hashtags\_35 | 239.0 |

Truncated to 30 rows

There should be 100 intervals for each second and the top hashtags for each of them should be in the file `top_hashtags_N` for `N` in 0,1,2,...,99 and the top hashtags should be based on the past 5 seconds window.

``` scala
dbutils.fs.head(s"${outputDirectory}/top_hashtags_11")
```

>     res15: String =
>     (#Nami,1)
>     (#WorldOctopusDay,1)
>     (#Terrebonne,1)
>     (#ヒグチユウコ,1)
>     (#シナモンメルツ,1)
>     (#مملكة_قلم_للدعم
>     #المقابيل_للدعم
>     #ذئب_سلمان_للدعم
>     #مملكه_شيخه_للدعم,1)
>     (#CCH32,1)
>     (#beBee,1)
>     (#WeeklyIdol,1)
>     (#SAMURAIBLUE,1)

>     twitter OAuth Credentials loaded
>     MyconsumerKey: String
>     MyconsumerSecret: String
>     Mytoken: String
>     MytokenSecret: String
>     import twitter4j.auth.OAuthAuthorization
>     import twitter4j.conf.ConfigurationBuilder

### Let's brainstorm a bit now

What could you do with this type of streaming capability?

-   marketing?
-   pharmaceutical vigilance?
-   linking twitter activity to mass media activity?
-   ...

Note that there are various Spark Streaming ML algorithms that one could easily throw at such `reduceByKeyAndWindow` tweet streams:

-   [Frequent Pattern Mining](https://spark.apache.org/docs/latest/mllib-frequent-pattern-mining.html)
-   [Streaming K-Means](https://databricks.com/blog/2015/01/28/introducing-streaming-k-means-in-spark-1-2.html)
-   [Latent Dirichlet Allocation - Topic Modeling](https://spark.apache.org/docs/latest/ml-clustering.html#latent-dirichlet-allocation-lda)

Student Project or Volunteer for next Meetup - let's check it out now:

HOME-WORK:

-   [Twitter Streaming Language Classifier](https://databricks.gitbooks.io/databricks-spark-reference-applications/content/twitter_classifier/index.html)

license: Unlicense

repository: SheperoMah/scalable-data-science

gha_license_id: Unlicense

repo_name: SheperoMah/scalable-data-science


text:
# Intro

`Field` is a layer to define outputs of each Functional. It is very much similar to Keras' `Dense` layer.

It is not necessary to be defined explicitly, however, if you are expecting multiple outputs, it is better to be defined using `Field`.

```python
from sciann import Field

Fx = Field(name='Fx', units=10)
```

---

{{autogenerated}}

repository: kkoocheki/sciann

gha_license_id: NOASSERTION

repo_name: kkoocheki/sciann


text:
Hattr
=====

[![Build Status](https://travis-ci.org/arempe93/hattr.svg?branch=master)](https://travis-ci.org/arempe93/hattr)
[![Gem Version](https://badge.fury.io/rb/hattr.svg)](https://badge.fury.io/rb/hattr)
[![Coverage Status](https://coveralls.io/repos/github/arempe93/hattr/badge.svg?branch=master)](https://coveralls.io/github/arempe93/hattr?branch=master)

A simple library for interacting with the PSQL `hstore` extension in `ActiveRecord`. Hattr supports coercion of `hstore` attributes based on a simple dsl and will eventually support more advanced features such as Rails validations and more configurable options.

## Basic Usage

Hattr was designed to be simple. All you need is to extend the `Hattr` module and start building your spec.

```ruby
class Television < ActiveRecord::Base
  extend Hattr

  hattr :metadata, :weight, type: Float
  hattr :metadata, :screen_class, type: Integer
end
```

Now when you call `metadata` on your `Television` object, instead of getting an ugly hash back that looks like this:

```ruby
{ "weight" => "35.6", "screen_class" => "55" }
```

You can get back data that matches your specification - like this:

```ruby
{ weight: 35.6, screen_class: 55 }
```

## Advanced Usage

While Hattr was designed to be simple, it was also meant to be flexible to many use cases.

### Attribute Options

#### :type

Sets the type of the attribute which defaults to `String` if not specified. Valid values include `String`, `Integer`, `Fixnum`, `Float`, `Symbol`, `Array`, and `Hash`. Errors may be raised if coercion cannot be completed

```ruby
class Person < ActiveRecord::Base
  extend Hattr

  hattr :metadata, :name    # :type == String
end
```
**Super advanced usage**

For `Array` and `Hash` types, you can specify the types of the keys/values. For example, for an array of numbers and a hash of strings mapped to floats, you can do:

```ruby
class Person < ActiveRecord::Base
  extend Hattr

  hattr :metadata, :array, type: Array[Fixnum]
  hattr :metadata, :hash, type: Hash[String => Float]
end
```

The inner values are subject to inclusion in the list above for normal types.

**NOT ADVANCED ENOUGH**

Since it recursively calls `typecast` on the values of the array and **values** of the hash, you can do some pretty crazy stuff

```ruby
class Person < ActiveRecord::Base
  extend Hattr

  hattr :metadata, :wtf, type: Hash[Symbol => Array[Hash[String => Hash[Symbol => Float]]]]
end
```

Also, in case you're curious, just plain `Hash` and `Array` in the "normal" types list map to `Hash[Symbol => String]` and `Array[String]` respectively behind the scenes.

### Group Options

#### :string_keys

Leaves hash keys as strings instead of symbolizing them. Defaults to `false` if `hattr_group` is not called or `:string_keys` is not specified.

```ruby
class Television < ActiveRecord::Base
  extend Hattr

  hattr_group :metadata, string_keys: true
  hattr :metadata, :weight, type: Float
  hattr :metadata, :screen_class, type: Integer
end
```

### Multiple Attributes

```ruby
class Person < ActiveRecord::Base
  extend Hattr

  hattr :address, :line1, type: String
  hattr :address, :line2, type: String
  hattr :address, :zip, type: Integer

  hattr :metadata, :age, type: Integer
  hattr :metadata, :weight, type: Float
end
```

## Contributing

I would like to add more options to attribute and group dealing with validation, but if you have an idea feel free to open an issue/pull request!

## Installation

#### With RubyGems

To install Hattr with RubyGems:

```
gem install hattr
```

#### With Bundler

To use Hattr with a Bundler managed project:

```
gem 'hattr'
```

## License

Released under the MIT license


text:
example-colorguard
===================

This example refers to the article on [nodejitsu.com][1] and reposted on [npmawesome.com][2] about [colorguard][3].

## Usage

Clone the example:

    git clone https://github.com/npmawesome/example-colorguard.git

Install dependencies:

    npm install

Run the test:

    npm test

Profit!

If you have questions, feel free to leave a comments!

[1]: http://blog.nodejitsu.com/npmawesome-keep-colors-sane-in-your-css-files-with-colorguard/
[2]: http://npmawesome.com/posts/2014-07-09-colorguard/
[3]: https://github.com/SlexAxton/css-colorguard

repository: npmawesome/example-colorguard

repo_name: npmawesome/example-colorguard


text:
# PiStrip
A Raspberry Pi based power strip.

# Setup
You will need:
- A Raspberry Pi 2 + wifi dongle, 3, Zero-W, or Zero + wifi dongle
- A 4 channel relay interface (these run about $5 on Amazon)
- A 5v 2a AC-DC adapter
- A power cable
- Wiring of sufficient gague to handle mains current
- 4 outlets
- Soldering supplies
- Wire nuts
- Electrical tape/liquid electrical tape
- Jump wires
- **Enough electrical knowledge to be confident that you're not going to start a fire messing with mains current**

# Building it
I'm actually going to forego a step by step tutorial on the grounds that if you can't figure out how to do the wiring on outlets and relays then you probably have no business messing with mains current. I don't want to be responsible for someone who shouldn't be screwing with the mains burning down their house, so I'm not gonna do it. I will, however, give a few tips:

- If you cut open a wall wart and add a pair of lines from the power cord to it, and then replace the output cable with a couple jump wires going to pins 2 (5v) and 6 (ground) on your pi, you can get away with only having one power cord instead of one to power the Pi and one for everything else.
- The GPIO pins used for the relay interface are:
   * 2 (5v in)
   * 6 (input ground)
   * 4 (5v out)
   * 9 (ground)
   * 31, 33, 35, 37 (relay channels)

# The Software

First, for convenience sake I suggest setting a static IP on your Pi (tutorials abound, not repeating here).

Log in, navigate to the directory where you've downloaded this. Run
  sudo apt-get install apache2 mariadb-server php5 php5-mysql python-mysqldb

Open web/dbconfig.php and set the database username and password. You can set them to whatever you want. You'll never have to remember them once you're done with this process.

Open /web/session.php and alter lines 7, 8, 20, and 25 to reflect your own username and password. Either comment out lines 3-5 or, if you prefer not to login when you're at home, alter them to match your own network.

Run:
```
  mysql -u root -p
```
When prompted use the password you specified during the mysql install.
Once in MySql, run
```
  source *path*/db.sql;
  CREATE USER '*user*'@'localhost' IDENTIFIED BY '*password*';
  GRANT select, update, delete, insert ON powerstrip.* to '*user*'@'*localhost*';
  exit
```
Remember to replace *user* and *password* in the above with the values you specified in dbconfig.php. You should now have a copy of my personal database. Now run:

```
sudo rm /var/www/html/index.html
sudo cp -r web/\* /var/www/html/
sudo cp init.py /bin
sudo cp checkSched.py /bin
sudo cp setpin.py /bin
sudo chmod +x /bin/*.py
```
Add the following lines to your crontab:

- \* * * * * checkSched.py
- @reboot init.py

Point a browser at your Pi. You should see the control interface for your power strip.

# TODO:
- Improve the scheduling interface
- Get a real authentication system instead of this hard-coded BS I made when I was in a hurry.

# Changes:

- v0.1 Orange Pi One version, unreleased
- v0.2 Initial public release.
    Ported over to the Raspberry Pi

repository: siskulous/PiStrip

repo_name: siskulous/PiStrip


text:
# Dynamic Trivia Game Demo

## What You Will Need
*  [Amazon Developer Account](http://developer.amazon.com/alexa)
*  [Amazon Web Services Account](http://aws.amazon.com/)
*  The sample code on [GitHub](https://github.com/alexa/alexa-cookbook/tree/master/feature-demos/skill-demo-dynamic-entities/).

## Setting Up the Demo
This folder contains the interaction model and skill code.  It is structured to make it easy to deploy if you have the ASK CLI already setup.  If you would like to use the Developer Portal, you can follow the steps outlined in the [Hello World](https://github.com/alexa/skill-sample-nodejs-hello-world) example, substituting the [Model](./models/en-US.json) and the [skill code](./lambda/custom/index.js) when called for.

## Running the Demo
To start the demo say "alexa open trivia game". It will start a quiz game immediately where you'll have to answer questions where the answers are dynamic entities retrieved from a quiz API. You can say "start" to start over or stop to see the game stats. You can say Next one or I don't know to skip the question.

> Note: We use the Open Trivia Database to fetch Q/As (https://opentdb.com)

> Note: To make things easier we elimate questions that have mixed alphanumeric answers (only letters are allowed in the answers)

> Note: Some answers can be non standard words (like the name of a Pokemon character) so in those cases the answer might be not recognized properly


text:
# DestinyEquipItemResult

## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**itemInstanceId** | **int** | The instance ID of the item in question (all items that can be equipped must, but definition, be Instanced and thus have an Instance ID that you can use to refer to them) | [optional]
**equipStatus** | **int** | A PlatformErrorCodes enum indicating whether it succeeded, and if it failed why. | [optional]

[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

repository: Yogarine/bungie-sdk-php

repo_name: Yogarine/bungie-sdk-php