text:
bender
======

Bender makes it easy to build load testing applications for services using protocols like HTTP,
Thrift, Protocol Buffers and many more. Bender provides a library of flexible, powerful primitives
that can be combined (with plain Go code) to build load testers customized to any use case and that
evolve with your service over time.

Bender provides two different approaches to load testing. The first, LoadTestThroughput, gives the
tester control over the throughput (QPS), but not over the concurrency. This one is very well
suited for services that are open to the Internet, like web services, and even backend Thrift or
Protocol Buffer services, since it will just keep sending requests, even if the service is
struggling. The second approach, LoadTestConcurrency, gives the tester control over the concurrency,
but not over the throughput. This approach is better suited to testing services that require lots of
concurrent connections, and need to be tested for resource limits.

That Bender is a library makes it flexible and easy to extend, but means it takes longer to create
an initial load tester. As a result, we've focused on creating easy-to-follow tutorials that are
written for people unfamiliar with Go, along with documentation for all the major functions in the
library.

## Getting Started

The easiest way to get started with Bender is to use one of the tutorials:

* [Thrift](https://github.com/pinterest/bender/blob/master/thrift/TUTORIAL.md)
* [HTTP](https://github.com/pinterest/bender/blob/master/http/TUTORIAL.md)

## Documentation

The package documentation is available on [godoc.org](http://godoc.org/github.com/pinterest/bender).
The function and data structure documentation is also available there.

## What Is Missing

Bender does not provide any support for sending load from more than one machine. If you need to
send more load than a single machine can handle, or you need the requests to come from multiple
physical hosts (or different networks, or whatever), you currently have to write your own tools. In
addition, the histogram implementation used by Bender is inefficient to send over the network,
unlike q-digest or t-digest, which we hope to implement in the future.

Bender does not provide any visualization tools, and has a relatively simple set of measurements,
including a customizable histogram of latencies, an error rate and some other summary statistics.
Bender does provide a complete log of everything that happens during a load test, so you can use
existing tools to graph any aspect of that data, but nothing in Bender makes that easy right now.

Bender only provides helper functions for HTTP and Thrift currently, because that is all we use
internally at Pinterest.

The load testers we have written internally with Bender have a lot of common command line arguments,
but we haven't finalized a set to share as part of the library.

## Comparison to Other Load Testers

#### JMeter

JMeter provides a GUI to configure and run load tests, and can also be configured via XML (really,
really not recommended by hand!) and run from the command line. JMeter uses the same approach as
LoadTestConcurrency in Bender, which is not a good approach to load testing services (see the Bender
docs and the Iago philosophy for more details on why that is). It isn't easy to extend JMeter to
handle new protocols, so it doesn't have support for Thrift or Protobuf. It is relatively easy to
extend other parts of JMeter by writing Java code, however, and the GUI makes it easy to plug all
the pieces together.

#### Iago

Iago is Twitter's load testing library and it is the inspiration for Bender's LoadTestThroughput
function. Iago is a Scala library written on top of Netty and the Twitter Finagle libraries. As a
result, Iago is powerful, but difficult to understand, extend and configure. It was frustration with
making Iago work that led to the creation of Bender.

#### The Grinder

The Grinder has the same load testing approach as JMeter, but allows scripting via Jython, which
makes it more flexible and extensible. The Grinder uses threads, which limits the concurrency at
which it can work, and makes it hard to implement things like Bender's LoadTestThroughput function.
The Grinder does have support for conveniently running distributed load tests.

repository: sabagostar-ex/bender

repo_name: sabagostar-ex/bender


text:
Ruby Gem for temporarily modifying the Mac OS Keychain configuration. Useful for shared environments or processes with transient Keychain requirements.

This utility was created specifically for use with Fastlane's match / resign utilities. The process requires a keychain, but since match uses a git repository as the "source of truth" there's no need for the keychain to exist after the re-signing.

## API

```ruby
class SpareKeys
    # Temporarily adds the specified keychain to the top of the search list, reverting it after the block is invoked.
    def self.use_keychain(keychain_path, clear_list = false, type = nil, domain = nil)
    end

    # Creates a secure temporary keychain and adds it to the top of the
    # search list, reverting the list and deleting the keychain after the block is invoked.
    def self.temp_keychain(clear_list = false, type = nil, domain = nil)
    end
end
```

## Usage

```ruby
SpareKeys.temp_keychain do |temp_keychain_path, temp_keychain_password|
  # The keychain list starts with +temp_keychain_path+
end

# Everything is back to normal now

SpareKeys.temp_keychain true do |temp_keychain_path, temp_keychain_password|
  # The keychain list is empty, apart from +temp_keychain_path+
end

# Everything is back to normal now

SpareKeys.temp_keychain true, "default" do |temp_keychain_path, temp_keychain_password|
  # The keychain list is empty, apart from +temp_keychain_path+
  # The default keychain has been set to temp_keychain_path
end

# Everything is back to normal now
```

license: MIT

repository: gdimartino/spare-keys

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: gdimartino/spare-keys


text:
---
layout: post
title: "Worklog: Running Race Calculator "
date: 2018-08-11 22:30:00
categories: [blog, worklog]
summary: "
I’m pairing with [Lucia](http://luciagm.net) to build a pet project that would let people estimate the race pace and total time when running races. It’s the perfect excuse to do two things:

- She’d  code all the project to keep improving her front-end skills

- I’d stretch the creativity muscle and create a small UI kit to facilitate side projects

I’ve been pondering about the idea of having a daily worklog, let’s give it a try! If you got comments or suggestions please do [share them](http://twitter.com/adrianmg).
"
---

### August 26, 2018 at 19:14
{:.anchor }

I’ve covered the **basic input controls** and the **results table**. Speaking of which, we’ve decided that the table would feature not only one pace time column but a few more, showing pace time if you run a few seconds faster and slower. The idea is helping people to guess their time **without fine tune seconds up and down** by having a broader range.

![UI components race calculator]({{ "/assets/images/posts/2018/race-calculator-05.png" | relative_url }})

For the first version we’re going to keep things pretty simple, so I was sketching basic empty cases where the layout would always remain the same with the controls on the left and the results table on the right. The table would become an empty case pointing out to the controls until the user starts generating results.

![Notebook sketch race calculator]({{ "/assets/images/posts/2018/race-calculator-06.jpg" | relative_url }})

On the **development side**, [Lucia](https://twitter.com/luuugm) has started to put together the [**React application**](https://github.com/lucia-gm/race-calculator-react). The functionality was already there so now she’s investing some time translating all the vanilla JavaScript functionality into React components.

### August 25, 2018 at 20:03
{:.anchor }

We’ve been discussing a little bit how to surface the information to the user. The equation is pretty straightforward: you always have **distance**, and then the user needs to set either a **total time** or a **pace time**, and we’d generate the results.

Initially, I was tempted to force the user to choose between two tabs/switch, but it felt wrong, so we’re going to keep things simple and present the info in three steps:

1. What’s the distance you’re running?
2. Do you want to calculate your pace or total time?
3. Set a time and hit ‘Calculate.’

![Notebook sketch race calculator]({{ "/assets/images/posts/2018/race-calculator-03.jpg" | relative_url }})

All these doubts appeared after reviewing how other online calculators present all the options at first level, so they’d automatically calculate the option you leave empty: distance, total time or pace time. I found that a bit confusing.

I’m also starting to explore how the interactions are going to behave and look. [Lucia](http://luciagm.net) wants to have something strong for the primary actions, something purplish. I’ve been using blue as the primary action so many years in different companies (now, at Microsoft) that I guess it’s a blow of fresh air my eyes 😅.

![UI exploration race calculator]({{ "/assets/images/posts/2018/race-calculator-04.png" | relative_url }})

### August 11, 2018 at 16:05
{:.anchor }

When thinking about the visual execution of new projects I always start with the basic unit of information which is the text and its type ramp. I’m also going back to native fonts since they run fast as hell though I may take the license to use Montserrat for big bold titles when representing the time and other cool data about your running stats.

I’d keep you posted!

![Type ramp for race calculator]({{ "/assets/images/posts/2018/race-calculator-02.png" | relative_url }})

### August 10, 2018 at 22:30
{:.anchor }

I’m trying to map the basic elements that would be needed in order to have the race calculator. I don’t feel 100% convinced of what should be the right approach: asking the user what they need to know (pace VS total time) or just give all the option and let them figure out. At this point, I’m towards defaulting to set a pace that would generate the total time and they would be able to switch from a pivot to the other option.

I’m sharing some rough sketches of the process from my notebook:

![Notebook sketch race calculator]({{ "/assets/images/posts/2018/race-calculator-01.jpg" | relative_url }})

gha_language: HTML

gha_license_id: MIT


text:
# auto_aws

An automatically generated AWS client along the lines of [aws-go](https://github.com/stripe/aws-go).

## WIP warning

I'm writing this to learn OCaml and it's not even close to usable yet.

## Notes

* When possible, string-valued fields are converted to variant types. The following conversions are applied when necessary:
  * Enum options are capitalized, because variant constructors must begin with capital letters.
  * Periods are converted to underscores,

repository: apatil/auto-aws

repo_name: apatil/auto-aws


text:
# ContactsPhoneNumbers [![Flattr this git repo](http://api.flattr.com/button/flattr-badge-large.png)](https://flattr.com/submit/auto?user_id=callpal&url=https://github.com/callpal/cordova-plugin-contacts-phone-numbers&title=cordova-plugin-contacts-phone-numbers&language=en&tags=github&category=software)

Cross-platform plugin for Cordova / PhoneGap to list all the contacts with at least a phone number.

## Add Contact ##
Use the  `navigator.contactsPhoneNumbers` with the method

  `add(data)`

  A full example could be:

```
   //
   //
   // after deviceready
   //
   //
   var name = "Call Pal";
   var firstNumber = {
     label: 'mobile',
     number: $scope.number
   };
   var secondNumber = {
     label: 'mobile',
     number: '17864500211'
   };
   var data = {
     name: name,
     phones: [firstNumber, secondNumber]
   };
   navigator.contactsPhoneNumbers.add(data);

```

## Installing the plugin ##

Use this repository
```
cordova plugin add https://github.com/CallPal/cordova-plugin-contacts-phone-numbers
```
## Using the plugin ##
The plugin creates the object `navigator.contactsPhoneNumbers` with the methods

  `list(success, fail)`

A full example could be:

```
   //
   //
   // after deviceready
   //
   //
   navigator.contactsPhoneNumbers.list(function(contacts) {
      console.log(contacts.length + ' contacts found');
      for(var i = 0; i < contacts.length; i++) {
         console.log(contacts[i].id + " - " + contacts[i].displayName);
         for(var j = 0; j < contacts[i].phoneNumbers.length; j++) {
            var phone = contacts[i].phoneNumbers[j];
            console.log("===> " + phone.type + "  " + phone.number + " (" + phone.normalizedNumber+ ")");
         }
      }
   }, function(error) {
      console.error(error);
   });

```

## JSON Response format

The success callback function contains an array of contacts.

Each entry contains:

   * the unique contact id
   * the name of the contact (first name, last name, display name)
   * an array containing the number, the normalizedNumber and the type of the number (```WORK```, ```MOBILE```, ```HOME``` or ```OTHER```)

Here is a sample of what you can get:

```
    [{
        "id": "1",
        "firstName": "Kate",
        "lastName": "Bell",
        "displayName": "Kate Bell",
        "phoneNumbers": [{
            "number": "(555) 564-8583",
            "normalizedNumber": "(555) 564-8583",
            "type": "MOBILE"
        }, {
            "number": "(415) 555-3695",
            "normalizedNumber": "(415) 555-3695",
            "type": "OTHER"
        }]
    }, {
        "id": "2",
        "firstName": "Daniel",
        "lastName": "Higgins",
        "displayName": "Daniel Higgins",
        "phoneNumbers": [{
            "number": "555-478-7672",
            "normalizedNumber": "555-478-7672",
            "type": "HOME"
        }, {
            "number": "(408) 555-5270",
            "normalizedNumber": "(408) 555-5270",
            "type": "MOBILE"
        }, {
            "number": "(408) 555-3514",
            "normalizedNumber": "(408) 555-3514",
            "type": "OTHER"
        }]
    }, {
        "id": "3",
        "firstName": "John",
        "lastName": "Appleseed",
        "displayName": "John Appleseed",
        "phoneNumbers": [{
            "number": "888-555-5512",
            "normalizedNumber": "888-555-5512",
            "type": "MOBILE"
        }, {
            "number": "888-555-1212",
            "normalizedNumber": "888-555-1212",
            "type": "HOME"
        }]
    }]
```

## Behaviour

The plugin retrieves **ONLY** the contacts containing one or more phone numbers. It does not allow to modify them (use [the official cordova contacts plugin for that](https://github.com/apache/cordova-plugin-contacts)).

With the official plugin, it is difficult and inefficient[1] to retrieve the list of all the contacts with at least a phone number (for Android at least). I needed a fastest way to retrieve a simple list containing just the name and the list of phone numbers.

If you need more fields like the email address or if you also need to retrieve the contacts without email address, we can add an option, open an issue and I'll see what I can do.

**[1]** When I say *difficult and inefficient*, it is because on Android, all your Gmail contacts are returned as a contact. [See this issue on stackoverflow](http://stackoverflow.com/questions/20406564/phonegap-contacts-api-android-return-only-phone-contacts-and-not-gmail-conta). With the official plugin you have to retrieve all the contacts and then iterate over the result to filter out what you want.

I executed a small benchmark on my Nexus 5 with Lollipop. The code calls both plugins and displays the result in the console. On this phone I have 1028 contacts but only 71 contacts have at least a phone number. Of course the performances depends on the number of contacts with phone numbers.

**cordova-plugin-contacts**

    *  1 call:
        try 1: 2.527s
        try 2: 2.581s
        try 3: 2.221s

        => average of 2.443s

    * 10 calls:
        try 1: 6.048s
        try 2: 9.196s
        try 3: 8.981s

        => average of 8.075s for 10 calls

**cordova-plugin-contacts-phone-numbers**

    *  1 call
        try 1: 0.145s
        try 2: 0.185s
        try 3: 0.286s

        => average of 0.205s

    * 10 calls:
        try 1: 1.195s
        try 2: 1.211s
        try 3: 1.351s

        => average of 1.252s for 10 calls

## iOS and Android

The plugin works with iOS and Android.

iOS does not provide a normalized number like Android. So number === normalizedNumber for iOS.

The Android code is heavily inspired from the official plugin with some tweaks to improve the perfomances.

## Contributing

Thanks for considering contributing to this project.

### Finding something to do

Ask, or pick an issue and comment on it announcing your desire to work on it. Ideally wait until we assign it to you to minimize work duplication.

### Reporting an issue

- Search existing issues before raising a new one.

- Include as much detail as possible.

### Pull requests

- Make it clear in the issue tracker what you are working on, so that someone else doesn't duplicate the work.

- Use a feature branch, not master.

- Rebase your feature branch onto origin/master before raising the PR.

- Keep up to date with changes in master so your PR is easy to merge.

- Be descriptive in your PR message: what is it for, why is it needed, etc.

- Make sure the tests pass

- Squash related commits as much as possible.

### Coding style

- Try to match the existing indent style.

- Don't mix platform-specific stuff into the main code.

## Licence ##

The MIT License

Copyright (c) 2013 Didier Baquier

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.


text:
# Just Animate

*Just Animate creates beautiful animations using the latest browser standards*

[![npm version](https://badge.fury.io/js/just-animate.svg)](https://badge.fury.io/js/just-animate)
[![Build Status](https://travis-ci.org/just-animate/just-animate.svg?branch=master)](https://travis-ci.org/just-animate/just-animate)
[![Downloads](https://img.shields.io/npm/dm/just-animate.svg)](https://www.npmjs.com/package/just-animate)

## Features

- over 75 preset animations
- hardware acceleration on Chrome and FireFox using the Web Animation API
- control multiple animations
- animation sequencing
- animation timelines
- randomness and staggered timing through property resolvers
- full animation control (reverse, pause, cancel, and seek)

> When this project gets to 100 🌟s, I'm going to work on morphing SVGs,  [so ^ star it please](https://github.com/just-animate/just-animate/stargazers).

## Getting Started

### Setup from CDN
 - For support in Internet Explorer, Edge, or Safari, include this script. Just Animate uses the Web Animation API and is not yet supported in these browsers.
    ```html
    <script src="https://cdnjs.cloudflare.com/ajax/libs/web-animations/2.2.2/web-animations.min.js"></script>
    ```

 - Include this script (change 1.0.7 to the current version to get the latest)
    ```html
    <script src="https://cdnjs.cloudflare.com/ajax/libs/just-animate/1.0.7/just-animate-all.min.js"></script>
    ```
 - ```just``` is available from
 - That's it!

### Setup for Webpack/Rollup/Browserify

Just Animate can also be installed through NPM for bundling.

```bash
npm i just-animate --save
```

Then it can be used like all other NPM libraries:

## Usage

|property|type|description|
|:-------------|:-------------|:-------------|
|css|CSSKeyframe[] or CSSProperty|An array of keyframes or an object of properties to animate|
|delay|Time or Function|Time before animation starts as '2s', '2000ms', 2000, '+=2s', '100 to 200 ms', or function() { return 2000; }|
|direction|string|Direction of the animation (normal, reverse, alternate)|
|easing|AnimationTimingFunction|Animation timing function (ease, ease-in, easeOutCubic, step(1,end))|
|fill|string|Animation fill mode (none, both, forwards, backwards)|
|from|Time|When to start the animation as '2s', '2000ms', or 2000|
|iterations|number|Number of iterations, defaults to 1|
|mixins|string or string[]|A preset or a list of presets to add to the animation(e.g. fadeIn, hinge, zoomOutLeft, etc.)|
|targets|AnimationTarget|An html selector, an Element, a NodeList, or a jQuery object. This can also be an array of any of those or a function that returns any of those.|
|to|Time|When to stop the animation written as '2s', '2000ms', or 2000|

 ```javascript
  just.animate({
    // css: [
    //   { opacity: 0 },
    //   { opacity: 1 }
    // ],
       css: {
           opacity: [0, 1]
       },

    // delay: '0ms',
    // delay: '+=100',
    // delay: function(ctx) {
    //     return 10 * ctx.index;
    // },

    // direction: 'normal',
    // direction: 'reverse',
    // direction: 'alternate',

    // easing: 'linear',
    // easing: 'cubic-bezier(0, 0, 1, 1)',
    // easing: 'step(1, end)',

    // fill: 'none',
    // fill: 'forwards',
    // fill: 'backwards',
    // fill: 'both',

    // from: 0,
    // from: '0ms',
    // from: '0.002s',

    // iterations: 1,
    // iterations: Infinity,

    // mixins: 'fadeIn',
    // mixins: ['fadeIn'],

       targets: '#animate-me',
    // targets: document.getElementById('animate-me),
    // targets: document.querySelectorAll('#animate-me),
    // targets: $('#animate-me),
    // targets: ['#animate-me'],
    // targets: () => document.getElementById('animate-me'),

    // to: 1000,
    // to: '1000ms',
       to: '1s'
  })
 ```

### Animate multiple targets by using an html selector, JQuery, or an array of targets

 ```javascript
just.animate({
       mixins: 'fadeIn',
       targets: '#animate-me',
    // targets: document.getElementById('animate-me),
    // targets: document.querySelectorAll('#animate-me),
    // targets: $('#animate-me),
    // targets: ['#animate-me'],
    // targets: () => document.getElementById('animate-me'),
    // targets: function() { return document.getElementById('animate-me'); },
});
 ```

### Animate multiple targets in sequence by chaining .animate() calls

 ```javascript
just.animate({
      mixins: 'fadeIn',
      targets: '#first',
  })
  .animate({
      mixins: 'fadeIn',
      targets: '.second'
  });

```

### Use multiple animations at the same time by passing in an array of animation options

  ```javascript
  just.animate([
    {
      mixins: 'fadeOutLeft',
      targets: '#first',
      to: '2s',
      delay() {
        // delay animation for a random # of ms between 0 and 1000
        return just.random(0, 1000);
      }
      // delay: '0 to 1000'
    },
    {
      mixins: 'fadeOutRight',
      targets: '#second',
      to: '1.5s'
    }
  ]);

```

### Create custom animations or secondary effects using the update function

  ```javascript
just.animate({
    targets: '#first',
    to: '2s',
    on: {
        update: function(ctx) {
            console.log(ctx.delta);            // delta time since the last update
            console.log(ctx.currentTime);      // current time of animation
            console.log(ctx.relativeDuration); // total duration of the animation (to - from)
            console.log(ctx.offset);           // absolute offset (time) of the animation from 0 to 1
            console.log(ctx.playbackRate);     // current playback rate
            console.log(ctx.computedOffset);   // relative offset (progress) of the animation from 0 to 1
            console.log(ctx.target);           // target of this animation
            console.log(ctx.index);            // target index of this animation
            console.log(ctx.targets);          // all targets of this animation
        }
    }
});

```

### Animate CSS transforms with easy to use shorthand properties

```javascript
just.animate({
    /* .. */
    css: [{
        perspective: '200px',                // camera distance from the z plane
        matrix:      '1, 1, 1, 1, 1px, 1px', // matrix transform function
        translate:   '20px, 30px',           // X, Y distance from origin
        translate3d: '20px, 30px, 40px'      // X, Y, Z distance from origin
        translateX:  '20px'                  // same as x
        translateY:  '20px'                  // same as y
        translateZ:  '20px'                  // same as z
        x:           '20px',                 // X distance from origin
        y:           '30px',                 // Y distance from origin
        z:           '40px',                 // X distance from origin
        skew:        '20deg',                // X, Y skew function
        skewX:       '20deg',                // X skew function
        skewY:       '20deg',                // Y skew function
        scale:        1.1,                   // X, Y scale
        scale3d:      1.3,                   // X, Y, Z scale
        scaleX:       1,                     // X scale
        scaleY:       1,                     // Y scale
        scaleZ:       1,                     // Z scale
        rotate:      '90deg',                // same as rotateZ
        rotate3d:    '90deg',                // X, Y, Z rotation
        rotateX:     '90deg',                // X rotation
        rotateY:     '90deg',                // Y rotation
        rotateZ:     '90deg'                 // Z rotation
    }]
});
```

#### Notes:
- The properties are evaluated in the order shown here.  I believe the order solves a large number of use cases.  The transform property can be used directly for other cases.
- Using aliased properties such as translateX and x together may result in unexpected results

### Use the player returned from .animate() to listen for events or other functions control the animations.

 ```javascript
var player = just.animate(/* ... */);

// event listeners
player.on('finish', function(ctx) { }); // fired when the animation finishes or .finish() is called
player.on('cancel', function(ctx) { }); // fired when .cancel() is called
player.on('pause',  function(ctx) { }); // fired when .pause() is called
player.on('update', function(ctx) { }); // fired each update cycle
player.on('iteration', function(ctx) { }); // fired each time the entire timeline is repeated
player.off('finish', function(ctx) { } ) // unregisteres a function from .on()

// or pass an object with all event listeners
player.on({
    finish(ctx) {},
    cancel(ctx) {},
    pause(ctx) {},
    update(ctx) {},
    iteration(ctx) {}
})

player.cancel();         // cancels animation and resets all properties
player.duration();       // returns duration of the animation (to-from) in milliseconds
player.finish();         // pauses animation and seeks to the end when moving forwards and the beginning when backwards
player.pause();          // pauses the animation
player.playState();      // returns the current play state 'playing', 'paused', etc.
player.reverse();        // reverses the direction of the animation

player.play();           // plays the animation
player.play(Infinity);   // plays the animation until canceled

// plays the timeline once forwards and then once backwards
player.play({
    iterations: 2,
    direction: 'alternate'
})

player.currentTime();    // returns the current time of the animations
player.currentTime(500); // seeks to 500 milliseconds

player.playbackRate();    // returns the current playbackRate
player.playbackRate(2); // changes the playback rate to 2 (2x speed)

player.animate({ });      // appends a new animation to this player
player.animate([ ]);      // appends a set of animations to this player
```

### Easings / Timing Functions
For convenience, Just Animate has the majority of the easings on http://easings.net/ built right in.  By default, the "ease" function is used just like in CSS.  If one of these included easings doesn't cut it, a string with a CSS timing function can be used instead as the easing property.

| Name        | Timing Function                           |
| ------------- |:-----------------------------------:|
|ease           | cubic-bezier(.25, .1, .25, 1)             |
|easeIn         | cubic-bezier(.42, 0, 1, 1)                |
|easeInBack     | cubic-bezier(.6, -.28, .735, .045)        |
|easeInCirc     | cubic-bezier(.6, .04, .98, .335)          |
|easeInCubic    | cubic-bezier(.55, .055, .675, .19)        |
|easeInExpo     | cubic-bezier(.95, .05, .795, .035)        |
|easeInOut      | cubic-bezier(.42, 0, .58, 1)              |
|easeInOutBack  | cubic-bezier(.68, -.55, .265, 1.55)       |
|easeInOutCirc  | cubic-bezier(.785, .135, .15, .86)        |
|easeInOutCubic | cubic-bezier(.645, .045, .355, 1)         |
|easeInOutExpo  | cubic-bezier(1, 0, 0, 1)                  |
|easeInOutQuad  | cubic-bezier(.455, .03, .515, .955)       |
|easeInOutQuart | cubic-bezier(.77, 0, .175, 1)             |
|easeInOutQuint | cubic-bezier(.86, 0, .07, 1)              |
|easeInOutSine  | cubic-bezier(.445, .05, .55, .95)         |
|easeInQuad     | cubic-bezier(.55, .085, .68, .53)         |
|easeInQuart    | cubic-bezier(.895, .03, .685, .22)        |
|easeInQuint    | cubic-bezier(.755, .05, .855, .06)        |
|easeInSine     | cubic-bezier(.47, 0, .745, .715)          |
|easeOut        | cubic-bezier(0, 0, .58, 1)                |
|easeOutBack    | cubic-bezier(.175, .885, .32, 1.275)      |
|easeOutCirc    | cubic-bezier(.075, .82, .165, 1)          |
|easeOutCubic   | cubic-bezier(.215, .61, .355, 1)          |
|easeOutExpo    | cubic-bezier(.19, 1, .22, 1)              |
|easeOutQuad    | cubic-bezier(.25, .46, .45, .94)          |
|easeOutQuart   | cubic-bezier(.165, .84, .44, 1)           |
|easeOutQuint   | cubic-bezier(.23, 1, .32, 1)              |
|easeOutSine    | cubic-bezier(.39, .575, .565, 1)          |
|elegantSlowStartEnd | cubic-bezier(.175, .885, .32, 1.275) |
|linear         | cubic-bezier(0, 0, 1, 1)                  |
|stepEnd        | steps(1, end)                             |
|stepStart      | steps(1, start)                           |

### Utilities

#### random()

```javascript
// returns a number (including decimals) between 0 and 100
just.random(0, 100)

// returns 0px to 100px (only whole numbers)
just.random(0, 100, 'px', true)

// returns a whole number between 0 and 100
var start = 0;
var end = 100;
var unit = undefined;
var onlyWholeNumbers = true;
just.random(start, end, unit, onlyWholeNumbers);
```

#### shuffle()

```javascript
// randomly returns fadeIn, bounceIn, or zoomIn
just.shuffle(['fadeIn', 'bounceIn', 'zoomIn']);
```

#### splitText()

Splits the text of an element into words and characters so each word or character can be animated separately. Formatting should be preversed, but each character and word is wrapped in its own element.

```javascript
var characterElements = just.splitText('#element1').characters;
var wordElements = just.splitText('#element1').words;
```

## Demos

- [All Mixins](http://codepen.io/notoriousb1t/full/vXZNvm/)
- [CodePen Demos](http://codepen.io/collection/ANzZxb/)

## Frameworks

Sometimes a little bit of framework integration can go a long way to making a project more livable. Just Animate has bindings for VueJS 2.0 that make it easy to use in HTML.  Read about it [here](./docs/vue.md)

#### JavaScript

```javascript
var app = new Vue({
  el: '#app',
  data: {
    message: 'Hello Vue!'
  }
});
```

## License

Just Animate is licensed under the MIT license. (http://opensource.org/licenses/MIT)

## How can you contribute?

 - make awesome things with Just Animate.  If you make it on CodePen or otherwise, send me a link so I can add it to the demos here.
 - create issues if there is a bug or an unexpected behavior (if it isn't reported, it probably won't get fixed)
 - contribute code and help me make this library great!
 - help with documentation.  It takes four times as long at least to build out docs.  When you contribute docs, you are helping out everyone including me.


text:
# Naive Quant

A naive tool for generate EdgeTPU-ready quantized tflite file.

## Limitations

- This tool does not support TensorFlow 2.x
- Not all keras models are supported since this tool uses Quantization Aware Training tool internally and the quantization process is so complicated.

## Installation

```bash
$ pip install git+https://github.com/ohtaman/naive_quant.git
```

## Usage

```python

import naivequant as nq

...

tflite_file = n.convert(
    keras_model_file,
    representative_dataset,
    input_ranges,
    default_ranges_stats
)

with open('model.tflite', 'wb') as o_:
    o_.write(tflite_file)

```


text:
---
title: Xamarin.Android error XA0112
description: XA0112 error code
ms.date: 10/02/2020
---
# Xamarin.Android warning XA0112

## Issue

The AAPT2 command-line tool was not found.

## Solution

Check for any `<Aapt2ToolPath>` elements in the _.csproj_ file and
remove them. Xamarin.Android redistributes its own recommended version
of AAPT2, so setting a custom value for the `Aapt2ToolPath` MSBuild
property is generally not recommended. Alternatively, try re-installing
the Xamarin workload in Visual Studio.

AAPT is part of the Android SDK and is used internally by
Xamarin.Android to process and compile resources into binary assets.
Learn more about AAPT and Android resources from the [Android
documentation][aapt].

AAPT2 is version 2.0 of AAPT. Learn more about AAPT2 from the
[Android documentation][aapt2].

[aapt]: https://developer.android.com/guide/topics/resources/accessing-resources.html
[aapt2]: https://developer.android.com/studio/command-line/aapt2

repository: xamarin/xamarin-android

branch_name: refs/heads/main

gha_language: C#

gha_license_id: NOASSERTION

repo_name: xamarin/xamarin-android


text:
# Imgur-.NET-SDK

This is an  .NET SDK for the Imgur API

This is an Beta and is currently working

Appveyor CI:

[![Build status](https://ci.appveyor.com/api/projects/status/3o5g55bkf0ouese0?svg=true)](https://ci.appveyor.com/project/Delaire/imgur-net-sdk)

#Setting up

you first need to get your api key and secret for this work and to initialise it:

    ApiRoot.Instance.Init("XXXXXXXX", "XXXXXX");

Here is example of how the api works with the GetEndPointEntityAsync method:

              	ApiRoot.Instance.GetEndPointEntityAsync<RootElement<Data_Format_we_will_recieve>>(
                                         new Data_to_get()
                                         {
                                             //Data parameters
                                         });

Here is an example for how you can make a call:

            var result = await ApiRoot.Instance.GetEndPointEntityAsync<RootElement<GalleryImage>>(
                new GetGalleryImage	()
                {
                    page = 0,
                    section = Section.hot.ToString(),  <-- this could be improved
                    showViral = true,
                    sort = Sort.top.ToString(),
                    window = WindowsSort.day.ToString(),
                });

Here is another example of how to do a Get Random Gallery Image

              	ApiRoot.Instance.GetEndPointEntityAsync<RootElement<GalleryAlbum>>(
                                         new GetRandomGalleryImages()
                                         {
                                             page = pageNumber
                                         });

This API Sdk is used in my personal windows phone imgur app 8gur which can be found here:
	http://windowsphone.com/s?appid=15913087-9ea1-4f42-92b0-8b2c1c764837

Author:
Damien Delaire

if you wish to help be my guest!


text:
# Bitcoin Miner Canaan Creative’s 2019 Revenue Nears $100M Ahead of IPO

Published at: **2019-11-07T07:52:00+00:00**

Author: **William Suberg**

Original: [Cointelegraph](https://cointelegraph.com/news/bitcoin-miner-canaan-creatives-2019-revenue-nears-100m-ahead-of-ipo)

Bitcoin (BTC) mining giant Canaan Creative has seen almost $100 million in revenue this year but remains over $30 million in the red.
According to an amendment form submitted on Nov. 5 as part of its registration to conduct an initial public offering (IPO) in the United States, Canaan revealed its Q3 net profit to be 94 million yuan ($13 million).
In total, the company generated revenue of just over $134 million in the first nine months of 2019, but weaker performance early on has still resulted in a net loss of around $31 million.
Canaan is due to conduct its pioneering IPO this month, subject to formal permission from the U.S. Securities and Exchange Commission, or SEC.
Executives previously attempted to launch an IPO in Hong Kong and China, but both processes fell through due to regulatory misgivings. Bitmain, one of Canaan’s largest competitors, has likewise tried and failed to complete an IPO so far.
Canaan, in fact, saw a healthier performance in 2018, posting net profits of just over $17 million on total revenues which approached $400 million.
As Cointelegraph reported, Bitcoin mining participants remain buoyant about the industry’s future perspectives. Rising Bitcoin hash rate continues a trend that is seeing record computing power devoted to validating and increasing the security of the Bitcoin network.
According to data from monitoring resource Blockchain, the hash rate has nonetheless dipped slightly in recent weeks. The figure currently stands at 90 quintillion hashes per second.

        Bitcoin network hash rate (one year). Source: Blockchain

A popular theory remains that as hash rate increases, so too will the Bitcoin price after a latency period.

license: Unlicense

repository: dsadlollioygjmzvcxv/tdc-exported

repo_name: dsadlollioygjmzvcxv/tdc-exported


text:
---
title: Configure CI/CD Pipeline - Cloud Services
description: Configure CI/CD Pipeline - Cloud Services
---

# Configuring your CI-CD Pipeline {#configure-ci-cd-pipeline}

## Understanding the Flow {#understanding-the-flow}

You can configure your pipeline from the **Pipeline Settings** tile in the [!UICONTROL Cloud Manager] UI.

The Deployment Manager is responsible for setting up the pipeline. When doing so, you first select a branch from the **Git Repository**.

To configure your pipeline, the user must:

* define the trigger that will start the pipeline.
* define the parameters controlling the production deployment.
* configure the performance test parameters.

## Setting up the Pipeline {#setting-up-the-pipeline}

>[!CAUTION]
>
>The pipeline cannot be setup until one program creation is complete and the Git repository has at least one branch.

Before you start to deploy your code, you must configure your pipeline settings from the [!UICONTROL Cloud Manager].

>[!NOTE]
>
>You can change the pipeline settings after initial set up.

## Configuring the Pipeline Settings from [!UICONTROL Cloud Manager] {#configuring-the-pipeline-settings-from-cloud-manager}

Once you have setup your program and have at least one environment using [!UICONTROL Cloud Manager] UI, you are ready to setup your deployment pipeline.

Follow these steps to configure the behavior and preferences for your pipeline:

1. Click **Setup Pipeline** to setup and configure your pipeline.

   ![](assets/set-up-pipeline1.png)

1. The **Setup Pipeline** screen displays. Select the branch and click **Next**.

   ![](assets/set-up-pipeline2.png)

1. Configure your deployment options.

   ![](assets/set-up-pipeline3.png)

   You can define the trigger to start the pipeline:

    * **Manual** - using the UI manually start the pipeline.
    * **On Git Changes** - starts the CI/CD pipeline whenever there are commits added to the configured git branch. Even if you select this option, you can always start the pipeline manually.

    During pipeline setup or edit, the Deployment Manager has the option of defining the behavior of the pipeline when an important failure is encountered in any of the quality gates.

   This is useful for customers who have the desire for more automated processes. The available options are:

   * **Ask every time** - This is the default setting and requires manual intervention on any Important failure.
   * **Fail immediately** - If selected, the pipeline will be cancelled whenever an Important failure occurs. This is essentially emulating a user manually rejecting each failure.
   * **Continue immediately** - If selected, the pipeline will proceed automatically whenever an Important failure occurs. This is essentially emulating a user manually approving each failure.

1. Click **Next** to access the **Testing** tab to define your testing criteria for your program.

   ![](assets/set-up-pipeline4.png)

1. Click **Save**. The *Overview* page now displays the **Deploy your Program** card. Click **Deploy** button to deploy your program.

   ![](assets/configure-pipeline5.png)

## Non-Production & Code Quality Only Pipelines

In addition to the main pipeline which deploys to stage and production, customers are able to set up additional pipelines, referred to as **Non-Production Pipelines**. These pipelines always execute the build and code quality steps. They can optionally also deploy to Adobe Managed Services environment.

On the home screen, these pipelines are listed in a new card:

1. Access the **Non-Production Pipelines** tile from the Cloud Manager home screen.

   ![](assets/configure-pipeline6.png)

1. Clicking on the **Add** button, to specify the Pipeline Name, Pipeline Type, and the Git Branch.

   Additionally, you can also set up Deployment Trigger and Important Failure Behavior from Pipeline Options.

   ![](assets/non-prod-pipe1.png)

1. Click **Save** and the pipeline is shown on the card on the home screen with three actions, as shown below:

   ![](assets/configure-pipeline8.png)

   * **Edit** - allows editing of the pipeline settings
   * **Build** - navigates to the execution page, from which the pipeline can be executed
   * **Manage Git** - allows the user to get the information necessary to access Cloud Manager Git repository

## The Next Steps {#the-next-steps}

Once you have configured the pipeline, you need to deploy your code.

Please see [Deploy your Code](deploy-code.md) for more details.


text:
# SQL Learning Notes

Wildcard

e.g

SELECT prod_id,prod-name FROM Products
WHERE
Prod_name LIKE 'Fish%' //all starts with fish
//accepting anything after fish
//microsoft Access uses *

or where
prod_name LIKE '%bean bag%' //as long as bean bag appear

using _ as wildcard

one char

//Access uses ? not _

WHERE prod_name LIKE '__ inch teddy bear';

[]// only supported by SQL server and Access

e.g find all contact start with J or M

FROM Customers
WHERE cust_contact LIKE '[JM]%'
ORDER BY cust_contact;

concatenate

"+ or || depends on server "

SELECT vend_name + '(' + vend_country + ')'
FROM Vendors
ORDER BY vend_name;

repository: Han-Haosen/MDNotes

repo_name: Han-Haosen/MDNotes


text:
# Master the Python Data Science Ecosytem

If you are looking to become an expert, check out my books:

* [Master the Fundamentals of Python][0]
* [Master Data Analysis with Python][1]
* [Master Machine Learning with Python][2]

They are all extremely comprehensive and offer lots of exercises with detailed solutions.

## This repository

Contains a tutorial on how to integrate pandas with scikit-learn using the ColumnTransformer

[0]: https://www.dunderdata.com/master-the-fundamentals-of-python
[1]: https://dunderdata.com/master-data-analysis-with-python
[2]: https://www.dunderdata.com/master-machine-learning-with-python


text:
# Peking University/Baidu - Autonomous Driving - CenterNet

This is a fork of [CenterNet repo](https://github.com/xingyizhou/CenterNet) aimed at using this architecture for aforementioned Kaggle's competition. Major changes:
* added new dataset `kaggle_cars`;
* added new pipeline `car_pose_6dof`;
* prerendering and using 3D location masks (the idea comes from 2018 paper *3D Pose Estimation for Fine-Grained Object Categories*);
* mixed precision training (Nvidia's Apex amp) and gradient accumulation for higher batch size;
* options to use SWA and weight decay for better generalization;

To learn more about the competition please visit [kaggle.com](https://www.kaggle.com/c/pku-autonomous-driving).

## Installation

Please refer to [INSTALL.md](readme/INSTALL.md) for installation instructions.

(OPTIONAL) You may want to install NVIDIA Apex to be able to use mixed precision training. Please, visit [this repo](https://github.com/NVIDIA/apex) for more info.

## Setup data

By default for `--dataset kaggle_cars` the script will search data in *data/pku-autonomous-driving*. You can create a symlink like this one:

~~~
$ ln -s /home/user/projects/kaggle_cars/input/pku-autonomous-driving /home/user/projects/kaggle_cars/centernet/data
~~~

To split images for train and validation create folder *split* with `train.txt, val.txt, ignore.txt`. Each file should contain a list of the image files with extensions. For example you may try this:

~~~
$ mkdir split
$ ls train_images | head -n 3800 > split/train.txt
$ ls train_images | tail -n 462 > split/val.txt
~~~

If you intend to use 3D location masks, you have to generate it first using another script from *tools* folder:

~~~
$ python src/tools/prepare_3d_loc_masks.py --num_workers 4 --norm_xyz 519.834,689.119,3502.94
~~~

By default it creates *train_3d_masks* folder in the dataset directory, but you can overwrite this behaviour providing another path with `--xyz_masks_dir`.

 If you want to load specific weights, pass it to `--load_model` .

## Save averaged model weights

Run train script with `--test --save_avg_weights`. The averaged weights file name ends with *_avg* suffix.

## Run predictions on test set

~~~
$ python test.py car_pose_6dof --exp_id car_pose_default --dataset kaggle_cars \
 --xyz_mask \
 --load_model ../exp/car_pose_6dof/car_pose_default/model_20_avg.pth --resume \
 --peak_thresh 0.5 --K 50 \
 --gpus 1 --trainval --not_prefetch_test
~~~

## Debug / visualisations

Use `--debug 4 --render_cars` to see rendered car models. Just add these parameters to train/test command lines. Adding `--debug_heatmap` will also visualise heatmaps.

## Citation

~~~
@inproceedings{zhou2019objects,
  title={Objects as Points},
  author={Zhou, Xingyi and Wang, Dequan and Kr{\"a}henb{\"u}hl, Philipp},
  booktitle={arXiv preprint arXiv:1904.07850},
  year={2019}
}

@inproceedings{wang20183d,
  title={3D Pose Estimation for Fine-Grained Object Categories},
  author={Wang, Yaming and Tan, Xiao and Yang, Yi and Liu, Xiao and Ding, Errui and Zhou, Feng and Davis, Larry S},
  booktitle={European Conference on Computer Vision Workshop},
  pages={619--632},
  year={2018},
  organization={Springer}
}
~~~


text:
# Changes

## 1.4.5

- [Fix] Add handling of no time remaining
- [Fix] Stopped warning for clear_low_battery message

## 1.4.4

- [Enhancement] Removed the maxRetries on the websocket so it will keep trying to reconnect forever

## 1.4.3

- [Enhancement] Added config.schame.json to enable Homebridge Config UI

## 1.4.1

- [Enhancement] Improved getToken and getDevices error handling


text:
# DKR command: Terraform

This is [terraform](https://www.terraform.io/) command for
[dkr](https://github.com/hekonsek/dkr).

## Packages

- `terraform`
- `helm` (required by Helm Terraform Helm module)
- `kubectl` (required by `helm`)
- `aws-iam-authenticator` (required by `kubectl` AWS authentication)
- `aws` cli v2 (required by Terraform AWS EKS module and `aws-iam-authenticator`)

## License

This project is distributed under [Apache 2.0 license](http://www.apache.org/licenses/LICENSE-2.0.html).

repository: hekonsek/dkr-terraform

repo_name: hekonsek/dkr-terraform


text:
## PropAuth: Property-based policy evaluation

[![Travis-CI Build Status](https://secure.travis-ci.org/psecio/propauth.png?branch=master)](http://travis-ci.org/psecio/propauth)

Performing evaluations on credentials for authentication or sets of permissions on users has its limitations. With these things you're restricted to evaluations like "has permission" or "credentials invalid". The goal behind `PropAuth` is to make these evaluations much more flexible and allow you to define reusable *policies* that can be evaluated against the provided user dynamically.

> Hey Laravel users, there's also a provider to help you integrate PropAuth into your application and Blade templates: [PropAuth-Provider](https://github.com/psecio/propauth-provider)

## Installation

You can install the library easily with Composer:

```bash
composer.phar install psecio/propauth
```

## Examples

```php
<?php

require_once 'vendor/autoload.php';

use \Psecio\PropAuth\Enforcer;
use \Psecio\PropAuth\Policy;

$enforcer = new Enforcer();
$myUser = (object)[
	'username' => 'ccornutt',
    'permissions' => ['test1'],
    'password' => password_hash('test1234', PASSWORD_DEFAULT)
];

$myPolicy = new Policy();
$myPolicy->hasUsername('ccornutt');

$result = $enforcer->evaluate($myUser, $myPolicy);
echo 'RESULT: '.var_export($result, true)."\n\n"; // result is true

// You can also chain the evaluations to make more complex policies
$myPolicy->hasUsername('ccornutt')->hasPermissions('test1'); // also true

// There's also a static method to help make the creation more concise
$myPolicy = Policy::instance()->hasUser('ccornutt')->hasPermissions('test1');

?>
```

> **NOTE:** All matching is treated as *AND* so **all** criteria must be true for the evaluation to pass.

## Allowed User Types

The PropAuth engine tries several different ways to get information from the user instance (properties) that should accommodate most of the common User class types out there. When checking for a property, the engine will, in this order:

- Look for a public property with the given name
- Look for a getter with the property name (ex: for "foo" it looks for "getFoo")
- Look for the "getProperty" method specifically and calls it with the property name

In the first code example, we're just creating a basic class (`stdClass`) and applying public properties so it would match with the first check for public properties.

## Verifying passwords

You can also use `PropAuth` to verify passwords as a part of your policy right along with the other evaluations. Here's an example of a policy that would verify the input for the user defined above:

```php
<?php
$myUser = (object)[
    'username' => 'ccornutt',
    'password' => password_hash('test1234', PASSWORD_DEFAULT)
];

$gate = new Gateway($myUser);
$subject = $gate->authenticate($password);

if ($subject !== false && $subject->can('policy1') === true) {
	echo 'They can, woo!';
}

?>
```

The password validation assumes the use of the [password hashing methods](http://php.net/manual/en/ref.password.php) and so requires PHP >=5.5 to function correctly. The plain-text password is given to the policy and hashed internally. Then the values are checked against the ones provided in the user for a match. In this case, if they put in either the wrong username or password, the policy evaluation will fail.

## How it checks properties

If you'll notice, we've called the `hasUsername` method on the `Policy` above despite it not being defined on the `User` class. This is handled by the `__call` magic method. It then looks for one of two key words: `has` or `not`. It determines which kind of check it needs to perform based on this.

- **has:** Tells the system to do an "equals" match, comparing the given value and the property value
- **not:** Tells the systems to do a "not equals" match

This gives you the flexibility to define custom policies based on the properties your user has and does not restrict it down to just a subset required by the object.

## Rules (`ANY` and `ALL`)

Your checks can have a second parameter after the value that further customizes the checks that it performs: `Policy::ANY` and `Policy:ALL`. These have different meanings based on the data in the property and the data defined. Here's a brief summary:

Property data type | Input Data type | ANY                                     | ALL
------------------ | --------------- | --------------------------------------- | -----------------------------------------------
string             | string          | equals (==)                             | equals (==)
string             | array           | input contains property                 | all input values equal property
array              | string          | property contains input                 | all property values equal input
array              | array           | any match of input with property values | property values exactly equals input

> **NOTE:** The `Policy::ANY` rule is the default, regardless of data input type. All matches are done as *exactly* equal, even arrays (so if the order of the values is different they won't match).

## Searching by "path"

PropAuth also allows you to search for the data to evaluate by a "path". Using this searching, you can recurse down through object and array data to locate just what you want rather than having to gather it before hand. Here's an example:

```php
<?php
class User
{
    public $permissions = [];
    public $username = ['user1', 'user2'];
}

class Perm
{
    public $name = '';
    protected $value = '';
    public $tests = [];

    public function __construct($name, $value)
    {
        $this->name = $name;
        $this->value = $value;
    }
    public function setTests(array $tests)
    {
        $this->tests = $tests;
    }

    public function getValue()
    {
        return $this->value;
    }
}
class Test
{
    public $title;
    public $foo = [];

    public function __construct($title)
    {
        $this->title = $title;
    }
    public function setFoos(array $foo)
    {
        $this->foo = $foo;
    }
}
class Foo
{
    public $test;

    public function __construct($test)
    {
        $this->test = $test;
    }
}

$policies = PolicySet::instance()
    ->add('test', Policy::instance()->find('permissions.tests.foo')->hasTest('t3'));

?>
```

In this case, we have a lot of nested data under the `User` instance that we want to evaluate. For this `test` policy, however, we only want one thing: the "test" values from the `Foo` objects. To fetch these we'd have to go through this process:

1. On the `User` instance, get the `permissions` property value
2. For each of the items in this set, get the `Test` instances that relate to them
3. Then, for each one of these tests, we want only the `Foo` instances related in a set.

While this can be done outside of the PropAuth library and just passed in directly for evaluation, the search "path" handling makes it easier. To perform all of the above, you just use the search path in the example above: `permissions.tests.foo`. This fetches all of the `Foo` instances then the `hasTest` method looks at the `test` value on the `Foo` objects to see if any match the `t3` value.

## Other examples

**All of the following examples evaluate to `true` based on the defined user.**

```php
<?php

$policy = new Policy();

#### POSITIVE CHECKS

// Check to see if the permission is in a set
$user = new User([
	'permissions' => ['test1', 'test2']
]);
$policy->hasPermissions('test1');

// Check to see if any permissions match
$policy->hasPermissions(['test3', 'test2', 'test5'], Policy::ANY);

// Check to see that the permissions match exactly
$policy->hasPermissions(['test1', 'test2'], Policy::ALL);

#### NEGATIVE CHECKS

// Check to see if the permission is NOT in the set
$policy->notPermissions('test5');

// Check to see if NONE of the permissions match
$policy->notPermissions(['test3', 'test5', 'test6'], Policy::ANY);

// Check to see if the permissions are NOT equal
$policy->notPermissions(['test4', 'test5'], Policy::ALL);
?>

```

## Using Callbacks

If you have some more custom logic that you need to apply, you might want to use the callback handling built into PropAuth. Much like the "has" and "not" of the property checks, there's "can" (result should be true) and "cannot" (result should be false) for callbacks. Here's an example of each:

```php
<?php
// Make a user
$myUser = (object)[
	'username' => 'ccornutt',
	'permissions' => ['test1']
];

// Make a post
$post = (object)[
	'title' => 'This is a test post',
	'id' => 1
];

$myPolicy = new Policy();
$myPolicy
    ->hasUsername(['ccornutt', 'ccornutt1'], Policy::ANY)
    ->can(function($subject, $post) {
		return ($post->id === 1);
    })
    ->cannot(function($subject, $post) {
		return (strpos($post->title, 'foobar') === false);
    });

$result = $enforcer->evaluate($myUser, $myPolicy, [ $post ]); // result is TRUE
?>
```

> **NOTE:** The additional parameters that are passed in to the `evaluate` method will be given to the closure check types in the same order they're given in the array. However, the first parameter will *always* be the subject (User) being evaluated.

## Policy Sets

It's also possible to defile a policy in a set, referenced by a key name (string). For example, if we wanted to create a simple policy that let a user with the username "testuser1" be able to perform an "edit post" action:

```php
<?php

$set = new PolicySet();
$set->add(
	'edit-post',
	Policy::instance()->hasUsername('testuser1')
);

// Or, using the instance method
$set = new PolicySet()
	->add('edit-post', Policy::instance()->hasUsername('testuser1'));
?>
```

Then, when we want to evaluate the user against this policy, we can use the `allows` and `denies` methods after injecting the set into the `Enforcer`:

```php
<?php
$myUser = (object)[
	'username' => 'testuser1',
	'permissions' => ['test1']
];
$enforcer = new Enforcer($set);

if ($enforcer->allows('edit-post', $myUser) === true) {
	echo 'Hey, you can edit the post - rock on!';
}

?>
```

## Using a Class & Method for Evaluation

You can also use a class and method for evaluation as a part of `can` and `cannot` checks similar to how the closures work. Instead of passing in the closure method like in the previous examples, you simply pass in a string with the class and method names separated by a double colon (`::`). For example:

```php
<?php

$policy = Policy::instance()->can('\App\Foo::bar', [$post]);

?>
```

In this example, you're telling it to try to create an instance of the `\App\Foo` class and then try to call the `bar` method on that instance. *Note:* the method does not need to be static despite it using the double colon. Much like the closures, the subject will be passed in as the first parameter. Additional information will be passed in as other parameters following this.

So, in our above example the method would need to look like this:

```php
<?php
namespace App;

class Foo
{
	public function foo($subject, $post)
	{
		/* evaluation here, return boolean */
	}
}

?>
```

## Loading Policies from an External Source

Defining policies in your code is good, but sometimes it just makes more sense to have them in an external location where you can load them as needed. Maybe you have a situation where only "Post" related policies need to be loaded and not everything across the entire site. `PropAuth` offers a "load DSL" method on the `Policy` class that can help here.

What's a DSL? A "domain specific language" lets you define a string in a certain format where `PropAuth` will understand how to parse it and create a policy instance from it. Let's start with an example and then break down the format:

```
hasUsername:ccornutt||notUsername:ccornutt1||hasPermissions:(test1,test2)[ANY]
```

You'll notice that there's similarities in the method names you'd call if you were making the policy yourself and what the DSL will recognize (like `hasUsername`). This DSL string defines the following policy:

- subject has the username of "ccornutt"
- subject does not have the username of "ccornutt1"
- subject has any of the following permissions: test1, test2

The logic is the same as if you were manually setting those methods on the policy object, just in a simplified way. Here's how it looks in code:

```php
<?php
$dsl = 'hasUsername:ccornutt||notUsername:ccornutt1||hasPermissions:(test1,test2)[ANY]';
$myPolicy = Policy::load($dsl);
?>
```

Simple right? Okay, so lets look at the format:

- the method/value pairs are split by the double pipe (`||`)
- the method name and value are then split by a colon (`:`)
- single string values are just put into the string, arrays are surrounded by parentheses and split with a comma
- modifiers (like `ANY` or `ALL`) are added to the end of the method/value pair, enclosed in brackets (`[]`)

Obviously this is only really for simpler checks where the criteria can be defined by strings and simple values, but it can be quite useful for a wide range of circumstances. Of course, you can always pull these as base policies and then add on to them manually once you have the `Policy` object created post-load.

## Using the Gateway interface for evaluation

In addition to the powerful features already listed here, the `PropAuth` library also provides a simplified interface for working with your user (subject) and its authentication and authorization.

First, let's take a look at authentication. It uses a `bcrypt` hashing method behind the scenes to evaluate the password:

```php
<?php
$myUser = (object)[
    'username' => 'ccornutt',
    'password' => password_hash('test1234', PASSWORD_DEFAULT)
];

$gate = new Gateway($myUser);
$subject = $gate->authenticate($password);

// Then we can check if the user is authenticated
echo 'Is authenticated? '.var_export($subject->isAuthed(), true)."\n";

?>
```

We create the `Gateway` class instance and then can call the `authenticate` method on it with the password provided. The script then assumes it can access the user's `password` property as a value on the object and makes a comparison. It will return a new instance of a `Subject` class if the authentication is successful or `false` if not. The `Subject` class is just a wrapper around your object (`$myUser` in this case). The original object can be fetched using the `Subject->getSubject()` method.

Additionally, you can provide a bit more context and use the `Gateway` interface for policy evaluation too. You simply define the policies as a part of the `Context` object in the constructor:

```php
<?php
$context = new Context([
	'policies' => [
		'policy1' => Policy::instance()->hasUsername('ccornutt')
	]
]);
$gate = new Gateway($myUser, $context);

// When we can call the "evaluate" method with the policies we want to check:
if ($gate->evaluate('policy1') === true) {
	echo 'Policy1 passes!';
}

// Or you can just add your own PolicySet instance and use "evaluate" the same way
$myPolicySet = new PolicySet()->add('edit-post', Policy::instance()->hasUsername('testuser1'));

$context = new Context([
	'policies' => $myPolicySet
]);

?>
```

Once you have your valid `Subject` instance, you can then check its abilities with the `can` and `cannot` methods:

```php
<?php
$myUser = (object)[
    'username' => 'ccornutt',
    'password' => password_hash('test1234', PASSWORD_DEFAULT)
];

$context = new Context([
	'policies' => [
		'policy1' => Policy::instance()->hasUsername('ccornutt')
	]
]);

$gate = new Gateway($myUser, $context);
$subject = $gate->authenticate($password);

if ($subject !== false && $subject->can('policy1') === true) {
	echo 'They can, woo!';
}

?>
```

The parameter on the `can` and `cannot` methods are policy names you've already defined in your context. If the `Policy` is defined as a closure with more complex logic, you can provide this option (or multiple options) as the second parameter:

```php
<?php
$post = (object)[
	'author' => 'ccornutt'
];
$set = PolicySet::instance()->add(
    'delete',
    Policy::instance()->can(function($subject, $post) {
        return ($subject->username == 'ccornutt' && $post->author == 'ccornutt');
    })
);
$context = new Context(['policies' => $set]);
$gate = new Gateway($myUser, $context);

$subject = $gate->authenticate('test1234');

if ($subject->can('delete', $post) === true) {
	echo 'They can delete it!';
}
?>
```

repository: psecio/propauth

repo_name: psecio/propauth


text:
{{{
"title": "Application site. Login and Quick Tour",
"date": "09-01-2016",
"author": "",
"attachments": [],
"contentIsHTML": false,
"sticky": true
}}}

### Login and Quick Tour

**In this article:**

* [Overview](#overview)
* [Audience](#audience)
* [Prerequisites](#prerequisites)
* [Login](#login)
* [Quick tour](#quick-tour)

 ### Overview
 This article makes a quick tour on login page and the panel after login to the platform in **Applications site**.
 If you want to administer your organization go to [Management site](../Administering Your Organization/admin-overview.md)

 ### Audience
 All users with Cloud Application Manager access.

 ### Prerequisites
 * Access to Cloud Application Manager, [Applications site](https://cam.ctl.io/login).

___

### Login

**Where to log in?**

* To access Cloud Application Manager in the cloud, browse to [Applications site](https://cam.ctl.io/login) or your Cloud Application Manager company URL.
* To access Cloud Application Manager inside a private network, ask your administrator for the login URL.

   **Note**: Enable JavaScript on the browser to use Cloud Application Manager. If not, some elements of the web interface may not work or render properly. We support Cloud Application Manager on the latest Chrome, Safari, Firefox, and Internet Explorer browsers.

**How to log in?**

You have a few options:

* Sign up for an Cloud Application Manager account with a username and password.
* Log in with an existing Google account.
* Log in with your current GitHub credentials if your org admin has enabled GitHub sign in for Cloud Application Manager.
Enter your company Active Directory credentials in the username, password fields if your admin enabled LDAP integration with Cloud Application Manager.

   **Note**: When you log in with your AD credentials, GitHub or Google accounts, Cloud Application Manager does not have access to your password. We use your email to create a profile and workspace for you.

![CAM Login Page](../../images/cloud-application-manager/getting-started-login-1.png)

___

### Quick Tour

When you log in, at the top you will see the global navigation bar. It has three sections: Service, Site and Context.
Notice you are in the **Applications site**. In the context section you can access to **Workspaces**.

* **Service**  Cloud Application Manager
* **Site**     Applications site
* **Context**  Organization, Cost Center or Workspace selected

On the left side, a menu guides you to the main areas of Cloud Application Manager

* **Dashboard**
* **Instances**
* **Boxes**
* **Providers**
* **Catalog**
* **Activity**

On the right side of the top navigation bar, you will find the links to **Help** and **My Account** menus.

* **My Account**
* **Help**

____

**Workspaces**

![CAM Workspace menu](../../images/cloud-application-manager/getting-started-login-2.png)

When you log in, you’re in your personal workspace. From the workspace drop-down, you can access team workspaces if you belong to them, or create one and add others to share Cloud Application Manager assets and collaborate. To learn more, see [Workspaces & Sharing](../Core Concepts/workspaces-and-collaboration.md).

___

**Dashboard**

![CAM workspace panel](../../images/cloud-application-manager/getting-started-login-8.png)

The Dashboard page displays a global vision of the content and uses of the current scope. There you can see the number of instances, boxes or providers it contains. In addition, there are two charts with the uses by day, week or month.

___

**Instances**

![CAM instances menu ](../../images/cloud-application-manager/getting-started-login-3.png)

* Click **New** to launch instances of boxes to a provider environment in the public or private cloud.
* Manage instances you’ve [launched](../Deploying Anywhere/deploying-managing-instances.md) through the web interface, the API, or on [any infrastructure using the Cloud Application Manager agent](../Deploying Anywhere/deploying-on-anyinfra.md). Here you can quickly [manage the lifecycle](../Deploying Anywhere/deploying-managing-instances.md) of several instances from the Bulk Actions menu or handle them individually from the gear menu.
* Find an instance by searching any part of its name, or click filtered views of instances you launched or that were shared with you. Or locate them by tags.

___

**Boxes**

The Boxes page shows everything you create including boxes shared with you.

![CAM Boxes menu](../../images/cloud-application-manager/getting-started-login-4.png)

* Click **New** to create a box. Automate configuration by selecting a box type: Script, Deployment Policy, CloudFormation, Container.
* Cloud Application Manager provides a public service box catalog, which you find when you filter by the public tag. Public boxes are available to all users. These are pre-configured, which means you can directly deploy or nest them in other boxes.  Examples of public boxes include MongoDB, Puppet, Chef Solo, Rails, Redis, RabbitMQ, WordPress among others.
   **Note**: Some public boxes require an Internet connection to install software. So if you’re using Cloud Application Manager as an appliance in your private datacenter without Internet access, these boxes will not deploy.
* Organize boxes as icons or as a list. Sort alphabetically by name, most recently viewed, or by the owner.
* Search for a box by name, owner, or filter by technology under Requirements. The owner is a user or a workspace.

___

**Providers**

![CAM providers menu](../../images/cloud-application-manager/getting-started-login-5.png)

* Connect to a provider to orchestrate deployments. Click **New** to add AWS, Azure, vSphere, Google Cloud, OpenStack, or CloudStack.
* Locate a provider through search or by type.
* Sync or delete provider accounts using Bulk Actions.

___

**Catalog**

![CAM catalog menu](../../images/cloud-application-manager/getting-started-login-9.png)

* Shows you all the public boxes available for all users, categorized as **Featured, Managed Services, Services, Plugins** and **Other** to easily locate and deploy.
* There is a feature where you can request to publish one of your own boxes to the catalog. It will become available in the catalog for any user in one of the existing categories or a new one.

___

**Activity**

![Activity page](../../images/cloud-application-manager/getting-started-login-10.png)

* Allows you to see (based on your scope and permissions) all of the relevant activity that has occurred in the platform, the type of event and the user that performed the event.
* Filter through activity by type of event or switch between all activity or current scope.
![Activity page, Filter dropdown](../../images/cloud-application-manager/activity-filter-list.png)

    * **Provider**, this option filters the result and shows only the provider's activity.
    * **Box**, this option filters the result and shows only the box's activity.
    * **Instance**, this option filters the result and shows only the activities that are related to the instances.
    * **Catalog**, this option filters the result and shows only the catalog's activity.

        > **note that you can combine these filters with each other on your own will.**

* Search by user name or action to narrow the results, or click on certain actions to navigate to the affected resource.

___

**My Account**

![CAM account menu](../../images/cloud-application-manager/getting-started-login-6.png)

* From the username drop-down, access your account profile where you can change your username or reset password if you’re using username and password to log in.
* Get tokens to use Cloud Application Manager via the CLI or API with Authentication Tokens.

___

**Help**

Please review the [troubleshooting tips](../Troubleshooting/troubleshooting-tips.md) for help. Or you may contact<EMAIL_ADDRESS>from within the Cloud Application Manager interface.

![CAM help/support menu](../../images/cloud-application-manager/getting-started-login-7.png)

For issues related to API calls, send the request body along with details related to the issue.

In the case of a box error, share the box in the workspace that your organization and Cloud Application Manager can access and attach the logs.
* Linux: SSH and locate the log at /var/log/elasticbox/elasticbox-agent.log
* Windows: RDP into the instance to locate the log at ProgramDataElasticBoxLogselasticbox-agent.log

repository: AnthonyHakim/PublicKB

path: /Cloud Application Manager/Getting Started/getting-started-login-quick-tour.md

repo_name: AnthonyHakim/PublicKB


text:
---
layout: post
date: 2017-09-17
title: Pandas Indexing
description: Comparing different pandas indexing method
categories: python
tags: python pandas
---

## Basic

There are few method we can access our DataFrame with pandas. There are `.ix`, `.loc` and `.iloc`. We will examine them one by one.

###### `.ix`

Since `.ix` is deprecated right now. And `.loc`, `.iloc` are expected replacements. `.ix` is a mixture of `.loc` and `.iloc` which supports mixed integer and label based indexing. We will focus more time with `.loc` and `iloc` instead of `.ix`.

###### `.loc`

Label-location based indexing

It has the basic format of `pd.DataFrame.loc[<row(s)>,<column(s)>]`. If we want to select single row of some DataFrame we created. We can use `myDataFrame.loc[0,:]`. Since it starts with number **0**. And colon (:) means selecting all columns. If we want to select certain rows or columns. We can use `myDataFrame.loc[[0,1,2],:]` or `myDataFrame.loc[0:2,:]` to select row 0,1,2 and all columns. And please notice that when we use `0:2` colon expression, it will include both endpoints (both 0 and 2), unlike `range(0,2)`, only include left endpoint, not right endpoint (only 0, not 2). Or we want to select certain columns, we can use `myDataFrame.loc[:,['col_name1','col_name3']]` will only select two columns of 'col_name1' and 'col_name2'. We also can select consecutive columns using `myDataFrame.loc[:,'col_name1':'col_name3']` to select three columns of 'col_name1' to 'col_name2'.

### Boolean Selection

To select observations with State column equal to 'New York', we can use `myDataFrame.loc[myDataFrame.State=='New York',:]`.

###### `.iloc`

Integer-location based indexing

It has the same format as `.loc` of `pd.DataFrame.iloc[<row(s)>,<column(s)>]`. To select single of multiple rows or columns, we can use `myDataFrame.iloc[:,[0,3]]` to select all rows and first column(0) and forth column(3). We can also use `0:3` colon expression. However in this case, it's inclusive from left endpoint but **exclusive** from right endpoint like `list(range(0,3))`. In this case `0:3` will show only 0,1,2 columns. You can also do a boolean selection like `.loc` in `.iloc`.


text:
# 🦴 Laravel Skeleton

[![Latest Version on Packagist](https://img.shields.io/packagist/v/lorisleiva/skeleton.svg)](https://packagist.org/packages/lorisleiva/skeleton)
[![GitHub Tests Action Status](https://img.shields.io/github/workflow/status/lorisleiva/package-skeleton-laravel/Tests?label=tests)](https://github.com/lorisleiva/package-skeleton-laravel/actions?query=workflow%3ATests+branch%3Amain)
[![Total Downloads](https://img.shields.io/packagist/dt/lorisleiva/skeleton.svg)](https://packagist.org/packages/lorisleiva/skeleton)

A skeleton repository for Laravel Packages.

## Installation

```bash
composer require lorisleiva/skeleton
```

## Usage

``` php
//
```

repository: secmohammed/package-skeleton-laravel

repo_name: secmohammed/package-skeleton-laravel


text:
# Client-go equivalent of Kubectl

## Introduction
Kubectl is a CLI tool for interacting to a  Kubernetes cluster.
Similarly, client-go is a library for interacting to a Kubernetes cluster that
can be used in go programs. Using client-go you can do CRUD operations on
Kubernetes resources or fetch information regarding the Kubernetes server.
This document is kind of a cheat sheet to know the exact method of the client-go
library to manipulate a particular API in Kubernetes.

Client -go library can be found [here](https://github.com/kubernetes/client-go)

## Goal

Consider following:
```bash
$ kubectl version
```
Above command prints the Kubernetes version. Now you are required to find
the Kubernetes version via a go program. Well, you can use methods available
in client-go to do that.

Now obivously, we can explore the client-go and find out the right method. But
sometimes it becomes little of search/explore job especially for people who are
just getting started with Kubernetes and client-go in general.

The cheat sheet should ease the process. I call this `Cleqku` cheat sheet.

## Cleqku Cheat Sheet

We first need an `clientset` object that can help talk to Kubernetes API.

Following is the code to do it :

```go
    var kubeconfig *string
	if home := homedir.HomeDir(); home != "" {
		kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
	} else {
		kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
	}
	flag.Parse()

	// use the current context in kubeconfig
	config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
	if err != nil {
		panic(err.Error())
	}
    // The clientset object can be used to talk to Kubernetes API
	clientset,err:=k8sclient.NewForConfig(config)
```

### **Fetching Kubernetes Server Version**

Kubectl Command :

```bash
$ kubectl version
```
Cleqku :

```go
version,err:=clientset.Discovery().ServerVersion()
```

### **Fetching Kubernetes API Resources,Versions And Groups**

Kubectl Command :

```bash
$ kubectl api-resource
$ kubectl api-versions
```
Cleqku :

```go
// Returns API group list
apiGrpList,err:=clientset.Discovery().ServerGroups()
// Returns API group and API resources list
apiGrpList,apiResList,err:=clientset.Discovery().ServerGroupsAndResources()
```

### **Fetching Kubernetes Resource List By GroupVersion**

Kubectl Command :

```bash
N/A
```
Cleqku :

```go
groupVersion:="apps/v1"
// Returns list of resources in `apps` group and `v1` version
resList,err:=clientset.Discovery().ServerResourcesForGroupVersion(groupVersion)
```

### **Fetching Kubernetes Resources With The Server Preferred Version**

Kubectl Command :

```bash
N/A
```
Cleqku :

```go
resList,err:=clientset.Discovery().ServerPreferredResources()
```

### **Fetching Kubernetes Namespaced Resources With The Server Preferred Version**

Kubectl Command :

```bash
N/A
```
Cleqku :

```go
resList,err:=clientset.Discovery().ServerPreferredNamespacedResources()
```

### **CRUD Operation On Resources**
Note:
- For CRUD operations on resources using Kubectl, please refer the
Kubectl [doc](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands)

### **CRUD Operation On ConfigMap Resource**
Cleqku:
```go
ns:="exampleNamespace"
// cmClient will have methods available to do CRUD
cmClient:=clientset.CoreV1().ConfigMaps(ns)
```

### **CRUD Operation On Events Resource**

Cleqku:
```go
ns:="exampleNamespace"
// eventsClient will have methods available to do CRUD
eventsClient:=clientset.CoreV1().Events(ns)
```

### **CRUD Operation On Endpoints Resource**

Cleqku:
```go
ns:="exampleNamespace"
epClient:=clientset.CoreV1().Endpoints(ns)
```

### **CRUD Operation On LimitRanges Resource**

Cleqku:
```go
ns:="exampleNamespace"
lrClient:=clientset.CoreV1().LimitRanges(ns)
```

### **CRUD Operation On Namespaces Resource**

Cleqku:
```go
nsClient:=clientset.CoreV1().Namespaces()
```

### **CRUD Operation On Nodes Resource**

Cleqku:
```go
nodeClient:=clientset.CoreV1().Nodes()
```


text:
#Allergies Section

Data model description - [Allergies](../allergies.md)

##JSON model sample

```javascript
[
    {
        "allergen": {
            "name": "Antibotic"
        },
        "date": [
            {
                "date": "1926-01-08T00:00:00Z",
                "precision": "day"
            },
            {
                "date": "1955-03-13T00:00:00Z",
                "precision": "day"
            },
            {
                "date": "1949-09-28T00:00:00Z",
                "precision": "day"
            }
        ],
        "reactions": {
            "reaction": {
                "name": "Vomiting"
            },
            "severity": "Severe"
        }
    },
    {
        "allergen": {
            "name": "Grasses"
        },
        "date": [
            {
                "date": "1973-05-13T00:00:00Z",
                "precision": "day"
            },
            {
                "date": "1996-07-20T00:00:00Z",
                "precision": "day"
            },
            {
                "date": "2008-09-27T00:00:00Z",
                "precision": "day"
            }
        ],
        "reactions": {
            "reaction": {
                "name": "Sneezing"
            },
            "severity": "Severe"
        }
    }
]
```

##Original CMS snippet

```text

--------------------------------
Self Reported Allergies

--------------------------------

Source: Self-Entered

Allergy Name: Antibotic

Type: Drugs

Reaction: Vomiting

Severity: Severe

Diagnosed: Yes

Treatment: Allergy Shots

First Episode Date: 01/08/1926

Last Episode Date: 03/13/1955

Last Treatment Date: 09/28/1949

Comments: Erythromycin

Allergy Name: Grasses

Type: Environmental

Reaction: Sneezing

Severity: Severe

Diagnosed: Yes

Treatment: Avoidance

First Episode Date: 05/13/1973

Last Episode Date: 07/20/1996

Last Treatment Date: 09/27/2008

Comments:

```

repository: mariaro/blue-button

repo_name: mariaro/blue-button


text:
# Profile-cleaner
Use profile-cleaner to reduce the size of browser profiles by organizing their sqlite databases using sqlite3's vacuum and reindex functions. The term "browser" is used loosely since profile-cleaner happily works on some email clients and newsreaders too.

## Supported Browsers
* Brave (stable, beta, dev and nightly) (https://aur.archlinux.org/packages/brave-bin)
* Chromium (stable, beta, and dev)
* Conkeror
* Edge
* Falkon
* Firefox (stable, beta, and aurora)
* Google-chrome (stable, beta, and dev)
* Heftig's version of Aurora (https://bbs.archlinux.org/viewtopic.php?id=117157)
* Icecat
* Icedove
* Inox (https://bbs.archlinux.org/viewtopic.php?id=198763)
* Librewolf
* Midori
* Newsboat
* Palemoon
* Qupzilla
* Seamonkey
* Thunderbird
* Tor-browser
* Vivaldi (stable and snapshot)

## Documentation
Consult the man page.

## Installation from Source
To build from source, see the included INSTALL text document.

## Installation from Distro Packages
* ![logo](http://www.monitorix.org/imgs/archlinux.png "arch logo")Arch via the official package in [community]
* ![logo](http://i.imgur.com/ooVjgFG.png "gentoo logo")Gentoo: See instructions below
* ![logo](https://i.postimg.cc/13Jzxh5C/nix.png "nixos logo")NixOS: in the official repos
* ![logo](http://wiki.codeblocks.org/images/8/8b/Slackware-logo_32.png "slack logo")Slackware: on [slackbuilds](http://slackbuilds.org/apps/profile-cleaner/)

### Gentoo Users
Profile cleaner is in the main repo:

    emerge -av app-admin/profile-cleaner

### Other Distros
If you are interested in packaging profile-cleaner for your favorite distro, please contact me.

## Dependencies
* bash
* bc
* findutils
* sqlite
* parallel

## Sample Output and Results Using an "Uncleaned" Firefox Profile:

	Size before: 154.17 Mbytes
	Size after:  99.11 Mbytes

	$ profile-cleaner f

	/home/facade/.mozilla/firefox/zee52krz.default/cookies.sqlite reduced by .87 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/signons.sqlite reduced by .41 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/downloads.sqlite reduced by .56 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/chromeappsstore.sqlite reduced by .03 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/urlclassifier3.sqlite reduced by 43.10 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/formhistory.sqlite reduced by .11 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/addons.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/permissions.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/content-prefs.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/extensions.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/webappsstore.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/OfflineCache/index.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/search.sqlite reduced by 0 Mbytes
	/home/facade/.mozilla/firefox/zee52krz.default/places.sqlite reduced by 9.98 Mbytes

	firefox profile reduced by 55.06 Mbytes

## Speed Comparison
28-Nov-2012		Profile-cleaner has been rewritten to use GNU Parallel for processing.

Here is a contrived example cleaning 6 firefox profiles comparing the classical version of Profile-cleaner which used for loops to process profiles to the new version:

* There are a total of 312 M in the 6 "dirty" profiles.
* Test machine is an Intel 3770K @ 4.50 GHz.
* After cleaning there is a total of 213 M in the 6 "clean" profiles.

new 4.07s user 0.42s system 318% cpu 1.411 total

old 8.15s user 0.43s system 95% cpu 8.942 total

Your mileage may vary.

gha_language: Shell


text:
# Few-Shot Text Classification

This repo accompanies the Cloudera Fast Forward report [Few-Shot Text Classification](https://few-shot-text-classification.fastforwardlabs.com/). It provides small library to fascilitate text classification using latent text embeddings with [Sentence-BERT](https://www.sbert.net/) as well as a simple application to explore text classification in several limited-labeled-data regimes.

The primary output of this repository is the Few-Shot Text Classification application, a prototype user interface for the latent text embedding classification method. It includes the ability to apply various models for both on-the-fly and few-shot classification on the AG News dataset.

![Few-Shot Text Classification app interface](images/fewshot_app.png)

Instructions are given both for general use (on a laptop, say), and for Cloudera CML and CDSW. We'll first describe what's here, then go through how to run everything.

## Structure

```
.
├── apps        # Small Streamlit application.
├── cml         # This folder contains scripts that facilitate the project launch on CML.
├── data        # This folder contains starter data, and is where text embeddings will live.
├── scripts     # This is where all the code that does something lives.
├── notebooks   # This contains several Jupyter/Colab notebooks that accompany the report and demonstrate basic usage.
└── fewshot     # A small library of useful functions.
```

There are also `images` and  `tests` directories that are unimportant and can be ignored. Let's examine each of the important folders in turn.

### `apps`
The application accompanying this project comes with a launcher script to assist launching an [Application](https://docs.cloudera.com/machine-learning/cloud/applications/topics/ml-applications.html) with CDSW/CML.
To launch the applications in another environment, run the code inside the launcher file, with the prefixed `!` removed.
You may need to specify different ports.

### `cml`
This script facilitates the automated project setup on CML and is triggered by the declarative pipeline as defined in the `.project-metadata.yaml` file found in the project's root directory.

### `fewshot`
```
fewshot
├── data
│   ├── loaders.py
│   └── utils.py
├── embeddings
│    ├── sentence_embeddings.py
│    └── word_embeddings.py
├── models
│    ├── few_shot.py
│    └── on_the_fly.py
├── eval.py
└── utils.py
```
The `data/loaders.py` is used in all scripts and notebooks, containing code that returns a specialized `Dataset` object that makes it easier to handle the original text, embeddings, and labels simultaneously.

The latent text embedding method relies on first embedding text with Sentence-BERT before performing any other steps. This code is found under `embeddings/sentence_embeddings.py`. More sophisticated methods incorporate word embeddings to augment the Sentence-BERT embeddings, and this code is under `embeddings/word_embeddings.py`.

There are two regimes in which we perform text classification and we include a model for each. `models/few_shot.py` contains code to train a model that incorporates _some_ labeled data, while `models/on_the_fly.py` computes a model that performs classification with _no labeled data at all_.

We also provide helper functions for generating predictions and computing metrics such as basic accuracy in `eval.py`. `utils.py` contains additional helper functions for I/O and serializing data.

### `scripts`

```
scripts
├── few-shot_text_classification.py
└── on-the-fly_text_classification.py
```
These scripts perform basic text classification for the various classification regimes and generate outputs that are used in the app. These outputs have been pre-trained and included with this repo.

### `notebooks`
```
notebooks
├── CreatingRedditDataset.ipynb
├── Wmap_Experiments.ipynb
└── Zmap_Experiments.ipynb
```
The `Wmap_Experiments` and `Zmap_Experiments` notebooks walk through analysis discussed in the accompanying [Few-Shot Text Classification report](https://few-shot-text-classification.fastforwardlabs.com/) and are intended for interactive learning purposes. These work best when run as a [Colab](https://colab.research.google.com/notebooks/intro.ipynb) (rather than as a Jupyter Notebook) to take advantage of free GPUs.

The `CreatingRedditDataset` notebook is included purely to document the steps taken when creating the **Reddit** dataset that we include with this module (located in the `data/reddit` directory.) It is not intended to be run directly.

## Performing text classification in limited-labeled-data regimes
To go from a fresh clone of the repo to the final state, follow these instructions in order.

### Installation
The code and applications within were developed against Python 3.6.9, and are likely also to function with more recent versions of Python. We relied on GPUs for much of the analysis and use a version of PyTorch optimized for CUDA 10.2.

To install dependencies, first create and activate a new virtual environment through your preferred means, then pip install from the requirements file. I recommend:

```python
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

In CML or CDSW, no virtual env is necessary. Instead, inside a Python 3 session (with at least 2 vCPU / 4 GiB Memory), simply run

```python
!pip3 install -r requirements.txt     # notice `pip3`, not `pip`
```
---
#### Note on GPUs
The `requirements.txt` file installs a GPU-compatible version of PyTorch. If GPUs are not in your future, it might be prudent to instead install a CPU-only version of torch (which is more lightweight) with the following command:

```python
!pip3 install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
```
(Remove the `!` for non-CML/CDSW environments.)

---

### Data
We use two datasets in several of the scripts and notebooks, although the few-shot application currently only allows interaction with the AG News dataset.

**AG News**

This is a collection of 127,600 news articles in four categories. The dataset is pulled from the open-source [Datasets repository](https://huggingface.co/datasets) maintained by HuggingFace. The `load_or_cache_data` function in the `data/loaders.py` file manages downloading this dataset on its first call, after which the dataset is cached and does not need to be downloaded again.

**Reddit**

This [dataset](https://www.aclweb.org/anthology/W17-4508/) contains nearly four million preprocessed submissions and comments from Reddit, collected between 2006 and 2016. Like AG News, it is also available on the HuggingFace Datasets repository, but it is **extremely large** and we do not recommend that you download it yourself. Instead, we provide curated subsamples of this dataset in the `data/reddit` directory, as well as a notebook (`CreatingRedditDataset.ipynb`) detailing how we performed the sampling.

### Scripts / Notebooks
To fit models and perform text classification experiments, one can either call the scripts in the `scripts` directory, or walk through a more detailed process in the notebooks. Their functionality is as follows:

* `scripts/on-the-fly_text_classification.py` performs **on-the-fly (zero-shot) text classification**; that is, text classification with *no labeled training examples*. This script generates a simple model (called a `Zmap`, stored in the `data/maps` directory) that is used in the next script as well as in the app. `Zmaps` are data-agnostic because they do not rely specifically on training data. We have already performed this analysis and include this output for you. Similar procedures are echoed in the `notebooks/Zmap_Experiments` notebook.

* `scripts/few-shot_text_classification.py` performs **few-shot text classification**; that is, text classification with *only a few labeled training examples.* This script generates a model known as a `Wmap`. `Wmaps` rely on training data and are thus specific to a given dataset. In the `data/maps` directory we include a `Wmap` trained on the AG News dataset, which is also used in the app. Similar procedures are echoed in the `notebooks/Wmap_Experiments` notebook.

To run scripts, follow this procedure in the terminal or a Session:

```python
!python3 scripts/on-the-fly_text_classification.py
!python3 scripts/few-shot_text_classification.py
```
(Remove the `!` for non-CML/CDSW environments.)

---
NOTE: The scripts are intended to be run on a GPU-enabled machine in order to complete in a timely fashion. On a CPU-only machine, these scripts can take from 3-6 hours to complete. With GPUs enabled, this reduces to a few minutes.

---

## Deploying on CML
There are three ways to launch this project on CML:

1. **From Prototype Catalog** - Navigate to the Prototype Catalog on a CML workspace, select the "Deep Learning for Question Answering" tile, click "Launch as Project", click "Configure Project"
2. **As ML Prototype** - In a CML workspace, click "New Project", add a Project Name, select "ML Prototype" as the Initial Setup option, copy in the repo URL, click "Create Project", click "Configure Project"
3. **Manual Setup** - In a CML workspace, click "New Project", add a Project Name, select "Git" as the Initial Setup option, copy in the repo URL, click "Create Project". Then, follow the installation instructions above.

## Additional information
### Tests

The `fewshot` module logic is partly covered by unittests.  To run all tests, use:

```bash
python -m unittest discover
```

We recommend running tests before committing any major changes.

The end-to-end test (`test_e2e.py`) will not work if files generated by `on-the-fly_text_classification.py` have not been generated.  (These files are checked in.)

### Formatting

We use black library to format code.  It is required that new changes to the library conform to this style.  To auto-format code, you can call:

```bash
./format_code.sh
```