text:
---
title:  "What are Archetypes?"
permalink: docs/what_are_archetypes.html
toplevel: "Getting Started: Quick Guide"
---

> Archetypes are encapsulated boilerplates for centralizing your project configurations, workflows and dependencies.

An archetype is an npm module template, which is a “superclass” of a module, think inheritance for npm modules but not one that is used to generate code files and then discarded. Archetypes provide something that is quite far off from what frameworks and app generators provide.

Let’s say that you are developing many different modules and all those modules require you to have very similar configs, scripts and dependencies, you can use the same archetype for all of them. With a generator like Yeoman, you would run the generator once, and it would output your module’s boilerplate and all is fine. However, if you ever wanted to change dependencies, and have that change propagate through to each and every module, you would have to manually go through each module and update those dependencies by hand.

> Archetypes help to solve the problem of updating module boilerplates, by providing a single source of truth, that all of your modules inherit from.

If after developing hundreds of modules ( say, React components ), you decide to change your build script, and that build script requires a new dependency, you can change it once in the archetype. Once all of the other modules run `npm install`, they will get the updated build scripts as well as the updated dependencies.

Electrode archetypes are a way for every electrode module to share common scripts, workflows, dependencies and best practices that can be updated in real time. That update will then propagate to every module that inherits from that specific archetype.

> Electrode archetypes:

> * [electrode-archetype-react-app] : Electrode Archetype for Apps.

> * [electrode-archetype-react-component] : Electrode archetype for react components.

> * [electrode-archetype-njs-module-dev] : Electrode archetype for server components.

#### Features

Archetypes come loaded with the latest technologies for developing, bundling, testing and deployment workflows.

* Generate/Build production bundles and assets.
* Transpile your JavaScript using Babel.
* Running eslint rules for client and server JavaScript and their tests.
* Running unit tests for client and server (Karma/Mocha).
* Running functional tests.
* Generate code coverage results for client and server (Istanbul).
* Run a webpack-dev-server for hot reloading of jsx changes.
* Single source of truth for all dependencies.
* Single source for all development, bundling, testing and deployment related workflows.

#### What configs exist inside archetypes?

* Webpack
* Babel
* Eslint
* Karma/Mocha/Istanbul

#### Webpack config setup
Let's peek under the hood into the webpack setup workflow.

*webpack.config.js* Webpack configuration entrypoint

```js

var _ = require("lodash");
var mergeWebpackConfig = require("webpack-partial").default;

var baseConfig = require("./base.js");
var defineConfig = require("./partial/define.js");
var optimizeConfig = require("./partial/optimize");
var localesConfig = require("./partial/locales");
var productionSourcemapsConfig = require("./partial/sourcemaps-remote");

module.exports = _.flow(
  mergeWebpackConfig.bind(null, {}, baseConfig),
  optimizeConfig(),
  localesConfig(),
  defineConfig(),
  productionSourcemapsConfig()
)();
```

*Webpack Base Config*

```js
var baseConfig = {
  cache: true,
  context: context,
  plugins: [
    new webpack.LoaderOptionsPlugin({
      options: {
        debug: false
      }
    })
  ],
  entry: appEntry(),
  output: {
    path: Path.resolve("dist", "js"),
    pathinfo: inspectpack, // Enable path information for inspectpack
    publicPath: "/js/",
    chunkFilename: "[hash].[name].js",
    filename: "[name].bundle.[hash].js"
  },
  resolve: {
    modules: [
      archetypeNodeModules,
      archetypeDevNodeModules,
      AppMode.isSrc && Path.resolve(AppMode.src.dir) || null
    ]
      .concat(archetype.webpack.modulesDirectories)
      .concat([process.cwd(), "node_modules"])
      .filter(_.identity),
    extensions: [".js", ".jsx", ".json"]
  },
  resolveLoader: {
    modules: [
      archetypeNodeModules,
      archetypeDevNodeModules,
      Path.resolve("lib"),
      process.cwd(),
      "node_modules"
    ].filter(_.identity)
  }
};
```

#### What workflows exist inside archetypes?
Workflows are tasks exposed via npm scripts & gulp as part of the [electrode-archetype-react-app], type `gulp` on your command prompt to see the full list of workflow tasks.

```
$ gulp
[16:56:36] Using gulpfile ~/walmart-oss/test-generator-electrode-app/test-app/gulpfile.js
[16:56:36] Starting 'help'...

Usage
  gulp [TASK] [OPTIONS...]

Available tasks
  build ............................ Build your app's client bundle for production
                                       tasks: ["build-dist"]
  build-analyze -------------------- Build your app's client bundle for production and run bundle analyzer
                                       tasks: ["build-dist","optimize-stats"]
                                       deps: [".optimize-stats"]
  build-browser-coverage ........... Build browser coverage
                                       tasks: ["clean-dist",".build-browser-coverage-1","build-dist:flatten-l10n","build-dist:clean-tmp"]
  build-dev-static ----------------- Build static copy of your app's client bundle for development
                                       tasks: ["clean-dist","build-dist-dev-static"]
  build-dist .......................   tasks: ["clean-dist","build-dist-min","build-dist:flatten-l10n","generate-service-worker","build-dist:clean-tmp"]
  build-webpack-stats-with-fullpath  Build static bundle with stats.json containing fullPaths to inspect the bundle on electrode-electrify

  etc ...
```

Let's dive a little deeper into an example workflow `gulp build` task that is provided by the archetype for a universal React application.

#### gulp build
This task produces a production ready js, css bundle. Running `gulp build` will run through a few other gulp tasks to help you build your application for a production environment generating a minified bundle ready to use.

The following tasks will be run in order:

* `clean-dist`:
    Delete the old `dist/` folder

* `build-dist-min`:
    Run webpack.config.js This particular production webpack config will be merged with the base config that is located [here.](https://github.com/electrode-io/electrode-archetype-react-app/blob/master/config/webpack/base.js)
    Each of the archetypes webpack configs use lodash's [flow](https://lodash.com/docs/4.16.4#flow) and [webpack-partial](https://github.com/webpack-config/webpack-partial) to compose together functions that merge partial webpack configs
    together with the base config allowing for a modular configuration file.

```javascript
// the production webpack config
// _.flow is used to compose functions
module.exports = _.flow(
  // use webpack-partial binding it to a the baseConfig
  mergeWebpackConfig.bind(null, {}, baseConfig),
  // optimize.js
  optimizeConfig(),
  // locals.js
  localesConfig(),
  // define.js
  defineConfig(),
  // sourcemaps-remote.js
  productionSourcemapsConfig()
)();
```

The production webpack config is merged together with the base config and the following partials:

   - [optimize.js](https://github.com/electrode-io/electrode-archetype-react-app/blob/master/config/webpack/partial/optimize.js): Apply webpack's dedupe plugin to deduplicate similar or equal files from the output bundle in addition to Uglifying the output using the UglifyJS plugin

   - [locales.js](https://github.com/electrode-io/electrode-archetype-react-app/blob/master/config/webpack/partial/locales.js): Handle possible dynamic requires within moment.js, so webpack does not include all of the language files

   - [define.js](https://github.com/electrode-io/electrode-archetype-react-app/blob/master/config/webpack/partial/define.js): Define a production environment for webpack so it can remove non production code.

   - [sourcemaps-remote.js](https://github.com/electrode-io/electrode-archetype-react-app/blob/master/config/webpack/partial/sourcemaps-remote.js): Create sourcemaps.

* `build-dist:flatten-l10n`:
    Run a script to extract localized messages from a file located in a temporary folder generated by the build and write them to a json file in the dist folder. This script is located [here.](https://github.com/electrode-io/electrode-archetype-react-app/blob/master/scripts/l10n/flatten-messages.js)

* `generate-service-worker`:
    Electrode's React application archetype has built in support for [service workers](https://developers.google.com/web/fundamentals/getting-started/primers/service-workers). This gulp task will take a service worker precache configuration file (if it exists, electrode does not provide it by default) and generate a new service worker file for caching of static assets with service workers.

* `build-dist:clean-tmp`:
    Delete the .tmp folder generated by the build.

The output of `gulp build` will look something like this:

```sh

Hash: 095d27720d0a224d3f5a
Version: webpack 1.13.2
Time: 6607ms
                                    Asset       Size  Chunks             Chunk Names
           bundle.095d27720d0a224d3f5a.js     246 kB       0  [emitted]  main
           style.095d27720d0a224d3f5a.css  182 bytes       0  [emitted]  main
../map/bundle.095d27720d0a224d3f5a.js.map    2.42 MB       0  [emitted]  main
../map/style.095d27720d0a224d3f5a.css.map  107 bytes       0  [emitted]  main
                     ../server/stats.json  233 bytes          [emitted]
                ../isomorphic-assets.json  239 bytes          [emitted]
```

[electrode-archetype-react-app]: https://github.com/electrode-io/electrode#app-archetype
[electrode-archetype-react-component]: https://github.com/electrode-io/electrode-archetype-react-component
[electrode-archetype-njs-module-dev]: https://github.com/electrode-io/electrode-archetype-njs-module-dev

extension: markdown

gha_license_id: NOASSERTION


text:
# gcpe-news-api-client
C# Client for BC Gov News API.
======================

Introduction
----------------
The code of this client is generated by autorest and allows a C# MVC web application to easily
consume the BC Government News API.  The build script pushes a nuget package, and does not produce any container images.

Repository Map
--------------
- .s2i : Source to image bash assemble script
- **Gov.News.Api.Client**: C# Dotnet Core class Library

Installation
------------
In an OpenShift "Tools" or build project, add the objects contained in the OpenShift\Templates folder.

`oc process -f build-template.json | oc create -f -`

- Once the build template is processed and imported, the following will need to be done:

1.  Set the NUGET_KEY environment variable with your NUGET Key. (if you do not do this, the build will not push the Nuget package to nuget.org)
2.

Developer Prerequisites
-----------------------

**Gov.News.Api.Client**
- Node.js
- Visual Studio 2017
- Dotnet Core 2.2

**DevOps**
- OpenShift Container Platform 3.6 or newer (Origin is supported)
- RedHat OpenShift tools
- Docker
- Centos Linux
- A familiarity with Jenkins

Development
-----------
A Visual Studio 2017 solution has been provided, however Visual Studio Pro is not required to run the build script.

*Important*

You will need to install Autorest prior to building the source.
`npm install -g autorest`

You may have to run the build more than once initially if using Visual Studio.

Removing a package
------------------
Note that nuget.org maps delete to unlist.

If you wish to remove a package from nuget.org or another nuget repository, issue the following command:

`nuget delete Gov.News.Api.Client <version-to-remove> -Source nuget.org -ApiKey <key>`

(key is your Nuget.org key)

Contribution
------------

Please report any [issues](https://github.com/bcgov/gcpe-news-api-client/issues).

[Pull requests](https://github.com/bcgov/gcpe-news-api-client/pulls) are always welcome.

If you would like to contribute, please see our [contributing](CONTRIBUTING.md) guidelines.

Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.

License
-------

    Copyright 2017 Province of British Columbia

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.

Maintenance
-----------

This repository is maintained by [GCPE](http://www.gov.bc.ca/).
Click [here](https://github.com/orgs/bcgov/teams/gcpe/repositories) for a complete list of our repositories on GitHub.

repository: WadeBarnes/gcpe-news-api-client

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: WadeBarnes/gcpe-news-api-client


text:
# Mongo wrapper

This oPack provides a wrapper, using [Mongo's Java Driver](https://mvnrepository.com/artifact/org.mongodb/mongo-java-driver), to access Mongo from OpenAF.

## Installing

To install:

````
opack install Mongo
````

## Using it

Check out: https://docs.openaf.io/docs/guides/medium/accessing-mongodb.html

repository: OpenAF/openaf-opacks

gha_language: JavaScript

repo_name: OpenAF/openaf-opacks


text:
Developing a Docker Container
=============================

To run a collaborator's tool (e.g. variant caller) on a compute cluster, there needs to be a "Dockerfile" which describes all dependencies and environmental setup required to run the tool.  This includes any and all software packages, symlinks, and environmental variables that are needed by the tool itself or needed by its dependencies.

Dockerfiles use a 'fork and commit' strategy similar to github. You can start from a predefined system, and make small alterations on top of it. Available images can be found at https://registry.hub.docker.com/.

The Mutect Build starts with the 'java' image, which contains java pre-installed.

```
FROM java

RUN apt-get update
RUN apt-get install -y zip wget
#
# Install samtools and the python vcf libraries for wrapper code
#
RUN apt-get install -y samtools python-pip
RUN pip install PyVCF

# We'll be working in /opt from now on
WORKDIR /opt

#
# Download and unpack Mutect
#
RUN wget https://github.com/broadinstitute/mutect/releases/download/1.1.5/muTect-1.1.5-bin.zip
RUN wget https://github.com/broadinstitute/picard/releases/download/1.122/picard-tools-1.122.zip

RUN unzip muTect-1.1.5-bin.zip
RUN unzip picard-tools-1.122.zip

# Link the picard tools to /opt/picard
RUN ln -s picard-tools-1.122 picard
```
From there, the system issues 'apt-get' commands (the Debian package management system) to install zip, wget, samtools and pip. Pip is then used to install PyVCF. Zip files for MuTect and Picard are downloaded and unzipped (in /opt, which was defined as the 'WORKDIR').

Once a docker file is built it should be tested on the collaborator's local host system using a pre-installed Docker service before submission to the target host system.  The following links should help in understanding how to install/configure the docker service, create a docker container, and run it on the docker instance.

Docker's own basic interactive tutorial:
https://www.docker.com/tryit/

A written step-by-step guide to creating docker files and running them is here:
https://www.digitalocean.com/community/tutorials/docker-explained-using-dockerfiles-to-automate-building-of-images

The Dockerfile reference manual
https://docs.docker.com/reference/builder/

A simple example of docker files produced for UCSC projects is the Broad MuTect variant caller tool:
https://github.com/ucscCancer/pcawg_tools/blob/master/tools/mutect/Dockerfile

Using Planemo to Develop Docker Containers
==========================================

${PLANEMO_DOCKER}

repository: kellrott/galaxy-tooldev-docs

gha_language: Python

repo_name: kellrott/galaxy-tooldev-docs


text:
# Comparator

Easily match on the ordering of two `Comparable` values.
```swift
switch Ordering(a, b) {
  case .ascending:  print("It's going up!")
  case .same:       print("They're no different.")
  case .descending: print("Down they go!")
}
```
You can also use the convenient operator `a <=> b` to create an `Ordering` value.

license: MIT

repository: JadenGeller/Comparator

repo_name: JadenGeller/Comparator


text:
---
layout: post
title: "Identity Crisis"
excerpt: "National"
categories: national
comments: false
share: true
---

![](http://netstorage.discovery.com/feeds/brightcove/asset-stills/nws/127749088467412934814901199_UncleSam5.jpg)

What are you? Are you white? A woman? A Muslim? An American? A Democrat?

Yes, I know you can be all of those things. But when push comes to shove, which one do YOU identify with the most?

This is a big topic that will be re-visited, but for the sake of this maiden post, I want to focus this on politics.

The Republican party, over the last year, has been scorching Donald Trump with statements that he is not fit, not trustworthy, has small hands, and, the real burn, not a true conservative.

 It’s the wrong message to send, and it’s not going to be effective. Always beware of the candidate for public office who has the quick and easy answer to a complicated problem.'

Rick Perry (Gov. Texas): 'Donald Trump is the modern-day incarnation of the 'Know Nothing' movement.'

Marco Rubio (Sen. Florida): I believe Donald Trump as our nominee is going to shatter and fracture the Republican Party and the conservative movement'

Note: I am no political wonk, but even I have heard of each of these guys before all of this stuff. In other words, they are important people in the Republican party.

So what do each one of these guys have in common? They all support Donald Trump as of right now.

Granted, all of these guys were running against him in the primaries at one time, but those statements are deep and highly abrasive.

My ultimate question is this: **what are you first, a Republican or an American?**

Even if you like Donald Trump, you like the bravado and think America needs a change of leadership, there is no question he is unqualified for the job. He doesn't know policy, how the government works, and the true responsibilities of the president. Even more importantly, he doesn't care to know. How do I know this? He has never held office in his life!! There is no possible way he knows any of this stuff.

What other job in the world allows someone to receive the highest position without any experience in the field? I've never been a doctor, can I be the head surgeon at Johns Hopkins? I've never taken an architecture class, can I build the WWII monument in DC? Hell, I wouldn't let someone pick the weeds in my yard unless they had a hoe. Donald Trump doesn't even have the proper hoe for this job.

So what are you: a Republican or an American?

Two months ago only 1% of active GOP politicians overtly supported Trump. Now, only 5 have openly not endorsed him. Make no mistake about it, Trump will get the support of McCain, Ryan, and others not because they like him, or feel he is a fit candidate, it's because they want to win.

So what are you: a Republican or an American?

I think we know the answer to that.

I like to think if the situation was reversed, if there was a 'D' by Trump's name, the Democrats would respond in a more appropriate way. But I am an idealist and have no proof of this. For the sake of sanity, I'll keep believing this because...well...LOOK AT THAT BIRD OVER THERE!!!!

gha_license_id: MIT


text:
## Learn How to Program Makeover
---

#### Nelia Terrazas and Dino Nguyen
#### Date of Completion

### Description

Redesign learnhowtoprogram.com in Sketch to be more functional and appealing

### Images

Mobile Original Sketches:
![](pics/rough-sketch1.jpg)

![](pics/rough-sketch2.jpg)

Desktop Original Sketches:
![](pics/rough-sketch3.jpg)

#### Mobile Version

![mobile user interface](pics/mobile-homepage.png)

![mobile user interface](pics/mobile-content.png)

#### Desktop Version

![](pics/desktop-homepage.png)

### User Stories

* As an Epicodus student, I want an easily navigable site so that I can quickly search/find the information that I need.
* As a non-Epicodus student/developer, I want easy access to relevant notes and lessons in each track in the form of a navigation bar/menu so that I can switch between references quickly.
* Other target audiences: parents, Epicodus teachers/admin, other coding bootcamp students/admins

## Usability Research and Feedback

* Need home icon to go back to homepage
* Make hit boxes bigger
* Add back button links to make wireframe cleaner

### Technologies and Resources

* Sketch

### Setup Instructions

* Download/Clone this repository: https://github.com/neliaterrazas/lhtp-makeover.git.
* Open up in Sketch.

### License

MIT License

Copyright (c) 2018 **_{ Dino Nguyen and Nelia Terrazas }_**

repository: neliaterrazas/lhtp-makeover

repo_name: neliaterrazas/lhtp-makeover


text:
# [Sum of odd numbers](https://www.codewars.com/kata/sum-of-odd-numbers "https://www.codewars.com/kata/55fd2d567d94ac3bc9000064")

Given the triangle of consecutive odd numbers:

```
             1
          3     5
       7     9    11
   13    15    17    19
21    23    25    27    29
...
```

Calculate the row sums of this triangle from the row index (starting at index 1) e.g.:
```
rowSumOddNumbers(1); // 1
rowSumOddNumbers(2); // 3 + 5 = 8
```

repository: ParanoidUser/codewars-handbook

branch_name: refs/heads/main

gha_language: Java

repo_name: ParanoidUser/codewars-handbook


text:
# data_Structures_Algorithms_Java
Learning log for data structure and algorithms using java

### What is a dataStructure?
- Organizes and stores data.
- Many data structures are available as there is no single data structure which can perform all tasks efficiently.

### What is an algorithm?
- An algorithm is something, which specifies the steps needed to perform a task.
- There can be many algorithms for accomplishing a task.
- An algorithm is not an implementation. There can be many implementations of the same algorithm.

## Arrays
- Used to store similar kind of data.
- Efficient while searching data if you know the index.

> Algorithm performance:
> Generally to check an algorithm performance, we will add code to check the start time and end time of a task.
> then specify the time taken to run the task, but this is not an effective way of measuring an algorithms performance.
> Older machines can do it very slow, newer machines or super computers can do it very fast even-though the algorithm is not great.

### How is algorithm performance measured?
- By the number of steps it takes to execute a task, this is called ** Time Complexity ** .
- Amount of memory consumed to run an algorithm which is called ** Memory Complexity ** .

> As memory is cheaper now a days, concern is more towards time complexity.

### Time Complexity
- Time complexity is considered for the worst case scenario.
- Performance of algorithms are calculated by comparing the worst case time complexity of those two.

### Big-O notation
- Expressing the way of complexity, related to the number of items, that the algorithm deals with.
- Written as capital alphabet 'O' followed by parentheses.
- Provides a way of comparing different algorithms with time complexity, in an objective manner, independent of hardware.

#### Example
### Steps to add sugar in tea:
1. Fetch the bowl with sugar.
2. Get a spoon.
3. Scoop out sugar with spoon.
4. Pour sugar from spoon into tea.
5. if you need more sugar repeat the steps 3 and 4.

### Calculating time complexity for above algorithm
Spoons of sugars required | Steps required (Time Complexity)
------------------------- | --------------------------------
1 | 4
2 | 6
3 | 8
4 | 10

### Converting to notation
- Number of desired sugars = n
- Total number of steps = 2n + 2
- As n grows the number of steps grows.
- The '2' in '2n' and '+2' remains constant, so they do not factor into the time complexity.
- The value of 'n' determines the growth rate.
- Hence the time complexity is ** 'O(n)' **
- This is called as linear complexity

### General Big-O values
Big-O  | Description | Notes
------ | ----------- | ------------------------------------------------------
O(n) | constant | Increase in number of steps is constant
O(logn) | logarithmic | logarithmic considered is base 2
O(n) | linear | Increase in number of steps will be linear, difference between any two in the sequence is same
O(nlogn)| n log-star n | <del> need to be updated </del>
O(n2) | quadratic | n to the power 2, or n squared


text:
---
title: Drill Down
permalink: /wizz-vis_latest_drill_down.html
version: latest
---

{% include image.html file="wizz-vis/drill_down.png" alt="Drill Down Feature" max-width=350 %}

We can create drill downs at each widget. The drill down has to be set in the `options` attribute. The drill down can contains more than one link, external resources and dashboard links.

```json
{
  "links": [
    {
      "url": "https://wizzie.io",
      "name": "Wizzie Web",
      "type": "absolute"
    },
    {
      "url": "/dashboards/1",
      "name": "Clients detail",
      "type": "dashboard"
    }
  ]
}
```

gha_language: HTML


text:
---
title: Cloud storages
type: guide
order: 103
---

You can integrate the popular cloud storage with Label Studio, collect new tasks uploaded to your buckets, and sync back annotation results to use them in your machine learning pipelines.

You can configure storage type, bucket and prefixes during the start of the server or during the runtime via UI on **Tasks** page.

You can configure one or both:

- _source storage_ (where tasks are stored)
- _target storage_ (where completions are stored)

The connection to both storages is synced, so you can see new tasks after uploading them to the bucket without restarting Label Studio.

The parameters like prefix or matching filename regex could be changed any time from the webapp interface.

> Note: Choose target storage carefully: be sure it's empty when you just start labeling project, or it contains completions that match previously created/import tasks from source storage. Tasks are synced with completions based on internal ids (keys in `source.json`/`target.json` files in your project directory), so if you accidentally connect to the target storage with existed completions with the same ids, you may fail with undefined behavior.

## Amazon S3

To connect your [S3](https://aws.amazon.com/s3) bucket with Label Studio, be sure you have programmatic access enabled. [Check this link](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#configuration) to learn more how to set up access to your S3 bucket.

### Create connection on startup

The following commands launch Label Studio, configure the connection to your S3 bucket, scan for existing tasks, and load them into the labeling app.

#### Read bucket with JSON-formatted tasks

```bash
label-studio start my_project --init --source s3 --source-path my-s3-bucket
```

#### Write completions to bucket

```bash
label-studio start my_project --init --target s3-completions --target-path my-s3-bucket
```

### Working with Binary Large OBjects (BLOBs)

When you are storing BLOBs in your S3 bucket (like images or audio files), you might want to use then as is, by generating URLs pointing to those objects (e.g. `gs://my-s3-bucket/image.jpg`)
Label Studio allows you to generate input tasks with corresponding URLs automatically on-the-fly. You can to this either specifying `--source-params` when launching app:

```bash
label-studio start my_project --init --source s3 --source-path my-s3-bucket --source-params "{\"data_key\": \"my-object-tag-$value\", \"use_blob_urls\": true, \"regex\": ".*"}"
```

You can leave `"data_key"` empty (or skip it at all) then LS generates it automatically with the first task key from label config (it's useful when you have only one object tag exposed).

### Optional parameters

You can specify additional parameters with the command line escaped JSON string via `--source-params` / `--target-params` or from UI.

#### prefix

Bucket prefix (typically used to specify internal folder/container)

#### regex

A regular expression for filtering bucket objects. Default is skipping all bucket objects (Use ".*" explicitly to collect all objects)

#### create_local_copy

If set true, the local copy of the remote storage will be created.

#### use_blob_urls

Generate task data with URLs pointed to your bucket objects(for resources like jpg, mp3, etc). If not selected, bucket objects will be interpreted as tasks in Label Studio JSON format, one object per task.

## Google Cloud Storage

To connect your [GCS](https://cloud.google.com/storage) bucket with Label Studio, be sure you have enabled programmatic access. [Check this link](https://cloud.google.com/storage/docs/reference/libraries) to learn more about how to set up access to your GCS bucket.

### Create connection on startup

The following commands launch Label Studio, configure the connection to your GCS bucket, scan for existing tasks, and load them into the app for the labeling.

#### Read bucket with JSON-formatted tasks

```bash
label-studio start my_project --init --source gcs --source-path my-gcs-bucket
```

#### Write completions to bucket

```bash
label-studio start my_project --init --target gcs-completions --source-path my-gcs-bucket
```

### Working with Binary Large OBjects (BLOBs)

When you are storing BLOBs in your GCS bucket (like images or audio files), you might want to use then as is, by generating URLs pointing to those objects (e.g. `gs://my-gcs-bucket/image.jpg`)
Label Studio allows you to generate input tasks with corresponding URLs automatically on-the-fly. You can to this either specifying `--source-params` when launching app:

```bash
label-studio start my_project --init --source gcs --source-path my-gcs-bucket --source-params "{\"data_key\": \"my-object-tag-$value\", \"use_blob_urls\": true, \"regex\": ".*"}"
```

You can leave `"data_key"` empty (or skip it at all) then LS generates it automatically with the first task key from label config (it's useful when you have only one object tag exposed).

### Optional parameters

You can specify additional parameters with the command line escaped JSON string via `--source-params` / `--target-params` or from UI.

#### prefix

Bucket prefix (typically used to specify internal folder/container)

#### regex

A regular expression for filtering bucket objects. Default is skipping all bucket objects (Use ".*" explicitly to collect all objects)

#### create_local_copy

If set true, the local copy of the remote storage will be created.

#### use_blob_urls

Generate task data with URLs pointed to your bucket objects(for resources like jpg, mp3, etc). If not selected, bucket objects will be interpreted as tasks in Label Studio JSON format, one object per task.


text:
## Mocha [![CircleCI status of freerange/mocha](https://circleci.com/gh/freerange/mocha.svg?style=shield)](https://app.circleci.com/pipelines/github/freerange/mocha) [![Gem Version](https://badge.fury.io/rb/mocha.svg)](http://badge.fury.io/rb/mocha)

### Description

* A Ruby library for [mocking](http://xunitpatterns.com/Mock%20Object.html) and [stubbing](http://xunitpatterns.com/Test%20Stub.html) - but deliberately not (yet) [faking](http://xunitpatterns.com/Fake%20Object.html) or [spying](http://xunitpatterns.com/Test%20Spy.html).
* A unified, simple and readable syntax for both full & partial mocking.
* Built-in support for MiniTest and Test::Unit.
* Supported by many other test frameworks.

### Intended Usage
Mocha is intended to be used in unit tests for the [Mock Object](http://xunitpatterns.com/Mock%20Object.html) or [Test Stub](http://xunitpatterns.com/Test%20Stub.html) types of [Test Double](http://xunitpatterns.com/Test%20Double.html), not the [Fake Object](http://xunitpatterns.com/Fake%20Object.html) or [Test Spy](http://xunitpatterns.com/Test%20Spy.html) types. Although it would be possible to extend Mocha to allow the implementation of fakes and spies, we have chosen to keep it focused on mocks and stubs.

### Installation

#### Gem

Install the latest version of the gem with the following command...

    $ gem install mocha

Note: If you are intending to use Mocha with Test::Unit or MiniTest, you should only setup Mocha *after* loading the relevant test library...

##### Test::Unit

```ruby
require 'rubygems'
gem 'mocha'
require 'test/unit'
require 'mocha/test_unit'
```

##### MiniTest

```ruby
require 'rubygems'
gem 'mocha'
require 'minitest/unit'
require 'mocha/minitest'
```

#### Bundler

If you're using Bundler, include Mocha in the `Gemfile` and then setup Mocha later once you know the test library has been loaded...

##### Test::Unit

```ruby
# Gemfile
gem 'mocha'

# Elsewhere after Bundler has loaded gems e.g. after `require 'bundler/setup'`
require 'test/unit'
require 'mocha/test_unit'
```

##### MiniTest

```ruby
# Gemfile
gem 'mocha'

# Elsewhere after Bundler has loaded gems e.g. after `require 'bundler/setup'`
require 'minitest/unit'
require 'mocha/minitest'
```

##### RSpec

RSpec includes a mocha adapter. Just tell RSpec you want to mock with `:mocha`:

```ruby
# Gemfile in Rails app
gem 'mocha'

# Within `spec/spec_helper.rb`
RSpec.configure do |config|
  config.mock_with :mocha
end
```

Note: There is no need to use a require statement to setup Mocha; RSpec does this itself.

##### Cucumber

```ruby
# In e.g. features/support/mocha.rb
require 'mocha/api'

World(Mocha::API)

Around do |scenario, block|
  begin
    mocha_setup
    block.call
    mocha_verify
  ensure
    mocha_teardown
  end
end
```

#### Rails

If you're loading Mocha using Bundler within a Rails application, you should setup Mocha manually e.g. at the bottom of your `test_helper.rb`.

##### MiniTest

Note that since Rails v4 (at least), `ActiveSupport::TestCase` has inherited from `Minitest::Test` or its earlier equivalents. Thus unless you are *explicitly* using Test::Unit, you are likely to be using MiniTest.

```ruby
# Gemfile in Rails app
gem 'mocha'

# At bottom of test_helper.rb (or at least after `require 'rails/test_help'`)
require 'mocha/minitest'
```

##### Other Test Framework

Follow the instructions for the relevant test framework in the [Bundler](#bundler) section, but ensure that the relevant Mocha file (`mocha/minitest`, `mocha/test_unit`, or `mocha/api`) is required **after** the test framework has been loaded, e.g. at the bottom of `test_helper.rb` or `spec_helper.rb`, or at least after `rails/test_help` has been required.

#### Known Issues

* In Mocha v1.10.0 an undocumented feature of `API#mock`, `API#stub` & `API#stub_everything` was changed. Previously when these methods were passed a single symbol, they returned a mock object that responded to the method identified by the symbol. Now Passing a single symbol is equivalent to passing a single string, i.e. it now defines the 'name' of the mock object.
* In Mocha v1.2.0 there is a scenario where stubbing a class method originally defined in a module hangs the Ruby interpreter due to [a bug in Ruby v2.3.1](https://bugs.ruby-lang.org/issues/12832). See #272. This was fixed in Mocha v1.2.1.
* Since v1.1.0 Mocha has used prepended modules internally for stubbing methods. There is [an obscure Ruby bug](https://bugs.ruby-lang.org/issues/12876) in many (but not all) versions of Ruby between v2.0 & v2.3 which under certain circumstances may cause your Ruby interpreter to hang. See the Ruby bug report for more details. The bug has been fixed in Ruby v2.3.3 & v2.4.0.
* Stubbing an aliased class method, where the original method is defined in a module that's used to `extend` the class doesn't work in Ruby 1.8.x. See stub_method_defined_on_module_and_aliased_test.rb for an example of this behaviour.
* 0.13.x versions cause a harmless, but annoying, deprecation warning when used with Rails 3.2.0-3.2.12, 3.1.0-3.1.10 & 3.0.0-3.0.19.
* 0.11.x versions don't work with Rails 3.2.13 (`TypeError: superclass mismatch for class ExpectationError`). See #115.
* Versions 0.10.2, 0.10.3 & 0.11.0 of the Mocha gem were broken. Please do not use these versions.

### Usage

#### Quick Start

```ruby
require 'test/unit'
require 'mocha/test_unit'

class MiscExampleTest < Test::Unit::TestCase
  def test_mocking_a_class_method
    product = Product.new
    Product.expects(:find).with(1).returns(product)
    assert_equal product, Product.find(1)
  end

  def test_mocking_an_instance_method_on_a_real_object
    product = Product.new
    product.expects(:save).returns(true)
    assert product.save
  end

  def test_stubbing_instance_methods_on_real_objects
    prices = [stub(:pence => 1000), stub(:pence => 2000)]
    product = Product.new
    product.stubs(:prices).returns(prices)
    assert_equal [1000, 2000], product.prices.collect {|p| p.pence}
  end

  def test_stubbing_an_instance_method_on_all_instances_of_a_class
    Product.any_instance.stubs(:name).returns('stubbed_name')
    product = Product.new
    assert_equal 'stubbed_name', product.name
  end

  def test_traditional_mocking
    object = mock('object')
    object.expects(:expected_method).with(:p1, :p2).returns(:result)
    assert_equal :result, object.expected_method(:p1, :p2)
  end

  def test_shortcuts
    object = stub(:method1 => :result1, :method2 => :result2)
    assert_equal :result1, object.method1
    assert_equal :result2, object.method2
  end
end
```

#### Mock Objects

```ruby
class Enterprise
  def initialize(dilithium)
    @dilithium = dilithium
  end

  def go(warp_factor)
    warp_factor.times { @dilithium.nuke(:anti_matter) }
  end
end

require 'test/unit'
require 'mocha/test_unit'

class EnterpriseTest < Test::Unit::TestCase
  def test_should_boldly_go
    dilithium = mock()
    dilithium.expects(:nuke).with(:anti_matter).at_least_once  # auto-verified at end of test
    enterprise = Enterprise.new(dilithium)
    enterprise.go(2)
  end
end
```

#### Partial Mocking

```ruby
class Order
  attr_accessor :shipped_on

  def total_cost
    line_items.inject(0) { |total, line_item| total + line_item.price } + shipping_cost
  end

  def total_weight
    line_items.inject(0) { |total, line_item| total + line_item.weight }
  end

  def shipping_cost
    total_weight * 5 + 10
  end

  class << self
    def find_all
      # Database.connection.execute('select * from orders...
    end

    def number_shipped_since(date)
      find_all.select { |order| order.shipped_on > date }.length
    end

    def unshipped_value
      find_all.inject(0) { |total, order| order.shipped_on ? total : total + order.total_cost }
    end
  end
end

require 'test/unit'
require 'mocha/test_unit'

class OrderTest < Test::Unit::TestCase
  # illustrates stubbing instance method
  def test_should_calculate_shipping_cost_based_on_total_weight
    order = Order.new
    order.stubs(:total_weight).returns(10)
    assert_equal 60, order.shipping_cost
  end

  # illustrates stubbing class method
  def test_should_count_number_of_orders_shipped_after_specified_date
    now = Time.now; week_in_secs = 7 * 24 * 60 * 60
    order_1 = Order.new; order_1.shipped_on = now - 1 * week_in_secs
    order_2 = Order.new; order_2.shipped_on = now - 3 * week_in_secs
    Order.stubs(:find_all).returns([order_1, order_2])
    assert_equal 1, Order.number_shipped_since(now - 2 * week_in_secs)
  end

  # illustrates stubbing instance method for all instances of a class
  def test_should_calculate_value_of_unshipped_orders
    Order.stubs(:find_all).returns([Order.new, Order.new, Order.new])
    Order.any_instance.stubs(:shipped_on).returns(nil)
    Order.any_instance.stubs(:total_cost).returns(10)
    assert_equal 30, Order.unshipped_value
  end
end
```

### Thread safety

Mocha currently *does not* attempt to be thread-safe.

#### Can I test multi-threaded code with Mocha?

The short answer is no. In multi-threaded code Mocha exceptions may be raised in a thread other than the one which is running the test and thus a Mocha exception may not be correctly intercepted by Mocha exception handling code.

#### Can I run my tests across multiple threads?

Maybe, but probably not. Partial mocking changes the state of objects in the `ObjectSpace` which is shared across all threads in the Ruby process and this access to what is effectively global state is not synchronized. So, for example, if two tests are running concurrently and one uses `#any_instance` to modify a class, both tests will see those changes immediately.

### Expectation matching / invocation order

Stubs and expectations are basically the same thing. A stub is just an expectation of zero or more invocations. The `Expectation#stubs` method is syntactic sugar to make the intent of the test more explicit.

When a method is invoked on a mock object, the mock object searches through its expectations from newest to oldest to find one that matches the invocation. After the invocation, the matching expectation might stop matching further invocations.

See the [documentation](https://mocha.jamesmead.org/Mocha/Mock.html) for `Mocha::Mock` for further details.

### Configuration

If you want, Mocha can generate a warning or raise an exception when:

* stubbing a method unnecessarily
* stubbing method on a non-mock object
* stubbing a non-existent method
* stubbing a non-public method

See the [documentation](https://mocha.jamesmead.org/Mocha/Configuration.html) for `Mocha::Configuration` for further details.

##### MOCHA_OPTIONS
`MOCHA_OPTIONS` is an environment variable whose value can be set to a comma-separated list, so that we can specify multiple options e.g. `MOCHA_OPTIONS=debug,use_test_unit_gem`.
Only the following values are currently recognized and have an effect:
* `debug`: Enables a debug mode which will output backtraces for each deprecation warning. This is useful for finding where in the test suite the deprecated calls are.

### Semantic versioning

* Every effort is made to comply with [semantic versioning](https://semver.org/).
* However, this only applies to the behaviour documented in the public API.
* The documented public API does *not* include the content or format of messsages displayed to the user, e.g. assertion failure messages.

### Useful Links

* [Official Documentation](https://mocha.jamesmead.org)
* [Source Code](http://github.com/freerange/mocha)
* [Mailing List](http://groups.google.com/group/mocha-developer)
* [James Mead's Blog](http://jamesmead.org/blog/)
* [An Introduction To Mock Objects In Ruby](http://jamesmead.org/talks/2007-07-09-introduction-to-mock-objects-in-ruby-at-lrug/)
* [Mocks Aren't Stubs](http://martinfowler.com/articles/mocksArentStubs.html)
* [Growing Object-Oriented Software Guided By Tests](http://www.growing-object-oriented-software.com/)
* [Mock Roles Not Objects](http://www.jmock.org/oopsla2004.pdf)
* [jMock](http://www.jmock.org/)

### Contributors

See this [list of contributors](https://github.com/freerange/mocha/graphs/contributors).

### Releasing a new version

* Update the RELEASE.md file with a summary of changes
* Bump the version in `lib/mocha/version.rb`
* Commit & push to GitHub
* Check CircleCI build is passing - https://app.circleci.com/pipelines/github/freerange/mocha

* Generate documentation:

```bash
$ MOCHA_GENERATE_DOCS=true bundle install

$ MOCHA_GENERATE_DOCS=true rake generate_docs
```
* Commit documentation & push to GitHub
* Sign in to rubygems.org and find API key - https://rubygems.org/profile/edit

```bash
$ curl -u <email-address> -H 'OTP:<one-time-password>' https://rubygems.org/api/v1/api_key.yaml > ~/.gem/credentials; chmod 0600 ~/.gem/credentials
```

* Release gem to Rubygems:

```bash
$ rake release
[runs tests]
mocha 1.2.0 built to pkg/mocha-1.2.0.gem.
Tagged v1.2.0.
Pushed git commits and tags.
Pushed mocha 1.2.0 to rubygems.org.
```

### History

Mocha was initially harvested from projects at [Reevoo](http://www.reevoo.com/). It's syntax is heavily based on that of [jMock](http://www.jmock.org).

### License

© Copyright James Mead 2006

You may use, copy and redistribute this library under the same terms as [Ruby itself](https://www.ruby-lang.org/en/about/license.txt) or under the [MIT license](https://mit-license.org/).

repository: freerange/mocha

gha_language: Ruby

repo_name: freerange/mocha


text:
# Contextualised Word Representations for Lexical Semantic Change

This is a repository for the paper:

Mario Giulianelli, Marco Del Tredici, and Raquel Fernández. 2020. [Analysing Lexical Semantic Change with Contextualised Word Representations](https://arxiv.org/abs/2004.14118). In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020). Association for Computational Linguistics_.

We also make public the [DUPS (Diachronic Usage Pair Similarity)](https://doi.org/10.5281/zenodo.3773250) dataset, which contains similarity judgements of English word usage pairs from different time periods, as described in the paper.

---

- Check the [semeval2020](https://github.com/akutuzov/semeval2020) repository with updated code for change detection on multiple languages and with multiple language models.


text:
---
layout: post
title: Speech by SMS Assoc Prof Ho Peng Kee at the 2010 Prix d'Excellence Awards Dinner
permalink: /news/speeches/speech-by-sms-assoc-prof-ho-peng-kee-at-the-2010-prix-d-excellence-awards-dinner
breadcrumb: Speeches
collection_name: news
---

5 Oct 2010 Posted in [Speeches](/news/speeches)

Dr Lim Lan Yuan, President of FIABCI Singapore,
Ms Gillian Tay, Chairperson of the Organising Committee,
Distinguished Guests,
Ladies and Gentlemen,

1. Good evening.  I am happy to join you this evening to celebrate Singapore’s winning of three Prix d’Excellence awards.  This is a significant achievement for Singapore, and for our real estate, construction and architectural fraternities. The Prix d’Excellence award is much sought after by developers, builders and architects around the world, as it is only conferred on developments that have demonstrated excellence in all of the real estate disciplines.

2. My warmest congratulations to Far East Organization, City Development Ltd, UOL Group Ltd, DP Architects, Team Design Architects and P & T Consultants Pte Ltd for winning top awards in the Office, Residential and Hotel categories. Indeed, it is noteworthy that Far East Organization is the only developer in the world with the highest number of wins. These achievements are testaments of the high standards achieved by Singapore’s developers and architects, reflecting the growing international recognition of our creativity, talent and quality.

**The Role of Developers and Architects**

{:start="3"}
3. Our vision for Singapore is a distinctive cosmopolitan city that is both a shining example of a global city of tomorrow and an endearing home for all.  As developers and architects, you play a critical role in shaping our constantly evolving cityscape. Your works form the elements of the urban backdrop to our daily life and help forge a distinctive Singapore skyline that Singaporeans can be proud of, and that visitors to our shores find memorable!

{:start="4"}
4. Whilst achieving this, you also need to create and enhance the working, living and public spaces within which Singaporeans and other residents living here interact with one another. The challenge therefore is that Singapore must not only be a city known for its dramatic and exciting cityscape, but is also an inclusive, liveable and vibrant city that is enjoyed and embraced by citizens and visitors alike.

**Architecture and Urban Design Excellence**

{:start="5"}
5. A quality urban environment helps elevate our quality of life and physical environment, bringing about tangible economic and financial dividends in attracting talents and businesses, more tourist arrivals, fostering creativity, and engendering a sense of rootedness among its citizens. Hence, a maturing and growing Singapore requires, without a doubt, excellence in architecture and urban design as it is increasingly a factor that enhances the experience of a city.  Attaining such excellence requires attention to be paid not just to high standards in design aesthetics and functionality, but also to the increasingly important elements of environmental sustainability, social inclusiveness and our heritage.

{:start="6"}
6. I am heartened by the progress Singapore has made in this area. I encourage you to continue to apply innovation and creative thinking to your design and building processes. Not only will this push the frontiers of design, it will also raise Singapore’s status as a global city abuzz with new ideas and energy.

{:start="7"}
7. On its part, the Government will play its part to incorporate the concepts of architectural excellence and good urban design in its work. Be it in our housing estates or large scale development projects such as Marina Bay, down to streets and open spaces, the Government is committed to a built environment that facilitates Singapore’s economic growth, whilst also meeting the needs of its people and the environment. The evidence of this commitment is there for all to see. In this regard, I look forward to your continual partnership with the Government to come up with new designs that will enhance our built environment.

{:start="8"}
8. In addition, the Government aims to create an environment that promotes the recognition and appreciation of good urban design.  To this end, the Urban Redevelopment Authority, or URA, has been working with the private sector to showcase local projects with excellent urban design through its Architecture and Urban Design Excellence programme, and the annual President’s Design Award. For instance, the URA is currently holding a ‘20 Under 45’ exhibition at the URA Centre to showcase the works of 20 locally-registered architects under the age of 45.  Through the exhibition, the URA hopes to raise the profile of locally-registered architectural talents and provide wider public and industry exposure for them

**Prix d’Excellence Awards**

{:start="9"}
9. In short, the entire fraternity must rally around and be involved as we press on resolutely towards our vision of a Singapore that is a world class city of the future. Thank you for your unwavering enthusiasm. The fact that Singapore has won so many Prix d’Excellence awards since 1997 shows that we do have rich talent and creativity here. My interaction with the young bears this out clearly.

{:start="10"}
10. Once again, my heartiest congratulations to all winners for being conferred these prestigious awards on a globally-recognised platform.  You make us proud and bring honour to Singapore. Your success stories showcasing the virtues of architecture and urban design excellence in Singapore will, I am sure, inspire more of such stories, especially amongst our developers and architects. It leaves me now to wish you every success in all your future undertakings as you continue to contribute to Singapore’s physical landscape.

{:start="11"}
11. Have a pleasant evening.

{:start="12"}
12. Thank you.

Last updated on 25 Nov 2012

repository: chungcuiyi/isomerpages-mlaw

repo_name: chungcuiyi/isomerpages-mlaw


text:
# camp-pie-cutting

New years pie cutting with random lucky strike win for [Rama Camp](https://rama-camp-pie.netlify.app/).

## Installation

```
yarn install
```

## Run Dev Server

```
yarn dev
```

## License
[MIT](https://choosealicense.com/licenses/mit/)

repository: menepet/camp-pie-cutting

repo_name: menepet/camp-pie-cutting


text:
# RateIITU Telegram bot

RateIITU professor notification-bot written on Ruby.

## Dependencies
- Ruby Version Manager
- Bundler

## Installation
- Install GPG keys:
```bash
$ gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
```
- Install RVM
```bash
$ \curl -sSL https://get.rvm.io | bash -s stable
```
In case you encounter an issues or want to know more, check [RVM official website (rvm.io)](rvm.io)

## Usage
First you have to clone this repository, and go into project folder.

```yml
$ git clone https://github.com/soundsnick/rate-bot.git
$ cd rate-bot
```

Configure `config/config.yml` setting your credits.

Run server using Ruby interpreter.
```bash
$ ruby server.rb
```
or Daemonize it using `nohup` utility
```bash
$ nohup ruby server.rb &
```

##  Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

## License
[MIT](https://choosealicense.com/licenses/mit/)

repository: soundsnick/rate-bot

repo_name: soundsnick/rate-bot


text:
---
layout: post
title:  "100 Days of SwiftUI - Day 2"
date:   2019-09-25 19:19:46 +1200
categories: ohdos
---
According to the rules for taking part in [100 Days of SwiftUI](https://www.hackingwithswift.com/100/swiftui), as soon as I've finished each day, I have to post about it.

The topic for Day 2 was: Complex data types.

This involved watching short videos on: Arrays; Sets; Tuples; Arrays vs sets vs tuples; Dictionaries; Dictionary default values; Creating empty collections; Enumerations; Enum associated values; Enum raw values; and Complex types: Summary.


text:
# Auto Profiler

This should be run after major updates to both test performance and also verify correctness.

## Rust

Profiling rust simulator and decoder

need to install https://github.com/flamegraph-rs/flamegraph

run `python3 benchmark.py` to profile decoders, by testing the logical error rate of several well-known cases and also the run time

run `COMPARE_WITH_VERSION=0.1.0 python3 benchmark.py` to compare with a specific version. This can help you to pinpoint possible erroneous logical error rate or performance drop.

repository: yuewuo/QEC-Playground

repo_name: yuewuo/QEC-Playground


text:
PicoScrape

A python package to scrape images from various websites.

Install via cmd : pip install picoscrape

[How to upload python package to<EMAIL_ADDRESS>

Details

Name : picoscrape

Version : 1.0

License : MIT License

Author : Anurag Gupta

Author Email<EMAIL_ADDRESS>
URL = https://github.com/anuraggupta29/picoscrape

Requirements : Selenium

Development Status : 3 - Alpha

Intended Audience :: Developers

Topic : Software Development - Build Tools

Description

This is a python package with functions to easily scrape images from various websites. It also has basic file editing functions and image manipulation functions to ease the process of acquiring image datasets for training models.

Use help(), dir() in python to get better insights on how to use this package,

Do make a pull request to contribute to this project

Currently, this package only supports image downloads from Unsplash.

Support for other websites and new functions to be added soon.


text:
# Arduino CI Scripts

This repos contains various scripts and tools related to running continuous integration (CI) checks on Arduino Library Repos. The operations include:

* checking formatting using using [clang-format](https://clang.llvm.org/docs/ClangFormat.html),
* generating documentation from source comments using using [Doxygen](https://www.doxygen.nl/), and
* building each example in the library for selected targets.

There is an associated guide available here:
https://learn.adafruit.com/the-well-automated-arduino-library/

## Adding GitHub Actions to Repo

To run these continuous integration checks on each push, pull-request or [repository dispatch](https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#create-a-repository-dispatch-event) using [GitHub actions](https://github.com/features/actions):

* Create a folder named `.github/worflows` in the root of the repo.
* Copy `example_actions.yml` into the above directory and rename it `githubci.yml`.
* Edit `githubci.yml` and change `PRETTYNAME` to the library repo name. Optionally, delete or comment out steps (using the `#` character), you don't want to include.
* Here's an example: [Adafruit_BME280_Library](https://github.com/adafruit/Adafruit_BME280_Library/blob/master/.github/workflows/githubci.yml)

## Controlling Test Behavior

The `build_platform.py` script is used to test each `.ino` example in the repo for selected build platforms. The [`ALL_PLATFORMS`](ci-arduino/blob/master/build_platform.py#L54) dictionary contains a listing of all available platforms and selected platform groups. By default, `main_platforms` is used. To select a specific platform or group, replace `main_platforms` in [`githubci.yml`](`example_actions.yml`) with the group or platform name.

Additionally, [UF2 files](https://github.com/microsoft/uf2) of the compiled sketches can be generated for supported platforms.

### Fine tuning test selection

The script behavior can be controlled using special filenames:

* `.PLATFORM_ID.test.skip` - Skip the specified platform. All others are tested.
* `.PLATFORM_ID.test.only` - Test the specified platform. All others are skipped.
* `.PLATFORM_ID.generate` - Generate UF2 of sketch for specified platform (if supported).

These are just empty files placed in an example folder. Replace `PLATFORM_ID` in the name with the key from [`ALL_PLATFORMS`](ci-arduino/blob/master/build_platform.py#L54). `metro_m0` from the following line in `build_platform.py`, for example:

```python
"metro_m0" : ["adafruit:samd:adafruit_metro_m0", "0x68ed2b88", None],
```

You can use several `.PLATFORM_ID.test.skip` or `.PLATFORM_ID.test.only` to exclude or include multiple platforms. For example:

* To **skip** testing on ESP8266, add a file named `.esp8266.test.skip`
* To test **only** the Arduino UNO, add a file named `.uno.test.only`
* To skip all and test **nothing**, add a file named `.none.test.only`
* To generate UF2s for PyPortal, add a file named `.pyportal.generate`

### Dependencies

Any library dependencies included in the [`library.properties`](https://arduino.github.io/arduino-cli/0.19/library-specification/#libraryproperties-file-format) are automatically installed before the tests are started. To install additional dependencies (e.g., those required for some examples but not the library itself) using [`arduino-cli`](https://arduino.github.io/arduino-cli/0.19/commands/arduino-cli_lib_install/), you could add additional steps to the `githubci.yml` file. For example:

```yaml
- name: Set configuration
  run: arduino-cli config set library.enable_unsafe_install true

- name: Install test dependencies
  run: arduino-cli lib install --git-url https://github.com/arduino-libraries/Servo --git-url https://github.com/arduino-libraries/Ethernet
```

Note: you'll only need to enable the [`enable_unsafe_install`](https://arduino.github.io/arduino-cli/0.32/configuration/#configuration-keys) option if you want to identify libraries using urls. This isn't necessary when using the library name.

## Formatting Check with Clang

The `run-clang-format.py` script is used to run [clang-format](https://clang.llvm.org/docs/ClangFormat.html) and check file formatting.
See [the guide](https://learn.adafruit.com/the-well-automated-arduino-library/formatting-with-clang-format) for details on installing `clang-format` to run formatting locally.
Even a single extra white space can cause the CI to fail on formatting.
You can typically just let clang do its thing and edit files in place using:

```
clang-format -i File_To_Format.cpp
```

## Documentation with Doxygen

The `doxy_gen_and_deploy.sh` script uses [Doxygen](https://www.doxygen.nl/) to generate and deploy documentation
for the library. Any issues, like missing documentation, will cause the CI to fail.
See the [the guide](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen) for details on installing and running doxygen locally. The guide also has some
[tips](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen-tips) on basic usage of doxygen markup within your code.

repository: makermelissa/ci-arduino

repo_name: makermelissa/ci-arduino


text:
# Smash Ultimate Controller Tools
 Smash Ultimate Controller Tools (UltConTool) is a sing-file c++ library for messing with controller profiles in Smash Ultimate

## Usage
 Just include uct.h

## Demo
 Compiling this repo gives you a demo app to test out the functions with


text:
# nyc-gtfs2geojson
Script to extract GeoJSON points and lines from NYC GTFS Data.  Extracts lines from `shapes.txt`, and points from `stops.txt`

## How to use
- Install dependencies `npm install`
- Start with an array of objects for GTFS data sources: `sources.js`.  Each has a name and a url.
- Download and unzip each source `node get.js`
- Transform each into GeoJson `node transform.js`
- Geojson is saved in `./geojson/{source.name}/{stops||lines}.geojson

## gtfs2geojson
This script uses a modified version of `[gtfs2geojson(https://github.com/tmcw/gtfs2geojson)`, a node package by Tom MacWright.  We have added parsing for `stops.txt` and will do a Pull Request back to the package soon.

## How we are using it
- The resulting GeoJSON files are included in this repo.  The github file URLS are the source for another ETL that loads the data into a PostGIS database for use in mapping applications.


text:
# Discussion Api Framework

Hello,
Weclome to Discssion Forum, a simple forum which can be used to disucssion on topics based on channel and ask your questions.

## License

The Lumen framework is open-sourced software licensed under the [MIT license](http://opensource.org/licenses/MIT)


text:
# Sidebear

![](./_images/Sidebear_Icon.png)

Robofont extension for manipulating your glyph’s sidebearings.

Sidebear installs itself in your Inspector, underneath the Glyph Info pane.

![](./_images/Sidebear_screen.png)

## Actions include:
`Enter Desired SB` You may enter numerical values or glyph names. If you place a comma `,` before or after the glyph name, the opposite SB will be matched. (For example, `,n` input on the LSB of glyph `u`  will match the RSB of the `n`.)

`Swap SBs`

`Make LSB = RSB`

`Center glyph` Make SBs equal (Set width may change ± 1)

`Make RSB = LSB`

##### Choose an increment, then:
`Contract both SBs` equally by that increment.

`Expand both SBs` equally by that increment.

ʕ·ᴥ· ʔ

Enjoy designing with your friendly Sidebear
Special thanks to Just van Rossum, Gustavo Ferreira, Frederik Berlaen, Frank Griesshammer, Erik van Blokland.


text:
# Emotion-recognition-and-prediction

A real time system to detect human emotions from image and voice and predict its reaction. This is a pre-trained model with an accuracy of **80.7%** over Facial Expression Recognition (FER) 2013 dataset. This project is the direct outcome of my graduation submission at MIT, Pune. The extraction of emotions from audio is done by using [pyAudio](https://github.com/tyiannak/pyAudioAnalysis)
**Reaction prediction module is under development and is not included yet**

# Prerequisites
```
Python 3
OpenCV
```
# Installation
```
pip install tensorflow numpy scipy h5py
```
Download the trained model from the links given in source/trained-model.txt
# Testing
```
python em_model.py
```


text:
# mongodb puppet module

## Overview

Installs mongodb on Ubuntu/Debian per 10gen [installation documentation](http://www.mongodb.org/display/DOCS/Ubuntu+and+Debian+packages).

## Usage

### class mongodb

Parameters:

* init: optionally specify the init script used, accepts sysv or upstart.

By default ubuntu is upstart and debian uses sysv.

Examples:

    class mongodb {
      init => 'sysv',
    }

## Supported Platforms

The module have been tested on the following operating systems. Testing and patches for other platforms are welcomed.

* Debian Wheezy

This module is under development and does not manage mongodb.conf at the moment.

repository: nanliu/puppetlabs-mongodb

repo_name: nanliu/puppetlabs-mongodb


text:
# DataFrameWriter

`DataFrameWriter[T]` is a high-level API for Spark SQL developers to describe "write path" of a structured query over rows of `T` type (i.e. how to [save a Dataset](#save) to a data source).

`DataFrameWriter` is available using [Dataset.write](Dataset.md#write) operator.

## Creating Instance

`DataFrameWriter` takes the following to be created:

*  [Dataset](Dataset.md)

`DataFrameWriter` is created when:

* [Dataset.write](Dataset.md#write) operator is used

##  DataFrame

When [created](#creating-instance), `DataFrameWriter` converts the [Dataset](#ds) to a [DataFrame](Dataset.md#toDF).

##  Name of Data Source

```scala
source: String
```

`source` is a short name (alias) or a fully-qualified class name to identify the data source to write data to.

`source` can be specified using `format` method:

```scala
format(
  source: String): DataFrameWriter[T]
```

Default: [spark.sql.sources.default](configuration-properties.md#spark.sql.sources.default) configuration property

##  insertInto

```scala
insertInto(
  tableName: String): Unit
```

`insertInto` requests the [DataFrame](#df) for the [SparkSession](Dataset.md#sparkSession).

`insertInto` tries to [look up the TableProvider](#lookupV2Provider) for the [data source](#source).

`insertInto` requests the [ParserInterface](sql/ParserInterface.md) to [parse](sql/ParserInterface.md#parseMultipartIdentifier) the `tableName` identifier (possibly multi-part).

In the end, `insertInto` uses the [modern](#insertInto-CatalogPlugin) or the [legacy](#insertInto-TableIdentifier) insert paths based on...FIXME

![DataFrameWrite.insertInto Executes SQL Command (as a Spark job)](images/spark-sql-DataFrameWrite-insertInto-webui-query-details.png)

`insertInto` [asserts that write is not bucketed](#assertNotBucketed) (with **insertInto** operation name).

!!! note
    [saveAsTable](#saveAsTable) and [insertInto](#insertInto) are structurally alike.

###  Modern Insert Path (CatalogPlugin)

```scala
insertInto(
  catalog: CatalogPlugin,
  ident: Identifier): Unit
```

`insertInto`...FIXME

###  Legacy Insert Path (TableIdentifier)

```scala
insertInto(
  tableIdent: TableIdentifier): Unit
```

`insertInto`...FIXME

###  AnalysisException

`insertInto` throws an `AnalysisException` when the [partitioningColumns](#partitioningColumns) are defined:

```text
insertInto() can't be used together with partitionBy(). Partition columns have already been defined for the table. It is not necessary to use partitionBy().
```

##  saveAsTable

```scala
saveAsTable(
  tableName: String): Unit
```

`saveAsTable` requests the [DataFrame](#df) for the [SparkSession](Dataset.md#sparkSession).

`saveAsTable` tries to [look up the TableProvider](#lookupV2Provider) for the [data source](#source).

`saveAsTable` requests the [ParserInterface](sql/ParserInterface.md) to [parse](sql/ParserInterface.md#parseMultipartIdentifier) the `tableName` identifier (possibly multi-part).

In the end, `saveAsTable` uses the [modern](#saveAsTable-TableCatalog) or the [legacy](#saveAsTable-TableIdentifier) save paths based on...FIXME

!!! note
    [saveAsTable](#saveAsTable) and [insertInto](#insertInto) are structurally alike.

###  Modern saveAsTable Path (TableCatalog)

```scala
saveAsTable(
  catalog: TableCatalog,
  ident: Identifier,
  nameParts: Seq[String]): Unit
```

###  Legacy saveAsTable Path (TableIdentifier)

```scala
saveAsTable(
  tableIdent: TableIdentifier): Unit
```

`saveAsTable` saves the content of a `DataFrame` to the `tableName` table.

###  AnalysisException

`saveAsTable` throws an `AnalysisException` when no catalog could handle the table identifier:

```text
Couldn't find a catalog to handle the identifier [tableName].
```

### Demo

```text
val ids = spark.range(5)
ids.write.
  option("path", "/tmp/five_ids").
  saveAsTable("five_ids")

// Check out if saveAsTable as five_ids was successful
val q = spark.catalog.listTables.filter($"name" === "five_ids")
scala> q.show
+--------+--------+-----------+---------+-----------+
|    name|database|description|tableType|isTemporary|
+--------+--------+-----------+---------+-----------+
|five_ids| default|       null| EXTERNAL|      false|
+--------+--------+-----------+---------+-----------+
```

##  save

```scala
save(): Unit
save(
  path: String): Unit
```

Saves a `DataFrame` (the result of executing a structured query) to a data source.

Internally, `save` uses `DataSource` to [look up the class of the requested data source](DataSource.md#lookupDataSource) (for the [source](#source) option and the [SQLConf](SessionState.md#conf)).

!!! note
    `save` uses [SparkSession](Dataset.md#sparkSession) to access the [SessionState](SparkSession.md#sessionState) and in turn the [SQLConf](SessionState.md#conf).

    ```text
    val df: DataFrame = ???
    df.sparkSession.sessionState.conf
    ```

`save`...FIXME

`save` throws an `AnalysisException` when requested to save to Hive data source (the [source](#source) is `hive`):

```text
Hive data source can only be used with tables, you can not write files of Hive data source directly.
```

`save` throws an `AnalysisException` when [bucketing is used](#assertNotBucketed) (the [numBuckets](#numBuckets) or [sortColumnNames](#sortColumnNames) options are defined):

```text
'[operation]' does not support bucketing right now
```

##  Looking up TableProvider

```scala
lookupV2Provider(): Option[TableProvider]
```

`lookupV2Provider` tries to [look up a TableProvider](DataSource.md#lookupDataSourceV2) for the [source](#source).

`lookupV2Provider` explicitly excludes [FileDataSourceV2](FileDataSourceV2.md)-based data sources (due to [SPARK-28396](https://issues.apache.org/jira/browse/SPARK-28396)).

`lookupV2Provider` is used when:

* `DataFrameWriter` is requested to [save](#save), [insertInto](#insertInto) and [saveAsTable](#saveAsTable)

## Review Me

[[methods]]
.DataFrameWriter API / Writing Operators
[cols="1,2",options="header",width="100%"]
|===
| Method
| Description

| <<bucketBy-internals, bucketBy>>
a| [[bucketBy]]

[source, scala]
----
bucketBy(numBuckets: Int, colName: String, colNames: String*): DataFrameWriter[T]
----

| <<csv-internals, csv>>
a| [[csv]]

[source, scala]
----
csv(path: String): Unit
----

| <<format-internals, format>>
a| [[format]]

[source, scala]
----
format(source: String): DataFrameWriter[T]
----

| <<jdbc-internals, jdbc>>
a| [[jdbc]]

[source, scala]
----
jdbc(url: String, table: String, connectionProperties: Properties): Unit
----

| <<json-internals, json>>
a| [[json]]

[source, scala]
----
json(path: String): Unit
----

| <<mode-internals, mode>>
a| [[mode]]

[source, scala]
----
mode(saveMode: SaveMode): DataFrameWriter[T]
mode(saveMode: String): DataFrameWriter[T]
----

| <<option-internals, option>>
a| [[option]]

[source, scala]
----
option(key: String, value: String): DataFrameWriter[T]
option(key: String, value: Boolean): DataFrameWriter[T]
option(key: String, value: Long): DataFrameWriter[T]
option(key: String, value: Double): DataFrameWriter[T]
----

| <<options-internals, options>>
a| [[options]]

[source, scala]
----
options(options: scala.collection.Map[String, String]): DataFrameWriter[T]
----

| <<orc-internals, orc>>
a| [[orc]]

[source, scala]
----
orc(path: String): Unit
----

| <<parquet-internals, parquet>>
a| [[parquet]]

[source, scala]
----
parquet(path: String): Unit
----

| <<partitionBy-internals, partitionBy>>
a| [[partitionBy]]

[source, scala]
----
partitionBy(colNames: String*): DataFrameWriter[T]
----

| <<sortBy-internals, sortBy>>
a| [[sortBy]]

[source, scala]
----
sortBy(colName: String, colNames: String*): DataFrameWriter[T]
----

| <<text-internals, text>>
a| [[text]]

[source, scala]
----
text(path: String): Unit
----
|===

`DataFrameWriter` is available using <<spark-sql-DataFrame.md#write, Dataset.write>> operator.

[source, scala]
----
scala> :type df
org.apache.spark.sql.DataFrame

val writer = df.write

scala> :type writer
org.apache.spark.sql.DataFrameWriter[org.apache.spark.sql.Row]
----

`DataFrameWriter` supports many <<writing-dataframes-to-files, file formats>> and <<jdbc, JDBC databases>>. It also allows for plugging in <<format, new formats>>.

`DataFrameWriter` defaults to <<parquet, parquet>> data source format. You can change the default format using [spark.sql.sources.default](configuration-properties.md#spark.sql.sources.default) configuration property or <<format, format>> or the format-specific methods.

```text
// see above for writer definition

// Save dataset in Parquet format
writer.save(path = "nums")

// Save dataset in JSON format
writer.format("json").save(path = "nums-json")

// Alternatively, use format-specific method
write.json(path = "nums-json")
```

In the end, you trigger the actual saving of the content of a `Dataset` (i.e. the result of executing a structured query) using <<save, save>> method.

[source, scala]
----
writer.save
----

[[internal-state]]
`DataFrameWriter` uses <<internal-attributes-and-corresponding-setters, internal mutable attributes>> to build a properly-defined _"write specification"_ for <<insertInto, insertInto>>, <<save, save>> and <<saveAsTable, saveAsTable>> methods.

[[internal-attributes-and-corresponding-setters]]
.Internal Attributes and Corresponding Setters
[cols="1m,2",options="header"]
|===
| Attribute
| Setters

| source
| [[source]] <<format, format>>

| mode
| [[mode-internal-property]] <<mode, mode>>

| extraOptions
| [[extraOptions]] <<option, option>>, <<options, options>>, <<save, save>>

| partitioningColumns
| [[partitioningColumns]] <<partitionBy, partitionBy>>

| bucketColumnNames
| [[bucketColumnNames]] <<bucketBy, bucketBy>>

| numBuckets
| [[numBuckets]] <<bucketBy, bucketBy>>

| sortColumnNames
| [[sortColumnNames]] <<sortBy, sortBy>>
|===

[[df]]
NOTE: `DataFrameWriter` is a type constructor in Scala that keeps an internal reference to the source `DataFrame` for the whole lifecycle (starting right from the moment it was created).

NOTE: Spark Structured Streaming's `DataStreamWriter` is responsible for writing the content of streaming Datasets in a streaming fashion.

=== [[runCommand]] Executing Logical Command(s) -- `runCommand` Internal Method

[source, scala]
----
runCommand(session: SparkSession, name: String)(command: LogicalPlan): Unit
----

`runCommand` uses the input `SparkSession` to access the <<SparkSession.md#sessionState, SessionState>> that is in turn requested to <<SessionState.md#executePlan, execute the logical command>> (that simply creates a [QueryExecution](QueryExecution.md)).

`runCommand` records the current time (start time) and uses the `SQLExecution` helper object to [execute the action (under a new execution id)](SQLExecution.md#withNewExecutionId) that simply requests the `QueryExecution` for the [RDD[InternalRow]](QueryExecution.md#toRdd) (and triggers execution of logical commands).

TIP: Use web UI's SQL tab to see the execution or a `SparkListener` to be notified when the execution is started and finished. The `SparkListener` should intercept `SparkListenerSQLExecutionStart` and `SparkListenerSQLExecutionEnd` events.

`runCommand` records the current time (end time).

In the end, `runCommand` uses the input `SparkSession` to access the <<SparkSession.md#listenerManager, ExecutionListenerManager>> and requests it to <<ExecutionListenerManager.md#onSuccess, onSuccess>> (with the input `name`, the `QueryExecution` and the duration).

In case of any exceptions, `runCommand` requests the `ExecutionListenerManager` to <<ExecutionListenerManager.md#onFailure, onFailure>> (with the exception) and (re)throws it.

NOTE: `runCommand` is used when `DataFrameWriter` is requested to <<save, save the rows of a structured query (a DataFrame) to a data source>> (and indirectly <<saveToV1Source, executing a logical command for writing to a data source V1>>), <<insertInto, insert the rows of a structured streaming (a DataFrame) into a table>> and <<createTable, create a table>> (that is used exclusively for <<saveAsTable, saveAsTable>>).

=== [[jdbc-internals]] Saving Data to Table Using JDBC Data Source -- `jdbc` Method

[source, scala]
----
jdbc(url: String, table: String, connectionProperties: Properties): Unit
----

`jdbc` method saves the content of the `DataFrame` to an external database table via JDBC.

You can use <<mode, mode>> to control *save mode*, i.e. what happens when an external table exists when `save` is executed.

It is assumed that the `jdbc` save pipeline is not <<partitionBy, partitioned>> and <<bucketBy, bucketed>>.

All <<options, options>> are overriden by the input `connectionProperties`.

The required options are:

* `driver` which is the class name of the JDBC driver (that is passed to Spark's own `DriverRegistry.register` and later used to `connect(url, properties)`).

When `table` exists and the <<mode, override save mode>> is in use, `DROP TABLE table` is executed.

It creates the input `table` (using `CREATE TABLE table (schema)` where `schema` is the schema of the `DataFrame`).

=== [[bucketBy-internals]] `bucketBy` Method

[source, scala]
----
bucketBy(numBuckets: Int, colName: String, colNames: String*): DataFrameWriter[T]
----

`bucketBy` simply sets the internal <<numBuckets, numBuckets>> and <<bucketColumnNames, bucketColumnNames>> to the input `numBuckets` and `colName` with `colNames`, respectively.

[source, scala]
----
val df = spark.range(5)
import org.apache.spark.sql.DataFrameWriter
val writer: DataFrameWriter[java.lang.Long] = df.write

val bucketedTable = writer.bucketBy(numBuckets = 8, "col1", "col2")

scala> :type bucketedTable
org.apache.spark.sql.DataFrameWriter[Long]
----

=== [[partitionBy]] `partitionBy` Method

[source, scala]
----
partitionBy(colNames: String*): DataFrameWriter[T]
----

CAUTION: FIXME

=== [[mode-internals]] Specifying Save Mode -- `mode` Method

[source, scala]
----
mode(saveMode: String): DataFrameWriter[T]
mode(saveMode: SaveMode): DataFrameWriter[T]
----

`mode` defines the behaviour of <<save, save>> when an external file or table (Spark writes to) already exists, i.e. `SaveMode`.

##  SaveMode

[cols="1,2",options="header",width="100%"]
|===
| Name
| Description

| `Append`
| [[Append]] Records are appended to existing data.

| `ErrorIfExists`
| [[ErrorIfExists]] Exception is thrown.

| `Ignore`
| [[Ignore]] Do not save the records and not change the existing data in any way.

| `Overwrite`
| [[Overwrite]] Existing data is overwritten by new records.
|===

=== [[sortBy-internals]] Specifying Sorting Columns -- `sortBy` Method

[source, scala]
----
sortBy(colName: String, colNames: String*): DataFrameWriter[T]
----

`sortBy` simply sets <<sortColumnNames, sorting columns>> to the input `colName` and `colNames` column names.

NOTE: `sortBy` must be used together with <<bucketBy, bucketBy>> or `DataFrameWriter` reports an `IllegalArgumentException`.

NOTE: <<assertNotBucketed, assertNotBucketed>> asserts that bucketing is not used by some methods.

=== [[option-internals]] Specifying Writer Configuration -- `option` Method

[source, scala]
----
option(key: String, value: Boolean): DataFrameWriter[T]
option(key: String, value: Double): DataFrameWriter[T]
option(key: String, value: Long): DataFrameWriter[T]
option(key: String, value: String): DataFrameWriter[T]
----

`option`...FIXME

=== [[options-internals]] Specifying Writer Configuration -- `options` Method

[source, scala]
----
options(options: scala.collection.Map[String, String]): DataFrameWriter[T]
----

`options`...FIXME

=== [[writing-dataframes-to-files]] Writing DataFrames to Files

CAUTION: FIXME

=== [[format-internals]] Specifying Data Source (by Alias or Fully-Qualified Class Name) -- `format` Method

[source, scala]
----
format(source: String): DataFrameWriter[T]
----

`format` simply sets the <<source, source>> internal property.

=== [[parquet]] Parquet

CAUTION: FIXME

NOTE: Parquet is the default data source format.

=== [[getBucketSpec]] `getBucketSpec` Internal Method

[source, scala]
----
getBucketSpec: Option[BucketSpec]
----

`getBucketSpec` returns a new <<spark-sql-BucketSpec.md#, BucketSpec>> if <<numBuckets, numBuckets>> was defined (with <<bucketColumnNames, bucketColumnNames>> and <<sortColumnNames, sortColumnNames>>).

`getBucketSpec` throws an `IllegalArgumentException` when <<numBuckets, numBuckets>> are not defined when <<sortColumnNames, sortColumnNames>> are.

```
sortBy must be used together with bucketBy
```

NOTE: `getBucketSpec` is used exclusively when `DataFrameWriter` is requested to <<createTable, create a table>>.

##  Creating Table

```scala
createTable(
  tableIdent: TableIdentifier): Unit
```

`createTable` [builds a CatalogStorageFormat](DataSource.md#buildStorageFormatFromOptions) per [extraOptions](#extraOptions).

`createTable` assumes the table being [external](CatalogTable.md#CatalogTableType) when [location URI](spark-sql-CatalogStorageFormat.md#locationUri) of `CatalogStorageFormat` is defined, and [managed](CatalogTable.md#CatalogTableType) otherwise.

`createTable` creates a [CatalogTable](CatalogTable.md) (with the [bucketSpec](CatalogTable.md#bucketSpec) per [getBucketSpec](#getBucketSpec)).

In the end, `createTable` creates a <<CreateTable.md#, CreateTable>> logical command (with the `CatalogTable`, <<mode, mode>> and the <<Dataset.md#planWithBarrier, logical query plan>> of the <<df, dataset>>) and <<runCommand, runs>> it.

NOTE: `createTable` is used when `DataFrameWriter` is requested to <<saveAsTable, saveAsTable>>.

=== [[assertNotBucketed]] `assertNotBucketed` Internal Method

[source, scala]
----
assertNotBucketed(operation: String): Unit
----

`assertNotBucketed` simply throws an `AnalysisException` if either <<numBuckets, numBuckets>> or <<sortColumnNames, sortColumnNames>> internal property is defined:

```
'[operation]' does not support bucketing right now
```

NOTE: `assertNotBucketed` is used when `DataFrameWriter` is requested to <<save, save>>, <<insertInto, insertInto>> and <<jdbc, jdbc>>.

##  Executing Logical Command for Writing to Data Source V1

```scala
saveToV1Source(): Unit
```

`saveToV1Source` creates a [DataSource](DataSource.md#apply) (for the [source](#source) class name, the [partitioningColumns](#partitioningColumns) and the [extraOptions](#extraOptions)) and requests it for the [logical command for writing](DataSource.md#planForWriting) (with the [mode](#mode) and the [analyzed logical plan](Dataset.md#logicalPlan) of the structured query).

!!! note
    While requesting the [analyzed logical plan](Dataset.md#logicalPlan) of the structured query, `saveToV1Source` triggers execution of logical commands.

In the end, `saveToV1Source` [runs the logical command for writing](#runCommand).

!!! note
    The [logical command for writing](DataSource.md#planForWriting) can be one of the following:

  * A [SaveIntoDataSourceCommand](logical-operators/SaveIntoDataSourceCommand.md) for [CreatableRelationProviders](CreatableRelationProvider.md)

    * An [InsertIntoHadoopFsRelationCommand](logical-operators/InsertIntoHadoopFsRelationCommand.md) for [FileFormats](datasources/FileFormat.md)

`saveToV1Source` is used when `DataFrameWriter` is requested to [save the rows of a structured query (a DataFrame) to a data source](#save).

=== [[assertNotPartitioned]] `assertNotPartitioned` Internal Method

[source, scala]
----
assertNotPartitioned(operation: String): Unit
----

`assertNotPartitioned`...FIXME

NOTE: `assertNotPartitioned` is used when...FIXME

=== [[csv-internals]] `csv` Method

[source, scala]
----
csv(path: String): Unit
----

`csv`...FIXME

=== [[json-internals]] `json` Method

[source, scala]
----
json(path: String): Unit
----

`json`...FIXME

=== [[orc-internals]] `orc` Method

[source, scala]
----
orc(path: String): Unit
----

`orc`...FIXME

=== [[parquet-internals]] `parquet` Method

[source, scala]
----
parquet(path: String): Unit
----

`parquet`...FIXME

=== [[text-internals]] `text` Method

[source, scala]
----
text(path: String): Unit
----

`text`...FIXME

=== [[partitionBy]] `partitionBy` Method

[source, scala]
----
partitionBy(colNames: String*): DataFrameWriter[T]
----

`partitionBy` simply sets the <<partitioningColumns, partitioningColumns>> internal property.

repository: myandpr/mastering-spark-sql-book

repo_name: myandpr/mastering-spark-sql-book


text:
# Migration Guide

## Migrating from v3 to v4

### Updated dependencies

The dependencies for this component have all been updated to the latest major versions.
If you have any conflicts while installing this version, you'll need to first update
its dependencies. See [the Bower config for these](./bower.json).

### JavaScript

o-gallery v4 uses [ES Modules over CommonJS](https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/) syntax.

```diff
-const Gallery = require('o-gallery');
+import Gallery from 'o-gallery';
```

However to continue using the CommonJS syntax, without a plugin like [babel-plugin-transform-es2015-modules-commonjs](https://babeljs.io/docs/en/babel-plugin-transform-es2015-modules-commonjs), add `.default`.

```diff
-const Gallery = require('o-gallery');
+const Gallery = require('o-gallery').default;
```

### Sass Mixins

The `oGallery` mixin now outputs all o-gallery styles. All other o-gallery mixins have been removed. Instead include `oGallery` once and use default o-gallery css classes.

### Colour Usecases
All [o-gallery colour usecases](https://github.com/Financial-Times/o-gallery/blob/v3.0.9/src/scss/_color-use-cases.scss) have been removed. Use [o-colors](https://registry.origami.ft.com/components/o-colors) directly.

## Migrating from v2 to v3

This update introduces the new major of o-colors. Updating to this new version will mean updating any other components that you have which are using o-colors. There are no other breaking changes in this release.

repository: Financial-Times/o-gallery

repo_name: Financial-Times/o-gallery


text:
Dart Utils
==========

A library for Dart developers.

Created from templates made available by Stagehand under a BSD-style
[license](https://github.com/dart-lang/stagehand/blob/master/LICENSE).

![Coverage](coverage_badge.svg)

## Usage

A simple usage example:

```dart
import 'package:dart_util/dart_util.dart';

main() {
}
```

## Features

* A number of pre-defined regular expressions for ease of reuse and correctness.


text:
<img src="steeriously.svg" width="400px" height="400px">

# Steeriously C++ Library

Steeriously. A dead-simple pure C++ library for adding steering behaviors to your game and multimedia entities.
All credit goes to Mat Buckland for writing "Programming AI by Example" and providing such great steering code.
All source is derived from "Programming AI by Example".
I can't recommend the book enough - you can buy the book here:
[Programming AI by Example](https://www.amazon.com/Programming-Example-Wordware-Developers-Library/dp/1556220782/ref=sr_1_1?ie=UTF8&qid=1512951593&sr=8-1&keywords=programming+ai+by+example)

Steeriously has one humble goal - to make steering behavior accessible and fun to program.
One way to achieve this is by redesigning the original code to expose a more general purpose API.
Tutorials and examples for using this library are available at:
[https://code.markrichards.ninja/cpp](https://code.markrichards.ninja/cpp)

## Features include components for the following steering behaviors:

* Seek
* Flee
* Arrive
* Evade
* Pursuit
* Offset Pursuit (formation)
* Wander
* Interpose
* Alignment, Separation, and Cohesion (for flocking, or "emergent" behavior)
* Hiding
* Path Following (not path finding...that would be like A*, these are precalculated paths)
* Obstacle Avoidance
* Wall Avoidance
* Zero dependencies

It is also possible to extend the library by creating new components using
lower-level features. All steering functionality is provided in templated functions,
so derive your new components from steer::Agent and combine functions to create
your own steering component. You will need to override the following functions:

`virtual bool on(steer::behaviorType behavior) = 0;`

`virtual steer::Vector2 Calculate() = 0;`

..and probably these too, but they are not required:

`virtual bool accumulateForce(steer::Vector2 &startingForce, steer::Vector2 forceToAdd);`

`virtual steer::Vector2 calculateWeightedSum();`

`virtual steer::Vector2 calculatePrioritized();`

...then implement an update function passing in your frame time:

`void Update(float dt);`

The components I implemented should give you what you need to implement your own. There
are certainly things you can do much differently than what is provided in the example
components.

More information, tutorials, and demos along with API documentation are available at [https://code.markrichards.ninja/cpp](https://code.markrichards.ninja/cpp).

Steeriously does not use a build file. However, you can simply place the files in your project and add
include paths accordingly using your favorite IDE. Documentation will need built with Doxygen - just issue the doxygen command on the steeriously.doxy file via command line.

license: Zlib

repository: pushbuttonreceivecode/Steeriously

repo_name: pushbuttonreceivecode/Steeriously


text:
# Budget Tracker Application

This mobile application helps to keep track of you daily budget goals.  Built with Ionic framework with Firebase for the backend database.  Ng-Cordova is added for future features like local storage.  I've added the Android platform to build native application.  You can add the iOS platform to tryout the application on your iPhones.  But I don't have any plans for building the app for iOS since I don't own an iPhone.

## Features
* User registration
* User Authentication
* Remote storage
* Graphic data charts

## Getting Started

Clone the repo and run npm install.  Type ionic serve to run the app in a browser.  Checkout the app through this link [Budget Tracker](https://msalas74.github.io/budgettracker/www/index.html#/)

## Built With

* Ionic - For the UI stuff
* Firebase - For the data stuff
* Vanilla JS - For the good stuff
* Gulp - To automate my build and stuff
* NodeJS - For the server side stuff
* NPM - To install all the dependencies...and stuff
* Sublime Text - Where it's all put together!
* Code Style - [![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg)](http://standardjs.com/)

## TODO List

* Create dummy log in account for users to test out the app.
* Modal to list items in a category. (Completed 7-31-16)
* Loader icons while app is retrieving data from server. (Completed 8-1-16)
* Upgrade Firebase from 2.2.4 to latest version.
* Add Test (Jasmine, Karma, and angular-mocks)
* Edit and Delete method. *Status 8-18-16* Delete method working.  Need to add Edit/Update method.
* Sorting list method.
* Establish method for saving data to local storage.
* Change color scheme.

## Authors

* **Matias Salas** - *Initial work* - [Website Portfolio](http://www.matiasis.me)

## License

This project is licensed under the ISC License - see the [LICENSE.md](LICENSE.md) file for details

license: ISC


text:
safename
========

Get safe file name from a given string.

![tests badge](https://travis-ci.org/jacoborus/safename.svg?branch=master)

## Example

```js
// require only in node/browserify
var safename = require( 'safename' );

safename('my file name.txt');
// => my_file_name.txt

safename('my file name.txt', '-');
// => my-file-name.txt

safename.low('my file name.txt');
// => my_file_name.txt

safename.middle('my file name.txt', '-');
// => my-file-name.txt

safename.dot('my file name.txt', '.');
// => my.file.name.txt
```

## Installation

Install with npm
```
$ npm install safename
```

Install with Bower
```
$ bower install safename
```

repository: pedroigor/keycloak

repo_name: pedroigor/keycloak


text:
# Google File System P1-5 notes

## Design Space

1. Component failures are the morm rather than the exception.
2. Files are huge by traditional standards. Multi-GB files are commmon.
3. Most files are mutated by appending new data rather than overwriting existing data.
4. Co-designing the applications and the file system API benifits the overall system by increasing our flexibility.
用中文总结一下就是，机器是普通机器，很容易出问题；文件会很大；以“附加”代替“覆写”，最后，面向应用设计系统，使得设计更加灵活。

## Design Assumptions

1. The System is built from many in expensive commodity components that may fail.
   * 这个和上面第一条相同，所以处理错误的能力对GFS非常重要。
2. The System stores a modest number of larger files.
   * 百万级的100MB文件，适量的Multi-GB文件，（我对文件大小其实没什么概念）
3. large streaming reads and small random reads。
    * [流式读取](https://groups.google.com/g/cs501pku/c/rVblGSxAUWE?pli=1)貌似对应的是顺序读取，与随机读取相对应。
4. The workloads also have many large sequential writes that append data to files.
5. The system must efficiently implement well-defined semantics for multiple clients that concurrently append to the same file.
6. Hight sustained bandwidth is more import tant than low latency.
   * 高带宽比低延迟重要，不懂，可以反应慢一些，但一旦建立连接就要非常稳定？

## Interface

* create
* delete
* open
* close
* read
* write

## Architecture

参照图一，一个master 用于管理数据调度，很多chunkservers 用于实际存储数据。
后面有提到，不会再master 上运行chunkserver 上的任务，避免master成为瓶颈。

## Consistency Model

* Guarantees by GFS / GFS
* Implication for Applications

repository: chasedreamsd/CruelSystem

repo_name: chasedreamsd/CruelSystem


text:
# Employee-Tracker

![Language](https://img.shields.io/github/languages/top/jiji-smith/Employee-Tracker/app.js)
## Title
Employee Tracker
## Description
You can track employee information here
## Table of Contents
* [Title](#title)
* [Description](#description)
* [Table of Contents](#table-of-contents)
* [Usage](#usage)
* [License](#license)
* [Contributing](#contributing)
* [Video](#video)
* [Tests](#tests)
* [Questions](#questions)

## Installation
Please check https://github.com/jiji-smith/Employee-Tracker
## Usage
Check the code, install npm, answer the questions!
## License
MIT License
## Contributing
Jiiiii
## Video
[Video Walkthrough: https://youtu.be/t04bJVhZMJg](https://youtu.be/t04bJVhZMJg)
## Tests

## Questions
Nope

repository: jiji-smith/Employee-Tracker

repo_name: jiji-smith/Employee-Tracker