text:
---
title: Sorting
page_title: Sorting
description: Check our "Sorting" documentation article for the RadPivotGrid {{ site.framework_name }} control.
slug: radpivotgrid-features-queryable-sorting
tags: sorting
published: True
position: 1
site_name: WPF
---

# Sorting

In this article you will find out how to sort your data based on the header names or the totals.

## Understanding Sorting Mechanism

__RadPivotGrid__ provides different sorting capabilities. You can sort your *RowGroupDescriptions* and *ColumnGroupDescriptions* ascending (from A to Z) and descending (from Z to A) based on the names of the properties they are showing or based on the aggregated description.

With the __Q1 2014 SP1__ version of __Telerik UI for__ __RadPivotGrid__ provides an additional option. The data can be shown the way it was received from the data source using the __SortOrder__ property set to __None__.

We will explain this with a simple __RadPivotGrid__:

![Rad Pivot Grid Features Queryable Sorting 01](images/RadPivotGrid_Features_Queryable_Sorting_01.png)

The __ColumnGroupDescription__ in this __RadPivotGrid__ is the *OrderDate*. By default the sorting of the columns is ascending (from A to Z) based on the headers: *1996*, *1997*, *1998*. You can easily change the sorting of the *OrderDate* ColumnGroupDescription and set it to descending (from Z to A). This will lead to the following result:

![Rad Pivot Grid Features Queryable Sorting 02](images/RadPivotGrid_Features_Queryable_Sorting_02.png)

The columns are reordered based on their names (check the green rectangle on the top of the image). You can apply sorting based on some of the __QueryablePropertyAggregateDescription__ you are using. In the example above you can apply sorting of the *OrderDate* ColumnGroupDescription in ascending (from A to Z) or descending (from Z to A) format based on the *Total Sum of Freight* (red rectangle) or *Total Sum of ShipVia* (blue rectangle). Here is the new view of __RadPivotGrid__ when the *OrderDate* ColumnGroupDescription is sorted in descending format based on the *Total Sum of Freight*:

![Rad Pivot Grid Features Queryable Sorting 03](images/RadPivotGrid_Features_Queryable_Sorting_03.png)

>importantThe sorting is set on the __ColumnGroupDescriptions__ or __RowGroupDescriptions__. You cannot set sorting on the __AggregateDescriptions__, but you can sort the columns or rows based on the aggregated values.

## Sort the data

The sorting can be applied in the __XAML__, in the code behind or even at runtime with the help of __RadPivotFieldList__.

* Sorting based on the GroupName (header) can be *Ascending (from A to Z)* or *Descending (from Z to A*). The default value is *Ascending*, but you can change it by using __SortOrder__ property:

#### __XAML__

{{region radpivotgrid-features-queryable-sort_1}}
	<pivot:QueryablePropertyGroupDescription PropertyName="ShipCountry" SortOrder="Ascending"/>
	<pivot:QueryableDoubleGroupDescription PropertyName="Freight" SortOrder="Descending"/>
	<pivot:QueryableDateTimeGroupDescription PropertyName="OrderDate" SortOrder="Ascending"/>
{{endregion}}

#### __C#__

{{region radpivotgrid-features-queryable-sort_1}}
	var shipCountryGroupDescription = new QueryablePropertyGroupDescription();
	shipCountryGroupDescription.PropertyName = "ShipCountry";
	shipCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending;

	var freightCountryGroupDescription = new QueryableDoubleGroupDescription();
	freightCountryGroupDescription.PropertyName = "ShipCountry";
	freightCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Descending;

	var orderDateCountryGroupDescription = new QueryableDateTimeGroupDescription();
	orderDateCountryGroupDescription.PropertyName = "OrderDate";
	orderDateCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending;
{{endregion}}

#### __VB.NET__

{{region radpivotgrid-features-queryable-sort_1}}
	Dim shipCountryGroupDescription = New QueryablePropertyGroupDescription()
	shipCountryGroupDescription.PropertyName = "ShipCountry"
	shipCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending

	Dim freightCountryGroupDescription = New QueryableDoubleGroupDescription()
	freightCountryGroupDescription.PropertyName = "ShipCountry"
	freightCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Descending

	Dim orderDateCountryGroupDescription = New QueryableDateTimeGroupDescription()
	orderDateCountryGroupDescription.PropertyName = "OrderDate"
	orderDateCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending
{{endregion}}

* If you need to avoid the data to be sorted anyhow, you can set the __SortOrder__ property to __None__. This way the data will appear the way it is was received from the data source:

#### __XAML__

{{region radpivotgrid-features-queryable-sort_3}}
	<pivot:QueryablePropertyGroupDescription PropertyName="ShipCountry" SortOrder="None"/>
	<pivot:QueryableDoubleGroupDescription PropertyName="Freight" SortOrder="None"/>
	<pivot:QueryableDateTimeGroupDescription PropertyName="OrderDate" SortOrder="None"/>
{{endregion}}

#### __C#__

{{region radpivotgrid-features-queryable-sort_3}}
	var shipCountryGroupDescription = new QueryablePropertyGroupDescription();
	shipCountryGroupDescription.PropertyName = "ShipCountry";
	shipCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.None;

	var freightCountryGroupDescription = new QueryableDoubleGroupDescription();
	freightCountryGroupDescription.PropertyName = "ShipCountry";
	freightCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.None;

	var orderDateCountryGroupDescription = new QueryableDateTimeGroupDescription();
	orderDateCountryGroupDescription.PropertyName = "OrderDate";
	orderDateCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.None;
{{endregion}}

#### __VB.NET__

{{region radpivotgrid-features-queryable-sort_3}}
	Dim shipCountryGroupDescription = New QueryablePropertyGroupDescription()
	shipCountryGroupDescription.PropertyName = "ShipCountry"
	shipCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.None

	Dim freightCountryGroupDescription = New QueryableDoubleGroupDescription()
	freightCountryGroupDescription.PropertyName = "ShipCountry"
	freightCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.None

	Dim orderDateCountryGroupDescription = New QueryableDateTimeGroupDescription()
	orderDateCountryGroupDescription.PropertyName = "OrderDate"
	orderDateCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.None
{{endregion}}

>Note that the __SortOrder None__ property is available with the __Q1 2014 SP1__ version of Telerik UI for {% if site.site_name == 'WPF' %}WPF{% endif %}{% if site.site_name == 'Silverlight' %}Silverlight{% endif %}.

* Sorting based on the GrandTotals can be *Ascending* or *Descending*. To set such sort mechanism you have to use the __SortOrder__ and __GroupComparer__ properties:

#### __XAML__

{{region radpivotgrid-features-queryable-sort_2}}
	<pivot:QueryablePropertyGroupDescription PropertyName="ShipCountry" SortOrder="Ascending">
	    <pivot:QueryablePropertyGroupDescription.GroupComparer>
	        <pivot:GrandTotalComparer AggregateIndex="0"/>
	    </pivot:QueryablePropertyGroupDescription.GroupComparer>
	</pivot:QueryablePropertyGroupDescription>
	<pivot:QueryableDoubleGroupDescription PropertyName="Freight" SortOrder="Descending">
	    <pivot:QueryablePropertyGroupDescription.GroupComparer>
	        <pivot:GrandTotalComparer AggregateIndex="1"/>
	    </pivot:QueryablePropertyGroupDescription.GroupComparer>
	</pivot:QueryableDoubleGroupDescription>
	<pivot:QueryableDateTimeGroupDescription PropertyName="OrderDate" SortOrder="Ascending">
	    <pivot:QueryablePropertyGroupDescription.GroupComparer>
	        <pivot:GrandTotalComparer AggregateIndex="0"/>
	    </pivot:QueryablePropertyGroupDescription.GroupComparer>
	</pivot:QueryableDateTimeGroupDescription>
{{endregion}}

#### __C#__

{{region radpivotgrid-features-queryable-sort_2}}
	var shipCountryGroupDescription = new QueryablePropertyGroupDescription();
	shipCountryGroupDescription.PropertyName = "ShipCountry";
	shipCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending;
	shipCountryGroupDescription.GroupComparer = new GrandTotalComparer() { AggregateIndex = 0 };

	var freightCountryGroupDescription = new QueryableDoubleGroupDescription();
	freightCountryGroupDescription.PropertyName = "Freight";
	freightCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Descending;
	freightCountryGroupDescription.GroupComparer = new GrandTotalComparer() { AggregateIndex = 1 };

	var orderDateCountryGroupDescription = new QueryableDateTimeGroupDescription();
	orderDateCountryGroupDescription.PropertyName = "OrderDate";
	orderDateCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending;
	orderDateCountryGroupDescription.GroupComparer = new GrandTotalComparer() { AggregateIndex = 0 };
{{endregion}}

#### __VB.NET__

{{region radpivotgrid-features-queryable-sort_2}}
	Dim shipCountryGroupDescription = New QueryablePropertyGroupDescription()
	shipCountryGroupDescription.PropertyName = "ShipCountry"
	shipCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending
	shipCountryGroupDescription.GroupComparer = New GrandTotalComparer() With {.AggregateIndex = 0}

	Dim freightCountryGroupDescription = New QueryableDoubleGroupDescription()
	freightCountryGroupDescription.PropertyName = "Freight"
	freightCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Descending
	freightCountryGroupDescription.GroupComparer = New GrandTotalComparer() With {.AggregateIndex = 1}

	Dim orderDateCountryGroupDescription = New QueryableDateTimeGroupDescription()
	orderDateCountryGroupDescription.PropertyName = "OrderDate"
	orderDateCountryGroupDescription.SortOrder = Telerik.Pivot.Core.SortOrder.Ascending
	orderDateCountryGroupDescription.GroupComparer = New GrandTotalComparer() With {.AggregateIndex = 0}
{{endregion}}

>The __AggregateIndex__ property is set based on the count of your aggregate descriptions. If you have two aggregates the first one will have __AggregateIndex = 0__ and the second - __AggregateIndex = 1__.

## Changing the Sorting at runtime

__RadPivotFieldList__ gives you the ability to change the sorting of a description at runtime. When you click on RowGroupDescription or ColumnGroupDescription a new popup opens with several sorting options. If you want to save the current sorting, but only change its order (for example the above __RadPivotGrid__ is sorted based on the Total Sum of Quantity in Descending order, and you want to change it to Ascending) you can use the first two options which are doing exactly this - just changing the __SortOrder__:

![Rad Pivot Grid Features Sorting 04](images/RadPivotGrid_Features_Sorting_04.png)

If you want to change the property based on which the sort is applied, then you can use the third option - More Sorting Options. A new RadWindow will be opened in which you can choose Ascending or Descending order of the sort. When you click the dropdown button you'll see a full list of the possible sorting basis:

![Rad Pivot Grid Features Queryable Sorting 06](images/RadPivotGrid_Features_Queryable_Sorting_06.png)

## See Also

 * [Getting Started]({%slug radpivotgrid-getting-started%})

 * [RadPivotFieldList]({%slug radpivotgrid-fieldlist%})

 * [QueryableDataProvider]({%slug radpivotgrid-data-queryable%})

 * [Populating with Data]({%slug radpivotgrid-populating-with-data%})

 * [QueryableDataProvider Overview]({%slug radpivotgrid-features-queryable-overview%})

repository: telerik/xaml-docs

gha_license_id: NOASSERTION

repo_name: telerik/xaml-docs


text:
# elementary-dark-plank
A theme for plank attempting to emulate Elementary OS's dark theme

## Installation

To install the theme, download and place the folder in ~/.local/share/plank/themes/

## Screenshots

license: MIT

repository: ChildishGiant/elementary-dark-plank

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: ChildishGiant/elementary-dark-plank


text:
formvalidation.js[![Build Status](https://api.travis-ci.org/chemerisuk/formvalidation.js.png?branch=master)](http://travis-ci.org/chemerisuk/formvalidation.js)
=================
HTML5 forms polyfill

Features
--------
* no additional dependencies
* doesn't require initialization call on each form
* every UI element and text content is fully customizable via CSS
* displays calendar control for `input[type="date"]`
* internationalization support

Example usage
-------------
All you need is to include several additional files on your page:
```html
<html>
  <head>
    ...
    <link href="formvalidation.css" rel="stylesheet"/>
    ...
  </head>
  <body>
    ...
    <script src="formvalidation.js"></script>
  </body>
</html>
```
No initialization is required, it just works. And even for dynamic content. So it's perfect for single-page websites with ajax navigation.

JavaScript API
--------------
Plugin uses standards-based javascript API to display validation messages programmatically. So you may use any source which describes HTML5 Constraint Validation API. For example look at the appropriate section on https://developer.mozilla.org/en-US/docs/HTML/Forms_in_HTML.

Internationalization
--------------------
All message strings are stored in css. To display them `data-i18n` attribute is used with combination of `:before` pseudoelement:
```css
[data-i18n="validity.value.missing"]:before { content: "Please fill this field" }
[data-i18n="validity.type.mismatch"]:before { content: "Value has illegal format" }
[data-i18n="validity.email.mismatch"]:before { content: "Should be a valid email" }
```
`:lang` selector allows to target specific language and localize strings for it:
```css
[data-i18n="validity.value.missing"]:lang(ru):before { content: "Это поле не может быть пустым" }
[data-i18n="validity.type.mismatch"]:lang(ru):before { content: "Введенное значение имеет недопустимый формат" }
[data-i18n="validity.email.mismatch"]:lang(ru):before { content: "Здесь должен быть правильный email" }
```
Therefore you need an appropriate `lang` attribute value for the `<html>` element to change plugin language.

TODOs
-----
1. add `input[type=number]` support with spin buttons
2. IE8 support?

Browser support
---------------
* Chrome
* Firefox
* Opera
* IE9+

It's possible to add IE8 support, but not IE6-7 (these dinosaurs do not support DOM object prototypes)

repository: aghed/bloom-private

repo_name: aghed/bloom-private


text:
This plugin will also forward in and load your ssh key so you can pull code that is key-protected.

## Installation

You can install this plugin by going into your app directory and running the normal

```
npm install kalabox-plugin-git --save
```

In order for your app to use the plugin you will also need to inform the app of its existence. This can be done in the `kalabox.json` file in your app root. Just add the plugin name to the `appPlugins` key.

```json
{
  "appName": "pressflow7",
  "appPlugins": [
    "my-hot-plugin",
    "kalabox-plugin-pressflow7-env",
    "kalabox-plugin-git"
  ],
}

```

## Usage

Run any git command you normally would but start it with `kbox`. Run it from the directory that contains the app you want to run it against or pass in the appname.

Examples

```
# Returns the version of git, must run from a directory that contains a kalabox app
kbox git version

# Clones a repo into /code for a D7 app
kbox pressflow7 git clone http://github.com/kalamuna/playbox.git ./

# Pulls down the latest code, must run from a directory that contains a kalabox app
kbox git pull
```

## Configuration

In your apps kalabox.json add the following to the pluginConf key to set the username and email that will be applied to your git commits

```json
    "kalabox-plugin-git": {
      "git-username": "William T. Riker",
      "git-email"<EMAIL_ADDRESS>    },
```

## Other Resources

* [API docs](http://api.kalabox.me/)
* [Test coverage reports](http://coverage.kalabox.me/)
* [Kalabox CI dash](http://ci.kalabox.me/)
* [Mountain climbing advice](https://www.youtube.com/watch?v=tkBVDh7my9Q)
* [Boot2Docker](https://github.com/boot2docker/boot2docker)
* [Syncthing](https://github.com/syncthing/syncthing)
* [Docker](https://github.com/docker/docker)

-------------------------------------------------------------------------------------
(C) 2015 Kalamuna and friends

repository: ManatiCR/kalabox-app-pantheon

gha_language: JavaScript

repo_name: ManatiCR/kalabox-app-pantheon


text:
# Deta Base Example

An example using [Deta Base](https://docs.deta.sh/docs/base/about) in a Next.js project.

## Deploy your own

Once you have access to [the environment variables you'll need](#step-2-setting-up-environment-variables), deploy the example using [Vercel](https://vercel.com?utm_source=github&utm_medium=readme&utm_campaign=next-example):

[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/vercel/next.js/tree/canary/examples/with-deta-base&project-name=with-deta-base&repository-name=with-deta-base&env=DETA_PROJECT_KEY&envDescription=The%20Deta%20Project%20Key%2C%20found%20in%20the%20Deta%20dashboard&envLink=https://github.com/vercel/next.js/tree/canary/examples/with-deta-base%23configuration)

## How to use

Execute [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app) with [npm](https://docs.npmjs.com/cli/init), [Yarn](https://yarnpkg.com/lang/en/docs/cli/create/), or [pnpm](https://pnpm.io) to bootstrap the example:

```bash
npx create-next-app --example with-deta-base with-deta-base-app
```

```bash
yarn create next-app --example with-deta-base with-deta-base-app
```

```bash
pnpm create next-app --example with-deta-base with-deta-base-app
```

## Configuration

### Step 1. Create a Deta Account

Create an account on [Deta](https://www.deta.sh/?ref=next.js). Save the default _Project Key_ which will be auto-generated on account creation.

### Step 2. Setting Up Environment Variables

Copy the `.env.local.example` file from this directory to `.env.local` (which will be ignored by Git):

```bash
cp .env.local.example .env.local
```

Then set each variable on `.env.local`:

- `DETA_PROJECT_KEY` should be the default _Project Key_ that you saved from step 1.

The resulting `env.local` file should look like this:

```bash
DETA_PROJECT_KEY=...
```

### Step 3. Run Next.js in development mode

```bash
npm install
npm run dev

# or

yarn install
yarn dev
```

Your todo app should be up and running on [http://localhost:3000](http://localhost:3000)! If it doesn't work, post on [GitHub discussions](https://github.com/vercel/next.js/discussions).

### Step 4. Deploy on Vercel

You can deploy this app to the cloud with [Vercel](https://vercel.com?utm_source=github&utm_medium=readme&utm_campaign=next-example) ([Documentation](https://nextjs.org/docs/deployment)).

#### Deploy Your Local Project

To deploy your local project to Vercel, push it to GitHub/GitLab/Bitbucket and [import to Vercel](https://vercel.com/new?utm_source=github&utm_medium=readme&utm_campaign=next-example).

**Important**: When you import your project on Vercel, make sure to click on **Environment Variables** and set them to match your `.env.local` file.

#### Deploy from Our Template

Alternatively, you can deploy using our template by clicking on the Deploy button below.

[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/vercel/next.js/tree/canary/examples/with-deta-base&project-name=with-deta-base&repository-name=with-deta-base&env=DETA_PROJECT_KEY&envDescription=The%20Deta%20Project%20Key%2C%20found%20in%20the%20Deta%20dashboard&envLink=https://github.com/vercel/next.js/tree/canary/examples/with-deta-base%23configuration)

branch_name: refs/heads/canary

gha_license_id: MIT


text:
# Employee CMS
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Description
A command-line CMS that allows users to add, update and remove entries about employees.
Runs on node.js and uses the mysql2 package for database logic.

## Contents
- [Installation](#installation)
- [Usage](#usage)
- [Credits](#credits)
- [License](#license)

## Installation
Node.js and mySQL are required to run this program:
- [Node.js](https://nodejs.org/)
- [MySQL](https://dev.mysql.com/downloads/installer/)

After both of these are installed, you can download/clone the project. In a command line interface, navigate to the root folder and use the command `npm i` to install all necessary dependencies.

## Usage
In `main.js`, the variable `db` contains the connection to a database. You'll need to enter your own credientials to connect to a specific database (or just change the password if it's on localhost).

Afterwards, use the command `npm start` to begin the program. You'll be given a list of options that can be used to manipulate the database, such as Add Employee, Remove Employee, and Update Employee.
Employee entires are built using existing roles and departments--you'll probably want to add some of each before building an Employee entry.
Simply follow the on screen intructions to enter data.

A demonstration can be seen [here](https://youtu.be/Ho6vi73GNDI).

### Limitations
Currently, if you remove a department that a customer is a part it will be replaced with null and cannot be updated. An update function is in development, but this project is pretty low priority and is more for demonstrative purposes (which is also why the login process is not streamlined!).

## Questions
If you have any questions about using or contibuting to the project, you can contact me via email or though github:
- Email<EMAIL_ADDRESS>- Github: [@jkohrt7](https://github.com/jkohrt7)

## Credits
- [jkohrt7 ](https://www.github.com/jkohrt7)
- [inquirer ](https://www.npmjs.com/package/inquirer)
- [mySQL2 ](https://www.npmjs.com/package/mysql2)

## License
[MIT](https://opensource.org/licenses/MIT)

branch_name: refs/heads/main


text:
# We have terminated the backend for this project.  Old Reshuffle projects can no longer be deployed.

# Simple Landing Page

Basic boilerplate for creating an app landing page. Visit /admin to manage the emails collected.
This project implemented with Reshuffle, Handlebars, REST API, gulp and bootstrap.

## Screenshot

### Home Page

### Admin Page

## Running locally

To run your express server locally run:

```bash
npm install
npm start
```

## Running remotely

Deploy your app to the Reshuffle cloud to run it remotely. Simply run:

```bash
npx reshuffle deploy
```

from the projects root directory

## Credits

This project was inspired from [startbootstrap-coming-soon](https://github.com/BlackrockDigital/startbootstrap-coming-soon.git)

repository: reshufflehq/simple-product-landing-page

gha_language: HTML

repo_name: reshufflehq/simple-product-landing-page


text:
# Version Discrepency Example

This example demos two federated apps that use two different versions of lodash.

- `app1` uses<EMAIL_ADDRESS>- `app2` uses<EMAIL_ADDRESS>and `lodash.nth`--a feature not available in<EMAIL_ADDRESS>
# Running Demo
Run `yarn`
Run `yarn start`.
This will build and serve both `app1` and `app2` on ports 3001 and 3002 respectively.

- [localhost:3001](http://localhost:3001/)
- [localhost:3002](http://localhost:3002/)
  <img src="https://ssl.google-analytics.com/collect?v=1&t=event&ec=email&ea=open&t=event&tid=UA-120967034-1&z=1589682154&cid=ae045149-9d17-0367-bbb0-11c41d92b411&dt=ModuleFederationExamples&dp=/email/VersionDiscrepency">

# Running Cypress E2E Tests

To run tests in interactive mode, run  `npm run cypress:debug` from the root directory of the project. It will open Cypress Test Runner and allow to run tests in interactive mode. [More info about "How to run tests"](../../cypress/README.md#how-to-run-tests)

To build app and run test in headless mode, run `yarn e2e:ci`. It will build app and run tests for this workspace in headless mode. If tets failed cypress will create `cypress` directory in sample root folder with screenshots and videos.

["Best Practices, Rules amd more interesting information here](../../cypress/README.md)

gha_language: TypeScript


text:
# Pony Tutorial

Welcome to the Pony tutorial! If you're reading this, chances are you want to
learn Pony. That's great, we're going to make that happen.

This tutorial is aimed at people who have some experience programming already.
It doesn't really matter if you know a little Python, or a bit of Ruby, or you
are a JavaScript hacker, or Java, or Scala, or C/C++, or Haskell, or OCaml: as
long as you've written some code before, you should be fine.

## What's Pony, anyway?

Pony is an object-oriented, actor-model, capabilities-secure programming
language. It's __object-oriented__ because it has classes and objects, like
Python, Java, C++, and many other languages. It's __actor-model__ because it
has _actors_ (similar to Erlang or Akka). These behave like objects, but they
can also execute code _asynchronously_. Actors make Pony awesome.

When we say Pony is __capabilities-secure__, we mean a few things:

* It's type safe. Really type safe. There's a mathematical proof and everything.
* It's memory safe. Ok, this comes with type safe, but it's still interesting.
There are no dangling pointers, no buffer overruns, heck, the language doesn't
even have the concept of _null_!
* It's exception safe. There are no runtime exceptions. All exceptions have
defined semantics, and they are _always_ handled.
* It's data-race free. Pony doesn't have locks or atomic operations or anything
like that. Instead, the type system ensures _at compile time_ that your
concurrent program can never have data races. So you can write highly
concurrent code and never get it wrong.
* It's deadlock free. This one is easy, because Pony has no locks at all! So
they definitely don't deadlock, because they don't exist.

We'll talk more about capabilities-security, including both __object
capabilities__ and __reference capabilities__ later.

## The Pony Philosophy: Get Stuff Done

In the spirit of [Richard Gabriel](http://www.jwz.org/doc/worse-is-better.html),
the Pony philosophy is neither "the-right-thing" nor "worse-is-better". It is
"get-stuff-done".

* Correctness. Incorrectness is simply not allowed. _It's pointless to try to
get stuff done if you can't guarantee the result is correct._

* Performance. Runtime speed is more important than everything except
correctness. If performance must be sacrificed for correctness, try to come up
with a new way to do things. _The faster the program can get stuff done, the
better. This is more important than anything except a correct result._

* Simplicity. Simplicity can be sacrificed for performance. It is more
important for the interface to be simple than the implementation. _The faster
the programmer can get stuff done, the better. It's ok to make things a bit
harder on the programmer to improve performance, but it's more important to
make things easier on the programmer than it is to make things easier on the
language/runtime._

* Consistency. Consistency can be sacrificed for simplicity or performance.
_Don't let excessive consistency get in the way of getting stuff done._

* Completeness. It's nice to cover as many things as possible, but completeness
can be sacrificed for anything else. _It's better to get some stuff done now
than wait until everything can get done later._

The "get-stuff-done" approach has the same attitude towards correctness and
simplicity as "the-right-thing", but the same attitude towards consistency and
completeness as "worse-is-better". It also adds performance as a new principle,
treating it as the second most important thing (after correctness).

## Guiding Principles

Throughout the design and development of the language the following principles
should be adhered to.

* Use the get-stuff-done approach.

* Simple grammar. Language must be trivial to parse for both humans and
computers.

* No loadable code. Everything is known to the compiler.

* Fully type safe. There is no "trust me, I know what I'm doing" coercion.

* Fully memory safe. There is no "this random number is really a pointer,
honest."

* No crashes. A program that compiles should never crash (although it may hang
or do something unintended).

* Sensible error messages. Where possible use simple error messages for
specific error cases. It is fine to assume the programmer knows the definitions
of words in our lexicon, but avoid compiler or other computer science jargon.

* Inherent build system. No separate applications required to configure or
build.

* Aim to reduce common programming bugs through the use of restrictive syntax.

* Provide a single, clean and clear way to do things rather than catering to
every programmer's preferred prejudices.

* Make upgrades clean. Do not try to merge new features with the ones they are
replacing, if something is broken remove it and replace it in one go. Where
possible provide rewrite utilities to upgrade source between language versions.

* Reasonable build time. Keeping down build time is important, but less
important than runtime performance and correctness.

* Allowing the programmer to omit some things from the code (default arguments,
type inference, etc) is fine, but fully specifying should always be allowed.

* No ambiguity. The programmer should never have to guess what the compiler
will do, or vice-versa.

* Document required complexity. Not all language features have to be trivial to
understand, but complex features must have full explanations in the docs to be
allowed in the language.

* Language features should be minimally intrusive when not used.

* Fully defined semantics. The semantics of all language features must be
available in the standard language docs. It is not acceptable to leave
behaviour undefined or "implementation dependent".

* Efficient hardware access must be available, but this does not have to
pervade the whole language.

* The standard library should be implemented in Pony.

* Interoperability. Must be interoperable with other languages, but this may
require a shim layer if non primitive types are used.

* Avoid library pain. Use of 3rd party Pony libraries should be as easy as
possible, with no surprises. This includes writing and distributing libraries
and using multiple versions of a library in a single program.

## More help

Working your way through the tutorial but in need of more help? Not to worry,
we have you covered.

If you are looking for an answer "right now", we suggest you give our IRC
channel a try. It's #ponylang on Freenode. If you ask a question, be sure to
hang around until you get an answer. If you don't get one, or IRC isn't your
thing, we have a friendly [mailing list](https://groups.io/g/pony+user) you can
try. Whatever your question is, it isn't dumb, and we won't get annoyed.

Think you've found a bug? Check your understanding first by writing the mailing
list. Once you know it's a bug,
[open an issue](https://github.com/ponylang/ponyc/issues).

# Help us

Found a typo in this tutorial? Perhaps something isn't clear? We welcome pull
requests against the tutorial:
[https://github.com/ponylang/pony-tutorial](https://github.com/ponylang/pony-tutorial).
Be sure to check out the
[contribution guidelines](https://github.com/ponylang/pony-tutorial/blob/master/CONTRIBUTING.md)
before opening your PR.

repository: nerdatmath/pony-tutorial

repo_name: nerdatmath/pony-tutorial


text:
---
title: "UBC Computer Vision Lab - Research"
layout: textlay
excerpt: "UBC Computer Vision Lab -- Research"
sitemap: false
permalink: /research/
---

# Research
Computer Vision group at UBC focuses on developing and building state-of-the-art algorithms and prototypes for visually intelligent systems. We tackle a broad spectrum of problems, from low-level vision to high-level visual semantic understanding. Applications are equally broad, spanning robotics, sport analytics, health-care, visual media curation and annotation.
{: .text-justify}

![]({{site.url}}{{site.baseurl}}/images/respic/crv16_1.png){: style="width: 300px; float: right; border: 10px"}

### Sport Analytics
Sports video analysis offers an opportunity to address a wide range of vision tasks, from tracking to re-identification to action recognition to understanding cooperative and competitive actions of groups of team members.  By tackling this wide range of problem in the constrained context of sports (hockey, soccer, football) we can make progress on the sub-problems (such as pose and action recognition) that are valuable in more general settings such as smart homes.
{: .text-justify}

### Human Pose Estimation and Activity Recognition
Ability to recover human pose and understand human behavior is a very difficult problem that dates back to the beginnings of Computer Vision. AI-driven automated systems will eventually need to interact with humans and require an ability to reason about and interpret our subtle behaviors. Such algorithms are also critical for variety of important applications, including in gaming, security, analytics, virtual retail and many others. The topics we are working on in this space are: person detection and tracking, human pose estimation, activity recognition, situation recognition, even recognition.
{: .text-justify}
<!--
![]({{ site.url }}{{ site.baseurl }}/images/respic/layers_fft.jpg){: style="width: 300px; float: left; border: 10px"}
-->

### Vision and Language
As humans, our main modality of perception is vision, and for communication — language. These two modalities are closely tied and enable our ability to navigate and engage in complex tasks. With the progress of AI, which has lead to substantial advances in both visual and natural language processing, we are looking to move from learning from a single modality like vision, language or sound, to building integrated autonomous algorithms and systems that can interact and learn from humans through language. The topics we are working on in this space are: image and video captioning, visual question answering and dialog systems, visual language grounding, alignment of text and video, natural language-based image/video search and retrieval, and zero-shot learning.
{: .text-justify}

### Visual Imagination (Generative) Models
Humans have amazing ability to be able to hypothesize and create visual depictions of described scenes or objects from novel views. This imagination or control hallucination is, at least in part, responsible for transference and our ability to efficiently learn concepts for few instantiations. With the progress of deep generative models, such as Variational Auto Encoders and Generative Adversarial Networks, we are starting to strive towards similar successes from automated visual systems. The topics we are working on in this space are: disentangled representations, image-to-image translation, generating realistic visual depictions from linguistic or layout description, video generation.
{: .text-justify}

branch_name: refs/heads/gh-pages


text:
## vagrant-workflow-software

Vagrant manifest to set up a VirtualBox instance for developing and
testing the deployment of workflow software for compute clusters and
workflow servers. See
[https://wiki.apidb.org/index.php/PreparingClusters](https://wiki.apidb.org/index.php/PreparingClusters)
for specifications.

### Requirements

__Software__

- Vagrant
- VirtualBox

### Usage

    git clone git@github.com:EuPathDB/vagrant-workflow-software.git
    cd vagrant-workflow-software
    vagrant up

### Vagrantfile

The `Vagrantfile` specifies a dictionary at the top.

    WF_SERVERS = {
      :el6 => {
        :vagrant_box     => 'puppetlabs/centos-6.6-64-nocm',
        :wf_hostname     => 'consign.vm',
        :wf_eupath_dir   => '/project/eupathdblab',
        :wf_user         => 'debbie',
        :wf_shared_group => 'eupathdblab',
      },
      :el7 => {
        :vagrant_box     => 'ebrc/centos-7-64-puppet',
        :wf_hostname     => 'elm.vm',
        :wf_eupath_dir   => '/eupath',
        :wf_user         => 'debbie',
        :wf_shared_group => 'eupa',
      },
    }

This allows defining properties for multiple Vagrant boxes. The above
example will set up CentOS 6 and CentOS 7 environments on two virtual
machines. These will be named according the top key values (`el6` and
`el7` in this example). These names will be used with `vagrant ssh` to
identify which managed machine to connect to.

`wf_hostname` can be given the name of an existing cluster. This permits
using an existing Puppet node manifest and leverages existing profile
configurations (e.g. `$eupath_dir/etc/bashrc`) that use hostname
conditionals. Note that different compute clusters use various OS
versions (e.g. CentOS 6 and CentOS 7). Be sure the `vagrant_box` in the
`WF_SERVERS` dictionary is configured for a Vagrant box with a matching
OS for the chosen `wf_hostname`. For example, the physical sapelo in
the datacenter uses CentOS 6 so be sure the virtual machine simulation
also uses CentOS 6.

`wf_user_path` should match `user_path` in the Puppet node manifest for
the host.

`wf_shared_group` should match `shared_group` in the Puppet node
manifest for the host.

`wf_user` can be any name. An account of this name will be created on
the virtual machine and its shell environment will be configured as a
typical workflow user. This account is just for testing on the virtual
machines, there is no corresponding value in the Puppet manifests.

### Simulating a workflow user

The vagrant box includes the account `debbie` whose `.bashrc` matches
that of a typical workflow user. Use this account to test running the
workflow software and ensure that the environment is set up correctly.
The `.bashrc` is on the guest VM is provisioned by Ansible from
`ansible/files/workflow_bashrc`. This file and the
[WorkflowSoftware wiki instructions](https://wiki.apidb.org/index.php/WorkflowSoftware)
should be kept in sync if you need to make changes.

To log in as `debbie`, run

    vagrant ssh el7 -- -l debbie

where `el7` is Vagrant box you want to connect to (as defined in the
Vagrantfile's `WF_SERVER` dictionary).

### Installing and Updating workflow software

The "sysadmin" of the workflow software infrastructure is the user
`vagrant`. This user should be used for updating the workflow software
and changing the deployment infrastructure. The `workpuppet` script is
used for routine updates. See the wiki for details.

The `workpuppet` script (see the [PreparingClusters
wiki](https://wiki.apidb.org/index.php/PreparingClusters)) is run on
each invocation of `vagrant
provision`. In this case `workpuppet` is run by Ansible which eats
stdout, so it can be difficult to troubleshoot problems. However, the
stderr and stdout is recorded in `~vagrant/workpuppet.log`, so that may
be of some use.

Alternatively you can ssh to the virtual machine and run the command
manually. This option allow you to monitor progress through stdout and
more closely simulates how cluster software is updated in the wild.

    vagrant ssh el7 -c 'source $admin_path/bashrc; workpuppet'

where `el7` is Vagrant box you want to connect to (as defined in the
Vagrantfile's `WF_SERVER` dictionary).

Interactive changes and gefingerpoken can, of course, also be done from
the `vagrant` shell. To log in as `vagrant`, run

    vagrant ssh el7

where `el7` is Vagrant box you want to connect to (as defined in the
Vagrantfile's `WF_SERVER` dictionary).

_Caution: When screwing around in terminal windows be very careful which
window you are working in. Because the virtual machine may have the same
name as a real server, and because you may have terminals open to both,
you want to be sure to keep track of the difference, lest you
accidentally do development changes on the real server. The user shells
on the virtual machine are configured with a distinctive command prompt
to help disambiguate virtual from real._

__The Development Environment__

Git cloning from git.apidb.org requires ssh key authentication. The
`Vagrantfile` specifies `config.ssh.forward_agent = true` to use your
host ssh agent - so you need an agent running on your Vagrant host.

The `init.sh` script that does the initial provisioning assumes a VM
that mounts the Vagrant project directory from host to `/vagrant` on the
guest. The `puppetlabs/centos-6.6-64-nocm` specified in the
`Vagrantfile` satisfies this requirement.

One of the temptations of having a virtualized development environment
is working off-campus or offline. Be aware that the provisioning and
many other operations require network access to resources behind UGA
firewalls and/or expect clients originating from trusted IP addresses.
If you don't meet expected network conditions, silent or cryptic
failures await you. If you work remotely, fire up UGA's VPN and you
should be OK.

### Manual Package Deployment

If you have a newly made RPM to test before adding to the YUM
repository, you can copy the RPM to the virtual machine and install it
manually.

A `wfpkginstall` script is included in the vagrant user's home `bin`
dir. This script can be used to install a single package without going
through Puppet and YUM repository. Manually place the desired rpm file
into `$admin_path/yum-workflow/<os_major_version>/x86_64/` and then run
the `wfpkginstall` script with the package name (not file name). This
simulates what Puppet will do, including processing the
`MANIFEST.EUPATH` file bundled with the software package.

*Limitation: You may encounter conflicts if you use `wfpkginstall` to
install a newer version of a package that is already installed. In such
a case you should add the new package to `$active_software` in the
`packages.pp` Puppet manifest (and the old version to
`$inactive_software`), and then run the `workpuppet` script.*

    wfpkginstall python27-2.7.10

**Tip:** The `$admin_path/yum-workflow/<os_major_version>/x86_64/`
directory is shared with the Vagrant host under the `scratch` directory
of the Vagrant project. The
[`vagrant-rpmbuild`](https://github.com/EuPathDB/vagrant-rpmbuild)
project - the recommended environment for generating workflow-software
rpms - places the rpm files built there under the `scratch` directory of
that project. Therefore you can use host tools (e.g. drag-and-drop) to
quickly copy the rpm file from the `rpmbuild` to the `workflow-software`
project and then run `wfpkginstall` on the guest.

**Tip:** Tab completion is available for `wfpkginstall`.

### Provisioning

The `init.sh` script initializes and bootstraps the
`${BASE_DIR}/sysadmin` directory with required infrastructure. It's
mostly copy/paste from the [PreparingClusters
wiki](https://wiki.apidb.org/index.php/PreparingClusters) instructions
(and the two should be kept in sync if you make changes) with a few
tweaks to aid Vagrant VM work. The design is such that
`${BASE_DIR}/sysadmin` is only built on the first `vagrant up`.
Subsequent `vagrant provision` invocations will skip `init.sh` if
`${BASE_DIR}/sysadmin` exists. So if if you need to re-provision
`${BASE_DIR}/sysadmin` you might as well `vagrant destroy` and start
over.

Invocations of `vagrant provision` after the initial setup will invoke
the `workpuppet` script. This script manages the YUM repo updates and
runs the `puppet apply`. You can skip this step by passing Ansible
`--skip-tags` via the `ANSIBLE_ARGS` environment variable.

    ANSIBLE_ARGS="--skip-tags=workpuppet" vagrant provision el7

The git repo of Puppet manifests is checked out to the guest
`/vagrant/scratch` mount point and symlinked in to the
`${BASE_DIR}/sysadmin` directory. The `/vagrant` volume is provided by
the host so its contents persist across `vagrant destroy` (reducing the
changes of losing uncommitted changes). This also has the side-effect
that Puppet manifest editing and Git management can be done on you local
host.

The `yum-workflow` directory is also symlinked in `${BASE_DIR}/sysadmin`
to the `/vagrant/scratch` mountpoint on the guest, shared with the host,
so it is persistent across `vagrant destroy`. This is just to save yum
package download time on subsequent provisionings. The directory can be
deleted manually if you want to test a complete provision from scratch.

### Re-provisioning

You can skip tagged tasks by passing Ansible `--skip-tags` via the
`ANSIBLE_ARGS` environment variable.

    ANSIBLE_ARGS="--skip-tags=workpuppet" vagrant provision el7

### Known Issues

- The disconnect after 'change sysadmin user group' no longer works
in modern Ansible. The 'meta: reset_connection' task seems to do
nothing in <IP_ADDRESS>.

- workpuppet run at the end exits cleanly but doesn't install the
bioinformatics software. Possibly due to
`Dependency File[/project/eupathdblab/workflow-software/tmp] has failures: true`
Running workpuppet manually succeeds. Maybe related to
failure of 'change sysadmin user group' above. (this seems to be the case
as waiting a couple of minutes after 'change sysadmin user group' fails
before running provision again (giving time for multiplexing to timeout)
seems to workaround this issue.

repository: VEuPathDB/vagrant-workflow-software

repo_name: VEuPathDB/vagrant-workflow-software


text:
# Midnight Flames
## A Visual Studio Code Theme

![Example Image](images/preview.png)

[![Github License](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/Nicholas-LeClair/midnight-flames/blob/master/LICENSE)
[![Github Release](https://img.shields.io/badge/Release-V1.1-green.svg)](https://github.com/Nicholas-LeClair/midnight-flames/releases)
[![VS Code Download](https://img.shields.io/badge/VS_Code-V1.1-orange.svg)](https://marketplace.visualstudio.com/items?itemName=Nickal.midnight-flames)

## Description
A clean dark theme with vibrant orange accents.

If you are interested in contributing here is a list of things i need help with:

- Feedback
- Syntax Customization

To get your ideas in, simply make a fork of this project and add your customizations. If i like what i see i'll add it into the next release and give credit to you in the description.

repository: Nicholas-LeClair/midnight-flames

repo_name: Nicholas-LeClair/midnight-flames


text:
##  Stress tests
If you are going to use the stress tests, you may adjust these functions to your needs:
```cpp
// stress_client.cpp
client_settings default_client_set();
server_settings default_server_set();
client_config default_client_config();
client_timeout default_client_timeout();
run_time();
```
Hint: If you are going to use many clients or test entries, do this incrementally since a few Sanitizers consumes a lot of memory & processing power.
If you would like, you may also add your own tests for a specific network traffic.
After you have compiled this, you will end up with "stress_client*" and "stress_server*".
One binary can be also run on a extra server, if you are changing the host.
This is overall not perfect, but if you got ideas or see improvements, i would be glad to hear from you.

##  Unit tests
The tests for "unit_common*" are splitted into:
- defaults/default_plain_client.cpp
- defaults/default_plain_server.cpp
- defaults/default_ssl_client.cpp (if SSL enabled)
- defaults/default_ssl_server.cpp (if SSL enabled)
- _main.cpp
- misc.cpp
- plain_client.cpp
- plain_server.cpp
- ssl_client.cpp (if SSL enabled)
- ssl_server.cpp (if SSL enabled)

The ssl files are actually almost the same as the plain, except a few sections.
The tests for "unit_utils*" are located in "unit_utils.cpp".

repository: julian-becker/radrpc

repo_name: julian-becker/radrpc


text:
# Structural Deep Embedding for Hyper-Networks

```
 @inproceedings{dhne_aaai18,
     author = {Tu, Ke and Cui, Peng and Wang, Xiao and Wang, Fei and Zhu, Wenwu},
     title = {Structural Deep Embedding for Hyper-Networks},
     booktitle = {AAAI},
     year = {2018}
   }
```

links
- [arXiv](https://arxiv.org/abs/1711.10146)
- [code](https://github.com/tadpole/DHNE)

gha_language: Jupyter Notebook


text:
# NutriPlotter
# Installation

### Overview
NutriPlotter is an Expo application, meaning it is designed to run natively on a range of devices, primarily smartphones running an Android OS, or iOS in the case of iPhones. The app is based around the concept of a “digital plate” that can be adjusted  in both portion size and food composition to provide an effective estimate of the nutritional value of the meal built in the app. This is the core focus of NutriPlotter, and its primary reason for conception and development. As such, the plate is the central point of the application for users, effectively serving as the focus of their main screen. The concept of the digital plate was introduced due to an aversion to weighing the foods out before eating, as is standard with most nutritional tracking apps. The digital plate concept was developed to avoid this, serving the dual purpose of reinforcing the “Balanced Plate” model used in recommending a healthy diet.

### Installation
The Git repository is the central storage location for our code, amongst other things. In order to get access to the code for the project on your local device, you should run the following command in the command prompt:

```bash
    git clone http://stgit.dcs.gla.ac.uk/tp3-2018-se4/dissertation
```

This will create a local copy of the repository on your machine. If you simply want to run the app to use, this should be your only interaction with Git.

If you want to run NutriPlotter, go to the folder titled “NutriPlotter” and run the following command to install dependencies (the code libraries which must be installed prior to running the app):

```bash
    npm install
```

Note: You will need to have a command prompt window open in the “NutriPlotter” folder in order for this to work. If you run the command elsewhere, it may produce unexpected results.

# Running the App
In the "Nutriplotter" folder, enter the following command (to run the application), which should be run in the same directory once the dependencies have finished installing:

```bash
    expo start
```

### Get the Published Version
The most recently published version of the app is available [here](https://expo.io/@eleondella/NutriPlotter)

Just scan the QR code with your phone to open it up.

### Running NutriPlotter on mobile
This command should open a browser window with the Metro Bundler for the app. On the left side of the screen, there is a number of options for launching your app. The easiest method is using the QR code, which can be scanned by the Expo mobile application for Android or iOS. The Expo application can be found on the App Store for iOS devices and is used to launch applications developed on the Expo framework. Once you have installed the Expo application itself and opened it, you will be given the option to scan a QR code to launch an application. You can also do this without opening the Expo application first on iOS by simply holding your finger on the QR code in the Camera app.

Alternatively, you can launch the app by making an account and signing into Expo on both your mobile device and the device hosting. You will then get the option to open your project directly upon opening the Expo mobile app. Signing into Expo also gives the added benefit of providing a default phone number or email address through which to send you a link to open the application, should you choose this route.

Once you’ve either opened a link to your project, or used a QR code (they point to the same place), you should see the bundler start after a short wait. Once the bundler has completed the task of building, the app will open automatically. This can take a little time, and sometimes the percentages of completion will jump around somewhat. This isn’t cause for concern and if you let it run the process to completion, the app should open normally.

### Resources

- official tutorial for [ReactJS](https://reactjs.org/)
- official tutorial for [Expo](https://docs.expo.io/versions/latest/)

### Contributing

- Eleonora Della    2244079D
- Matthew Smith     2260469S
- Han Loo, Nicholas 2288527L
- Peter Macaldowie  2258785M
- Soma Froghyar     2267217F

repository: PMacald/NutriPlotter

repo_name: PMacald/NutriPlotter


text:
# Changelog

## master

- FIXED: undefined local variable or method `source_address_inet6' (GH-40). [Thanks @simsicon]

- FIXED: Fixed bug on parsing multiple nameservers on different lines (GH-45). [Thanks @nicholasren]

- CHANGED: Dropped duplicate query ID filter. Query ID is now randomically generated but it's not guaranteed to be unique (GH-39). [Thanks @ebroder]

- CHANGED: require 'net/dns' is now the preferred way to load the library (GH-37). [Thanks @johnroa]

- CHANGED: Removed setup.rb installation script.

## Release 0.7.1

- FIXED: Invalid file permissions on several files (GH-35) [Thanks @jamespharaoh]

## Release 0.7.0

- ADDED: Added (experimental) Support for HINFO record.

- FIXED: Use Net::DNS::Resolver::Error class (not ResolverError, which does not exist).

- FIXED: Cleaned up require dependency and recursive require statements.

- FIXED: Use RbConfig instead of obsolete and deprecated Config (GH-28, GH-33) [Thanks @shadowbq, @eik3]

- FIXED: SRV record not required by Net::DNS::RR (GH-27) [Thanks @sebastian]

- FIXED: Resolver now supports IPv6 (GH-32) [Thanks @jamesotron]

- FIXED: Net::DNS::RR::PTR references an invalid parameter (GH-19) [Thanks @dd23]

- FIXED: Net::DNS::Question changes input arguments (GH-7) [Thanks @gfarfl]

- CHANGED: Refactoring unit test to follow most used Ruby conventions.

- CHANGED: Rewritten and simplified Net::DNS::Classes. Improved test harness.

- CHANGED: Removed Jeweler development dependency.

- CHANGED: The library is now compatible with Bundler.

- CHANGED: Minimum supported Ruby version changed to Ruby 1.8.7.

- CHANGED: Rescue NameError so unsupported record types only result in a warning.

- CHANGED: Renamed Net::DNS::Resolver#send to Net::DNS::Resolver#query to avoid overriding default meaning of send method.

## Release 0.6.1

- ADDED: Net::DNS::Packet#to_s method (alias of #inspect)

- FIXED: typo in lib/net/dns/rr/ptr.rb [Thanks Chris Lundquist]

- FIXED: warning: method redefined; discarding old inspect (GH-3) [Thanks Kevin Baker]

- FIXED: issue with rescue ArgumentError (GH-5) and with IPAddr handling (GH-6)

## Release 0.6.0

*WARNING:- If you are upgrading from a previous minor release, check out the Compatibility issue list below.

- FIXED: Added missing #to_s method to Net::DNS::Question.

- FIXED: Compatibility with Ruby 1.9

- FIXED: Types regexp order issue

- CHANGED: Refactoring unit test to follow most used Ruby conventions

- CHANGED: default timeout is now 5 seconds for both UDP and TCP

- CHANGED: Moved main dns.rb file to lib/net folder as default for GEMs. In this way it can be autoloaded when the gem is required.

### Compatibility issues

- CHANGED: RR#set_stype scope is now private to prevent invalid usage.

- CHANGED: DnsTimeout#timeout now raises LocalJumpError instead of DnsTimeoutArgumentError when block is missing.

- CHANGED: Renamed Net::DNS::RR::Types::Types to Net::DNS::RR::Types::TYPES to follow Ruby coding standards.

## Release 0.4

- many bug fixes (thanks guys!)
- a whole new class Net::DNS::Header::RCode
- new methods in Net::DNS::Resolver class to do AXFR queries
- a new SRV resource record written by Dan Janowski
- more documentation written and corrected

license: Ruby,MIT

detected_licenses: Ruby MIT

gha_language: CSS


text:
---
jupytext:
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.12
    jupytext_version: 1.9.1
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

```{code-cell}
:tags: [remove-cell]

    import pyslim, tskit, msprime
    from IPython.display import SVG
    import numpy as np

    ts = pyslim.load("example_sim.trees")
    tables = ts.tables
```

```{eval-rst}
.. currentmodule:: pyslim
```

(sec_python_api)=

# Python API

This page provides detailed documentation for the methods and classes
available in pyslim.

## Methods

pyslim provides tools for transforming tree sequences:

```{eval-rst}
.. autosummary::

  recapitate
  convert_alleles
  generate_nucleotides
```

## Additions to the tree sequence

The {class}`tskit.TreeSequence` class represents a sequence of trees
that describes the genealogical history of a population.
Here, we describe pyslim's additions to this class.

### The SLiM Tree Sequence

:::{eval-rst}
.. autoclass:: pyslim.SlimTreeSequence()
    :members:
:::

## Metadata

SLiM-specific metadata is made visible to the user by ``.metadata`` properties.
For instance:
```{code-cell}
ts.node(4).metadata
```
shows that the fifth node in the tree sequence was given pedigree ID ``982740`` by SLiM,
is *not* a null genome, and has ``genome_type`` zero, which corresponds to an autosome
(see below).

### Annotation

These two functions will add default SLiM metadata to a tree sequence (or the
underlying tables), which can then be modified and loaded into SLiM.

:::{eval-rst}
.. autofunction:: pyslim.annotate_defaults
:::

:::{eval-rst}
.. autofunction:: pyslim.annotate_defaults_tables
:::

### Provenances

:::{eval-rst}
.. autoclass:: pyslim.ProvenanceMetadata
   :members:
:::

:::{eval-rst}
.. autofunction:: pyslim.get_provenance
:::

(sec_constants_and_flags)=

## Constants and flags

:::{eval-rst}
.. data:: NUCLEOTIDES == ['A', 'C', 'G', 'T']

   Nucleotide states in nucleotide models are encoded as integers (0, 1, 2, 3),
   so a nucleotide encoded as ``k`` refers to nucleotide
   ``pyslim.NUCLEOTIDES[k]``.

repository: chriscrsmith/pyslim

repo_name: chriscrsmith/pyslim


text:
# RAILS with JS Project - Healthy Smoothie

Welcome to Healthy Smoothie! It's just a nice app to display smoothies. User can manage and share their smoothie (make them private or public), user friendly and fast!

## Installation

Copy all folders and files

And then execute:

    $ bundle install

Migrate databases and seed:

    $ rake db:migrate
    $ rake db:seed

## Usage

Start server by running:

    $ rails s

Login with admin credentials<EMAIL_ADDRESS>// admin)
One user is created through `seed` file<EMAIL_ADDRESS>// user)

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/kheyro/healthy-smoothie.

repository: kheyro/healthy-smoothie

repo_name: kheyro/healthy-smoothie


text:
---
title: |
  Compute quick cluster analysis
rdname: sjc.qclus
date: 2015-10-22
output: html_document
layout: article
category: sjplot
images:
 - /allYourFigureAreBelongToUs/figure/source/1991-05-31-sjc.qclus//sjc.qclus-1.png
 - /allYourFigureAreBelongToUs/figure/source/1991-05-31-sjc.qclus//sjc.qclus-2.png
---

{% highlight r %}
## Not run:
##D # k-means clustering of mtcars-dataset
##D sjc.qclus(mtcars)
##D
##D # k-means clustering of mtcars-dataset with 4 pre-defined
##D # groups in a faceted panel
##D sjc.qclus(airquality,
##D           groupcount = 4,
##D           facetCluster = TRUE)
## End(Not run)

# k-means clustering of airquality data
# and saving the results. most likely, 3 cluster
# groups have been found (see below).
airgrp <- sjc.qclus(airquality)
{% endhighlight %}

{% highlight text %}
## Clustering Gap statistic ["clusGap"].
## B=100 simulated reference sets, k = 1..10
##
##  --> Number of clusters (method 'Tibs2001SEmax', SE.factor=1): 3
{% endhighlight %}

![plot of chunk sjc.qclus](/allYourFigureAreBelongToUs/figure/source/1991-05-31-sjc.qclus/sjc.qclus-1.png)

{% highlight r %}
# "re-plot" cluster groups, without computing
# new k-means cluster analysis.
sjc.qclus(airquality,
          groupcount = 3,
          groups = airgrp$classification)
{% endhighlight %}

![plot of chunk sjc.qclus](/allYourFigureAreBelongToUs/figure/source/1991-05-31-sjc.qclus/sjc.qclus-2.png)


text:
# PIG DICE    :pig:
-----------------
**Created by Tony Kihara**
------------------
## DESCRIPTION :message:
 ~~~
 It is a game that mainly entails rolling of dices among two parties.
 It record  the highest points of one player and declares him/her the winner.
 It is addictive and it can be played by anyone both adults and children.
 ~~~
------------------------------------
## FEATURES
The main features used in making of the games is:
* images:
        ![image](img/background.jpg)      ![image](img/game.gif)
        ![image](img/fail.gif)            ![image](img/new-game.gif)
        ![image](img/play.gif)             ![image](img/rules.gif)
        ![image](img/success.gif)
* The languages used in the game are:
    -[HTML](https://www.w3schools.com/html/)
    -[CSS](https://www.w3schools.com/css/css_intro.asp)
    -[Javascript](https://javascript.info/)
------------------------------------------------------------------
## INSTRUCTION OF USE
To use the codes of the game:
~~~~
$ git clone https://github.com/Kihara-tony/Pig-dice.git
$ atom .
~~~~
## BDD
|Behavoir|input|output|
|------:|------:|-----:|
|To show your name|Tony|Player Tony|
|To add up the digits|roll=3|total=9|
|To show the winner|After three attempts each|Tony you won|

------------------------------
## CONTACT
To reach me use the contacts below:
~~~
*<EMAIL_ADDRESS>* Call:<PHONE_NUMBER>
~~~
----------------------------------------
## LISENCE
Copyright 2019 Tony Kihara.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

repository: Kihara-tony/Pig-dice

repo_name: Kihara-tony/Pig-dice


text:
More than 3 years ago, we posted a comparative study about [Looping over Pandas data](https://www.architecture-performance.fr/ap_blog/looping-over-pandas/) using a CPU. Because a lot of things evolved since 2018, this post is kind of an update. For example Pandas tag version was `0.23.3` at that time, it is now `1.4.0`. Also, we added some more options.

Here is a list of all the options tested in the following:
- Pandas built-in vectorization
- pandas.DataFrame.iterrows
- pandas.DataFrame.apply
- pandas.DataFrame.itertuples
- numpy.apply_along_axis
- numpy.vectorize
- map
- swifter
- dask.dataframe.map_partitions
- polars.DataFrame.apply
- polars built-in vectorization
- Numba
- Numba parallel
- Cython
- Cython parallel

## Introduction

### Motivation

The focus is on looping over the rows of a Pandas dataframe holding some numerical data. All the elements of the dataframe are of `np.float64` data type. A function is applied to each row, taking the row elements as input, either as distinct scalar arguments, as an array, or as a Pandas Series. The computation returns a scalar value per row, so that the process eventually returns a numeric Pandas series with same index as the original dataframe. For this post, a toy function computing the determinant of a 3-by-3 symmetric real matrix is used:

$$\begin{equation*}
M =
\begin{pmatrix}
m_{1,1} & m_{1,2} & m_{1,3} \\\\
m_{1,2} & m_{2,2} & m_{2,3} \\\\
m_{1,3} & m_{2,3} & m_{3,3}
\end{pmatrix}
\end{equation*}$$

$$|M| = m_{1,1} (m_{2,2} \, m_{3,3} - m_{2,3}^2) + m_{1,2} \, (2 \,  m_{2,3} \, m_{1,3} - m_{1,2} \, m_{3,3}) - m_{2,2} \, m_{1,3}^2$$

Here is a pure python implementation of the row-wise function:

```python
def det_sym33(m11, m12, m13, m22, m23, m33):
    """Compute the determinant of a real symmetric 3x3 matrix
    given its 6 upper triangular coefficients.
    """
    return (
        m11 * (m22 * m33 - m23**2)
        + m12 * (2.0 * m23 * m13 - m12 * m33)
        - m22 * m13**2
    )
```

### Imports

```python
from itertools import cycle
from time import perf_counter

import cython
import dask.dataframe as dd
from numba import jit, float64, njit, prange
import numpy as np
import perfplot
import pandas as pd
import polars as pl
import swifter

%load_ext Cython
%load_ext line_profiler

SD = 124  # random seed
rng = np.random.default_rng(SD)  # random number generator
```

    Python version       : 3.9.10
    IPython version      : 8.0.1
    polars    : 0.12.19
    numpy     : 1.21.5
    swifter   : 1.0.9
    pandas    : 1.4.1
    cython    : 0.29.28
    numba     : 0.55.1
    perfplot  : 0.10.1
    dask      : 2022.2.1

Computations are performed on a laptop with an 8 cores Intel i7-7700HQ CPU @ 2.80GHz, running Linux.

### Timing function

This function is returning the best elapsed time over `r` trials, and the result of the computation:

```python
def timing(func, df, r=10):
    timings = []
    for i in range(r):
        start = perf_counter()
        s = func(df)
        end = perf_counter()
        elapsed_time = end - start
        timings.append(elapsed_time)
    return s, np.amin(timings)
```

### Create a dataframe with random floats

We start by creating a medium-sized dataframe to perform a first comparison. We will later compare the most efficient methods with longer dataframes.

```python
%%time
n = 100_000  # dataframe length
column_names = ['m11', 'm12', 'm13', 'm22', 'm23', 'm33']
df = pd.DataFrame(data=rng.random((n, 6)), columns=column_names)
```

    CPU times: user 10.2 ms, sys: 143 µs, total: 10.4 ms
    Wall time: 7.99 ms

```python
df.head(3)
```

      m11
      m12
      m13
      m22
      m23
      m33

      0
      0.785253
      0.785859
      0.969136
      0.748060
      0.655551
      0.938885

      1
      0.178614
      0.588647
      0.442799
      0.348847
      0.330929
      0.159369

      2
      0.989463
      0.257111
      0.715765
      0.505885
      0.664111
      0.702342

### Row-wise functions

We now create 3 different row-wise functions, with different argument types for the input row values:
- distinct scalar arguments
- an array
- a Pandas Series

```python
def det_sym33_scalars(m11, m12, m13, m22, m23, m33):
    return (
        m11 * (m22 * m33 - m23**2)
        + m12 * (2.0 * m23 * m13 - m12 * m33)
        - m22 * m13**2
    )
```

```python
def det_sym33_array(m):
    return (
        m[0] * (m[3] * m[5] - m[4] ** 2)
        + m[1] * (2.0 * m[4] * m[2] - m[1] * m[5])
        - m[3] * m[2] ** 2
    )
```

```python
def det_sym33_series(s):
    return (
        s.m11 * (s.m22 * s.m33 - s.m23**2)
        + s.m12 * (2.0 * s.m23 * s.m13 - s.m12 * s.m33)
        - s.m22 * s.m13**2
    )
```

In the following, we may try several of these row-wise functions for a given dataframe looping method, depending on how the dataframe rows are returned by the method.

## Pandas built-in vectorization

First we are going to use the built-in vectorization operations from Pandas. In the present case the row-wise computation is straightforward and can be performed with basic universal functions applied to the entire columns. This does not make use of any row-wise function, but allows us to have a reference timing.

```python
def pandas_vectorize(df):
    return (
        df.m11 * (df.m22 * df.m33 - df.m23**2)
        + df.m12 * (2.0 * df.m23 * df.m13 - df.m12 * df.m33)
        - df.m22 * df.m13**2
    )
```

We store the resulting Pandas Series into the `det_ref` variable, in order to check that the later computations lead to the same result:

```python
det_ref, t = timing(pandas_vectorize, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0044875 s

## pandas.DataFrame.iterrows

We know that the `iterrows` method is kind of slow (see the previous [post](https://aetperf.github.io/2018/07/03/Looping-over-Pandas-data.html) for example). It loops over dataframe rows as [index, Series] pairs. Here, we are going to compare the 3 different argument types of the row-wise function, given that the row returned by `iterrows` is a Pandas Series:
- scalar values
- a Numpy array
- a Pandas Series

|function name | method | returning rows as | row-wise function | argument type  |
|---|--------|------------------|--------------|---|
|iterrows_scalars | pd.DataFrame.iterrows | pd.Series | df_iterrows_scalars | float64  |
|iterrows_array| pd.DataFrame.iterrows | pd.Series | df_iterrows_array   | array  |
|iterrows_series| pd.DataFrame.iterrows | pd.Series | df_iterrows_series  | pd.Series  |

### Scalar arguments

```python
def iterrows_scalars(df):
    det = np.zeros(len(df), dtype=np.float64)
    i = 0
    for _, row in df.iterrows():
        det[i] = det_sym33_scalars(row.m11, row.m12, row.m13, row.m22, row.m23, row.m33)
        i += 1
    return pd.Series(det, index=df.index)
```

```python
det, t_scalars = timing(iterrows_scalars, df, r=3)
print(f"Elapsed time: {t_scalars:8.7f} s")
```

    Elapsed time: 5.6840575 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### numpy.ndarray argument

```python
def iterrows_array(df):
    det = np.zeros(len(df), dtype=np.float64)
    i = 0
    for _, row in df.iterrows():
        det[i] = det_sym33_array(row.values)
        i += 1
    return pd.Series(det, index=df.index)
```

```python
det, t_array = timing(iterrows_array, df, r=3)
print(f"Elapsed time: {t_array:8.7f} s")
```

    Elapsed time: 2.4847183 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### pandas.Series argument

```python
def iterrows_series(df):
    det = np.zeros(len(df), dtype=np.float64)
    i = 0
    for _, row in df.iterrows():
        det[i] = det_sym33_series(row)
        i += 1
    return pd.Series(det, index=df.index)
```

```python
det, t_series = timing(iterrows_series, df, r=3)
print(f"Elapsed time: {t_series:8.7f} s")
```

    Elapsed time: 7.8285359 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### Comparison

```python
ax = (
    pd.DataFrame(
        data=[{"scalars": t_scalars, "array": t_array, "Series": t_series}],
        index=["elapsed_time"],
    )
    .T.sort_values(by="elapsed_time", ascending=False)
    .plot.bar(legend=False, alpha=0.75, rot=45)
)
_ = ax.set(
    title="iterrows() with 3 different argument types",
    xlabel="Argument type",
    ylabel="Elapsed time (s)",
)
```

We observe that using the array of values from the pandas.Series as argument of the row-wise function, and accessing the data with indices is the fastest method when using `.iterrows()`. However, all three `iterrows` methods are very slow.

## pandas.DataFrame.apply

The `apply` method also iterates over dataframe rows (with the `axis=1` argument), returning either a Series [default] or an array (with `raw=True`). Here is what Pandas' [documentation](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.apply.html) says about it:
> the passed function will receive ndarray objects instead. If you are just applying a NumPy reduction function this will achieve much better performance.

|function name| method | returning rows as | row-wise function | argument type  |
|----|--------|------------------|--------------|---|
|apply_series| pd.DataFrame.apply | pd.Series | det_sym33_series | pd.Series  |
|apply_array| pd.DataFrame.apply | np.ndarray | det_sym33_array | array  |

### pandas.Series argument

```python
def apply_series(df):
    return df.apply(det_sym33_series, raw=False, axis=1)
```

```python
det, t_series = timing(apply_series, df, r=3)
print(f"Elapsed time: {t_series:8.7f} s")
```

    Elapsed time: 5.1164269 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### numpy.ndarray argument

```python
def apply_array(df):
    return df.apply(det_sym33_array, raw=True, axis=1)
```

```python
det, t_array = timing(apply_array, df, r=3)
print(f"Elapsed time: {t_array:8.7f} s")
```

    Elapsed time: 0.2638231 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### Comparison

```python
ax = (
    pd.DataFrame(
        data=[{"array": t_array, "Series": t_series}],
        index=["elapsed_time"],
    )
    .T.sort_values(by="elapsed_time", ascending=False)
    .plot.bar(legend=False, alpha=0.75, rot=45)
)
_ = ax.set(
    title="apply() with 2 different argument types",
    xlabel="Argument type",
    ylabel="Elapsed time (s)",
)
```

Indeed using arrays instead of Series is way faster! But still a lot slower than pandas built-in vectorization.

## pandas.DataFrame.itertuples

The `itertuples` method allows the iteratation over dataframe rows, returning them as namedtuples. Thus, the row values can either be accessed by name or by index. The function `det_sym33_scalars` and `det_sym33_series` are used in the former case, and `det_sym33_array` in the latter.

|function name| method | returning rows as | row-wise function | argument type  |
|-------|--------|------------------|--------------|---|
|itertuples_scalars| pd.DataFrame.itertuples | namedtuple | det_sym33_scalars | float64  |
|itertuples_array| pd.DataFrame.itertuples | namedtuple | det_sym33_array | array  |
|itertuples_series| pd.DataFrame.itertuples | namedtuple | det_sym33_series | pd.Series  |

### Scalar arguments

```python
def itertuples_scalars(df):
    det = np.zeros(len(df), dtype=np.float64)
    i = 0
    for row in df.itertuples():
        det[i] = det_sym33_scalars(row.m11, row.m12, row.m13, row.m22, row.m23, row.m33)
        i += 1
    return pd.Series(det, index=df.index)
```

```python
det, t_scalars = timing(itertuples_scalars, df)
print(f"Elapsed time: {t_scalars:8.7f} s")
```

    Elapsed time: 0.1051879 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### numpy.array argument

When row values are accessed by index, we need to account for the dataframe index, which is indexed by 0.

```python
def itertuples_array(df):
    det = np.zeros(len(df), dtype=np.float64)
    i = 0
    for row in df.itertuples():
        det[i] = det_sym33_array(row[1:])
        i += 1
    return pd.Series(det, index=df.index)
```

```python
det, t_array = timing(itertuples_array, df)
print(f"Elapsed time: {t_array:8.7f} s")
```

    Elapsed time: 0.1136810 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### pandas.Series argument

```python
def itertuples_series(df):
    det = np.zeros(len(df), dtype=np.float64)
    i = 0
    for row in df.itertuples():
        det[i] = det_sym33_series(row)
        i += 1
    return pd.Series(det, index=df.index)
```

```python
det, t_series = timing(itertuples_series, df)
print(f"Elapsed time: {t_series:8.7f} s")
```

    Elapsed time: 0.1084384 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### Comparison

```python
ax = (
    pd.DataFrame(
        data=[{"scalars": t_scalars, "array": t_array, "Series": t_series}],
        index=["elapsed_time"],
    )
    .T.sort_values(by="elapsed_time", ascending=False)
    .plot.bar(legend=False, alpha=0.75, rot=45)
)
_ = ax.set(
    title="itertuples() with 3 different argument types",
    xlabel="Argument type",
    ylabel="Elapsed time (s)",
)
```

We get kind of similar results with the 3 different argument types. Once again, we can say than `itertuples` is to be prefered over `iterrows` and `apply`.

## numpy.apply_along_axis

Let's try out this method allowing to apply a function to a 1D slice along a given axis (rows of a 2D array in our case).

|function name| method | returning rows as | row-wise function | argument type  |
|-----|--------|------------------|--------------|---|
|np_apply_along_axis| np.apply_along_axis | np.ndarray | det_sym33_array | array  |

```python
def np_apply_along_axis(df):

    return pd.Series(
        np.apply_along_axis(func1d=det_sym33_array, axis=1, arr=df.values),
        index=df.index,
    )
```

```python
det, t = timing(np_apply_along_axis, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.2435897 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

Well this is rather slow and disapointing...

## numpy.vectorize

Numpy vectorize evaluates the row-wise function over each element of the input numpy array(s). However, note the warning on the [NumPy documentation](https://numpy.org/doc/stable/reference/generated/numpy.vectorize.html):
> The vectorize function is provided primarily for convenience, not for performance. The implementation is essentially a for loop.

|function name| method | returning rows as | row-wise function | argument type  |
|--------|--------|------------------|--------------|---|
|np_vectorize_scalars| np.vectorize | float64 | det_sym33_scalars | float64  |
|np_vectorize_array| np.vectorize | np.ndarray | det_sym33_array | array  |

### Scalar arguments

```python
def np_vectorize_scalars(df):

    return pd.Series(
        np.vectorize(det_sym33_scalars)(df.m11, df.m12, df.m13, df.m22, df.m23, df.m33),
        index=df.index,
    )
```

```python
det, t_scalars = timing(np_vectorize_scalars, df)
print(f"Elapsed time: {t_scalars:8.7f} s")
```

    Elapsed time: 0.0498839 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### numpy.ndarray argument

In this case we need to add a `signature` argument in order to specify the input shape of the row-wise function.

```python
def np_vectorize_array(df):

    return pd.Series(
        np.vectorize(det_sym33_array, signature="(n)->()")(
            df[["m11", "m12", "m13", "m22", "m23", "m33"]].values
        ),
        index=df.index,
    )
```

```python
det, t_array = timing(np_vectorize_array, df)
print(f"Elapsed time: {t_array:8.7f} s")
```

    Elapsed time: 0.2720111 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

### Comparison

```python
ax = (
    pd.DataFrame(
        data=[{"scalars": t_scalars, "array": t_array}],
        index=["elapsed_time"],
    )
    .T.sort_values(by="elapsed_time", ascending=False)
    .plot.bar(legend=False, alpha=0.75, rot=45)
)
_ = ax.set(
    title="numpy.vectorize() with 2 different argument types",
    xlabel="Argument type",
    ylabel="Elapsed time (s)",
)
```

The version with scalar arguments is quite interesting, faster than `itertuples`.

## map

Let's use the standard `map()` Python method.

|function name| method | returning rows as | row-wise function | argument type  |
|--------|--------|------------------|--------------|---|
|map_scalars| map | float64 | det_sym33_scalars | float64  |

```python
def map_scalars(df):
    return pd.Series(
        map(det_sym33_scalars, df.m11, df.m12, df.m13, df.m22, df.m23, df.m33),
        index=df.index,
    )
```

```python
det, t = timing(map_scalars, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0679905 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

Performance seems to be similar between `map_scalars` and `np_vectorize_scalars.`

## Swifter

[Swifter](https://github.com/jmcarpenter2/swifter) is a "package which efficiently applies any function to a pandas dataframe or series in the fastest available manner". we use the `raw=True` argument. Here is a quote from the [documentation ](https://github.com/jmcarpenter2/swifter/blob/master/docs/documentation.md):

> raw : bool, default False
> False : passes each row or column as a Series to the function.
> True : the passed function will receive ndarray objects instead. If you are just applying a NumPy reduction function this will achieve much better performance.

|function name| method | returning rows as | row-wise function | argument type  |
|--------|--------|------------------|--------------|---|
|swifter_apply| swifter.apply | np.ndarray | det_sym33_array | array  |

```python
def swifter_apply(df):
    return df.swifter.progress_bar(False).apply(det_sym33_array, raw=True, axis=1)
```

```python
det, t = timing(swifter_apply, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.2725223 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

The computation is rather slow, we are probably missing something here and did not use this package correctly?

## dask.dataframe.map_partitions

We tried to use the great [dask](https://github.com/dask/dask) library with the `map_partitions` method. Unfortunately, we did not completely figure out how to handle the `meta` argument. Here is the description from the [documentation](https://docs.dask.org/en/stable/generated/dask.dataframe.DataFrame.map_partitions.html):

> meta : pd.DataFrame, pd.Series, dict, iterable, tuple, optional
>    An empty pd.DataFrame or pd.Series that matches the dtypes and column names of the output. This metadata is necessary for many algorithms in dask dataframe to work. For ease of use, some alternative inputs are also available. Instead of a DataFrame, a dict of {name: dtype} or iterable of [name, dtype] can be provided [note that the order of the names should match the order of the columns]. Instead of a series, a tuple of [name, dtype] can be used. If not provided, dask will try to infer the metadata. This may lead to unexpected results, so providing meta is recommended. For more information, see dask.dataframe.utils.make_meta.

The following implementation does work but there might be more efficient ways to do it...

|function name| method | returning rows as | row-wise function | argument type  |
|--------|--------|------------------|--------------|---|
|dask_df_map_partitions| dd.map_partitions| pd.Series | det_sym33_series |  pd.Series  |

```python
def dask_df_map_partitions(df, n_jobs=8):
    ddf = dd.from_pandas(df, npartitions=n_jobs)
    return pd.Series(
        ddf.map_partitions(
            det_sym33_series, meta=pd.Series(dtype=np.float64)
        ).compute(),
        index=df.index,
    )
```

```python
det, t = timing(dask_df_map_partitions, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0254871 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

Not so bad but still slower than Pandas built_in vectorization. Here is what can be found on [dask's documentaion](https://docs.dask.org/en/stable/generated/dask.dataframe.from_pandas.html?highlight=from_pandas):
> Note that, despite parallelism, Dask.dataframe may not always be faster than Pandas. We recommend that you stay with Pandas for as long as possible before switching to Dask.dataframe.

Also, we guess that there might be a dataframe copy from Pandas to Dask? This method is probably more recommended when dealing with very large dataframes distributed on clusters.

## polars.DataFrame.apply

[Polars](https://github.com/pola-rs/polars) is a fast multi-threaded DataFrame library written in Rust but also available in Python and Node.js. Here we are going to use `DataFrame.apply()` which allows to apply a custom function over the rows of a Polars dataFrame. The rows are passed as tuple. However, note this warning from Polars' [documentation](https://pola-rs.github.io/polars/py-polars/html/reference/api/polars.DataFrame.apply.html):
> Beware, this is slow.

|function name| method | returning rows as | row-wise function | argument type  |
|--------|--------|------------------|--------------|---|
|polars_apply| polars.DataFrame.apply() | tuple | det_sym33_array | array  |

```python
def polars_apply(df):
    df_pl = pl.from_pandas(df)
    det_pl = df_pl.apply(det_sym33_array)
    det_pd = det_pl.to_pandas()["apply"]
    det_pd.index = df.index
    det_pd.name = None
    return det_pd
```

```python
det, t = timing(polars_apply, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0902959 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

This is not so bad but one can wonder if a large part of the elapsed time is not spent converting the dataframe from Pandas to Polars and backward. Let's measure the elepased time per line with a line profiler:

```python
%lprun -f polars_apply polars_apply(df)
```

    Timer unit: 1e-06 s

    Total time: 0.229353 s
    File: /tmp/ipykernel_16266/3940900507.py
    Function: polars_apply at line 1

    Line #      Hits         Time  Per Hit   % Time  Line Contents
    ==============================================================
         1                                           def polars_apply(df):
         2         1       6545.0   6545.0      2.9      df_pl = pl.from_pandas(df)
         3         1     221243.0 221243.0     96.5      det_pl = df_pl.apply(det_sym33_array)
         4         1       1542.0   1542.0      0.7      det_pd = det_pl.to_pandas()["apply"]
         5         1         14.0     14.0      0.0      det_pd.index = df.index
         6         1          9.0      9.0      0.0      det_pd.name = None
         7         1          0.0      0.0      0.0      return det_pd

Most of the time is actually spent within the `apply` method. However, we believe that there is a copy process in the `from_pandas` step, which would not be not optimal regarding memory usage?

## Polars vectorize

We can also use Polars built-in vectorization the same way we did with Pandas. This does not make use of the previous row-wise functions.

```python
def polars_vectorize(df):
    df_pl = pl.from_pandas(df)
    det_pl = (
        df_pl.m11 * (df_pl.m22 * df_pl.m33 - df_pl.m23**2)
        + df_pl.m12 * (2.0 * df_pl.m23 * df_pl.m13 - df_pl.m12 * df_pl.m33)
        - df_pl.m22 * df_pl.m13**2
    )
    det_pd = det_pl.to_frame().to_pandas()["m11"]
    det_pd.index = df.index
    det_pd.name = None
    return det_pd
```

```python
det, t = timing(polars_vectorize, df)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0090223 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

If we run the line profiler, we can observe that most of the elapsed time is now spent converting the dataframe between Pandas and polars:

```python
%lprun -f polars_vectorize polars_vectorize(df)
```

    Timer unit: 1e-06 s

    Total time: 0.014961 s
    File: /tmp/ipykernel_16266/1430159236.py
    Function: polars_vectorize at line 1

    Line #      Hits         Time  Per Hit   % Time  Line Contents
    ==============================================================
         1                                           def polars_vectorize(df):
         2         1       7295.0   7295.0     48.8      df_pl = pl.from_pandas(df)
         3         1          1.0      1.0      0.0      det_pl = (
         4         3       2915.0    971.7     19.5          df_pl.m11 * (df_pl.m22 * df_pl.m33 - df_pl.m23**2)
         5         1        551.0    551.0      3.7          + df_pl.m12 * (2.0 * df_pl.m23 * df_pl.m13 - df_pl.m12 * df_pl.m33)
         6         1       2363.0   2363.0     15.8          - df_pl.m22 * df_pl.m13**2
         7                                               )
         8         1       1807.0   1807.0     12.1      det_pd = det_pl.to_frame().to_pandas()["m11"]
         9         1         17.0     17.0      0.1      det_pd.index = df.index
        10         1         11.0     11.0      0.1      det_pd.name = None
        11         1          1.0      1.0      0.0      return det_pd

## Numba

[Numba](https://numba.pydata.org/) makes Python code fast! Here we use the Numba `@jit` decorator with `nogil=True` for the Numba dedicated row-wise function. In the `apply_func_numba` function, we basically implement the loop over the dataframe rows.

```python
@jit(
    float64(float64, float64, float64, float64, float64, float64),
    nogil=True,
)
def det_sym33_numba(m11, m12, m13, m22, m23, m33):
    return (
        m11 * (m22 * m33 - m23**2)
        + m12 * (2.0 * m23 * m13 - m12 * m33)
        - m22 * m13**2
    )

@jit
def apply_func_numba(col_m11, col_m12, col_m13, col_m22, col_m23, col_m33):
    n = len(col_m11)
    det = np.empty(n, dtype="float64")
    for i in range(n):
        det[i] = det_sym33_numba(
            col_m11[i], col_m12[i], col_m13[i], col_m22[i], col_m23[i], col_m33[i]
        )
    return det

def numba_loop(df):
    det = apply_func_numba(
        df["m11"].to_numpy(),
        df["m12"].to_numpy(),
        df["m13"].to_numpy(),
        df["m22"].to_numpy(),
        df["m23"].to_numpy(),
        df["m33"].to_numpy(),
    )
    return pd.Series(det, index=df.index)
```

```python
det, t = timing(numba_loop, df, r=100)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0002950 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

Well this is the fastest implementation so far.

## Numba parallel

Now we use the Numba `@njit` decorator along with `prange` looping. By default, all available cores are used. The previous row-wise function `apply_func_numba` is used here.

```python
@njit(parallel=True)
def apply_func_numba_para(col_m11, col_m12, col_m13, col_m22, col_m23, col_m33):
    n = len(col_m11)
    det = np.empty(n, dtype="float64")
    for i in prange(n):
        det[i] = det_sym33_numba(
            col_m11[i], col_m12[i], col_m13[i], col_m22[i], col_m23[i], col_m33[i]
        )
    return det

def numba_loop_para(df):
    det = apply_func_numba_para(
        df["m11"].to_numpy(),
        df["m12"].to_numpy(),
        df["m13"].to_numpy(),
        df["m22"].to_numpy(),
        df["m23"].to_numpy(),
        df["m33"].to_numpy(),
    )
    return pd.Series(det, index=df.index)
```

```python
det, t = timing(numba_loop_para, df, r=100)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0002439 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

The parallel version is not a large improvement over the sequantial Numba version. This might be because of the relatively small size of the dataframe.

## Cython

[Cython](https://cython.org/) is a mix between C and Python. Writing Cython code is a little more involving than writing pure Python. Note that in the following we used `%%cython` magic command at the begining of the notebook cell to compile the Cython code [also we loaded the Cython extension at the begining of the notebook].

```cython
%%cython

cimport cython
import numpy as np
import pandas as pd

@cython.binding(False)
@cython.boundscheck(False)
@cython.wraparound(False)
@cython.initializedcheck(False)
cdef void loop_cython(double[:] det, double[:] m11, double[:] m12, double[:] m13, double[:] m22, double[:] m23, double[:] m33) nogil:

    cdef Py_ssize_t i

    for i in range(m11.shape[0]):
        det[i] = m11[i] * (m22[i] * m33[i] - m23[i] ** 2) + m12[i] * (2.0 * m23[i] * m13[i] - m12[i] * m33[i]) - m22[i] * m13[i] ** 2

cpdef cython_loop(df):

    det = np.zeros_like(df.m11.values)
    loop_cython(det, df.m11.values, df.m12.values, df.m13.values, df.m22.values, df.m23.values, df.m33.values)

    return pd.Series(det, index=df.index)
```

```python
det, t = timing(cython_loop, df, r=100)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0004649 s

```python
pd.testing.assert_series_equal(det, det_ref)
```

This is not as fast as Numba but close.

## Cython parallel

Similarly to what we did with Numba, we use here the `prange` looping function. Here is a quote from the [documentation](https://cython.readthedocs.io/en/latest/src/userguide/parallelism.html):

> cython.parallel.prange[[start,] stop[, step][, nogil=False][, schedule=None[, chunksize=None]][, num_threads=None]]
> This function can be used for parallel loops. OpenMP automatically starts a thread pool and distributes the work according to the schedule used.
> Thread-locality and reductions are automatically inferred for variables.

```cython
%%cython --compile-args=-fopenmp --link-args=-fopenmp

cimport cython
from cython.parallel import prange

import numpy as np
import pandas as pd

@cython.binding(False)
@cython.boundscheck(False)
@cython.wraparound(False)
@cython.initializedcheck(False)
cdef void loop_cython_para(double[:] det, double[:] m11, double[:] m12, double[:] m13, double[:] m22, double[:] m23, double[:] m33) nogil:

    cdef Py_ssize_t i

    for i in prange(m11.shape[0], nogil=True):
        det[i] = m11[i] * (m22[i] * m33[i] - m23[i] ** 2) + m12[i] * (2.0 * m23[i] * m13[i] - m12[i] * m33[i]) - m22[i] * m13[i] ** 2

cpdef cython_loop_para(df):

    det = np.zeros_like(df.m11.values)
    loop_cython_para(det, df.m11.values, df.m12.values, df.m13.values, df.m22.values, df.m23.values, df.m33.values)

    return pd.Series(det, index=df.index)
```

```python
det, t = timing(cython_loop_para, df, r=100)
print(f"Elapsed time: {t:8.7f} s")
```

    Elapsed time: 0.0003041 s

The parallel version is a little faster that the sequential one but we should test it on larger dataframes.

## Global comparison

We start by comparing all methods over small size dataframes.

### Small size dataframes

```python
def compare_all_methods(rng, funcs, df_sizes):

    column_names = ["m11", "m12", "m13", "m22", "m23", "m33"]
    time_df = pd.DataFrame()
    for n in df_sizes:

        df = pd.DataFrame(data=rng.random((n, 6)), columns=column_names)

        d = {}
        for func in funcs:
            _, t = timing(func, df)
            d[func.__name__] = t
        df_tmp = pd.DataFrame(d, index=[n])
        time_df = pd.concat((time_df, df_tmp), axis=0)

    return time_df
```

```python
funcs = [
    pandas_vectorize,
    iterrows_array,
    apply_array,
    itertuples_scalars,
    np_apply_along_axis,
    np_vectorize_scalars,
    map_scalars,
    swifter_apply,
    dask_df_map_partitions,
    polars_apply,
    polars_vectorize,
    numba_loop,
    numba_loop_para,
    cython_loop,
    cython_loop_para,
]

time_df = compare_all_methods(rng, funcs, df_sizes=[1_000, 10_000])
c_max = time_df.iloc[-1]
c_max = c_max.sort_values(ascending=False)
columns = c_max.index.values.tolist()
time_df = time_df[columns]
```

```python
ax = time_df.plot.bar(stacked=False, figsize=(16, 8), logy=True, rot=0)
_ = ax.set_ylim(1.0e-5)
_ = ax.set(
    title="Timing comparison of various dataframe looping methods",
    xlabel="Dataframe length",
    ylabel="Elapsed_time [s] (log scale)",
)
```

Only the Numba and Cython methods are significantly faster than Pandas' built_in vectorization (`pandas_vectorize`)!

```python
pd.options.display.float_format = "{:,.2e}".format
```

```python
time_df.T
```

      1000
      10000

      iterrows_array
      2.50e-02
      2.76e-01

      swifter_apply
      4.08e-03
      4.41e-02

      apply_array
      3.56e-03
      3.75e-02

      np_apply_along_axis
      2.79e-03
      3.71e-02

      dask_df_map_partitions
      1.11e-02
      1.92e-02

      itertuples_scalars
      1.66e-03
      1.56e-02

      polars_apply
      2.82e-03
      1.21e-02

      map_scalars
      9.42e-04
      9.08e-03

      np_vectorize_scalars
      7.03e-04
      6.39e-03

      polars_vectorize
      2.39e-03
      2.30e-03

      pandas_vectorize
      1.50e-03
      1.52e-03

      numba_loop_para
      1.68e-04
      1.84e-04

      cython_loop_para
      1.43e-04
      1.81e-04

      cython_loop
      1.06e-04
      1.20e-04

      numba_loop
      8.01e-05
      9.84e-05

### Medium to large size dataframes

```python
funcs = [
    pandas_vectorize,
    numba_loop,
    numba_loop_para,
    cython_loop,
    cython_loop_para,
]

time_df = compare_all_methods(rng, funcs, df_sizes=[1_000_000, 10_000_000])
c_max = time_df.iloc[-1]
c_max = c_max.sort_values(ascending=False)
columns = c_max.index.values.tolist()
time_df = time_df[columns]
```

```python
ax = time_df.plot.bar(stacked=False, figsize=(16, 8), logy=True, rot=0)
_ = ax.set_ylim(1.0e-5)
_ = ax.set(
    title="Timing comparison of various dataframe looping methods",
    xlabel="Dataframe length",
    ylabel="Elapsed_time [s] (log scale)",
)
```

```python
time_df.T
```

      1000000
      10000000

      pandas_vectorize
      3.95e-02
      4.10e-01

      numba_loop
      3.92e-03
      7.33e-02

      cython_loop
      4.40e-03
      5.29e-02

      cython_loop_para
      3.21e-03
      3.55e-02

      numba_loop_para
      2.35e-03
      2.53e-02

Let's focus on the Numba and Cython methods.

```python
funcs = [
    numba_loop,
    numba_loop_para,
    cython_loop,
    cython_loop_para,
]

out = perfplot.bench(
    setup=lambda n: pd.DataFrame(data=rng.random((n, 6)), columns=column_names),
    kernels=[(lambda df: func)(df) for func in funcs],
    labels=[func.__name__ for func in funcs],
    n_range=[10**k for k in range(3, 9)],
)
```

```python
def plot_timings(out, figsize=(12,12)):
    labels = out.labels
    ms = 10
    fig = plt.figure(figsize=figsize)
    ax = fig.add_subplot(1, 1, 1)

    for i, label in enumerate(labels):
        plt.loglog(out.n_range, out.timings_s[i], "o-", ms=ms, label=label)
    markers = cycle(
        ("", "o", "v", "^", "<", ">", "s", "p", "P", "*", "h", "X", "D", ".")
    )
    for i, line in enumerate(ax.get_lines()):
        marker = next(markers)
        line.set_marker(marker)
    plt.legend()
    plt.grid("on")
    _ = ax.set(
        title="Timing comparison of various dataframe looping methods",
        xlabel="Dataframe length (log scale)",
        ylabel="Elapsed_time [s] (log scale)",
    )

    return ax
```

```python
ax = plot_timings(out)
```

## Conclusion

Pandas built-in vectorization is a very good solution when possible. If we want to be faster with no extra pain, Numba is the best solution. Cython is about as fast as Numba, flexible, but more involving.

gha_language: SCSS


text:
# Foodbank3

Foodbank3 is an app that helps people in need of food place requests which donors can fulfill.

To install and start the app run:
    git clone
    bundle install
    rails db:migrate
    rail s
Then open the browser and redirect to 'http://localhost:3000'


text:
# Changelog

## Halley 3.x.x

### Breaking Changes
* `Game::startGame()` no longer takes API as a parameter. Use `Game::getAPI()` instead or storing it from this method.
* `Halley::Maybe<T>` is now deprecated. Use `std::optional<T>` instead.
* `EntityStage::createWorld()` no longer takes `createComponent` and `createSystem` arguments.
* Removed `Sprite::getOriginalSize()`. Use `Sprite::getUncroppedSize()` instead.
* Removed `Sprite::getRawSize()`. Use `Sprite::getSize()` instead.
* Removed `Halley::AABB`. Use `Halley::Rect4f` instead.
* `UIRoot` constructor changed, takes const `Halley::HalleyAPI&` instead of `const AudioAPI*`.
* `UITextInput` constructor changed, no longer takes `std::shared_ptr<InputKeyboard>`.
* `UISpinControl` constructor changed, no longer takes `std::shared_ptr<InputKeyboard>`.
* `UIWidget` can no longer be focused by calling `UIWidget::setFocus(true)`. Use `UIWidget::focus()` instead.
* `Halley::Keys` renamed to `Halley::KeyCode` (old name is still available as an alias, but is deprecated).
* `Halley::KeyCode` and `Halley::KeyMods` are now strongly typed (`enum class`).
* Removed `Halley::KeyCode::Escape`. Use `Halley::KeyCode::Esc` instead.
* Removed `Halley::KeyCode::Return`. Use `Halley::KeyCode::Enter` instead.
* Several other less common keys in `Halley::KeyCode` have been removed or renamed. See `input_keys.h` for the current list.

repository: amzeratul/halley

branch_name: refs/heads/develop

repo_name: amzeratul/halley