text:
---
id: 987
title: Running Unity3D webplayer on your linux box
date: 2014-03-11T00:35:32+00:00
author: acidjunk
layout: post
guid: http://www.renedohmen.nl/blog/?p=987
permalink: /2014/03/running-unity3d-webplayer-on-your-linux-box/
categories:
  - Computerz
header-img: "img/post-bg-01.jpg"
---
As NaCl builds are not working at all in Unity4.3 and Flash is most likely to be removed from future Unity versions the only native way to play Unity games on your Linux box is via a standalone game. But with the same technique that enables watching Netflix on Ubuntu it’s also possible to install the Unity Webplayer plguin for Chrome and Firefox.

First remove previous PPA as they will no be updated anymore. You can skip this step if you didn’t install older compholio/pipelight versions.

sudo add-apt-repository --remove ppa:mqchael/pipelight
sudo add-apt-repository --remove ppa:mqchael/pipelight

Now you are ready to add the new multiplugin pipelight PPA’s:

sudo add-apt-repository ppa:pipelight/stable
sudo apt-get update
sudo apt-get install --install-recommends pipelight-multi
sudo pipelight-plugin --update

Note: you will see a dialog asking you to confirm the install of the Microsoft core-fonts; this is a package form the Ubuntu multiverse repository. It’s enabled by default on recent Ubuntu distro’s. Instructions to enable it on older distro’s can be found [here](http://askubuntu.com/questions/89096/how-do-i-enable-the-multiverse-repository).

Go ahead and enable silverlight, this is not needed for Unity webplayer, but watching Netflix is relaxing and it can be used to test the whole system.

sudo pipelight-plugin --enable silverlight

(wait for wine to finish installing)

Now you need an Chrome extension called “User-Agent Switcher for Chrome” so you can force the user agent to “Windows Firefox 15”. You can even spoof it on a permanent basis for the domain netflix.com.

[](http://www.renedohmen.nl/blog/wp-content/uploads/2014/03/Schermafdruk-van-2014-03-11-005155.png)

After that enable the Unity3d plugin:

sudo pipelight-plugin --enable unity3d

(wait for wine to finish installing)

Important: set you user agent to Safari5, OSX; this is the only user-agent that worked for me!

Now try it with some games:

http://www.kongregate.com/games/flippfly/race-the-sun

http://dev.formatics.nl/tictactoe

[](http://www.renedohmen.nl/blog/wp-content/uploads/2014/03/Schermafdruk-van-2014-03-11-011958.png)


text:
---
layout: post
title: How to create a Time Series graph with Angular and D3
excerpt: "I recently implemented a time series graph in a web app that uses Angular on the front end and I wanted to share what I learned."
modified: 2016-11-24
tags: [tutorial, Angular, D3, JavaScript]
comments: true
---

While building my senior capstone project, [iSprinkle](https://github.com/cagmz/iSprinkle), I needed a way to display a time series graph of data.
A working demo can be found on my [Codepen](https://codepen.io/cagmz/pen/qqRGoY) and a gist over on [Github](https://gist.github.com/cagmz/eca1fae44b0c4d9de522104607e4f474). Here's what I learned.

Implementing a time series graph into an existing Angular app requires a few dependecies:

* [D3](https://github.com/cagmz/iSprinkle), a data visualization library
* [NVD3](http://nvd3.org/), a reusable chart library for D3
* [Angular-nvD3](http://krispo.github.io/angular-nvd3/#/), for AngularJS directives
* [Moment.js](http://momentjs.com/), a library for working with JavaScript dates/times

Note: As of this writing, nvd3 supports D3 versions 3.5.3 and above, but not D3 4.x .

In this minimal example, our Angular app and controllers are named like so:

~~~ html

		<nvd3 options="options" data="data"></nvd3>

~~~

However, because some dependecies depend on others, we'll need to make that they load in a particular order:

~~~ html

    <link rel="stylesheet" type="text/css" href="https://cdnjs.cloudflare.com/ajax/libs/nvd3/1.8.4/nv.d3.min.css">

~~~

Finally, most of the code for the graph is found within the `home` controller. In `$scope.options`, we'll find the configuration options for the chart. The actual data for the chart is stored in `$scope.data`, which should contain one or more time series objects in this format:

~~~ javascript
// xy_coordinate_pairs is a time series and should be an array of x, y coordinate pairs
// {x: xVal, y: yVal }
return [{
  values: xy_coordinate_pairs,
  key: 'Chart Key',
  color: '#1b75ba'
}];
~~~

In this example, the `generateTimeSeries` function is called to

~~~ javascript
// 1) Declare app
var nvd3_timeseries = angular.module('nvd3_timeseries', ['nvd3']);

// 2) Declare controller
nvd3_timeseries.controller('demo', ['$scope', function($scope) {

    // 3) Chart configuration
    $scope.options = {
        chart: {
            type: 'lineChart',
            height: 450,
            margin: {
                top: 20,
                right: 20,
                bottom: 40,
                left: 55
            },
            x: function(d) {
            /*
            Moment.js will attempt to parse the x coordinate
            and return a value in milliseconds
            */
                return moment.utc(d.x).valueOf();
            },
            y: function(d) {
                return d.y;
            },
            useInteractiveGuideline: true,
            xAxis: {
                axisLabel: 'Date',
                tickFormat: (function(d) {
                    // 3.1) Format the x ticks
                    return d3.time.format('%-m/%-d/%-Y')(new Date(d));
                })
            },
            yAxis: {
                axisLabel: 'Cats',
                axisLabelDistance: -10
            },
            callback: function(chart) {}
        },
        title: {
            enable: true,
            text: 'Time Series graph'
        }
    };

    $scope.dateRange = function dateRange(startDate, endDate) {
        // http://stackoverflow.com/a/23796069
        var dates = [];

        var currDate = startDate.clone().startOf('day');
        var lastDate = endDate.clone().startOf('day');

        while (currDate.add(1, 'days').diff(lastDate) <= 0) {
            dates.push(currDate.clone().toDate());
        }

        return dates;
    };

    // 4) Chart data
    $scope.data = generateTimeSeries();

    function generateTimeSeries() {
        // Generate an array of dates for plotting
        var dateRange = $scope.dateRange(moment().subtract(7, 'days'), moment());

        // Plottable will be an array of (x, y) objects
        var plottable = dateRange;

        plottable.forEach(function to_utc(date, index) {
            console.log('a[' + index + '] = ' + date);
            // set x value to date and y to a random value
            var rand = Math.round((Math.random() * (100 - 1) + 1));
            var point = {
                x: date,
                y: rand
            };
            // Replace original element
            plottable[index] = point;
        });

        // Return the plottable array for plotting
        return [{
            values: plottable,
            key: 'Cats',
            color: '#1b75ba'
        }];
    };
}]);
~~~

As an aside, although this chart is initiated with data instantly, we can set an empty data value like so:

~~~ javascript
$scope.data = [{
    "key": "Some key",
    "values": []
}];
~~~

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
- [Task](#task)
   - [Useful links](#useful)
   - [Close port 5000](#close-port)
   - [Manage Docker](#manage)
   - [Manage Docker as a non-root user](#non-root)
   - [Start containers automatically](#start-container)
   - [Stop all docker containers](#stop-all)
   - [Manage Docker Compose tool](#docker-compose)

---
### Task
   - Docker configuration and administration.

---
### Useful Docker links

   - [How To Install and Use Docker on Ubuntu 18.04](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04)
   - [Post-installation steps for Linux](https://docs.docker.com/install/linux/linux-postinstall)
   - [Get Started, Part 1: Orientation and setup](https://docs.docker.com/get-started)
   - [Start containers automatically](https://docs.docker.com/config/containers/start-containers-automatically)

Docker Compose links:
   - [Docker compose tutorial for beginners by example [all you need to know]](https://youtu.be/4EqysCR3mjo)
   - [What is Docker Compose | How to create docker compose file](https://youtu.be/HUpIoF_conA)

---
### Close port 5000

Close port 5000 for security reasons.
You should login via SSH graphical interface
`ssh -X user@website -p port` or via X2Go Client
and then use `http://localhost:5000` to connect to DIGITS.

```shell script
# Open port 5000 for NVIDIA DIGITS (not secure)
##sudo ufw allow 5000
# Close port 5000 for NVIDIA DIGITS
sudo ufw delete allow 5000
# Check the status
sudo ufw status
```

---
### Manage Docker
[Manage Docker containers](https://docs.docker.com/engine/reference/commandline/container/)

```shell script
# To show only running containers
docker ps
# or
docker container ls

docker container stop 6827ed3a784f  # stop container

# To show all containers
docker ps -a
# or
docker container ls -a

# Check status of the docker service. Press "Q" key to exit.
sudo systemctl status docker
# or
systemctl is-active docker
```

---
### Manage Docker as a non-root user

If you don’t want to preface the `docker` command with `sudo`,
create a Unix group called `docker` and add users to it.

:exclamation: **Only trusted users should be allowed
to control your Docker daemon.** :exclamation:

```shell script
# Show docker group
cat /etc/group | grep "docker"
# Create the docker group if it doesn't exist
sudo groupadd docker
# Add trusted users to the docker group.
# I don't know, is it possible to add existing group to docker group?
sudo usermod -aG docker $USER
# Log out and log back.

# Verify that you can run docker commands without sudo.
docker run hello-world

# To delete user from a docker group
sudo deluser username docker
```

---
### Start containers automatically

It starts container, but not [Nvidia DIGITS](old/12_Nvidia_DIGITS.md) itself.

```shell script
# Autostart
docker run -dit --restart unless-stopped nvidia/digits:latest

# Stop autostart
docker run -dit --restart no nvidia/digits:latest
```

---
### Stop all docker containers

```shell script
sudo docker kill $(sudo docker ps -q)
```

---
### Manage Docker Compose tool

[Docker Compose](https://docs.docker.com/compose) is a tool
for defining and running multi-container Docker applications.

```shell script
# Install Docker Compose tool. WARNING: current version could be different.
sudo curl -L https://github.com/docker/compose/releases/download/1.24.0-rc1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Test the installation
docker-compose --version
```


text:
---
id: index
title: Setup
sidebar_label: Overview
slug: /setup
---

A typical RIasC setup consists of at least one central server and multiple distributed agent nodes.
The setup is therefore is organized in two steps:

- [Setup of Server(s)](./server.md)
- [Setup of Agents](./agent.md)

Both steps can be automated by Ansible an open-source software provisioning, configuration management, and application-deployment tool enabling infrastructure as code

repository: petraraussi/riasc

repo_name: petraraussi/riasc


text:
---
layout: page
title: About
permalink: /about/
---

Hi! I'm cow-co, and I do a whole bunch of different things in my spare time. This blog is where I kinda brain-dump my thoughts and guides on how to do various things, such as security, machine learning, and physics.

I have a few different GitHub accounts. My main one is [cow-co](https://github.com/cow-co), where I work on web apps and assorted other miscellaneous things. I also have [cybersecmoo](https://github.com/cybersecmoo), where I work on cybersecurity projects, and [sci-cow](https://github.com/sci-cow), where I work on high-energy physics simulations.

gha_language: CSS

gha_license_id: MIT


text:
# Migration from SDK v1 to SDK v2

## Installation

Installation involves adjusting your `package.json` to depend on version `2.0.0`.

```shell
npm install --save amazon-chime-sdk-js@2
```

## Interface changes

Version 2 of the Amazon Chime SDK for JavaScript makes a small number of interface
changes, as well as removing some deprecated interfaces.

In many cases you should not need to adjust your application code at all. This will be the case if:

* You do not implement `AudioVideoFacade` or `DeviceController` yourself.
* You do not explicitly call `enableWebAudio` on any instances of `DeviceController` or
  `AudioVideoFacade`, or use the `MeetingSessionConfiguration` field `enableWebAudio`.
* You already handle errors in `chooseAudioInputDevice` with `catch`.
* You neither directly call, nor implement, the `bindAudio*` methods on `AudioMixController`.

If your application does not meet all four criteria, read on.

### Removing `enableWebAudio`

The `enableWebAudio` method on `DefaultDeviceController` would produce unexpected results if
called after the first audio device was selected, and as a synchronous API it was not possible
to reimplement it to behave correctly.

Additionally, it was not documented how the API should behave if Web Audio were to be disabled
while in use, or how it should be implemented correctly by other `DeviceController` classes.

This API method has been removed entirely, along with the corresponding field on
`MeetingSessionConfiguration`. The `MeetingSession` will no longer call `enableWebAudio` on the
corresponding `DeviceController`.

Applications should instead use the constructor argument added to
`DefaultDeviceConfiguration` to enable Web Audio at point of construction.

If your code looked like this:

```typescript
const configuration = new MeetingSessionConfiguration(…);
configuration.enableWebAudio = true;
…
const deviceController = new DefaultDeviceController(logger);
this.meetingSession = new DefaultMeetingSession(configuration, logger, deviceController);
this.audioVideo = this.meetingSession.audioVideo;
```

change it to

```typescript
const configuration = new MeetingSessionConfiguration(…);
…
const deviceController = new DefaultDeviceController(logger, { enableWebAudio: true });
this.meetingSession = new DefaultMeetingSession(configuration, logger, deviceController);
this.audioVideo = this.meetingSession.audioVideo;
```

If your code looked like this:

```typescript
const configuration = new MeetingSessionConfiguration(…);
…
const deviceController = new DefaultDeviceController(logger);
this.meetingSession = new DefaultMeetingSession(configuration, logger, deviceController);
this.audioVideo = this.meetingSession.audioVideo;
…
const enableWebAudio = await checkWhetherWebAudioNeeded();
deviceController.enableWebAudio(enableWebAudio);
```

change it to

```typescript
const configuration = new MeetingSessionConfiguration(…);
// …
const enableWebAudio = await checkWhetherWebAudioNeeded();
const deviceController = new DefaultDeviceController(logger, { enableWebAudio });
this.meetingSession = new DefaultMeetingSession(configuration, logger, deviceController);
this.audioVideo = this.meetingSession.audioVideo;
…
```

---

### Update `bindAudioElement()`, `bindAudioStream()` and `bindAudioDevice()` to return `Promise<void>` instead of `boolean`
The `bindAudioElement()` API in `AudioMixControllerFacade` was previously a synchronous function which used to return `boolean`.
Under the hood, it used to call [`setSinkId()`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/setSinkId)
which is an asynchronous function.

As part of this change, three APIs in `AudioMixController` were changed to return `Promise<void>`. These APIs are listed below.

```typescript
// Old Syntax
bindAudioElement(element: HTMLAudioElement): boolean;
bindAudioStream(stream: MediaStream): boolean;
bindAudioDevice(device: MediaDeviceInfo | null): boolean;

// New Syntax
bindAudioElement(element: HTMLAudioElement): Promise<void>;
bindAudioStream(stream: MediaStream): Promise<void>;
bindAudioDevice(device: MediaDeviceInfo | null): Promise<void>;
````

Additionally, `AudioMixController`'s constructor now takes an additional optional `logger` parameter.
If the `logger` is passed, it will log the errors for any of the operations in `AudioMixController`.

```typescript
constructor(private logger?: Logger) {}
```

If your code looked like this

```typescript
audioMixController.bindAudioDevice({ deviceId: sinkId });
audioMixController.bindAudioElement(new Audio());
audioMixController.bindAudioStream(destinationStream.stream);
```

change it to

```typescript
try {
  await audioMixController.bindAudioDevice({ deviceId: this.sinkId });
} catch (e) {
  console.error('Failed to bind audio device', e);
}
try {
  await audioMixController.bindAudioElement(new Audio());
} catch (e) {
  console.error('Failed to bind audio element', e);
}
await audioMixController.bindAudioStream(destinationStream.stream);
```

Additionally, if the browser does not support [`setSinkId()`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/setSinkId) API, the `bindAudioDevice()` will throw an
error with a message 'Cannot select audio output device. This browser does not support setSinkId.'
Check https://caniuse.com/?search=setSinkId for the setSinkId browser support.

---

### Introducing `AudioInputDevice` and `VideoInputDevice`

These two types describe `DeviceController`'s methods for selecting audio and
video inputs respectively. They both include the space of `Device`s, which are the 'intrinsic'
device kinds provided by the browser: identifiers, constraints, and streams. `AudioInputDevice`
extends these by adding the concept of an `AudioTransformDevice`, which can rewrite an inner
device on request, and optionally provide an arbitrary Web Audio graph to use as an additional
transformation pipeline.

You can use `AudioTransformDevice` to implement effects such as reverb, gain, _etc_.

Because `chooseAudioInputDevice` and `chooseVideoInputDevice` now have new type signatures, if
you implement the related interfaces (`AudioVideoFacade` or `DeviceController`) yourself, you will need to adjust your code.

If you have an implementation like:

```typescript
class MyDeviceController implements DeviceController {
  async chooseAudioInputDevice(device: Device): Promise<void> {
    // device must be a string, stream, constraints, or null.
    // …
    return permission;
  }
}
```

you must change your code to be:

```typescript
import {
  // …
  isAudioTransformDevice,
} from 'amazon-chime-sdk-js';

class MyDeviceController implements DeviceController {
  async chooseAudioInputDevice(device: AudioInputDevice): Promise<void> {
    if (isAudioTransformDevice(device)) {
      // Handle transform devices, should you need to.
      throw new Error('My app does not use transform devices.');
    }
    // Previous code can remain unchanged.
    // …
  }
}
```

A similar adjustment is needed for `VideoInputDevice`. v2.0 does not include any
`VideoTransformDevice`s.

If you use the type `Device` for a field or variable and pass that value to
`choose{Audio,Video}InputDevice`, your code should continue to work without changes.

---

### Introducing the `GetUserMediaError` type

v2.0 introduces new error types which will be thrown in case of any failures in `chooseAudioInputDevice` and `chooseVideoInputDevice` APIs in the `DeviceController`.

Here is the list of new error classes introduced in version 2.0:

* `GetUserMediaError`
  * `NotFoundError`
  * `NotReadableError`
  * `OverconstrainedError`
  * `PermissionDeniedError`
  * `TypeError`

Here is an example of handling `PermissionDeniedError`:

```typescript
try {
  await this.audiovideo.chooseAudioInputDevice(device);
} catch (e) {
  if (e instanceof PermissionDeniedError) {
    console.error('Permission denied', e);
  }
}
```

---

### Introducing `supportsSetSinkId()` in `DefaultBrowserBehavior`

This new helper API returns a boolean `true` if the browser supports [`setSinkId()`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/setSinkId), otherwise returning `false`.

Here is how the code would look like:

```typescript
if (!this.browserBehavior.supportsSetSinkId()) {
  throw new Error(
    'Cannot select audio output device. This browser does not support setSinkId.'
  );
} else {
  console.log('The browser supports setSinkId() API');
}
```

---

### Deprecating legacy screen share

From version 2.0 onwards, the Amazon Chime SDK for JavaScript will no longer include the deprecated screen share API identified by `ScreenShareFacade` and `ScreenShareViewFacade` and all related code.
Customers should use our Video Based Content Sharing detailed in our [Content Share guide](https://aws.github.io/amazon-chime-sdk-js/modules/contentshare.html).

repository: aws/amazon-chime-sdk-js

branch_name: refs/heads/main

gha_language: TypeScript

repo_name: aws/amazon-chime-sdk-js


text:
FontShrink.cpp
Stephen Fried, November 2016

Reduce the size of an Arduino custom font by only including the characters you need in the header file.

This program will parse through the information found in an Adafruit Arduino GFX font definition header file
and extract only the bitmap information for the chcracters it needs. By eliminating the bitmap information
for unneeded characters (or "glyphs", in the header definition) you can dramatically reduce the amount
of Arduino memory needed to store the font.

INSTRUCTIONS:

1. Copy the header file you are trying to shrink (e.g. FreeSansBold24pt7b.h) to a separate directory.
You'll want to work on a copy of the header file, not the original.

2. Edit the header file to remove the keyword "PROGMEM" from the structure definitions. Most C/C++ compilers don't
understand the keyword and will fail to compile the code. The program will re-insert the PROGMEM keyword in the appropriate place as it builds
the output.

3. Go through this program and change every reference to the font name or font file to the name of your font. Look for
places that say "// ********** EDIT THIS **********". Be careful, some of the font name references are embedded as part of
a longer variable or struct name. Change ONLY the font name, leaving the rest of the variable/struct name in tact.

4. Edit the "wantedCharacters" variable definition below to include only the characters you are looking for.

5. Compile the program. I used Visual Studio to build this as a WIN32 console application, but the code should be generic and straightforward enough that you can easily
port it to your language/compiler/platform of choice.

6. Run the program, redirecting the standard output stream (stdout) to the new header file. For example:

C:\Users\me> FontShrink.exe > FreeSansBold24pt7b-mini.h

7. If you want to see runtime messages, uncomment the line that says "//#define __DEBUG__" and recompile. This will include printf debug statements
into the program. The debug statements will print to the standard error stream (stderr), so it won't affect the resulting header file sent to stdout.

8. Take the resulting header file ("i.e. FreeSansBold24pt7b-mini.h") and include that in your Arduino sketch.

This program was built for a specific use case and has not been extensively tested. However, feel free to adapt the code to your own needs.

This program is released into the wild as open-source software under Unilicense. See the file UNILICENSE.txt for terms and conditions.

license: Unlicense

repository: HandyGuySoftware/FontShrink

repo_name: HandyGuySoftware/FontShrink


text:
---
title: Function timeout in Python using the signal module
date: 2016-02-05 00:00:00 +0100
categories: [Python, timeout]
tags: [python, signal]
---

## The problem

Sometimes you may want to impose a timeout on a Python function. Why would you want to do such a thing? Let's say you're computing something but you know there are some hopeless scenarios where the computation just takes too long, and you'd be OK to just skip them and go on with the rest of the workflow.

For an illustration, the figure below shows several tasks. Those that take longer than the specified timeout should be aborted (orange) and the remaining tasks that take a reasonable amount of time should be executed normally (green).

![A timeout is a cutoff for the duration of a task](/assets/img/posts/function_timeout.png){: width="600"}

There are several ways in which setting a timeout on a function may be achieved such that the execution continues past the timed-out method. We will be examining two solutions here:
* in this post we will be using [`signal`] module;
* in the [next post][np] we will be using the [`multiprocessing`] module.

## Solution using the signal module

### What are signals?

[Signals][signal] are a form of [inter-process communication][ipc] that only applies to [POSIX][]-compliant operating systems. Note that Microsoft Windows is _not_ POSIX-compliant, so this solution cannot be used when running Python on Windows.

Signals can be regarded as software interrupts sent from the kernel to a process in order to inform it that a special event took place. The process receiving the signal can choose to handle it in a specific way (if the program was written with this intention, that is). Otherwise, signals are handled in a default manner specified by the default signal handlers. For example, when you press Ctrl + C in your Linux terminal to stop a running program, you are in fact sending it the SIGINT signal. The default handler for SIGINT is to stop process execution.

Check out this [article][stackabuse] for more information on handling UNIX signals in Python.

### Using signals to set a timeout

Suppose we have a method `do_stuff()` that can sometimes be very time-consuming. We'll be keeping this very simple:

```python
import time

def do_stuff(n):
    time.sleep(n)
```

Let's say we only want to run `do_stuff()` to completion if it finishes in less than 6 seconds. With the `signal` module, this can be achieved if we set a timer (an "alarm") for 6 seconds just before calling `do_stuff()`. If the timer runs out before the function completes, SIGALRM is sent to the process. We will therefore be using `signal.alarm(6)` to set a timer for 6 seconds before calling `do_stuff()`. Note that the argument to `signal.alarm()` must be an integer. Let's check what happens:

```python
import signal
import time

def do_stuff(n):
    time.sleep(n)
    print('slept for {}s'.format(n))

def main():
    signal.alarm(6)
    do_stuff(2)
    do_stuff(5)
    do_stuff(6)

if __name__ == '__main__':
    main()
```

Here is the output:

```
$ python timeout_signal.py
slept for 2s
Alarm clock
$ echo $?
142
```

What happened? Well, the timer was set to 6 seconds and it finally ran out, one second before the second call to `do_stuff()` would have normally finished. The process exits with code 142 (SIGALRM). Let us change the `main()` function to reset the alarm after each call to `do_stuff()`:

```python
def main():
    signal.alarm(6)
    do_stuff(2)
    signal.alarm(0)

    signal.alarm(6)
    do_stuff(5)
    signal.alarm(0)

    signal.alarm(6)
    do_stuff(6)
    signal.alarm(0)
```

Note that `signal.alarm(delay)` arms a timer for `delay` seconds. This means that if `do_stuff()` takes exactly `delay` seconds to complete, SIGALRM gets transmitted nonetheless.

We now obtain:

```
slept for 2s
slept for 5s
Alarm clock
```

Next, we will define a handler for SIGALRM. A **handler** is a function that "handles a signal" in the specific way we instruct it to behave. User-defined handlers are used to override the default signal handlers. For example, suppose you want your program to ask the user to confirm her desire to quit the program when she presses Ctrl + C in the terminal. In this case you'd need a SIGINT handler that only exits upon confirmation. Note that signal handlers must respect a fixed prototype. To quote from the [Python documentation][`signal`]:

> The handler is called with two arguments: the signal number and the current stack frame (...).

Even if a a signal handler does not use these two arguments, they must be present in the handler's prototype (and no other arguments may be passed). Here is our simple handler; it just throws a `TimeoutError`:

```python
def handle_timeout(sig, frame):
    raise TimeoutError('took too long')
```

This handler only makes sense if it is registered for SIGALRM. Registering `handle_timeout()` for SIGALRM should be added to the `main()` function of the script above. Here is how to do it:

```python
signal.signal(signal.SIGALRM, handle_timeout)
```

By re-running our script, we can see that it now stops with a `TimeoutError`:

```
slept for 2s
slept for 5s
Traceback (most recent call last):
  File "timeout_signal.py", line 31, in <module>
    main()
  File "timeout_signal.py", line 26, in main
    do_stuff(7)
  File "timeout_signal.py", line 10, in do_stuff
    time.sleep(n)
  File "timeout_signal.py", line 6, in handle_timeout
    raise TimeoutError('took too long')
TimeoutError: took too long
```

This is great, we have an exception now -- and exceptions are something we can deal with in Python.

### Making execution continue past the timeout

Next, let's handle the `TimeoutError`. We will change our script such that it loops indefinitely and at each iteration through the loop it attempts to `do_stuff()` for a random number of seconds between 1 and 10. If `do_stuff()` is called with 6 seconds or more, then SIGALRM is sent and handled by raising a `TimeoutError`. We catch that `TimeoutError` and continue execution until hitting Ctrl + C. As an added bonus, we also include a handler for SIGINT (Ctrl + C).

```python
import sys
import random
import signal
import time

def handle_sigint(sig, frame):
    print('SIGINT received, terminating.')
    sys.exit()

def handle_timeout(sig, frame):
    raise TimeoutError('took too long')

def do_stuff(n):
    time.sleep(n)

def main():
    signal.signal(signal.SIGINT, handle_sigint)
    signal.signal(signal.SIGALRM, handle_timeout)

    max_duration = 5

    while True:
        try:
            duration = random.choice([x for x in range(1, 11)])
            print('duration = {}: '.format(duration), end='', flush=True)
            signal.alarm(max_duration + 1)
            do_stuff(duration)
            signal.alarm(0)
        except TimeoutError as exc:
            print('{}: {}'.format(exc.__class__.__name__, exc))
        else:
            print('slept for {}s'.format(duration))

if __name__ == '__main__':
    main()
```

So how does the execution continue past the first timeout? As we've seen above, we installed a handler for SIGALRM (lines 12-13 and 22) that raises a `TimeoutError`. Exception handling is performed in the `main()` function inside an infinite loop (lines 26-36). If `do_stuff()` succeeds, the script displays a message informing the user for how long the function ran (lines 35-36). If the `TimeoutError` is caught, it is simply displayed and the script continues.

Here is how the output might look like:

```
duration = 3: slept for 3s
duration = 1: slept for 1s
duration = 10: TimeoutError: took too long
duration = 7: TimeoutError: took too long
duration = 5: slept for 5s
duration = 9: TimeoutError: took too long
duration = 2: slept for 2s
duration = 5: slept for 5s
duration = 1: slept for 1s
duration = 6: TimeoutError: took too long
duration = 2: slept for 2s
duration = 10: ^CSIGINT received, terminating.
```

### Drawbacks

Well, _it works_ but there are two problems with this solution:

1. As mentioned in the introduction to signals, this mechanism is only present on UNIX-like systems. If the script needs to run in a classic Windows environment, the `signal` module is not suitable.
2. A SIGALRM can arrive at any time; however, its handler may only be ran between **atomic** instructions. By definition, atomic instructions cannot be interrupted. So if the timer runs out during such an operation, even though SIGALRM is sent, it won't be handled until that long computation you've been trying to abort finally completes. Typically, when using external libraries implemented in pure C for performing long computations, the handling of SIGALRM may be delayed.

## Conclusion

In this post we've seen a simple solution involving UNIX signals that may be used in some situations to set a timeout on a Python function. However, this solution is less than ideal for two reasons: the operating system must be POSIX-compliant and it can only work between atomic operations. In the [next post][np] we will examine a better solution using the `multiprocessing` module.

## Further reading

* [Signals][signal] (Wikipedia)
* [`signal`][] (Python documentation)
* [Handling UNIX signals in Python][stackabuse] (Frank Hofmann on stackabuse)

<!-- links -->

[`signal`]: https://docs.python.org/3/library/signal.html
[`multiprocessing`]: https://docs.python.org/3/library/multiprocessing.html
[np]: {% post_url 2016-02-08-function-timeout-in-python-multiprocessing %}
[signal]: https://en.wikipedia.org/wiki/Signal_(IPC)
[ipc]: https://en.wikipedia.org/wiki/Inter-process_communication
[POSIX]: https://en.wikipedia.org/wiki/POSIX
[stackabuse]: https://stackabuse.com/handling-unix-signals-in-python/


text:
# UPDATE
### This has been deprecated, as this functionality [is possible natively in IINA](https://twitter.com/lhc70000/status/1010360204158894081) (which offers several improvements over this approach). This repository is archived for reference and learning purposes, only.

# Wadsworth for IINA
### Jump around the timeline using keyboard shortcuts

[Forked from Wadsworth for VLC](https://github.com/verbiate/Wadsworth-for-VLC)

On YouTube, pressing 0-9 will jump you to a percentage of the timeline (e.g. pressing 3 will jump to 30% of the way through the video). In an effort to recreate this functionality, this is a set of scripts that enable jumping to a specific percentage location of the timeline in the [IINA video player for macOS](https://github.com/lhc70000/iina). You can use other tools (like System Preferences or Alfred) to assign keyboard shortcuts to each script and thereby replicate this functionality. An AppleScript is also included to create these shortcuts for you in System Preferences.

### INSTALLATION STEPS:
The easiest way to install is to use the included Automator Services to bring this functionality to IINA, and then use the keyboard shortcut AppleScript to automatically add keyboard shortcuts in System Preferences. Alternatively, there are also AppleScripts available that perform the same functions as the Automator Services, should you have a use for them.

## 1. Install the Automator Services

**INSTRUCTIONS:**
1. Copy the Automator scripts from [here](https://github.com/verbiate/Wadsworth-for-IINA/tree/master/01%20Automator%20scripts) into the user's Library > Services folder (~/Library/Services)
2. Double click on one of the newly copied Automator scripts (this causes the scripts to appear in the Services menu)
3. The scripts can now be found inside the IINA > Services menu

**KNOWN ISSUES:**
* In order to jump to a new point on the timeline, IINA requires that the mouse cursor moves over the playback window to make the playhead visible. This script works around this by moving the mouse cursor over the window for a split second. If a user is moving their mouse while triggering the shortcut, it's possible for the mouse to miss the window. This will cause a spike in CPU usage until the mouse cursor enters the playback window.

## 2. Create Keyboard Shortcuts
_This is an AppleScript which creates keyboard shortcuts. This is not required—you can choose to create your own keyboard shortcuts, or none at all (?), but this creates them for you automatically._

**INSTRUCTIONS:**
1. Run the Create Keyboard Shortcuts AppleScript
2. The script will ask you to grant Script Editor [permissions to Assistive Services](https://developer.apple.com/library/content/documentation/LanguagesUtilities/Conceptual/MacAutomationScriptingGuide/AutomatetheUserInterface.html#//apple_ref/doc/uid/TP40016239-CH69-SW1)
3. After granting permissions in System Preferences > Security & Privacy > Privacy > Accessibility, run the script again
4. The script will now create shortcuts for stops on the timeline from 0-90%
5. After this, you can now jump to any point of the timeline by pressing Command+Option and a number (e.g. 3 for 30%)

repository: verbiate/Wadsworth-for-IINA

repo_name: verbiate/Wadsworth-for-IINA


text:
---
layout: page
title: Consulting Services
# subtitle: I'm available for consulting services
---

On this page you'll find services I can provide to companies and other research institutions.

## Research & Development

I'm offering several consulting services related to R&D activities. All the services are primarily within field of
cyber security and information security.

R&D is a very specific and important field in which, at least in Croatia, neither research institutions nor
companies are good at with a few exceptions. The reason is that this field combines research that is specialty
of research institutions with development that is specialty of companies.

#### Idea Development

If you are thinking about starting a new R&D project and you need someone to help you with steering your
thoughts while discussing what could be done, or you need help in identifying research topics or issues,
then I can help you. Some of the project ideas I helped to articulate are:

* creating a tool to train incident handlers,
* increasing security of application when deployed in cloud environment.

I'm currently working on some other ideas, too. But since they are still in the stage of idea I won't talk
about them for now.

#### Project Application

#### Project Execution - Research Services


text:
# Highlight Ads on Google Search Results

This simple extension to highlight the ads in Google search results.

> ## Google’s ads just look like search results now

You might have noticed that Google slowly changed how they present ads in the search result. Now it is very difficult to differentiate an ad from an organic search result.
![](images/g-ads-change.jpg)
_source(https://twitter.com/GinnyMarvin/status/757606109322633216)_

You can read more about it on TechCrunch:
https://techcrunch.com/2020/01/23/squint-and-youll-click-it/

## What can we do? Write a plugin to highlight ads again!

Yes, its really easy to write a chrome extension to highlight the ads on Google search results, so that you can easily identify the ads from organic results. Just like this:

![extenstion-screenshot](images/extenstion-screenshot.png)

## Isn't it easier to use some existing extension for this?

Yes, it is. But not a lot.
Also, why should you trust someone else to inject random code into your browser while you could do it in a couple of minutes?

You just need 2 files.

1. [manifest.json](https://github.com/rohithb/highlight-ads-on-google-search-result/blob/master/manifest.json)
2. A css file with required styles. ([styles.css](https://github.com/rohithb/highlight-ads-on-google-search-result/blob/master/src/styles.css))

Refer the files on [github](https://github.com/rohithb/highlight-ads-on-google-search-result) for better understanding.

## Installing the extension

To install an unpacked extension:

1. Open the Extension Management page by navigating to **chrome://extensions**.
   (The Extension Management page can also be opened by clicking on the Chrome menu, hovering over More Tools then select Extensions.)
2. Enable Developer Mode by clicking the toggle switch next to Developer mode.
3. Click the **LOAD UNPACKED** button and select the extension directory.

## Warning

Since we are following a very simple approach, if Google changes the class names for the ads, the extension will stop working. But don't worry. Just open the inspect elements tool and update the class name in the css file with the new one.


text:
# Hyperscript/TSX reviver

This is a fork of the `sinuous/h` package from Sinuous. It was ported to
TypeScript, simplified in a few places, and now uses a general `api.patch()`
method to support DOM updates with any reactive library. There's no reactivity
baked into `haptic/dom`. Haptic configures reactivity in the primary `haptic`
package where it pairs this reviver with `haptic/state`.

It's 964 bytes min+gzip on its own.

Designed to be explicit, type safe, and interoperable with the Sinuous
ecosystem. The Sinuous repository lists some [community packages][1].

This `haptic/dom` package exports a vanilla JSX namespace that doesn't expect or
support any reactive functions as elements or attributes. To use a reactive
library of your choice, repeat how the `haptic` package configures `api.patch`
and the JSX namespace for `haptic/state`.

## `h(tag: Tag, props?: unknown, ...children: unknown[]): El | undefined`

```ts
type El = Element | Node | DocumentFragment;
type Tag = El | Component | [] | string;
type Component = (...args: unknown[]) => El | undefined;
```

The hyperscript reviver. This is really standard. It's how trees of elements are
defined in most frameworks; both JSX and traditional/transpiled `h()`-based
system alike.

The only notable difference from other frameworks is that functions inlined in
JSX will be serialized to strings. This is a deviation from Sinuous' reviver
which automatically converts any function to a `computed()` (their version of a
computed-signal) in order to support DOM updates. Haptic's reviver explicitly
only targets wires inlined in JSX - any non-wire function is skipped.

The following two uses of `h()` are equivalent:

```tsx
import { h } from 'haptic/dom';

document.body.appendChild(
  h('div', { style: 'margin-top: 10px;' },
    h('p', 'This is content.')
  )
);

document.body.appendChild(
  <div style='margin-top: 10px;'>
    <p>This is content</p>
  </div>
);
```

## `api: { ... }`

The internal API that connects the functions of the reviver allowing you to
replace or configure them. It was ported from Sinuous, and maintains most of
their API-compatibility, meaning Sinuous community plugins should work.

Read `./src/dom/index.ts` for the available methods/configurations.

Typically the usecase is to override methods with wrappers so work can be done
during the render. For example, I wrote a package called _sinuous-lifecycle_
that provides `onAttach` and `onDetach` lifecycle hooks and works by listening
to API calls to check for components being added or removed.

Here's a simple example:

```tsx
import { api } from 'haptic';

const hPrev = api.h;
api.h = (...rest) => {
  console.log('api.h:\t', rest);
  return hPrev(...rest);
};

const addPrev = api.add;
api.add = (...rest) => {
  console.log('api.add:\t', rest);
  return addPrev(...rest);
};

<p>This is a <em>test<em/>...</p>
```

This will log:

```
api.h:   ["em", null, "test"]
api.add: [<em>, "test"]
api.h:   ["p", null, "This is a", <em>, "..."]
api.add: [<p>, "This is a"]
api.add: [<p>, <em>]
api.add: [<p>, "..."]
```

## `svg(closure: () => Node)): El | undefined`

Tells Haptic's reviver to create SVG namespaced DOM elements for the duration of
the closure. This means it uses `document.createElementNS` instead of the usual
HTML `document.createElement`. Without this, elements like `<title>` would have
very different behaviour.

Usage:

```tsx
import { h, svg } from 'haptic'; // or 'haptic/dom';

cosnt <Page> = () =>
  <p>HTML text with an add icon {svg(() =>
    <svg viewBox="0 0 15 15" fill="none" width="15" height="15">
      <path d="M7.5 1v13M1 7.5h13" stroke="#000"/>
    </svg>
  )} inlined in the sentence.
  </p>;

document.body.appendChild(<Page/>);
```

[1]: https://github.com/luwes/sinuous#community

repository: eramax/haptic

repo_name: eramax/haptic


text:
Let's now look at the contents of the hook script to see how it works. To view the contents of the hook script run:

```execute
cat event-subscription/hooks/monitor-service-accounts.sh
```

The contents should be:

```
#!/usr/bin/env bash

if [[ $1 == "--config" ]] ; then
  cat <<EOF
configVersion: v1
kubernetes:
- apiVersion: v1
  kind: ServiceAccount
  executeHookOnEvent:
  - Added
  - Modified
  - Deleted
  namespace:
    nameSelector:
      matchNames:
      - $SESSION_NAMESPACE
EOF
else
  for k in $(jq '. | keys | .[]' $BINDING_CONTEXT_PATH); do
    type=$(jq -r ".[$k].type" $BINDING_CONTEXT_PATH)
    if [[ $type == "Event" ]] ; then
      watchEvent=$(jq -r ".[$k].watchEvent" $BINDING_CONTEXT_PATH)
      serviceAccountName=$(jq -r ".[$k].object.metadata.name" $BINDING_CONTEXT_PATH)
      echo ">>>>>> ${watchEvent} serviceaccount ${serviceAccountName} <<<<<<"
    fi
  done
fi
```

There are two parts to this script. The first part is executed when the script is run using the ``--config`` option. The ``shell-operator`` uses this to find out in what cases the hook script should be executed to handle an event.

The first thing the ``shell-operator`` therefore does on start up is to go through all the files in the hooks directory and if a file is an executable program, be that a script, or a compiled binary, it will run the program with the ``--config`` option.

To see what is generated for the hook script above, run:

```execute
event-subscription/hooks/monitor-service-accounts.sh --config
```

The output should be:

```
configVersion: v1
kubernetes:
- apiVersion: v1
  kind: ServiceAccount
  executeHookOnEvent:
  - Added
  - Modified
  - Deleted
  namespace:
    nameSelector:
      matchNames:
      - {{session_namespace}}
```

What the configuration in this case says is that the hook script should be executed whenever an ``ADDED``, ``MODIFIED`` or ``DELETED`` event occurs on a resource type with API version ``v1`` and kind ``ServiceAccount``. The events of interest are specified by the ``executeHookOnEvent`` property.

Since we are interested in all three event types, the ``executeHookOnEvent`` property could have been left out as the default is to create a subscription for all three event types. The example includes the property still, just so you can see how it can be set.

The ``namespace`` property is used for this workshop environment as you only have access to the single namespace associated with your workshop session. Using the name selector we are able to restrict the ``shell-operator`` to only look out for changes to the resource in the context of that namespace. If the ``namespace`` property were left out, it would attempt to monitor all namespaces in the cluster. In either case, the instance of ``shell-operator`` needs to be provided with an access token with suitable roles granted via RBAC, to query the resources in the desired scope, namespace or cluster.

If this operator were being run for the whole cluster, you could therefore have got away with using just:

```
configVersion: v1
kubernetes:
- apiVersion: v1
  kind: ServiceAccount
```

repository: liyongxian/lab-shell-operator

repo_name: liyongxian/lab-shell-operator


text:
# Project Name
Maria & Irene Team: Profile and Hobbies

# Project Description

Maria &  Irene Team: Profile and Hobbies is a project created by a two classmates from Mulitmedia Authoring 1 program. The main purpose of this project is to build a page app that includes a description of Maria and Irene's backgrounds, their major hobbies and interests.

# Installation

* Create a repository with a README file in GitHub.
* Copy URL.
* Clone repository in terminal.
* Update README file.
* Built a Home Page using text editor.
* Create a branch: Feature Home Page.
* Merge.
* Update README file.
* Built  new page  maria.html file.
* Create a new branch with maria.html file and merge.
* Invite Irene2022 to collaborate
* Create a Pull Request
* Merge

# MIT License
Copyright 2020 Luz Maria Lopera/ Irene Vinu

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

repository: lulopera/profile-hobbies

gha_language: HTML

repo_name: lulopera/profile-hobbies


text:
# Spin-weighted spherical harmonics

Spin-weighted spherical harmonics (SWSHs) generalize the standard
spherical-harmonic functions.  In particular, there is a spin weight
$s$ associated with each class of SWSHs ${}_{s}Y_{\ell,m}$, and $s=0$
corresponds to the standard spherical harmonics $Y_{\ell,m}$.  They
can be thought of as special cases of the
[Wigner $\mathfrak{D}$ matrices](/WignerDMatrices).  This allows
us to use the representation property of the $\mathfrak{D}$ matrices
to understand how SWSHs rotate.  More importantly, if a spin-weighted
function is expanded in a basis of SWSHs, we can derive a formula for
the coefficients of that expansion with respect to a rotated frame.
These properties will all be derived and discussed below.

This discussion is essentially an updated version of the one found in
[my paper](http://arxiv.org/abs/1302.2919) of a couple years ago.  For
more references, see the citations therein.

## Spin-weighted functions

Spin-weighted functions on the sphere are unusual objects.  They take
values in the complex numbers $\mathbb{C}$.  One would be forgiven,
therefore, for thinking that the field's value at a point is a scalar.
The key property of a scalar is its trivial transformation law, which
spin-weighted functions do not obey.[^1] To be more precise, imagine
we have some function ${}_{s}f(\boldsymbol{n})$, where
$\boldsymbol{n}$ is the vector from the origin to the corresponding
point on the sphere, and $s$ is the spin weight.  Now, if we rotate
our basis, we have a new function ${}_{s}f'$; the same rotation gives
us a new field point $\boldsymbol{n}'$.  A scalar function obeys
\begin{equation}
  \label{eq:ScalarTransformation}
  {}\_{0}f'(\boldsymbol{n}') = {}\_{0}f(\boldsymbol{n})
\end{equation}
On the other hand, a spin-weighted function obeys the somewhat
stranger transformation law
\begin{equation}
  \label{eq:SpinWeightedTransformation}
  {}\_{s}f'(\boldsymbol{n}') = {}\_{s}f(\boldsymbol{n})\, e^{-i\, s\, \gamma}
\end{equation}
for some angle $\gamma$.  The most important example of this
transformation is a rotation about the $\boldsymbol{n}$ vector (in the
*positive*, *right-handed* sense) through an angle $\gamma$.[^2]

It may be surprising that a transformation for which $\boldsymbol{n}'
= \boldsymbol{n}$ actually changes the value of a spin-weighted
function.  The reason for this strange behavior is that spin-weighted
functions are actually contractions of tensors with various terms of
the form $(\boldsymbol{\vartheta} + i\,
\boldsymbol{\varphi})/\sqrt{2}$, or the conjugate of that term.  In
this sense, the spin-weighted functions are actually tensor components
with respect to some coordinate basis.  So it makes sense that the
components should transform under coordinate transformations.

## Different ways of thinking about spin-weighted functions

It seems a little inconsistent that we should write spin-weighted
functions as functions of $\boldsymbol{n}$ alone; strictly speaking,
it seems like it would make more sense to write
${}_{s}f(\boldsymbol{n}, \boldsymbol{\vartheta},
\boldsymbol{\varphi})$.  In fact, we can do a little better.  Those
three vectors are a little redundant; all the information provided by
them is actually carried in a single rotation operator:
\begin{align}
  \boldsymbol{\vartheta} &= \mathcal{R} \{\vec{x}\}, \\\\
  \boldsymbol{\varphi} &= \mathcal{R} \{\vec{y}\}, \\\\
  \boldsymbol{n} &= \mathcal{R} \{\vec{z}\}.
\end{align}
And so we can consider the spin-weighted functions not to be functions
of points on the sphere, but functions of rotation operators.
Given that the group of unit quaternions (rotors) is so vastly
preferable as a way of representing rotations, we will frequently
write spin-weighted functions as functions of a rotor
${}_{s}f(\mathbf{R})$.

Here, the relation to the standard way of writing functions on the
sphere is clear.  ${}_{s}f(\vartheta, \varphi)$ is the function's
value at the point given by coordinates $(\vartheta, \varphi)$.  But
for spin-weighted functions, there is an implicit dependence on the
standard tangent basis at that point, $(\boldsymbol{\vartheta},
\boldsymbol{\varphi})$.  By writing the spin-weighted function as a
function of the rotor $\mathbf{R}$, we are simultaneously

 1. encoding the point $(\vartheta, \varphi)$ as $\mathbf{R}\,
    \vec{z}\, \mathbf{R}^{-1}$, and
 2. encoding the dependence on the tangent basis as $\mathbf{R}\,
    \vec{x}\, \mathbf{R}^{-1}$ and $\mathbf{R}\, \vec{y}\,
    \mathbf{R}^{-1}$.

Moreover, this will make the relationship between SWSHs and the Wigner
$\mathfrak{D}$ matrices very simple, and remove apparent singularities
associated with the coordinates $(\vartheta, \varphi)$.  (In fact,
this approach will also be very helpful with the more general
Newman-Penrose formalism and Lorentz transformations.)

As a side note, a common mathematical device for simultaneously
recording a position on the sphere and the orientation of the tangent
basis is called the
["Hopf fibration"](http://en.wikipedia.org/wiki/Hopf_fibration).  This
is a fiber bundle, where the base space is the sphere $S^2$
(representing positions on the sphere), and the fibers are circles
$S^1$ (corresponding to orientations of the tangent basis at that
point).  The total space is simply $S^3$.  This, not coincidentally,
is also the space of unit quaternions.  So we see that the Hopf
fibration is just another way of looking at rotors.

## SWSHs in terms of Wigner's $\mathfrak{D}$ matrices

It is important to make contact with other sources in the literature,
to ensure that we use the same conventions---or at least understand
how our conventions differ.  First, we note that
[as shown before](/index), the standard spherical coordinates
$(\vartheta, \varphi)$ and the corresponding standard tangent basis
vectors can be expressed simply in the form of the rotor
\begin{equation}
  R\_{(\vartheta, \varphi)} = e^{\varphi \vec{z}/2}\, e^{\vartheta \vec{y}/2}.
\end{equation}
Or, to put this in a form more useful for us, we have
\begin{equation}
  \mathbf{R}\_a = e^{i\,\varphi/2}\, \cos\frac{\beta}{2},
  \qquad
  \mathbf{R}\_b = e^{-i\,\varphi/2}\, \sin\frac{\beta}{2}.
\end{equation}
Plugging these expressions into the expression for $\mathfrak{D}$
given [here](/WignerDMatrices.html#mjx-eqn-eqD_RaGeqRb), for example,
and comparing to the expressions in
[the data-formats paper](http://arxiv.org/abs/0709.0093), we see that
SWSHs can be written as
\begin{equation}
  \label{eq:SWSHFromWignerD}
  {}\_{s}Y_{\ell,m} (\vartheta, \varphi)
  =
  {}\_{s}Y_{\ell,m} (\mathbf{R}\_{(\vartheta, \varphi)})
  =
  (-1)^s\, \sqrt{\frac{2\ell+1} {4\pi}}\,
  \mathfrak{D}^{(\ell)}\_{m,-s} \left(\mathbf{R}\_{(\vartheta, \varphi)}
  \right).
\end{equation}

## Rotating SWSHs

Now, using Eq. \eqref{eq:SWSHFromWignerD} and the representation
property of Wigner's $\mathfrak{D}$ matrices, we can show how to
rotate the SWSH functions, and show the transformation of modes of a
decomposition of a general spin-weighted function by SWSHs.

We start of with the fact that the $\mathfrak{D}$ matrices form a
representation of the rotation group.  That is, given any two rotors
$\mathbf{R}_1$ and $\mathbf{R}_2$, we have
\begin{equation}
  \mathfrak{D}^{(\ell)}\_{m',m} (\mathbf{R}\_1\, \mathbf{R}\_2) =
  \sum_{m''}
  \mathfrak{D}^{(\ell)}\_{m',m''} (\mathbf{R}\_1)\,
  \mathfrak{D}^{(\ell)}\_{m'',m} (\mathbf{R}\_2).
\end{equation}
From Eq. \eqref{eq:SWSHFromWignerD}, we see that it is possible to
rewrite this as
\begin{equation}
  \label{eq:SWSHTransformation1}
  {}\_{s}Y\_{\ell,m} (\mathbf{R}\_1\, \mathbf{R}\_2) =
  \sum\_{m'}
  \mathfrak{D}^{(\ell)}\_{m,m'} (\mathbf{R}\_1)\,
  {}\_{s}Y\_{\ell,m'}(\mathbf{R}\_2).
\end{equation}
In principle, this is the rotation law needed for SWSHs.

As we saw above, spin-weighted functions should be regarded as
functions on the rotation group, rather than as functions on points of
the sphere.  Unfortunately, this is not the standard treatment; a
canonical tangent basis is assumed from the coordinate system.  So we
need to write the arguments of the SWSHs in terms of spherical
coordinates.  But there's a wrinkle here.  If we assume that the
argument of the SWSH on the left-hand side above can be written as
spherical coordinates $\mathbf{R}_{(\vartheta', \varphi')}$, the
argument of the SWSH on the right-hand side cannot be written purely
as spherical coordinates; there is some additional rotation needed.
In general, then, we must write the transformation in the form
\begin{equation}
  \mathbf{R}\_{(\vartheta', \varphi')} = \mathbf{R}\_{\text{phys}}\,
  \mathbf{R}\_{(\vartheta, \varphi)}\, e^{\gamma\, \vec{z}/2}.
\end{equation}
The final term in this equation represents an *initial* rotation about
the $\vec{z}$ basis vector, which is equivalent to a *final*
rotation about the vector joining the origin to the point
$(\vartheta', \varphi')$.  This term must be included because that is
the only way the equation can be true for general sets of angles and
overall rotation $\mathbf{R}_{\text{phys}}$.  We have placed a
subscript "$\text{phys}$" on this overall rotor because, as will be
discussed below, this represents a transformation of the physical
system.

This $\gamma$ factor at first appears troubling.  It is a function of
both $\mathbf{R}_{\text{phys}}$ and $(\vartheta, \varphi)$.  In fact,
if we insert our rotations into Eq. \eqref{eq:SWSHTransformation1}, we
obtain a surprising result.  Using the known behavior of spin-weighted
functions we obtain
\begin{align}
  {}\_{s}Y\_{\ell,m} (\mathbf{R}\_{(\vartheta', \varphi')}) &=
  \sum\_{m'}
  \mathfrak{D}^{(\ell)}\_{m,m'} (\mathbf{R}\_{\text{phys}})\,
  {}\_{s}Y\_{\ell,m'}(\mathbf{R}\_{(\vartheta, \varphi)}\, e^{\gamma\,
  \vec{z}/2})
  \nonumber \\\\ \label{eq:SWSHRotation}
  &= \sum\_{m'}
  \mathfrak{D}^{(\ell)}\_{m,m'} (\mathbf{R}\_{\text{phys}})\,
  {}\_{s}Y\_{\ell,m'}(\mathbf{R}\_{(\vartheta, \varphi)})\, e^{-i\, s\, \gamma}.
\end{align}
The spin-weighted spherical harmonics, regarded as functions of the
spherical coordinates alone, *do not* transform among themselves under
a representation of the rotation group---except in the familiar case
of $s=0$!

This curious factor of $e^{-i\, s\, \gamma}$, however, is precisely
what is needed to ensure that *modes* of an expansion in SWSHs *do*
transform among themselves under rotations.  To be more precise, let
us take a function ${}_{s}f$, with a well defined spin weight $s$.
This can be expanded in the basis of SWSHs as
\begin{equation}
  {}\_{s}f(\mathbf{R}\_{(\vartheta, \varphi)}) = \sum\_m f^{\ell,m}
  {}\_{s}Y\_{\ell,m}(\mathbf{R}\_{(\vartheta, \varphi)}).
\end{equation}
Now, there is another field ${}_{s}f'$ representing the physically
rotated field having the same magnitude at $(\vartheta', \varphi')$
as ${}_{s}f$ has at $(\vartheta, \varphi)$.  This has a similar
expansion:
\begin{equation}
  {}\_{s}f'(\mathbf{R}\_{(\vartheta', \varphi')}) = \sum\_m f'^{\ell,m}
  {}\_{s}Y\_{\ell,m}(\mathbf{R}\_{(\vartheta', \varphi')}).
\end{equation}
Now, recalling from Eq. \eqref{eq:SpinWeightedTransformation} the
relation ${}_{s}f'(\mathbf{R}_{(\vartheta', \varphi')}) =
{}_{s}f(\mathbf{R}_{(\vartheta, \varphi)})\, e^{-i\, s\, \gamma}$, we
can combine the last three equations.
\begin{align}
  {}\_{s}f'(\mathbf{R}\_{(\vartheta', \varphi')})
  &=
  \sum\_m f'^{\ell,m} {}\_{s}Y\_{\ell,m}(\mathbf{R}\_{(\vartheta', \varphi')})
  \\\\ &=
  \sum\_m f'^{\ell,m} \sum\_{m'}
  \mathfrak{D}^{(\ell)}\_{m,m'} (\mathbf{R}\_{\text{phys}})\,
  {}\_{s}Y\_{\ell,m'}(\mathbf{R}\_{(\vartheta, \varphi)})\, e^{-i\, s\, \gamma}
  \\\\ &=
  \sum\_{m',m} f'^{\ell,m'}
  \mathfrak{D}^{(\ell)}\_{m',m} (\mathbf{R}\_{\text{phys}})\,
  {}\_{s}Y\_{\ell,m}(\mathbf{R}\_{(\vartheta, \varphi)})\, e^{-i\, s\, \gamma}
  \\\\ &=
  {}\_{s}f(\mathbf{R}\_{(\vartheta, \varphi)})\, e^{-i\, s\, \gamma}
  \\\\ &=
  \sum\_m f^{\ell,m} {}\_{s}Y\_{\ell,m}(\mathbf{R}\_{(\vartheta,
  \varphi)})\, e^{-i\, s\, \gamma}
\end{align}
Using orthogonality of the SWSHs, we can pull out the transformation
law for the modes:
\begin{equation}
  f^{\ell,m} = \sum\_{m'} f'^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m}
  (\mathbf{R}\_{\text{phys}}).
\end{equation}
Note that the spin dependence $e^{-i\, s\, \gamma}$ has canceled out.
We can also rewrite this as
\begin{equation}
  f'^{\ell,m} = \sum\_{m'} f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m}
  (\mathbf{R}^{-1}\_{\text{phys}}).
\end{equation}
This is the more interesting formula, because we assume that we've
started with the field ${}_{s}f$, and wish to derive the modes of the
rotated field ${}_{s}f'$.

We should note the meaning of $\mathbf{R}_{\text{phys}}$, now that we
understand how it fits into the transformations.  It is a rotation
taking a physical system $f$ with some value at the point $(\vartheta,
\varphi)$ into a physical system $f'$ with that value at the point
$(\vartheta', \varphi')$.  We can think of the rotor
$\mathbf{R}^{-1}_{\text{phys}}$ as the *same* rotation, but applied to
the coordinate basis itself, while leaving the physical system in
place.  We will therefore write it as $\mathbf{R}_{\text{basis}}$---the
rotation being applied to the basis itself, while leaving the physical
system fixed.  The field ${}_{s}f$ is measured with respect to the
original basis, and ${}_{s}f'$ is the field measured with respect to
this rotated basis.  Then we have
\begin{equation}
  f'^{\ell,m} = \sum\_{m'} f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m}
  (\mathbf{R}\_{\text{basis}}).
\end{equation}

## Composing rotations

The final detail we need to understand is the composition of
rotations.  We can write down a basic rule:

 * rotations of the physical system compose by left-multiplication;
 * rotations of the coordinate basis compose by right-multiplication.

Thus, for example, if we have an initial rotation of the physical
system $\mathbf{R}_{\text{p}1}$ and then apply a second rotation of the
physical system $\mathbf{R}_{\text{p}2}$, this is equivalent to
applying a single rotation $\mathbf{R}_{\text{p}2}\,
\mathbf{R}_{\text{p}1}$ (note the ordering).  Similarly, if we have an
initial rotation of the coordinate basis $\mathbf{R}_{\text{b}1}$ and
then apply a second rotation of the coordinate basis
$\mathbf{R}_{\text{b}2}$, this is equivalent to applying a single
rotation $\mathbf{R}_{\text{b}1}\, \mathbf{R}_{\text{b}2}$.

It is instructive to see explicitly how these rotations play out in
terms of the rotation of modes.  We start out with modes $f^{\ell,m}$
in the initial system; we then apply the first rotation (of either the
physical system or the basis) to obtain the modes $f'^{\ell,m}$; and
finally apply the second rotation to obtain the modes $f''^{\ell,m}$.

First, for rotations of the physical
system:
\begin{align}
  f''^{\ell,m}
  &=
  \sum\_{m''} \left( f'^{\ell,m''} \right)\, \mathfrak{D}^{(\ell)}\_{m'',m}
  (\mathbf{R}^{-1}\_{\text{p}2}) \\\\
  &=
  \sum\_{m'',m'} \left( f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m''}
  (\mathbf{R}^{-1}\_{\text{p}1}) \right)\, \mathfrak{D}^{(\ell)}\_{m'',m}
  (\mathbf{R}^{-1}\_{\text{p}2}) \\\\
  &=
  \sum\_{m'} f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m}
  (\mathbf{R}^{-1}\_{\text{p}1}\, \mathbf{R}^{-1}\_{\text{p}2}) \\\\
  &=
  \sum\_{m'} f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m}
  \left( (\mathbf{R}\_{\text{p}2}\, \mathbf{R}\_{\text{p}1})^{-1} \right)
\end{align}
Next, for rotations of the basis with respect to which the modes are
measured:
\begin{align}
  f''^{\ell,m}
  &=
  \sum\_{m''} \left( f'^{\ell,m''} \right)\, \mathfrak{D}^{(\ell)}\_{m'',m}
  (\mathbf{R}\_{\text{b}2}) \\\\
  &=
  \sum\_{m'',m'} \left( f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m''}
  (\mathbf{R}\_{\text{b}1}) \right)\, \mathfrak{D}^{(\ell)}\_{m'',m}
  (\mathbf{R}\_{\text{b}2}) \\\\
  &=
  \sum\_{m'} f^{\ell,m'}\, \mathfrak{D}^{(\ell)}\_{m',m}
  (\mathbf{R}\_{\text{b}1}\, \mathbf{R}\_{\text{b}2})
\end{align}
Thus, we see that applying the rotations in the natural order is fine,
but if we wish to keep track of what those rotations were, we need to
remember that rotations do not commute.

[^1]: We will frequently assume generality with respect to $s$ when
      making statements such as this.  In this case, $s=0$ fields are
      actually scalars, and do actually transform trivially.

[^2]: Footnote 2 of
      [the NR data-formats paper](http://arxiv.org/abs/0709.0093)
      stipulates that $\boldsymbol{m}^a = (\boldsymbol{\vartheta}^a +
      i\, \boldsymbol{\varphi}^a) / \sqrt{2}$, while Eq. (3.1) of
      [Newman-Penrose's introduction of SWSHs](http://link.aip.org/link/?JMP/7/863/1)
      states that $\boldsymbol{m}' = \boldsymbol{m}\, e^{i\, \psi}$.
      It is not hard to see that a rotation through an angle $\gamma$
      in the positive sense about the vector pointing to $(\vartheta,
      \varphi)$ gives us $\boldsymbol{m} \mapsto \boldsymbol{m}' =
      \boldsymbol{m}\, e^{-i\, \gamma}$.  Thus, if we are to use the
      standard NR conventions, we will have some negative signs
      relative to Newman and Penrose.  (Though Newman and Penrose were
      never specific enough for us to actually be in conflict with
      them; essentially $\gamma=-\psi$, which is perfectly cromulent.
      It's also reasonable to view this as a distinction between
      active and passive transformations; we assume passive in this
      case.)

repository: moble/spherical

gha_language: Python

repo_name: moble/spherical


text:
# **Predict Future Sales**

**GOAL**

- The task is to forecast the total amount of products sold in every shop for the test set. Note that the list of shops and products slightly changes every month. Creating a robust model that can handle such situations is part of the challenge.

**DATASET**

Dataset can be downloaded from [here](https://www.kaggle.com/c/competitive-data-science-predict-future-sales/data).

**WHAT I HAD DONE**

- Step 1: Data Preprocessing & Exploration
- Step 3: Data Training & Model Creation
- Step 4: Performance Evaluation

**MODELS USED**

-  Linear Regression
-  Random Forest Regressor

**LIBRARIES NEEDED**

- numpy
- pandas
- seaborn
- matplotlib
- scikit-learn
- Labelencoder

**Accuracy of different models used**

By using Linear Regression I got
 ```python
    Test set Accuracy:   0.42878283190980937
 ```

By using Random Forest Regressor I got
 ```python
    Test set Accuracy:  0.9324485724217306
 ```

 **CONCLUSION**

Performance of  random forest regressor is better in terms of accuracy as compared to linear regression.

**Author**

[Harshita Nayak](https://github.com/harshita9621)


text:
# Input Validation for ERTP - A Walkthrough

## Requirements

We want ERTP to be robust against malicious code in the same vat, so the ERTP must be robust without the kind of checks that we get for free in intervat communication going through `@endo/marshal` and `@agoric/swingset-vat`. In other words, the only tools at our disposal are being in a Hardened JS environment, and helpers that we import and call directly, such as `passStyleOf` from `@endo/marshal`.

## Malicious Behavior

We identify two personas: the creator of an issuerKit, and a user of the asset type. A user is someone who holds a purse or payment of that brand, or describes something in terms of an amount of that brand. A creator might be vulnerable to an attack by users and vice versa. Users might be vulnerable to other users.

The malicious behavior we would like to prevent is:

1. Users stealing funds from other users
2. Users minting who do not have access to the mint
3. The creator of an issuerKit revoking assets from the holders of purses and payments

## Entry points in ERTP

There are three entry points in ERTP:
1. packages/ERTP/src/amountMath.js
2. packages/ERTP/src/issuerKit.js
3. packages/ERTP/src/typeGuards.js

This document will only analyze `AmountMath.coerce`, but the
methodology for validating inputs remains the same for the ERTP functions.

`AmountMath.coerce` takes a `brand` and an `amount` and returns a new
`amount`, but throws if the amount's brand is not identical to the
`brand` argument, or if the amount is not a valid amount generally.
(Note that we do not check whether the `assetKind` according to the
`brand` argument matches the implicit `assetKind` of the `amount`
argument's value. This is because checking the `assetKind` according
to the `brand` would currently require an asynchronous call on the
brand.)

For `coerce` to work according to spec, if the user passes in an
invalid brand or an invalid amount, the function should throw.

## Valid amounts

A valid amount is an object record with two properties: `brand` and
`value`. `brand` is a Remotable with a few methods, such as
`isMyIssuer`, but for the purposes of a valid amount, we allow any
Remotable and do not call any of these methods (we would have to call
them asynchronously in order to check them, since the `brand` might be
a remote object, so we do not check them.)

The value is either a bigint Nat in the case of AssetKind.NAT (we formerly allowed numbers for backwards compatibility, but now only bigints are allowed) or an array of Structures in the case of AssetKind.SET. The array of Structures might include any combination of other Structures, “undefined”, “null”, booleans, symbols, strings, pass-by-copy records, pass-by-copy arrays, numbers, bigints,  Remotables, and errors. Importantly, a Structure cannot contain promises.

## Invalid Amounts

There are a number of ways in which amounts can be invalid:

### Not an object

* **The danger**: we’ll get unexpected failures later on in our use of the amount rather than failing fast at the start as intended.

### Not a CopyRecord

* **The danger**: an object that isn’t a `copyRecord` can have
  `getters` that return different values at different times, so
  checking the value would be no guarantee that the value will be the
  same when accessed later. For example:

```js
const object1 = {};
let checked = false;
Object.defineProperty(object1, 'value', {
 get() {
   if (checked) {
     return 1000000n;
   } else {
     checked = true;
     return 1n;
   }
 },
});
```
```sh
> object1.value
1n
> object1.value
1000000n
```

`Object.freeze(object1)` is not helpful here, as the `getter` is not changing, the value that is being returned is. `harden` also does not prevent `value` from changing. When harden does encounter an accessor property during its traversal, it does not "read" the property, and thereby does not call the getter. Rather, it proceeds to recursively harden the getter and setter themselves. That's because, for an accessor property, its current value isn't, for this purpose, considered part of its API surface, but rather part of the behavior of that API. Thus, we need an additional way to ensure that the value of `checked` cannot change.

`passStyleOf` throws on objects with accessor properties like the
object defined above. (In the future, `passStyleOf` may allow far
objects (passStyleOf === 'remotable') to have getters. When that
change happens, `passStyleOf` would not throw in that case.)

### Is a CopyRecord and is a proxy

* **The dangers**:
  1) A proxy can throw on any property access. Any time it does *not* throw, it *must* return the same
     value as before.
  2) A proxy can mount a reentrancy attack. When the property is
     accessed, the proxy handler is called, and the handler can
     reenter the code while the original is still waiting for that
     property access to return.

We *assume* we will address proxy-based reentrancy by other, lower-level means.
Specifically, `passStyleOf(x) === 'copyRecord'` or
`passStyleOf(x) === 'copyArray'` would guarantee that `x` is not a proxy, protecting
against dangers #1 and #2. We should note that proxy-based reentrancy
is not a threat across a vat boundary because it requires synchronous
access.

## Invalid brands

A brand can be "wrong" if:
1. It's not a Remotable
2. Its methods don’t adhere to the expected Brand API
3. It misbehaves (i.e. answering isMyIssuer with different responses)

In determining whether a brand is "valid", we only check that the brand is a remotable. This means that a brand
that passes our input validation could still have the wrong API or misbehave.

## Who hardens?
It is the responsibility of the sender/client (the creator of an
issuerKit or user of ERTP) to harden. AmountMath does not harden
inputs.

## The implementation

In `AmountMath.coerce(brand, allegedAmount)`, we do the following:
1. assert that the `brand` is a remotable
2. assert that the `allegedAmount` is a `copyRecord`
3. destructure the `allegedAmount` into `allegedBrand` and
   `allegedValue`
4. Assert that the `brand` is identical to the `allegedBrand`
5. Call `AmountMath.make(brand, allegedValue)`, which:
    * Asserts that the `brand` is a remotable, again.
    * Asserts that the `allegedValue` is a `copyArray` or a `bigint`
    * Gets the appropriate MathHelpers
    * Calls `helpers.doCoerce(allegedValue)`, which either asserts
      that the value is a `Nat bigint` or that the value is a
      `copyArray` `structure` with no duplicate elements
11. Return a new `amount`

Thus, we ensure that the `brand` is valid by checking that it is a
remotable. We ensure that the `allegedAmount` is a copyRecord with
valid `brand` and `value` properties.

If `allegedAmount` were a proxy, we would either throw in step 3 (the
destructuring), or we successfully get both the `allegedBrand` and
`allegedValue` and never touch the proxy again. Currently, we do not
attempt to prevent proxy-based reentrancy, so defending against danger
#1 is the full extent of our defense.

repository: Agoric/agoric-sdk

gha_language: JavaScript

repo_name: Agoric/agoric-sdk


text:
# benchdnn

**benchdnn** is a standalone correctness and performance benchmark for
[Intel(R) Math Kernel Library for Deep Neural Networks (Intel(R) MKL-DNN)](http://github.com/intel/mkl-dnn) library.
The purpose of the benchmark is extended and robust correctness verification of
the primitives provided by MKL-DNN. So far **benchdnn** supports convolutions
and inner products of different data types. It also implicitly tests reorders.

## License
**benchdnn** is licensed under
[Apache License Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).

## Usage (main driver)

**benchdnn** itself is a driver for different implementation specific
harnesses. So far it has harness for Intel MKL-DNN convolution, inner product,
reorder, batch normalization, and harness for testing itself.
The usage:
```
    $ ./benchdnn: [--HARNESS] [--mode=MODE] [-vN|--verbose=N] HARNESS-OPTS
```
where:

 - `HARNESS` is either `conv` [default], `ip`, `reorder`, `bnorm`, `rnn` or `self`

 - `MODE` -- string that contains flags for benchmark mode.
   - Use `C` or `c` for correctness (used by default), and `P` or `p` for performance.
     - ... of mkl-dnn implementations
   - For `--conv`, an `A` in `--mode` will loop over all available mkl-dnn implementations
   - For `--conv`, a `T` in `--mode` will loop over alt benchdnn reference impls.
     - `--conv --mode=PT` will do timing runs *just* for alt benchdnn reference impls.

 - `N` -- verbose level (integer from 0 [default] to ...)

 - `HARNESS-OPTS` are passed to the chosen harness

Returns `0` on success (all tests passed), and non-zero in case of any error
happened.

## Usage (convolution harness)

The usage:
```
    [harness-knobs] [conv-desc] ...
```

where *harness-knobs* are:

 - `--cfg={f32, u8s8u8s32, ...}` configuration (see below), default `f32`
 - `--dir={FWD_D (forward data), FWD_B (forward data + bias),FWD_I (forward data inference), BWD_D (backward data), BWD_W (backward weights), BWD_WB (backward weights + bias)}` direction, default `FWD_B`
 - `--alg={DIRECT, WINO}` convolution algorithm, default DIRECT
 - `--merge={NONE, RELU}` merged primitive, default NONE (nothing merged)
 - `--attr="attr_str"` convolution attributes (see in the section below), default `""` (no attributes set)
 - `--mb=N` override minibatch that is specified in convolution description, default `0` (use mb specified in conv desc)
 - `--match=regex` check only convolutions that match with regex, default is `".*"`. Notice: Windows may only interpret string arguments surrounded by double quotation marks.
 - `--skip-impl="str1[:str2]..."` skip implementation (see mkldnn_query_impl_info_str), default `""`
 - `--allow-unimpl=true|false` do not treat unimplemented configuration as an error, default `false`
 - `--perf-template=template-str` set template for performance report (see section *Performance measurements*)
 - `--reset` reset all the parameters set before to default one
 - `-vN|--verbose=N` verbose level, default `0`
 - `--batch=file` use options from the given file (see in subdirectory)

and *conv-desc* is convolution description. The canonical form is:
```
    gXmbXicXihXiwXocXohXowXkhXkwXshXswXphXpwXdhXdwXnS
```
Here X is a number and S is string (n stands for name). Some of the parameters
might be omitted if there is either default one (e.g. if g is not specified
**benchdnn** uses 1) or if the can be computed automatically (e.g. output shape
can be derived from the input one and kernel). Also if either width or height
is not specified than it is assumed height == width. Special symbol `_` is
ignored, hence maybe used as delimiter. See `str2desc()` in conv/conv_aux.cpp
for more details and implicit rules :^)

The attribute string *attr_str* is defined as (new lines for readability):
```
    [irmode={nearest,down};]
    [oscale={none,common,per_oc}[:scale];]
    [post_ops='[{relu,sum[:sum_scale]};]...';]
```

Here `irmode` defines the rounding mode for integer output (default is nearest).

Next, `oscale` stands for output_scales. The first parameter is the policy that
is defined below. The second optional parameter is a scale that specifies
either the one common output scale (for `none` and `common` polices) or a
starting point for `per_oc` policy, which uses many scales. The default scale
is 1.0. Known policies are:

  - `none` (default) means no output scales set (i.e. scale = 1.)
  - `common` corresponds to `mask=0` with common scale factor
  - `per_oc` corresponds to `mask=1<<1` (i.e. output channels) with different scale factors

*attr_str* is used for `--mode` C[orrectness] or P[erformance], not for T[est].

Next, `post_ops` stands for post operation sequence. Currently supported post
ops are:

  - `relu` with no parameters (i.e. corresponding scale is 1., alg = eltwise_relu, alpha = beta = 0.)
  - `sum` with optional parameter scale (default 1.)

### convolution configurations (aka precision specification)

`--cfg` option specifies what convolution would be used in terms of data type.
Also it defines all the magic with data filling inside. For integer type
saturation is implicitly implied.

Finally configuration defines threshold for computation errors (ideally we
want keep it 0 and it seems to work for now).

The table below shows cases supported by Intel MKL-DNN and corresponding
configurations for **benchdnn**:

|src type | wei type | dst type | acc type | cfg          | notes
|:---     |:---      |:---      |:---      |:---          |:---
| f32     | f32      | f32      | f32      | f32          | inference optimized for sse4.2+, training avx2+
| s16     | s16      | s32      | s32      | s16s16s32s32 | optimized for processors with support of 4vnni, forward pass only (aka FWD_D, FWD_B)
| s32     | s16      | s16      | s32      | s32s16s16s32 | optimized for processors with support of 4vnni, backward wrt data only (aka BWD_D)
| s16     | s32      | s16      | s32      | s16s32s16s32 | optimized for processors with support of 4vnni, backward wrt weights (aka BWD_W, BWD_WB)
| u8      | s8       | f32      | s32      | u8s8f32s32   | optimized for processors with support of avx512vl, forward pass only (aka FWD_D, FWD_B)
| u8      | s8       | s32      | s32      | u8s8s32s32   | same notes as for u8s8s32s32
| u8      | s8       | s8       | s32      | u8s8s8s32    | same notes as for u8s8s32s32
| u8      | s8       | u8       | s32      | u8s8u8s32    | same notes as for u8s8s32s32

## Performance measurements

**benchdnn** supports custom performance report. Template is passed via
command line and consists of terminal and nonterminal symbols. Nonterminal
symbols are printed as is. Description of terminal symbols is given below.
There is also a notion of modifiers (marked as @) that change meaning of
terminal symbols, e.g. sign '-' means minimum of (in terms of time). See
table of modifiers below.

> **caution:** threads have to be pinned in order to get consistent frequency

| abbreviation  | description
|:------------  |:-----------
| \%d           | problem descriptor
| \%D           | expanded problem descriptor (conv parameters in csv format)
| \%n           | problem name
| \%z           | direction
| \%@F          | [\*] effective cpu frequency computed as clocks[@] / time[s]
| \%O           | number of ops required (padding is not taken into account)
| \%@t          | time in ms
| \%@c          | [\*] time in ops
| \%@p          | ops per second
| \%i           | convolution implementation string

| modifier  | description
|:--------  |:-----------
|           | default
| -         | min (time) = default
| 0         | avg (time)
| +         | max (time)
| --------- | ----------
| K         | Kilo (1e3)
| M         | Mega (1e6)
| G         | Giga (1e9)

[\*] (time stamp counter code may be disabled on some platforms)

The definition of expanded problem descriptor is:
`g,mb,ic,ih,iw,oc,oh,ow,kh,kw,sh,sw,ph,pw`.

The default template can be found in conv/bench_conv.cpp that is defined as
`perf,\%n,\%d,\%GO,\%GF,\%-t,\%-Gp,\%-t,\%-Gp,\%0t,%0Gp,\%i`. That will produce the following output
in CSV format:
```
string: perf
convolution name
full conv-desc
number of giga ops calculated
[\*] effective cpu frequency in GHz (amb clocks[min] / time[min])
minimum time spent in ms
best gigaops (since it corresponds to mimimum time)
average time spent in ms
average gigaops (since it corresponds to average time)
convolution implementation name
```

## Examples

Run the set of f32 forward convolutions from inputs/conv_all file w/ bias and default minibatch:
```ShellSession
    $ ./benchdnn --conv \
        --cfg=f32 --dir=FWD_B --batch=inputs/conv_all
```

Run the same but with merged ReLU:
```ShellSession
    $ ./benchdnn --conv \
        --cfg=f32 --dir=FWD_B --merge=RELU --batch=inputs/conv_all
```

Run the same as previous but also measure performance:
```ShellSession
    $ ./benchdnn --conv --mode=CORRnPERF \
        --cfg=f32 --dir=FWD_B --merge=RELU --batch=inputs/conv_all
```

> **note**: instead of `CORRnPERF` one can use `CP`, `PC`, `cp`, or `pc`

Run a set of f32 backward convolutions wrt weights with kh=3 and
verbose level set to 2:
```ShellSession
    $ ./benchdnn --conv -v2 \
        --cfg=f32 --dir=BWD_W --match='.*kh3[^0-9].*' --batch=inputs/conv_all
```

Run a set of u8s8u8s32 backward convolutions wrt data but skip all
the convolutions that will use reference or gemm-based implementation:
```ShellSession
    $ ./benchdnn --conv \
        --cfg=u8s8u8s32 --dir=BWD_B --skip-impl='ref:gemm' --batch=inputs/conv_all
```

Run explicitly specified 1st forward convolution (including bias) from Alexnet
with the minibatch set to 4, verbose level set to 1 for two given
configurations (`u8s8u8s32` and `f32`):
```ShellSession
    $ ./benchdnn --conv -v1 \
        --mb=4 --dir=FWD_B \
        --cfg=u8s8u8s32 ic3ih227iw227_oc96oh55ow55_kh11kw11_sh4sw4ph0pw0_n"alexnet:conv1" \
        --cfg=f32 ic3ih227iw227_oc96oh55ow55_kh11kw11_sh4sw4ph0pw0_n"alexnet:conv1"
```

Run the 1st Alexnet convolution **comparing** all convolution
**implementations** except for the reference one. `--mode`
setting is for All Performance. The spec was shortened
because when height and width of things are equal, you only need to specify
one of them.
```ShellSession
    $ ./benchdnn --conv --mode=AP \
        --dir=FWD_D --skip-impl=ref ic3ih227_oc96oh55_kh11_sh4ph0_n"alexnet:conv1"
```
(On my desktop, the above yielded
```Text linenos:false
    benchdnn init ... omp_max_thr=12 OK

    perf,alexnet:conv1,"--dir=FWD_D ic3ih227oc96oh55kh11sh4nalexnet:conv1",0.421661,0.814615,517.62,0.890699,473.404,"_jit_avx2_convolution_fwd_t"
    perf,alexnet:conv1,"--dir=FWD_D ic3ih227oc96oh55kh11sh4nalexnet:conv1",0.421661,2.61064,161.516,2.85301,147.795,"_jit_sse42_convolution_fwd_t"
    0:PASSED __REPRO: --dir=FWD_D ic3ih227oc96oh55kh11sh4nalexnet:conv1
    tests:1 impls:3 passed:2 skipped:1 mistrusted:0 unimplemented:0 failed:0
```
- For this convolution, `avx2` was over 3x faster than the `sse2` (473 vs 148
  Mops).  Prefacing the command with `OMP_NUM_THREADS=1`, I got 81 vs 27 Mops.
- If you were to runn with `--mode=CAP`, the final output line should read
  `correct:2` instead of `passed:2`.
- If you were to run with `--mode=CAPT`, and you have defined reference loops other
  than the normal ones in `benchdnn/conv/ref_conv.cpp`, then you *may* see lines like:
  `TEST #1 ... time 471.092000 ms CORRECT`

Run batch file for different algorithms (assuming the file only specifies
convolutions and does not include harness options that would override ones
passed in the command line). Also ignore mkldnn_unimplemented errors in case of
Winograd:
```ShellSession
    $ ./benchdnn --conv \
        --alg=DIRECT --batch=convs.in \
        --allow-unimpl=true \
        --alg=WINO   --batch=convs.in
```

Run a set of u8s8u8s32 forward convolutions w/o bias, skipping
reference implementations and not triggering unimplemented as an error, with
one common output scale set to 0.5 with rounding mode set to down
(via attributes):
```ShellSession
    $ ./benchdnn --conv \
        --cfg=u8s8u8s32 --dir=FWD_D --skip-impl="ref" --allow-unimpl=true \
        --attr="irmode=down;oscale=common:.5" --batch=inputs/conv_all
```

Almost the same as above (with minor changes), but also add post operation
sequence **(relu, then sum with scale .3, then relu)** using
attributes/mkldnn_post_ops_t:
```ShellSession
    $ ./benchdnn --conv \
        --cfg=u8s8s32s32 --dir=FWD_B \
        --attr="oscale=common:.5;post_ops='relu;sum:.3;relu'" --batch=inputs/conv_all
```

## Notations / Glossary / Abbreviations

|Abbreviation   | Description
|:---           |:---
| src           | Source image (input image for forward convolution)
| wei           | Weights (aka filter)
| bia           | Bias
| dst           | Destination image (output image for forward convolution)
| acc           | Accumulation (typically in terms of data type)
| ic, oc        | Input/Output channels (aka feature maps)
| id, ih, iw    | Input depth[1], height, width
| od, oh, ow    | Output depth[1]height, width
| kd, kh, kw    | Kernel (filter, weights) depth[1], height, width
| sd, sh, sw    | Convolution stride over depth[1], height, width (stride>=1)
| pd, ph, pw	| Convolution top, left, near padding (0=none)
| dd, dh, dw    | Dilation of image region covered by kernel (0 => x1)
| mb            | Minibatch (amount of images processed at once)
| g             | Groups (a way to reduce the amount of computations, see Alexnet topology)
| FWD\_{D,B}     | forward w/o and w/ bias
| BWD\_{D,W,WB}  | backward wrt data, weights, and weights and bias
| DIRECT, WINO  | convolution algorithm: direct or Winograd based
| NONE, RELU    | merged primitives: nothing or ReLU

## Performance testing

A number of approaches to vectorizing and parallelizing the convolutions can be
tested via --mode=PT (Performance Test).  The general approach was to replace
conditional tests in inner loops with exactly calculated loop limits in a more
outer loop.

Condition hoisting was tricky for two reasons:

- signed integer division in C rounds to zero, so just manipulating the
  mathematical equations won't in general work.
  - a set of round-to-negative-infinity routines ease developing a set
    of mathematically correct hoists.
- stride and dilation sometimes involves a modulo condition.
  - this leads to a pair of linear Diophantine equations, whose efficient
    solution involves running the extended Euclid algorithm.

After hoisting with \c idiv.hpp division routines, one can derive alternative
formulations that will work with signed/unsigned integers by keeping all
intermediate quantities >= 1.  Then one can get *if* -based formulas that would
also be valid for unsigned calculation, as well as begin able to use C
integer division without problems with zero or negatives.  The *idiv*
approach is actually pretty fast on Intel hardware since it produces
branchless code (cmov) and is written to allow the compiler to use
masking instructions for some other *if* statements.

In coding for the SX chipset, the compiler needed much hand-holding, but the
resulting benchdnn impls (for gnchw data format) also worked better than what
I had arrived at for Intel.

Todays Intel performance tests on 6-core avx2 yielded 100x speedup for a
'minialex' test in FWD, but BWD\_D and BWD\_WB still lag at only 7-12x speedup.

## Usage (batch normalization harness)

The usage:
```
    ./benchdnn --bnorm [harness-knobs] bnorm-desc ...
```

where *harness-knobs* are:

 - `--mb=N` override minibatch that is specified in batch normalization description, default `0` (use mb specified in bnorm-desc)
 - `--dir={FWD_D (forward data /training), FWD_I (forward data /inference), BWD_D (backward data), BWD_DW (backward data + weights)}` direction, default `FWD_D`
 - `--dt={f32, s32, ...}` base data type, default `f32`
 - `--fmt={nchw, nChw16c, ...}` data layout, default `nchw`
 - `--flags=[|G|S|R]` batch normalization flags, default `none` (G -- global stats, S -- use scale shift, R -- fuse with ReLU)
 - `--attr="attr_str"` attributes (see in the convolution section above), default `""` (no attributes set)
 - `--match=regex` check only convolutions that match with regex, default is `".*"`. Notice: Windows may only interpret string arguments surrounded by double quotation marks.
 - `--skip-impl="str1[:str2]..."` skip implementation (see mkldnn_query_impl_info_str), default `""`
 - `--perf-template=template-str` set template for performance report (very similar to the convolution one)
 - `--reset` reset all the parameters set before to default one
 - `-vN|--verbose=N` verbose level, default `0`
 - `--batch=file` use options from the given file (see in subdirectory)

and *bnorm-desc* is a batch normalization description. The canonical form is:
```
    mbXicXihXiwXepsYnS
```
Here X is an integer number, Y is a real number, and S is string (n stands for
name). Special symbol `_` is ignored, hence maybe used as delimiter. There are
some implicit rules:
 - if mb is omitted set mb to 2

 - if iw is omitted set iw to ih (and vice versa)

 - if eps is omitted set eps to 1./16

## Installation

**benchdnn** is automatically built with Intel MKL-DNN. For the convenience one
may build **benchdnn** using cmake or make.

## Essence of convolution testing

Intel MKL-DNN supports different data types, such as single precision floating
point (`mkldnn_f32`), signed/unsigned integer of different length
(`mkldnn_{s,u}{8,16,32}`). We need to cover all those cases by tests. It is
essential to test real convolution sizes, since Intel MKL-DNN provides
different optimizations depending on convolution parameters, so there is no
one unified approach inside, which means it would not be enough to test only
few convolutions (aka unit tests).

But even for given convolution the correctness convolution test is not as
simple as it might seem to be at first sight. One of the biggest problem we
encountered is numerical instability. For every output point a lot of
operations may happen. For instance on backward propagation with respect to
filter each filter point requires `mb * oh * ow` operations (see *Notation*
section below). That big amount of compute operations may lead to either
integer overflow or accuracy loss if initial data was chosen inadequately.

These two main things complicate testing. **benchdnn** tries to address these
issues by using integers for initialization with uniform distribution in a
range `[cfg->f_min .. cfg->f_max]`, with the step `cfg->f_step`
(see `struct dt_conf_t` in conv/conv.hpp). `f_min` and `f_max` are chosen so
that most of the result would belong `[cfg->min .. cfg->max]` range. Also
for floating point all integers in both ranges have exact representation (i.e.
the absolute numbers are less than `2^size_of_mantissa`). Uniform distribution
leads to have result uniformly distributed and quite small `f_min/f_max` keep
the result in a reasonable range. Yet another trick: not all the points are
initialized with non-zero values: see `fill_{src,wei,bia,dst}` in
conv/conv.cpp.

## Further plans

Please see TODO.md in **benchdnn** root directory for development plans.

## Issues and contributions

We welcome community contributions to **benchdnn** as well as Intel MKL-DNN.
If you have any ideas or issues please submit an issue or pull request. For
clarity please include ''benchdnn: '' in the title.

## Inspiration

bench{yet another 3 letters where the first one equals second)...

repository: necla-ml/gen-dnn

gha_license_id: NOASSERTION

repo_name: necla-ml/gen-dnn


text:
## Project: Logs Analysis

#### Description
This is the first project in Udacity's Full Stack Web Developer Nanodegree Program.

The task is to create a reporting tool that prints out reports( in plain text) based on the data in the given database (database provided by Udacity).

The Python script `logs_analysis.py` in this project uses the `psycopg2` library to query the database and produce a report that answers the following three questions:

1. What are the most popular three articles of all time?
    * Which articles have been accessed the most? Present this information as a sorted list with the most popular article at the top.
2. Who are the most popular article authors of all time?
    * That is, when you sum up all of the articles each author has written, which authors get the most page views? Present this as a sorted list with the most popular author at the top.
3. On which days did more than 1% of requests lead to errors?
    * The log table includes a column status that indicates the HTTP status code that the news site sent to the user's browser.

The plain text file `logs_analysis_output.txt` in this project is used to display the output; that is a copy of what the program prints out.

---
#### Things to have/do before running the code
##### System Requirements
This project makes use of Linux-based virtual machine (VM).
- [Vagrant](https://www.vagrantup.com/)
- [Oracle VM VirtualBox](https://www.virtualbox.org/)
- [Git](https://git-scm.com/)

##### System Setup
1. Git Clone  [fullstack-nanodegree-vm](https://github.com/udacity/fullstack-nanodegree-vm)
    * Further Instructions provided on the above Udacity's repo on how to run the vagrant box.
    * Python & PostgreSQL database server are already installed and started in the VM
2. Download & extract the [sql dump](https://d17h27t6h515a5.cloudfront.net/topher/2016/August/57b5f748_newsdata/newsdata.zip) and run ```psql -d news -f newsdata.sql``` **within the vagrant machine** to import the data.
3. Git Clone this repo

##### Create Views
By first running the command `psql -d news`, create views within the vagrant machine
```sql
CREATE OR REPLACE VIEW logs_view AS
SELECT to_char(time,'DD-MON-YYYY') as date,
count(*) as log_count
FROM log GROUP BY date;
```

```sql
CREATE OR REPLACE VIEW error_logs_view AS
SELECT to_char(time,'DD-MON-YYYY') as date, count(*) as
error_count FROM log WHERE STATUS = '404 NOT FOUND'
GROUP BY date;
```

##### Running the code
`cd` into the project directory and run the below code within the vagrant machine
```
python logs_analysis.py
```

repository: vaibhav-sidapara/logs-analysis

repo_name: vaibhav-sidapara/logs-analysis


text:
# Wine Dataset Script
A python script to read an excel sheet containing a dataset of wines and extract the ones which are drinkable for 6 years or more into a new excel file, the ones below 5 years into another excel file and the one with no data on when to drink them into another excel file.

## Technologies
* Python 3.8
* Pandas 1.0.1
* xlrd 1.2.0
* openpyxl 3.0.3

## Running Program
1. Install pandas

Windows
-------
   ```
      python remove_zero_columns.py
      python check_if_drinkable_or_not.py
      python get_wine_attributes.py
      python association_rule.py

   ```

 Mac
--------
  ```
      python3 remove_zero_columns.py
      python3 check_if_drinkable_or_not.py
      python3 get_wine_attributes.py
      python3 association_rule.py
  ```

repository: Williano/Data-Mining

repo_name: Williano/Data-Mining