text:
[![DOI](https://zenodo.org/badge/227604330.svg)](https://zenodo.org/badge/latestdoi/227604330)

# *pa* - Performance (Change) Analysis

*pa* is an efficient tool for analyzing performance data based on statistical simulation, i.e., bootstrap.
The performance data can come from measurements such as performance tests or benchmarks.

There are two main ways how to analyze performance data with *pa*:
1. Single version analysis: statistic (e.g., mean) + variability (confidence interval of the statistic)
2. Analysis between two versions: confidence interval ratio of a statistic (e.g., mean)

Inspired by [1], *pa* employs a Monte-Carlo technique called *bootstrap* [2] to estimate the population confidence interval from a given sample.
It uses hierarchical random re-sampling with replacement [3].
In the context of performance analysis, these hierarchical levels correspond to levels where measurement repetition happens to have reliable results.
These hierarchical levels are:
1. invocation
2. iteration
3. fork
4. trial
5. instance

Higher levels are composed of multiple occurrences of lower levels, i.e., an iteration (level 2) consists of many invocations (level 1), and so on.

## Requirements and Installations

*pa* requires only Go with version 1.14 or higher ([Install Page](https://golang.org/doc/install)).

Install *pa* by running `go get github.com/chrstphlbr/pa`.

## Usage

### Command Line Interface

*pa* comes with a simple command line interface (optional flags in `[...]` with their defaults):

```bash
pa  [-bs 10000] [-is 0] [-sl 0.01] [-st mean] [-os] [-m 1] [-tra id:id] \
    file_1 \
    [file_2 ... file_n]
```

If 1 file (`file_1`) is provided, the single version analysis is performed, i.e., the confidence intervals of a single performance experiment is computed.

If multiple files are provided, the two version analysis is performed:
the confidence intervals for both versions *and* the confidence interval ratio between the two versions is computed.
In the simple case, 2 files are provided, `file_1` for version 1 and `file_2` for version 2.
It is also possible to provide multiple files per version (of equal number) by setting the flag `-m`.

Note that the files **MUST** be sorted alphabetically by their benchmarks (see section "Input Files").

Flags:
* `-bs` defines the number of bootstrap simulations, i.e., how many random samples are taken to estimate the population distribution
* `-is` defines how many invocation samples are used (0 takes the mean across all invocations of an iteration, -1 takes all invocations, and > 0 for number of samples).
* `-sl` defines the significance level.
The confidence level for the confidence intervals is then `1-sl`.
The default is 0.01 which corresponds to a 99% confidence level.
* `-st` defines the statistic for which a confidence interval is computed.
The default is `mean`.
Another option is `median`.
* `-os` defines whether the statistic, as set by `-st`, is included in the output file.
* `-m` sets the number of files per version (control and test group).
For example, if `-m 3` *pa* expects 6 files, where `file_1`, `file_2`, and `file_3` belong to version 1, and `file_4`, `file_5`, and `file_6` belong to version two.
* `-tra` defines the transformation(s) applied to the benchmark results (i.e., the file(s)),
in the form of `transformer1:transformer2`,
where `transformer1` is applied to the first (control) group and `transformer2` is applied to the second (test) group (if it exists).
Transformers can be one of `id` (identity, no transformation) or `f0.0` ('f' for factor followed by a user-specified float64 value)

### Input Files

*pa* expects CSV input files of the following form.
For JMH benchmark results, the tool [bencher](https://github.com/chrstphlbr/bencher) can transform JMH JSON output to this CSV file format.
```
project;commit;benchmark;params;instance;trial;fork;iteration;mode;unit;value_count;value
```

The columns represent the following values:
1. `project` is the project name
2. `commit` is the project version, e.g., a commit hash
3. `benchmark` is the name of the fully-qualified benchmark method
4. `params` are the performance parameters (not the function/method parameters) of the benchmark in comma-separated form.
Every parameter consists of a name and a value, separated by an equal sign (`name=value`).
For example JMH supports performance parameters through its `@Param` annotation
5. `instance` is the name of the instance or machine (level 5)
6. `trial` is the number of the trial (level 4)
7. `fork` is the fork number (level 3).
For example JMH supports forks through their `@Fork` annotation
8. `iteration` is the iteration number within a fork (level 2)
9. `mode` is the benchmark mode.
For exmaple JMH supports average time `avgt`, throughput `thrpt`, or sample time `sample`
10. `unit` is the measurement unit of the benchmark value.
Depending on the `mode`, the measurement unit can be ns/op for average time or op/s for throughput
11. `value_count` is the number of invocations (level 1) the `value` occurred in this iteration.
Every iteration can have multiple values (i.e., invocations), which are presented as a histogram.
Each histogram value corresponds to one CSV row, and the occurrences of this value is defined by `value_count`.
12. `value` is the performance metric with a certain `unit`

**IMPORTANT**: the input files must be sorted by `benchmark` and `params`, otherwise the tool will not work correctly.
This is because input files can be *large* and, therefore, *pa* works on file input streams.

### Output

*pa* writes the results in CSV form to stdout.
The output can contain 3 types of CSV rows:
* rows starting with `#` are comments
* empty rows
* all other rows are CSV rows

The columns are:
1. `benchmark` is the name of the benchmark
2. `params` are the function/method parameters of the benchmark.
*pa* does not populate this column, because the input format does not provide the function/method parameters
3. `perf_params` is a comma-separated list of performance parameters.
See column `params` of the input files for comparison
4. `st` is the statistic the confidence interval is for.
Can be "mean" or "median"
5. `ci_l` is the lower bound of the confidence interval
6. `ci_u` is the upper bound of the confidence interval
7. `cl` is the confidence level of the confidence interval

#### Single Version Analysis

The output file is a CSV with the following columns (without `-os`):
```
benchmark;params;perf_params;ci_l;ci_u;cl
```

And with the statistic, as set by `-os`, it has the following columns:
```
benchmark;params;perf_params;st;ci_lower;ci_u;cl
```

#### Two Version Analysis

The output file is a CSV with the following columns (without `-os`):
```
benchmark;params;perf_params;v1_ci_l;v1_ci_u;v1_cl;v2_ci_l;v2_ci_u;v2_cl;ratio_s;ratio_ci_l;ratio_ci_u;ratio_cl
```

And with the statistic, as set by `-os`, it has the following columns:
```
benchmark;params;perf_params;v1_st;v1_ci_l;v1_ci_u;v1_cl;v2_st;v2_ci_l;v2_ci_u;v2_cl;ratio_st;ratio_ci_l;ratio_ci_u;ratio_cl
```

Compared to the single version analysis, the two version analysis has three or four (with or without `-os`) columns, for both versions (`v1` and `v2`) and the confidence interval for the ratio between the two versions (`ratio`).

## References

[1] T. Kalibera and R. Jones, “Quantifying performance changes with effect size confidence intervals”, University of Kent, Technical Report 4–12, June 2012. Available: [URL](http://www.cs.kent.ac.uk/pubs/2012/3233)

[2] A. C. Davison and D. V. Hinkley, “Bootstrap methods and their application”

[3] S. Ren, H. Lai, W. Tong, M. Aminzadeh, X. Hou, and S. Lai, “Nonparametric bootstrapping for hierarchical data”, Journal of Applied Statistics, vol. 37, no. 9, pp. 1487–1498, 2010. Available: [DOI](https://doi.org/10.1080/02664760903046102)

repository: chrstphlbr/pa

repo_name: chrstphlbr/pa


text:
# hungrytummy - app
A self service ordering app for restaurant (Point Of Sale Project) using ReactNative, Express JS, and MySQL

## Important
This project is still in the very early development stage. If you want to use for production, use it with your own risk.
More feature will be Coming Soon.

## Screen

![Image](https://content.screencast.com/users/dyahh/folders/hungrytummy/media/86573a51-1f8e-49e8-b6b9-2c536e4746e7/happytummy-app.gif)

## [SLIDE POWERPOINT](https://docs.google.com/presentation/d/1mwoFzlRdpy90vKnyY-qPoKbsnmRXQM4CVvltne0hmVk/edit?usp=sharing)

## Features
* Add table number
* Get menu by category
* Add Order
* Get Bill

## Why?
* It's React Native, so it supports android & iOS by default!
* Simple Code, so you can easily contribute to it.

## Tech Stack
* React Native for the Mobile Frontend
* Express Js as the Backend
* MySQL as the database

## Prerequisites
* Make sure you had been install NodeJs in your system https://nodejs.org/
* Then install React Native https://facebook.github.io/react-native/
* Express Js as the backend https://expressjs.com

## Installation & Configuration
Follow this step to install

**Frontend:**
```
$ git clone https://github.com/dyahnrchls/hungrytummy.git
$ cd hungrytummy
$ npm install
$ react-native run-android #for android
```

**Back-end:**
```
$ git clone https://github.com/dyahnrchls/restaurant-API.git
$ cd restaurant-API
$ npm install
$ npm start
running at localhost:3001
```

## Support Me :)
* Hire Me https://www.linkedin.com/in/dyahnrchls

----

## License

[MIT](https://choosealicense.com/licenses/mit/)

license: MIT

repository: dyahnrchls/hungrytummy-app

branch_name: refs/heads/master

extension: md

gha_language: JavaScript

license_type: permissive

repo_name: dyahnrchls/hungrytummy-app


text:
---
languages:
- java
page_type: sample
description: "The MSAL Android preview gives your app the ability to begin using the
Microsoft identity platform."
products:
- azure
- azure-active-directory
- ms-graph
urlFragment: msal-android-app
---

# MSAL Android app calling Microsoft Graph

>[!IMPORTANT]
>DEPRECATED - Please see new Samples for MSAL 1.0

* [Java Sample](https://github.com/Azure-Samples/ms-identity-android-java)
* [Kotlin Sample](https://github.com/Azure-Samples/ms-identity-android-kotlin)

| [Getting Started](https://docs.microsoft.com/azure/active-directory/develop/guidedsetups/active-directory-android)| [Library](https://github.com/AzureAD/microsoft-authentication-library-for-android) | [API Reference](http://javadoc.io/doc/com.microsoft.identity.client/msal) | [Support](README.md#community-help-and-support)
| --- | --- | --- | --- |

![Build Badge](https://identitydivision.visualstudio.com/_apis/public/build/definitions/a7934fdd-dcde-4492-a406-7fad6ac00e17/500/badge)

The MSAL Android preview gives your app the ability to begin using the
[Microsoft identity platform](https://aka,ms/aaddev) by supporting [Microsoft Azure Active Directory](https://azure.microsoft.com/services/active-directory/) and [Microsoft Accounts](https://account.microsoft.com) in a converged experience using industry standard OAuth2 and OpenID Connect. This sample demonstrates all the normal lifecycles your application should experience, including:

* Get a token for the Microsoft Graph
* Device-wide SSO and Conditional Access suport through the Auth Broker
* Refresh tokens automatically
* Call the Microsoft Graph
* Sign out the user

## Scenario

This app is a multi-tenant app meaning it can be used by any Azure AD tenant or Microsoft Account.  It demonstrates how a developer can build apps to connect with enterprise users and access their Azure + O365 data via the Microsoft Graph.  During the auth flow, end users will be required to sign in and consent to the permissions of the application, and in some cases may require an admin to consent to the app.  The majority of the logic in this sample shows how to auth an end user and make a basic call to the Microsoft Graph.

![Topology](./images/topology1.png)

## Example

```Java
// Initialize your app with MSAL
PublicClientApplication pApp = new PublicClientApplication(
            this.getApplicationContext(),
            R.raw.auth_config);

// Perform authentication requests
pApp.acquireToken(getActivity(), SCOPES, getAuthInteractiveCallback());

// ...

// Get tokens to call APIs like the Microsoft Graph
authenticationResult.getAccessToken();
```

## Register your App

To begin registering your app, begin at the [Azure portal](https://aka.ms/MobileAppReg)

To create an app,
1. Click `New Registration`.

2. Name your app, select the audience you're targeting, and click `Register`.
    - This sample is focused on `Accounts in any organization and personal Microsoft accounts`, but can be modified to support any audience.
    - Do not register a Redirect URI.

3. Click `Authentication` > `Add Platform` > `Android`.
    - Enter the Package Name from your Android studio project.
    - Generate a Signature Hash. Refer to the portal for instructions.

4. Hit the `Configured` button. Store the ***MSAL Configuration*** for the next steps.

## Steps to Run

1. Clone the code.
    ```
    git clone https://github.com/Azure-Samples/active-directory-android-native-v2
    ```
2. Open Android Studio, and select *open an existing Android Studio project*. Find the cloned project and open it.

3. Configure the sample code in `auth_config.JSON` and `AndroidManifest.xml`.
    - Copy and paste the ***MSAL Configuration*** JSON from the Azure portal into `auth_config.JSON`.
    - Inside the `AndroidManifest.xml`, replace `android:host` and `android:path` with the same info registered above.
        - `auth_config.JSON` contains this information as a reference inside the `redirect_uri` field.
        - The Signature Hash should NOT be URL encoded in the `AndroidManifest.xml`.

5. Select *Build* > *Clean Project*.

6. Select *Run* > *Run 'app'*.

## Feedback, Community Help, and Support

We use [Stack Overflow](http://stackoverflow.com/questions/tagged/msal) with the community to
provide support. We highly recommend you ask your questions on Stack Overflow first and browse
existing issues to see if someone has asked your question before.

If you find and bug or have a feature request, please raise the issue
on [GitHub Issues](../../issues).

To provide a recommendation, visit
our [User Voice page](https://feedback.azure.com/forums/169401-azure-active-directory).

## Contribute

We enthusiastically welcome contributions and feedback. You can clone the repo and start
contributing now. Read our [Contribution Guide](Contributing.md) for more information.

This project has adopted the
[Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see
the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact
[opencode<EMAIL_ADDRESS>with any additional questions or comments.

## Security Library

This library controls how users sign-in and access services. We recommend you always take the
latest version of our library in your app when possible. We
use [semantic versioning](http://semver.org) so you can control the risk associated with updating
your app. As an example, always downloading the latest minor version number (e.g. x.*y*.x) ensures
you get the latest security and feature enhanements but our API surface remains the same. You
can always see the latest version and release notes under the Releases tab of GitHub.

## Security Reporting

If you find a security issue with our libraries or services please report it
to [secure<EMAIL_ADDRESS>with as much detail as possible. Your
submission may be eligible for a bounty through the [Microsoft Bounty](http://aka.ms/bugbounty)
program. Please do not post security issues to GitHub Issues or any other public site. We will
contact you shortly upon receiving the information. We encourage you to get notifications of when
security incidents occur by
visiting [this page](https://technet.microsoft.com/en-us/security/dd252948) and subscribing
to Security Advisory Alerts.

repository: hamiltonha/ms-identity-android-native

repo_name: hamiltonha/ms-identity-android-native


text:
# ip-data

Typings for the following API: http://ip-api.com/docs/api:json

Demo usage:

```ts
import getIPData from "@rbxts/ip-data";
import { ServerLocation } from "MyRemoteEvents"
const UnknownLocationStr = "Server Location: Unknown";

getIPData()
	.then(
		myData => {
			return myData.status === "success"
				? "Server Location: %s, %s".format(myData.regionName, myData.country)
				: UnknownLocationStr;
		},
		() => {
			return UnknownLocationStr;
		},
	)
	.then(locationString => {
		ServerLocation.FireAllClients(locationString);
		Players.PlayerAdded.Connect(plr => ServerLocation.FireClient(plr, locationString));
	});
```

license: ISC

repository: Validark/Roblox-TS-Libraries

gha_language: Lua

gha_license_id: ISC

repo_name: Validark/Roblox-TS-Libraries


text:
WebCola [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
=======

JavaScript constraint based layout for high-quality graph visualization and exploration
using D3.js and other web-based graphics libraries.

[Homepage with code and more examples](http://marvl.infotech.monash.edu/webcola)

Note: While D3 adaptor supports both D3 v3 and D3 v4, WebCoLa's interface is styled like D3 v3. Follow the setup in our homepage for more details.

Installation
------------

#### Browser:
```html
<!-- Minified version -->
<script src="http://marvl.infotech.monash.edu/webcola/cola.min.js"></script>
<!-- Full version -->
<script src="http://marvl.infotech.monash.edu/webcola/cola.js"></script>
```

The minified version can also be accessed from [jsDelivr](https://cdn.jsdelivr.net/gh/tgdwyer/WebCola/WebCola/cola.min.js).

#### Npm:

	npm install webcola --save

You can also install it through npm by first adding it to `package.json`:

    "dependencies": {
      "webcola": "latest"
    }
Then by running `npm install`.

#### Bower:

	bower install webcola --save

If you use TypeScript, you can get complete TypeScript definitions by installing [tsd 0.6](https://github.com/DefinitelyTyped/tsd) and running `tsd link`.

Building
--------

*Linux/Mac/Windows Command Line:*

 - install [node.js](http://nodejs.org)

 - from the WebCola directory:

        npm install

 - build, minify and test:

        npm run build
        npm run test

This creates the `cola.js` and `cola.min.js` files in the `dist` directory and runs tests.

*Visual Studio:*

 - get the [typescript plugin](http://www.typescriptlang.org/#Download)
 - open webcola.sln

Running
-------

*Linux/Mac/Windows Command Line:*

Install the Node.js http-server module:

    npm install -g http-server

After installing http-server, we can serve out the example content in the WebCola directory.

    npm run website
    http-server website

The default configuration of http-server will serve the exampes on [http://localhost:8080](http://localhost:8080).

repository: tgdwyer/WebCola

gha_language: TypeScript

gha_license_id: MIT

repo_name: tgdwyer/WebCola


text:
# Data Vault dApp

Tokenize and store files on the blockchain.

Example Dapp for use by Dapp Deployer.

To install:
Run `yarn`

If running with dapp deployer, configure the deployer.conf to match your local folder setup.

Otherwise migrate using `truffle` as in:
```
truffle migrate --network development --reset
```

repository: GrahamMcBain/tool-dapploy-nodejs

repo_name: GrahamMcBain/tool-dapploy-nodejs


text:
---
description: Receive event notifications with webhooks
---

# Webhooks

Snapshot uses webhooks to notify your application when an event happens. Webhooks are particularly useful for asynchronous events like when a proposal is created, when it starts, or when it ends.

The webhook server sends a request for any new event, the request is sent to the configured URL with `POST` method and the event object as body.

**Here is an example of the event object:**

```javascript
{
  id: 'proposal/QmZ21uS8tVucpaNq2LZCbZUmHhYYXunC1ZS2gPDNWwPWD9',
  event: 'proposal/created',
  space: 'yam.eth',
  expire:<PHONE_NUMBER>
}
```

**Here are the possible events:**

`proposal/created`
When a new proposal is created

`proposal/start`
When the voting period for a proposal starts.

`proposal/end`
When the voting period for a proposal ends.

`proposal/deleted`
When a proposal is deleted by the author or an admin of the space.

### Subscribe to events

If you want to subscribe to webhooks please contact us on Discord or Telegram in the \#helpdesk channel. You will need to provide a URL to receive the webhook request.

repository: octavioamu/snapshot-docs

repo_name: octavioamu/snapshot-docs


text:
What we did: dig into this awards data
Where we are now: analyzing awards data
Where we're going next: Larger data set on the cluster

## Review

Last lecture I forgot to mention `apply`, that operates over the margins of a matrix, or array more generally.

## Object sizes in memory

How do we calculate how large a matrix / array is?

```{r}
n = 1000
p = 500
x = matrix(rnorm(n * p), nrow = n)

typeof(x)

# Expert knowledge!
bytes_per_double = 8

expected_size = n * p * bytes_per_double
```

Is the object the expected size?

```{r}
object.size(x)

object.size(x) - expected_size
```

No, the actual object size is _slightly_ larger.
Why?

Because R has some memory overhead per object, a few bytes of metadata hanging out for every object.

Question: How large will it be if we store this in a general container, like a list?

```{r}
xl = lapply(x, identity)
object.size(xl)

as.numeric(object.size(xl) / object.size(x))
```

The list is 8 times bigger than the efficient matrix form.

Which one is faster?

```{r}
library(microbenchmark)
sum(x)
microbenchmark(sum(x))
microbenchmark(do.call(sum, xl))
```

Lesson:
Using efficient data structures results in small objects and fast code.

Which is more efficient?
- Millions of objects   :(
- Millions of data elements inside one object :)

The latter is far more efficient, both in storage and computation.

## Matrix

Everyone loves matrices.
Seriously, they're a huge topic in computing.

Matrix package, object oriented programming

Question: From your math classes, what kind of matrices do you know that have special structure?

Classes:
- symmetric
- diagonal
- banded
- orthogonal

Question: What kind of methods should we write for matrices?

Methods:
- `+, -, %*%`
- decompositions

Object oriented programming is about organization and extensibility.

R has _several_ object oriented systems.
S3 is the basic one, plot, summary

```{r}
> plot
function (x, y, ...)
UseMethod("plot")
<bytecode: 0x2c790a8>
<environment: namespace:graphics>
```

The Matrix package uses the S4 system.
Many good references

The [Matrix manual](https://cran.r-project.org/web/packages/Matrix/Matrix.pdf) itself is substantial- 200+ pages.
Explain a little bit what each part does:
- Class name
- Slots
- Extends
- Methods
- See Also
- Examples

The S4 machinery lives in the `methods` package, so we should load that up.

```{r}
library(methods)

library(Matrix)

m = Matrix(1:9, nrow = 3)

class(m)
```

So what's a dgeMatrix?

```{r}
class?dgeMatrix
```

A general class for dense matrices and lists many methods that are defined for this class.

```{r}
str(m)
```

This shows the underlying components, called __slots__, that together comprise an object of that class.
We can peer into the implementation and access the slots using the `@` operator.

```{r}
m@x
m@Dim

class(m@x)
class(m@Dim)
```

This shows how the data is actually stored as a one dimensional numeric vector.

There are three ways to create objects of some class:

```{r}
m = Matrix(1:9, nrow = 3)

as(m, "matrix")

new("dgeMatrix", x = rnorm(9), Dim = c(3L, 3L))
```

Question: Will a diagonal matrix be smaller than a dense general matrix?
Hopefully, it only has to store n entries.

```{r}
n = 1000
x = Matrix(rnorm(n * n), nrow = n)

# More practice: How large should the data in here be?
n * n * bytes_per_double

# We're off by ~1KB because of the overhead of having objects in R.
object.size(x)
```

Let's construct a diagonal matrix from this.

Question: Can we use `as`?
It's a little ambiguous.

```{r}
# fails
d = as(x, "ddiMatrix")

# Works, we're being explicit about what we want.
d = Diagonal(x = diag(x))
```

Conversely, it's not ambiguous to convert a diagonal matrix into a dense matrix, because there is no loss of information.

```{r}
ddense = as(d, "dgeMatrix")
```

How large should the diagonal matrix be?

```{r}
# Data should be
n * bytes_per_double

# But there's some overhead
object.size(d)
```

Are diagonal matrices closed under matrix multiplication?
Yes.

```{r}
d2 = d %*% d
class(d2)
```

Do you think diagonal matrices are faster than dense matrices?

```{r}
library(microbenchmark)

microbenchmark(x %*% x, times = 10L)

microbenchmark(d %*% d, times = 10L)
```


text:
Java Program to Display a polygon having n edges, where n is the number of points in the polygon.

Author: Bharatesh Chakravarthi
Affiliation: VE Lab, Chung Ang University, Seoul, South Korea.

repository: adithyakoundinya/VTKExamples

gha_language: HTML

repo_name: adithyakoundinya/VTKExamples


text:
# Unicorefuzz

Fuzzing the Kernel using AFL Unicorn

## Unicorefuzz Setup
* Install Python
* Clone afl-unicorn and follow installation instructions
* `pip install -r requirements.txt`

## Debug Kernel Setup (Skip this if you know how this works)

* Create a qemu-img and install your preferred OS on there through qemu
* An easy way to get a working userspace up and running in QEMU is to follow the steps described by syzkaller, namely [create-image.sh](https://github.com/google/syzkaller/blob/90c8f82ae8f12735e0e06d422dfea80758aaf0a5/tools/create-image.sh)
* For kernel customization you might want to clone your preferred kernel version and compile it on the host. This way you can also compile your own kernel modules (e.g. example_module).
* In order to find out the address of a loaded module in the guest OS you can use `cat /proc/modules` to find out the base address of the module location. Use this as the offset for the function where you want to break. If you specify MODULE and BREAKOFFSET in the `config.py`, it should use `./get_mod_addr.sh` to start it automated.
* You can compile the kernel with debug info. When you have compiled the linux kernel you can start gdb from the kernel folder with `gdb vmlinux`. After having loaded other modules you can use the `lx-symbols` command in gdb to load the symbols for the other modules (make sure the .ko files of the modules are in your kernel folder). This way you can just use something like `break function_to_break` to set breakpoints for the required functions.
* In order to compile a custom kernel for Arch, download the current Arch kernel and set the .config to the Arch default. Then set `DEBUG_KERNEL=y`, `DEBUG_INFO=y`, `GDB_SCRIPTS=y` (for convenience), `KASAN=y`, `KASAN_EXTRA=y`. For convenience, we added a working [example_config](./example_config) that can be place to the linux dir.
* To only get necessary kernel modules boot the current system and execute `lsmod > mylsmod` and copy the mylsmod file to your host system into the linux kernel folder that you downloaded. Then you can use `make LSMOD=mylsmod localmodconfig` to only make the kernel modules that are actually needed by the guest system. Then you can compile the kernel like normal with `make`. Then mount the guest file system to `/mnt` and use `make modules_install INSTALL_MOD_PATH=/mnt`. At last you have to create a new initramfs, which apparently has to be done on the guest system. Here use `mkinitcpio -k <folder in /lib/modules/...> -g <where to put initramfs>`. Then you just need to copy that back to the host and let qemu know where your kernel and the initramfs are located.
* Setting breakpoints anywhere else is possible. For this, set `BREAKADDR` in the `config.py` instead.

## Run

- ensure a target gdbserver is reachable, for example via `./startvm.sh`
- adopt `config.py`:
- provide the target's gdbserver network address in the config to the probe wrapper
- provide the target's target function to the probe wrapper and harness
- make the harness put AFL's input to the desired memory location by adopting the `place_input` func `config.py`
- add all EXITs
- start `./probe_wrapper.py`, it will (try to) connect to gdb.
- make the target execute the target function
- run `./start_afl.py`

Putting afl's input to the correct location must be coded invididually for most targets.
However with modern binary analysis frameworks like IDA or Ghidra it's possible to find the desired location's address.

The following `place_input` method places at the data section of `sk_buff` in `key_extract`:

```python
    # read input into param xyz here:
    rdx = uc.reg_read(UC_X86_REG_RDX)
    util.map_page_blocking(uc, rdx) # ensure sk_buf is mapped
    bufferPtr = struct.unpack("<Q",uc.mem_read(rdx + 0xd8, 8))[0]
    util.map_page_blocking(uc, bufferPtr) # ensure the buffer is mapped
    uc.mem_write(rdx, input) # insert afl input
    uc.mem_write(rdx + 0xc4, b"\xdc\x05") # fix tail
```

## Gotchas
A few things to consider.

### FS_BASE and GS_BASE

Unicorn does not offer a way to directly set model specific registers. Since kernels use them to store pointers to process- or cpu-specific memory, Unicorefuzz makes unicorn execute `wrmsr` to set the `fs` and `gs` registers to the appropriate values.
To do this, you'll have to provide a non-used `SCRATCH_ADDR` in the `config.py`.

### Improve Fuzzing Speed

Right now, the Unicorefuzz `probe_wrapper.py` needs to be manually restarted after an amount of pages has been allocated. Allocated pages don't propagate back to the forkserver parent automatically and would need be reloaded from disk for each iteration.

### CMPXCHNG16b
Unicorn does not [handle CMPXCHNG16b correctly](https://github.com/unicorn-engine/unicorn/issues/1095).
Until that is addressed, we needed to hook each cmpxchng16b with a python reimplementation.
This is super hacky and should only be used if really needed.
Instead, try to start fuzzing after the instruction or do anything else.
If you do need it, add all occurrences you want to replace to the `config.py`

repository: sogolo/unicorefuzz

repo_name: sogolo/unicorefuzz


text:
# Ansible Role: Docker

An Ansible role to install Docker (CE or EE) on various Linux distributions.

## Requirements

#### Supported distributions

| Distribution | Version            |
| ------------ | ------------------ |
| CentOS       | 7                  |
| Debian       | Buster 10          |
|              | Stretch 9 (stable) |
| Fedora       | 29                 |
|              | 28                 |
| Ubuntu       | Cosmic 18.10       |
|              | Bionic 18.04 (LTS) |
|              | Xenial 16.04 (LTS) |

## Role Variables

```yaml
# Flag to uninstall old versions of Docker
docker_uninstall_old_versions: false

# Edition can be one of: 'ce' (Community Edition) or 'ee' (Enterprise Edition)
docker_edition: ce
# Docker and containerd packages
docker_packages:
  - "docker-{{ docker_edition }}"
  - "docker-{{ docker_edition }}-cli"
  - containerd.io
# State can be one of: 'present' or 'latest'
docker_state: present

# List of users to be added to the docker group
docker_users:
  - "{{ lookup('env', 'USER') }}"

# Flag to configure Docker to start on boot
docker_start_on_boot: true
```

#### Debian / Ubuntu

```yaml
# Architecture can be one of: 'amd64', 'armhf', 'arm64', 'ppc64el' or 's390x'
docker_arch: amd64
# APT release channel can be one or many of: 'stable', 'nightly' or 'test'
docker_channels:
  - stable
```

## Dependencies

None.

## Example Playbook

```yaml
- name: Install Docker
  hosts: localhost
  become: yes

  roles:
    - ansible-role-docker
```

repository: anton-boyanov/ansible-role-docker

repo_name: anton-boyanov/ansible-role-docker


text:
---
layout: default
---

# 1996-11-15 - 3DES export stories? [was: British Telecom merger with MCI]

## Header Data

From: Eric Blossom<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: ebc2c5652436486334adbefa8969fa5dab8e00ef3a0a38f912745a10b68d9281

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1996-11-15 21:46:06 UTC

Raw Date: Fri, 15 Nov 1996 13:46:06 -0800 (PST)

## Raw message

```
{% raw  %}From: Eric Blossom<EMAIL_ADDRESS>Date: Fri, 15 Nov 1996 13:46:06 -0800 (PST)
To<EMAIL_ADDRESS>Subject: 3DES export stories? [was: British Telecom merger with MCI]
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

> On Mon, 4 Nov 1996, Adam Shostack wrote:
>
> > Phil Karn failed to get an export license for 3des for foriegn offices
> > of Qualcomm, staffed by Americans.  See
> > www.eff.org/pub/Crypto/ITAR_export/nsa_3des_export_denial_0396.letter
>
> [whoops]
> That's unusual - certainly, for the bigger companies it seems to be pretty
> automatic, especially for NATO countries. Of course, it could just be that
> Phil is "known to the authorities".

Sorry that the follow up is so late,  but can anyone give me *any*
examples where export of 3DES has been approved?  Please be as specific
as possible as to destination countries, use, etc.  First hand
knowledge is preferred.

Thanks,
Eric Blossom

{% endraw %}
```

## Thread

+ Return to [November 1996](/archive/1996/11)

+ Return to "[Adam Shostack <adam@homeport.org>](/authors/adam_shostack_adam_at_homeport_org_)"
+ Return to "["David Lesher / hated by RBOC's in 5 states" <wb8foz@wauug.erols.com>](/authors/david_lesher__hated_by_rbocs_in_5_states_wb8foz_at_wauug_erols_com_)"
+ Return to "[Eric Blossom <eb@comsec.com>](/authors/eric_blossom_eb_at_comsec_com_)"
+ Return to "["Peter D. Junger" <junger@pdj2-ra.F-REMOTE.CWRU.Edu>](/authors/peter_d_junger_junger_at_pdj2ra_fremote_cwru_edu_)"
+ Return to "[ph93szh@brunel.ac.uk](/authors/ph93szh_at_brunel_ac_uk)"
+ Return to "[Simon Spero <ses@tipper.oit.unc.edu>](/authors/simon_spero_ses_at_tipper_oit_unc_edu_)"
+ Return to "[The Deviant <deviant@pooh-corner.com>](/authors/the_deviant_deviant_at_poohcorner_com_)"

+ 1996-11-04 (Mon, 4 Nov 1996 04:00:13 -0800 (PST)) - [British Telecom merger with MCI](/archive/1996/11/e8971e99248bab44cf72e1be7fcb50c8271fb0d4c8e4d0c1d8cf7f64123db84b) - _"Peter D. Junger"<EMAIL_ADDRESS>  + 1996-11-04 (Mon, 4 Nov 1996 06:32:04 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/8b550798e60b2c35eb1f247c58d8db7dd2d905f518f6983bd491e88535507684) -<EMAIL_ADDRESS>  + 1996-11-04 (Mon, 4 Nov 1996 12:28:56 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/974435e5c641ce81886a0950a16eb0958a51c3276600f9ffaaaa599406495c5d) - _Simon Spero<EMAIL_ADDRESS>    + 1996-11-05 (Mon, 4 Nov 1996 19:31:09 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/7d254147fa98724e881e0ff2b756bfd6effb3f1c5c2a86d9deccac47f09c50dc) - _Adam Shostack<EMAIL_ADDRESS>      + 1996-11-05 (Tue, 5 Nov 1996 06:04:35 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/3cd889145f20c086fb99b07be22ba513fa9670e17a020289ffd019364cd65789) - _Simon Spero<EMAIL_ADDRESS>        + 1996-11-06 (Tue, 5 Nov 1996 16:49:27 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/f4f0c9c781326bd18708d00fc45a70e534834e65c13cf7f8d103d311f18c866b) - _"David Lesher / hated by RBOC's in 5 states"<EMAIL_ADDRESS>      + 1996-11-05 (Tue, 5 Nov 1996 06:06:56 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/e83f110a15141128c88b8786251a84d9668e144cd4ec4186bb7b50d544d228a6) - _Simon Spero<EMAIL_ADDRESS>        + 1996-11-05 (Tue, 5 Nov 1996 12:52:04 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/51ab5d2ea0289f4c2a1c390c6b56e73d3d40f6b92991b7ba8a03cfe79643dddb) - _Adam Shostack<EMAIL_ADDRESS>        + 1996-11-15 (Fri, 15 Nov 1996 13:46:06 -0800 (PST)) - 3DES export stories? [was: British Telecom merger with MCI] - _Eric Blossom<EMAIL_ADDRESS>  + 1996-11-04 (Mon, 4 Nov 1996 14:22:32 -0800 (PST)) - [Re: British Telecom merger with MCI](/archive/1996/11/5332053ff8dff128c0a6478bbb8875d1ccfa155e86812e175049f106f67aa84b) - _The Deviant<EMAIL_ADDRESS>


text:
# Week 10 - Methods

Methods will make up one of the most important sections of your final year project, they ensure the repeatability of your work. They are documented as a **past tense** account of the tools, instruments and processes that were used to deliver your project. While in reality studies, including your own, won't actually be repeated, methods are still important as they allow the ways in which you conducted your study to be scrutinised. Speaking broadly, for your final year project, your methods will be split between process and evaluative (see Figure 1). I shall briefly explore these different method types below.

![](./assets/different_methods.png)
>> Figure 1, process vs evaluative methods

### Process methods

Process methods will make up a simple section of your report. They are simply a list of important tools and management processes that you used to deliver your project. For instance, you may have used:

- Git and GitHub to manage version control
- Travis to take care of continuous integration and deployment
- FireBase to host your work
- Trello and a Gant chart to handle task management
- OneNote as a research log

**IMPORTANT:** There are a couple of common mistakes that students make. **Firstly**, don't endlessly talk about alternative tools or techniques that you never actually used. For instance, if you used Trello for project management, you just need to document your justifications for using this tool. Do not fall into the trap of listing several other project management tools, this severs no purpose. **Secondly**, don't tell us that you used a team software development methodology (e.g. agile, spiral, waterfall...) to manage your project. These methods are not suitable as you will be working alone so why use a team methodology. While you can certainly use these development methodologies as inspiration, ensure you adapt them to the needs of your project. While process methods sit in the background and would have assisted you in delivering your project, evaluative methods measure the successes of your project.

### Evaluative methods

Evaluative methods can be split between qualitative, quantitative and mixed (a mixture of qualitative and quantitative). Qualitative data is non-numerical and can be used to get rich contextual user opinion on your project artefact. More often than not, data of a qualitative nature will be used to answer a research question. Quantitative data, on the other hand, is numeric and is used to prove or disprove a hypothesises. Regardless of whether you pick qualitative, quantitative or mixed methods, you are going to need to consider how you actually gather this data.

### Data Collection

In order to collect data you need to set up some form of experiment or planned evaluation of your artefact. This may be getting some targeted end users to test your web app or software.  Next, let us consider some example methods that you can use to evaluate the outcomes of this experiment or user evaluation.

####  Quantitative Methods

 Some example quantitative methods include:

- Request per Second (Web Based)
- System Load Average
- Memory Usage
- Response Time
- Metrics from auditing tools (e.g. [Light House](https://developers.google.com/web/ilt/pwa/lab-auditing-with-lighthouse))
- Usability Lab Tests (e.g. gaze plots, eye tracking), ensure these methods serve a purpose, do not just use them because they are easily available

A good paper to read that utilises quantitative methods is by [Olanrewaju Et al.](https://www.researchgate.net/publication/312737643_An_Empirical_Study_of_the_Evolution_of_PHP_MVC_Framework)  (2015). The paper compares the performance of different PHP frameworks, however, the methods they use could easily be generalised to other types of projects. A final point of note, it is possible to use a quantitative scale to measure human perception. You've all seen these strongly agree to strongly disagree type questions. While you can, by all means, collect data in this manner, I recommend you use it to complement other methods. Collecting data in this manner means you may miss nuanced opinions and gather little in the way insight. If you're focused purely on the human opinion you should consider mixed or qualitative methods.

### Qualitative Methods

Some qualitative methods include:

- Open question surveys
- Focus groups
- Interviews

While qualitative data is arguably easier to gather than quantitative data, it is a more time consuming undertaking to analyse it. Firstly, you need to get the data into a suitable format for analysis. For instance, if you are using focus groups or interviews, this means recording then transcribing the recording to allow for textual analysis. Finally, you need to understand the data and tell the story that it reveals. It is common to organise the data into broad themes, such an approach is know as thematic analysis (google it).

In summary, whether you use qualitative, quantitative or mixed methods, ensure you use methods that actually help you evaluate your artefact.  It is common to use several methods, but ensure that every method actually tells you something useful about your artefact. If you are unsure then speak to your supervisor/tutor.

#### References

[Olanrewaju, R.F., Islam, T. and Ali, N., 2015. An empirical study of the evolution of php mvc framework. In Advanced Computer and Communication Engineering Technology (pp. 399-410). Springer, Cham.](https://www.researchgate.net/publication/312737643_An_Empirical_Study_of_the_Evolution_of_PHP_MVC_Framework)


text:
# S3 Spaces Sync

Synchronize your WordPress media library with [DigitalOcean Spaces](https://www.digitalocean.com/?refcode=5c4f2a9f0908) Container.

## Description

This WordPress plugin syncs your media library with DigitalOcean Spaces. It's allows you
to simuntanously upload and delete files, replacing public media URL with relative cloud
storage links. You can choose between two options, to keep local copy of the files, or to
delete them and keep files only in cloud storage.

In order to use this plugin, you have to create a DigitalOcean Spaces API key.

You may now define constants in order to configure the plugin. If the constant is defined,
it overwrites the value from settings page.

### Contants Description

```txt
S3_SPACE_KEY                  : DigitalOcean Spaces key
S3_SPACE_SECRET               : DigitalOcean Spaces secret
S3_SPACE_ENDPOINT             : DigitalOcean Spaces endpoint
S3_SPACE_CONTAINER            : DigitalOcean Spaces container
S3_SPACE_STORAGE_PATH         : Path to the file in the storage, will appear as a prefix
S3_SPACE_STORAGE_FILE_ONLY    : Keep files only in DO Spaces or not (true|false)
S3_SPACE_STORAGE_FILE_DELETE  : Remove files in DO Spaces on delete or not  (true|false)
S3_SPACE_FILTER               : Regular expression filter
S3_SPACE_UPLOAD_URL_PATH      : Full url to the files, WP Constant
S3_SPACE_UPLOAD_PATH          : Path to the local files, WP Constant
```

There is a known issue with the built in Wordpress Image Editor, it will not upload changed images.
Know how to fix this, PR welcome.

### Example Configuration

```php
/**
 * S3 Space Configuration
 */
define( 'S3_SPACE_KEY',        'xxxxxxxxxxxxxxxxxxxx' );
define( 'S3_SPACE_SECRET',     'xxxxxxxxxxxxxxxxxxxx' );
define( 'S3_SPACE_ENDPOINT',   'https://sgp1.digitaloceanspaces.com' );
define( 'S3_SPACE_CONTAINER',  'mywordpress' );
```

## Installation

1. Upload plugin directory to `/wp-content/plugins/`
2. Activate plugin through 'Plugins' menu in WordPress
3. Go to `Settings -> S3 Spaces Sync` and set up plugin

If plugin has been downloaded from GitHub, you have to install vendor components via `composer update`.

## Credits

This plugin forked from [keeross/DigitalOcean-Spaces-Sync](https://github.com/keeross/DigitalOcean-Spaces-Sync)
and licensed under [MIT License](./license.txt).


text:
# Hucpa

[![Code Climate](https://codeclimate.com/github/tomekw/hucpa/badges/gpa.svg)](https://codeclimate.com/github/tomekw/hucpa) [![Gem Version](https://badge.fury.io/rb/hucpa.svg)](https://badge.fury.io/rb/hucpa) [![CircleCI](https://circleci.com/gh/tomekw/hucpa.svg?style=svg)](https://circleci.com/gh/tomekw/hucpa)

A JRuby wrapper to [HikariCP](https://github.com/brettwooldridge/HikariCP) - "zero-overhead" production ready JDBC connection pool.
This is / will be a part of a bigger effort, like a dedicated library to use SQL and / or ActiveRecord integration.

Please note the project support only JRuby (tested with <IP_ADDRESS>+) on Java 8.

The public API is subject to change before version `1.0.0`.

## Installation

Add this line to your application's Gemfile:

```ruby
gem "hucpa"
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install hucpa

## Configuration options

See [HikariCP Documentation](https://github.com/brettwooldridge/HikariCP#configuration-knobs-baby) for a detailed description.
Not all HikariCP-defined configuration options are currently supported. To add support for infrequently used options please
raise an issue or file a pull request.

| Option                  | Required | Default value | Notes                                                                   |
| ----------------------- | :------: | ------------- | ----------------------------------------------------------------------- |
| `adapter`               | Yes(1)   | -             | Symbol                                                                  |
| `auto_commit`           | No       | `true`        | Boolean                                                                 |
| `connection_test_query` | No       | -             | String                                                                  |
| `connection_timeout`    | No       | `30_000`      | Integer, greater than or equal to 250, in miliseconds                   |
| `database_name`         | No       | -             | String                                                                  |
| `idle_timeout`          | No       | `600_000`     | Integer, 0 (disabled) or greater than or equal to 10000, in miliseconds |
| `jdbc_url`              | Yes(1)   | -             | String                                                                  |
| `max_lifetime`          | No       | `1_800_000`   | Integer, 0 (disabled) or greater than or equal to 30000, in miliseconds |
| `maximum_pool_size`     | No       | `10`          | Integer, greater than or equal to 1                                     |
| `minimum_idle`          | No       | `10`          | Integer, greater than or equal to 1                                     |
| `password`              | Yes      | -             | String                                                                  |
| `pool_name`             | No       | -             | String                                                                  |
| `server_name`           | No       | -             | String                                                                  |
| `username`              | Yes      | -             | String                                                                  |

`(1)` - either `adapter` or `jdbc_url` has to be provided.

## Supported adapters and corresponding datasource class names

| Adapter          | Datasource class name                              |
| ---------------- | -------------------------------------------------- |
| `db2`            | `com.ibm.db2.jcc.DB2SimpleDataSource`              |
| `derby`          | `org.apache.derby.jdbc.ClientDataSource`           |
| `fdbsql`         | `com.foundationdb.sql.jdbc.ds.FDBSimpleDataSource` |
| `firebird`       | `org.firebirdsql.pool.FBSimpleDataSource`          |
| `h2`             | `org.h2.jdbcx.JdbcDataSource`                      |
| `hsqldb`         | `org.hsqldb.jdbc.JDBCDataSource`                   |
| `mariadb`        | `org.mariadb.jdbc.MySQLDataSource`                 |
| `mysql`          | `com.mysql.jdbc.jdbc2.optional.MysqlDataSource`    |
| `oracle`         | `oracle.jdbc.pool.OracleDataSource`                |
| `pgjdbc_ng`      | `com.impossibl.postgres.jdbc.PGDataSource`         |
| `postgresql`     | `org.postgresql.ds.PGSimpleDataSource`             |
| `sqlite`         | `org.sqlite.JDBC`                                  |
| `sqlserver_jtds` | `net.sourceforge.jtds.jdbcx.JtdsDataSource`        |
| `sqlserver`      | `com.microsoft.sqlserver.jdbc.SQLServerDataSource` |
| `sybase`         | `com.sybase.jdbcx.SybDataSource`                   |

## Usage

Install the database driver, for PostgreSQL:

```ruby
gem "jdbc-postgres"
```

Load the the database driver if needed, for PostgreSQL:

```ruby
require "jdbc/postgres"
Jdbc::Postgres.load_driver
```

Configure the connection pool:

```ruby
# Using the adapter option
options = {
  adapter: :postgresql,
  database_name: "hucpa",
  password: "hucpa",
  server_name: "postgres",
  username: "hucpa"
}

# Using the jdbc_url option
options = {
  jdbc_url: "jdbc:postgresql://postgres/hucpa",
  password: "hucpa",
  username: "hucpa"
}

connection_pool = Hucpa::ConnectionPool.new(options)
```

Use the connection pool with the `with_connection` API:

```ruby
answer = connection_pool.with_connection do |connection|
  result_set =
    connection
      .create_statement
      .execute_query("SELECT 42 AS answer")

  result_set.next and result_set.get_int("answer")
end

answer
=> 42
```

Or use the connection pool with the "classic" API:

```ruby
datasource = connection_pool.open

# Explicitly obtain the DB connection
connection = datasource.connection

result_set =
  connection
    .create_statement
    .execute_query("SELECT 42 AS answer")

answer = result_set.next and result_set.get_int("answer")

# Explicitly release the DB connection
connection.close

answer
=> 42
```

Close the connection pool:

```ruby
connection_pool.close
```

## Development

Build the Docker image:

    $ docker-compose build

Create services:

    $ docker-compose create

Run specs:

    $ docker-compose run --rm app rspec spec

Run console:

    $ docker-compose run --rm app irb

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/tomekw/hucpa. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

repository: guizmaii/hucpa

repo_name: guizmaii/hucpa


text:
---
templateKey: blog-post
title: Hello There
date: 2018-04-20T20:25:08.981Z
description: This is short description
tags:
  - admin
---
Cool rich text editor

Welcome.

You successfully created your Strapi application.

You are looking at: ./public/index.html.

Your built-in admin panel is available at /admin.

Create your first API

Easily generate a complete API with controllers, models and routes using:
 $ strapi generate:api <apiName>

Resources

You'll probably also want to learn how to customize your application, set up security and configure your data sources.

For more help getting started, check out:

Documentation

Repository on GitHub

news on Twitter

Real-time support on Slack


text:
---
title: inspirations
type: page
layout: text
draft: true
---

# Ideas

- can we create eschar / geb imagery with font-awesome (like hands drawing eachother for regen / two-way workflow)
- spark notes mode for docs, (the abstract / outline for all pages in a condensed form, for easily discovering content)
  - auto gen, use first N chars when spark field not in page params

# other dev tools & saas sites to be inspired by

### bun.sh

https://bun.sh/

- up and running from the landing page
- totally developer focused
- start with features, new thinking

repository: hofstadter-io/hof

gha_language: Go

repo_name: hofstadter-io/hof


text:
Material
========
###### 　　[English](https://github.com/ray-cast/ray-mmd/blob/developing/Materials/README.md)   [中文文档](https://github.com/ray-cast/ray-mmd/blob/developing/Materials/README_chs.md)

　　This document is designed to help those who wanted to quickly get up to speed in `Ray-MMD`, the PBR (physically-based-rendering) is a rendering pipeline around the physics that has rapidly gained popularity, so the first thing you need to know what is the PBR term.

　　There are three forms of PBR's material (physically-based material), we'll be present the **albedo**/**smoothness**/**metalness** pipeline, and not the **albedo**/**roughness**/**metalness**, because of the **roughness** does not be easy to understand and to use, and not the **diffuse**/**specular**/**gloss** pipeline, because it's not PBR's term, and then i'll go over all of the entries commonly used in `Ray-MMD` and also attach links to the wiki for more details.

　　Currently, you can create a new material by duplicate a `material_2.0.fx` and if you want to create a variety of textures, current software such as dDO, nDO, Substance Designer, Substance Painter, Photoshop, etc, that will do much to create these conditions make the process a bit easier and help you create your own materials.

[![link text](../Shader/screenshots/material.jpg)](https://raw.githubusercontent.com/ray-cast/ray-mmd/master/Shader/screenshots/material.jpg)

ALBEDO:
------
　　`Albedo` is also called `base color`, defines the overall color of the material, thus uses all 3 color channels, the albedo maps are [normalized value](https://en.wikipedia.org/wiki/Unit_vector) in the [sRGB](https://en.wikipedia.org/wiki/SRGB) color-space and clamped between `0.0` and `1.0`

* ##### ALBEDO_MAP_FROM
    You can use a `linear-color` and `sRGB-texture` to change the colors in your model by set the following `code` to the `ALBEDO_MAP_FROM`.

    `0` . Parameter fetch from `linear-color` from the `const float3 albedo = 1.0`.
    `1` . You can use a `srgb-image` (bmp, png, jpg, tga, dds) by enter a relative and absolutely path to the `ALBEDO_MAP_FILE`.
    `2` . You can use a `animation srgb-image` (gif, apng) by enter a relative and absolutely path to the `ALBEDO_MAP_FILE`.
    `3` . Parameter fetch from `Texture` from the `pmx`.
    `4` . Parameter fetch from `Sphere map` from the `pmx`.
    `5` . Parameter fetch from `Toon map` from the `pmx`.
    `6` . Parameter fetch from `avi` or `screen` from the `DummyScreen.x` inside `extension` folder.
    `7` . Parameter fetch from `Ambient Color` from the `pmx`.
    `8` . Parameter fetch from `Specular Color` from the `pmx`.
    `9` . Parameter fetch from `Specular Power` from the `pmx`. // `Only for smoothness`

* ##### ALBEDO_MAP_UV_FLIP
    You can flip your texture for the `X` and `Y` axis mirror by set `code` to the `ALBEDO_MAP_UV_FLIP`

    `0` . None
    `1` . Flip axis `X`
    `2` . Flip axis `Y`
    `3` . Flip axis `X` and `Y`

* ##### ALBEDO_MAP_APPLY_SCALE
    You can apply the color from `albedo` to change the colors in your texture by set `code` to the `ALBEDO_MAP_APPLY_SCALE`

    `0` . None
    `1` . map values * albedo
    `2` . map values ^ albedo (The `^` is exponential operations of `X` and `Y`)

* ##### ALBEDO_MAP_APPLY_DIFFUSE
    Texture colors to multiply with `diffuse` from the `PMX`.

* ##### ALBEDO_MAP_APPLY_MORPH_COLOR
    Texture colors to multiply with color from the (`R+`/`G+`/`B+`) morph controller. see `PointLight.pmx` for more information

* ##### ALBEDO_MAP_FILE
    If `ALBEDO_MAP_FROM` is `1` or `2`, you will need to enter the path to the texture resource.

    ##### For example :
    ###### 1. If the xxx.png and material.fx is inside same folder
    * `You can set the xxx.png to the ALBEDO_MAP_FILE like : #define ALBEDO_MAP_FILE "xxx.png"`
    ###### 2. If the xxx.png is inside parent path of the material.fx
    * `You can set the xxx.png to the ALBEDO_MAP_FILE like : #define ALBEDO_MAP_FILE "../xxx.png"`
    ###### 3. If the xxx.png is inside other path from parent path of the material.fx
    * `You can set the xxx.png to the ALBEDO_MAP_FILE like : #define ALBEDO_MAP_FILE "../other path/xxx.png"`
    ###### 4. If the xxx.png is inside your desktop or other disk
    * `You can set the xxx.png to the ALBEDO_MAP_FILE like : #define ALBEDO_MAP_FILE "C:/Users/User Name/Desktop/xxx.png"`

    ##### Tips:
    * Using `../` instead of parent folder
    * Change all `\` to `/`.

* ##### const float3 albedo = 1.0;
    If `ALBEDO_MAP_FROM` is `0` or `ALBEDO_MAP_APPLY_SCALE` is `1`, you need to set color to the `const float3 albedo = 1.0;`.

    ##### For example:
    ###### 1. If the red is [normalized value](https://en.wikipedia.org/wiki/Unit_vector), it can be set to albedo like:
    * `const float3 albedo = float3(1.0, 0.0, 0.0);`
    ###### 2. If the red is [unnormalized value](https://en.wikipedia.org/wiki/Unit_vector), it can be set to albedo like:
    * `const float3 albedo = float3(255, 0.0, 0.0) / 255.0;`
    ###### 3. If the color is fetched from your monitor, you need to convert the color from [sRGB](https://en.wikipedia.org/wiki/SRGB) to [linear color-space](https://en.wikipedia.org/wiki/SRGB) by `color ^ gamma`
    * Convert the `srgb color-space` from normalized value to `linear color-space` like:
      * `const float3 albedo = pow(float3(r, g, b), 2.2);`
    * Convert the `srgb color-space` from unnormalized value to `linear color-space` like:
      * `const float3 albedo = pow(float3(r, g, b) / 255.0, 2.2);`

* #### albedoMapLoopNum
    You can tile your texture for the `X` and `Y` axis separately by change `albedoMapLoopNum = float2(x, y)`
    ##### For example:
    ###### 1. If `X` and `Y` are the same numbers:
    * `const flaot albedoMapLoopNum = 2;`
    OR
    * `const flaot2 albedoMapLoopNum = 2;`
    ###### 2. Otherwise (2 is `X`-axis, 3 is `Y`-axis):
    * `const flaot2 albedoMapLoopNum = float2(2, 3);`

SubAlbedo:
--------------
* ##### ALBEDO_SUB_ENABLE
    You can apply second value for `base color` change by set the `code` to `ALBEDO_SUB_ENABLE`

    `0` . None
    `1` . albedo * albedoSub
    `2` . albedo ^ albedoSub
    `3` . albedo + albedoSub
    `4` . melanin
    `5` . Alpha Blend

* ##### ALBEDO_SUB_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### ALBEDO_SUB_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### ALBEDO_SUB_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### ALBEDO_SUB_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float3 albedoSub = 0.0 ~ 1.0;
* ##### const float2 albedoSubMapLoopNum = 0.0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

Alpha:
----------------
　　It has no effect on opaque objects.

* ##### ALPHA_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### ALPHA_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### ALPHA_MAP_SWIZZLE
    The ordering of the data fetched from a `texture` from the `code`.

    `0` . Fetch data from `R` channel
    `1` . Fetch data from `G` channel
    `2` . Fetch data from `B` channel
    `3` . Fetch data from `A` channel

    ##### For example:
    ###### 1. If `Smoothness` map is inside `Red` channel, you can simply set it as:
    * `#define SMOOTHNESS_MAP_SWIZZLE 0`
    ###### 2. If `Smoothness` map is inside `Greed` channel, you can simply set it as:
    * `#define SMOOTHNESS_MAP_SWIZZLE 1`

* ##### ALPHA_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float alpha = 0.0 ~ 1.0;
* ##### const float2 alphaMapLoopNum = 0.0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

Normal:
-------------
　　The normal maps alters the topography of the mesh and alters the angle of the light to add shadows to it in the details,
this map is almost always uses tangent-space map with three channels in the most time, but the influences of some other factors such as old pipeline and bandwidth,
this allows you to use different type to access the different texture of the normal map, see the `NORMAL_MAP_TYPE` for more information,
in order to calculate the light in the real-time, All input models must have the normals else will result a problem with the white edge,
that looks like some white edges on your model, so you can put the scene in the `PMXEditor` and
check the scene that all `normals` are not `zero-length` (XYZ are the same equal to zero) to be used for model.

* ##### NORMAL_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### NORMAL_MAP_TYPE
    Other parameter types for `normal`, see UE4 [docs](https://docs.unrealengine.com/latest/INT/Engine/Rendering/LightingAndShadows/BumpMappingWithoutTangentSpace/index.html) for `PerturbNormalLQ` and `PerturbNormalHQ`.

    `0` . Calculate world-space normal from RGB tangent-space map.
    `1` . Calculate world-space normal from RG  compressed tangent-space map.
    `2` . Calculate world-space normal from Grayscale bump map by `PerturbNormalLQ` (Low  Quality). It has no effect on small objects.
    `3` . Calculate world-space normal from Grayscale bump map by `PerturbNormalHQ` (High Quality).
    `4` . Calculate world-space normal from RGB world-space map.

* ##### NORMAL_MAP_UV_FLIP (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### NORMAL_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float normalMapScale = 0 ~ inf;
* ##### const float2 normalMapLoopNum = 0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

SubNormal
-------------
　　this entry is designed to add detail to a base normal map in a consistent way,
that pack two normal map into a normal map by using [Reoriented Normal Mapping](https://www.shadertoy.com/view/4t2SzR),
and also, you can see the [docs](http://blog.selfshadow.com/publications/blending-in-detail/) for more information.

* ##### NORMAL_SUB_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### NORMAL_SUB_MAP_TYPE (see [NORMAL_MAP_TYPE](#NORMAL_MAP_TYPE))
* ##### NORMAL_SUB_MAP_UV_FLIP (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### NORMAL_SUB_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float normalSubMapScale = 0.0 ~ inf;
* ##### const float normalSubMapLoopNum = 0.0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

Smoothness
-------------
　　Smoothness maps determines the unevenness of surface, this is always a grayscale map with mono channel,
but there is almost never grayscale map used, and as such only uses the `R` channel in the `RGBA` map as default channel,
also, you can specify what channel will happen for the default channel by sets `code` to the `SMOOTHNESS_MAP_SWIZZLE`, and
it's almost a time when a material_2.0.fx is used, it'll fetched data from a `SpecularPower` from the `PMX` file
and convert the `SpecularPower` to `Smoothness` as default value.

* ##### SMOOTHNESS_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### SMOOTHNESS_MAP_TYPE
    Other parameter types for `Smoothness`

    `0` . `Smoothness` (from Frostbite / CE5 textures)
    `1` . Calculate `Smoothness` from Roughness by `1.0 - Roughness ^ 0.5` (from UE4/GGX/SubstancePainter2)
    `2` . Calculate `Smoothness` from Roughness by `1.0 - Roughness`       (from UE4/GGX/SubstancePainter2 with linear roughness)

* ##### SMOOTHNESS_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### SMOOTHNESS_MAP_SWIZZLE (see [ALPHA_MAP_SWIZZLE](#ALPHA_MAP_SWIZZLE))
* ##### SMOOTHNESS_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### SMOOTHNESS_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float smoothness = 0.0 ~1.0;
* ##### const float2 smoothnessMapLoopNum = 1.0; (see [albedoMapLoopNum](#albedoMapLoopNum))

Metalness:
-------------
　　Metalness is one method of determining reflectivity and what part of the texture is a metal,
used to instead of old pipeline such as specular highlight map, the metalness maps are always a grayscale map with mono channel,
but there is almost never grayscale map used, and as such only uses the `R` channel in the `RGBA` map as default channel,
also, you can specify what channel will happen for the default channel by sets `code` to the `METALNESS_MAP_SWIZZLE`

* ##### METALNESS_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### METALNESS_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### METALNESS_MAP_SWIZZLE (see [ALPHA_MAP_SWIZZLE](#ALPHA_MAP_SWIZZLE))
* ##### METALNESS_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### METALNESS_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float metalness = 0.0 ~ 1.0;
* ##### const float2 metalnessMapLoopNum = 1.0; (see [albedoMapLoopNum](#albedoMapLoopNum))

Specular:
-------------
　　Specular maps aren't environment and sphere maps, only modifies the `base reflectivity` for the model,
that is used for control over the colors of the reflection, and there have two type of specular map that are `RGB` and `grayscale`,
but they have no effect when the `metalness` is greater than zero, and that RGB type of specular map will not work with when `CUSTOM_ENABLE` is not equal to zero,
so you can use the grayscale map instead of `RGB` by sets `code` to the `SPECULAR_MAP_TYPE`,
and if you don't feel like the model to reflect the specular color, you can set zero to `const float3 specular = 0.0;`

* ##### SPECULAR_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### SPECULAR_MAP_TYPE
    Other parameter types for Specular

    `0` . Calculate reflection coefficient from specular color by `F(x) = 0.08*(x  )` (from UE4 textures)
    `1` . Calculate reflection coefficient from specular color by `F(x) = 0.16*(x^2)` (from Frostbite textures)
    `2` . Calculate reflection coefficient from specular grays by `F(x) = 0.08*(x  )` (from UE4 textures)
    `3` . Calculate reflection coefficient from specular grays by `F(x) = 0.16*(x^2)` (from Frostbite textures)
    `4` . Using reflection coefficient (`0.04`) instead of specular value (`0.5`), Available when `SPECULAR_MAP_FROM` at `0`

* ##### SPECULAR_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### SPECULAR_MAP_SWIZZLE (see [ALPHA_MAP_SWIZZLE](#ALPHA_MAP_SWIZZLE))
* ##### SPECULAR_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### SPECULAR_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))
* ##### const float3 specular = 0.5;
    Anything less than `2%` is physically impossible and is instead considered to be shadowing
    For example: The reflectance coefficient is equal to `F(x) = (x - 1)^2 / (x + 1)^2`
    Consider light that is incident upon a transparent medium with a refractive index of `1.5`
    That result will be equal to `(1.5 - 1)^2 / (1.5 + 1)^2` = `0.04` (or `4%`).
    Specular to reflection coefficient is equal to `F(x) = 0.08*x`, if the `x` is equal to `0.5` the result will be `0.04`.
    So default value is `0.5` for `0.04` coefficient and clamped value between `0.0` ~ `1.0`

* ##### const float2 specularMapLoopNum = 1.0; (see [albedoMapLoopNum](#albedoMapLoopNum))

Occlusion
-------------
　　The ambient occlusion (AO) is an effect that approximates the attenuation of environment light due to occlusion.
Bacause sky lighting from many directions, cannot simply to calculating shadows in the real-time.
A simply way able to replaced by using `occlusion map` and `SSAO`,
and if you don't want `diffuse` and `specular`, you can set zero to the `const float occlusion = 1.0`.

* ##### OCCLUSION_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))

* ##### OCCLUSION_MAP_TYPE
    Other parameter types for `Occlusion`

    `0` . Fetch `ambient occlusion` from linear color-space
    `1` . Fetch `ambient occlusion` from sRGB   color-space
    `2` . Fetch `ambient occlusion` from linear color-space from second UV set
    `3` . Fetch `ambient occlusion` from sRGB   color-space from second UV set

* ##### OCCLUSION_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### OCCLUSION_MAP_SWIZZLE (see [ALPHA_MAP_SWIZZLE](#ALBEDO_MAP_UV_FLIP))
* ##### OCCLUSION_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))

* ##### const float occlusion = 0.0 ~ 1.0;
* ##### const float2 occlusionMapLoopNum = 0.0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

Parallax:
-------------
　　You can use a `height map` but the `parallax` does not work with vertex displacement in the `DX9`

* ##### PARALLAX_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))

* ##### PARALLAX_MAP_TYPE
    Other parameter types for `parallax`

    `0` . calculate without transparency
    `1` . calculate parallax occlusion with transparency and best `SSDO`

* ##### PARALLAX_MAP_UV_FLIP  (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### PARALLAX_MAP_SWIZZLE  (see [ALPHA_MAP_SWIZZLE](#ALPHA_MAP_SWIZZLE))
* ##### PARALLAX_MAP_FILE (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float parallaxMapScale = 0.0 ~ inf;
* ##### const float2 parallaxMapLoopNum = 0.0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

Emissive
-------------
　　You can add a light source in MMD (PointLight or others), And key it as part of emissive of the model, and the same color set it to your light source and emissive color

* ##### EMISSIVE_ENABLE
* ##### EMISSIVE_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### EMISSIVE_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### EMISSIVE_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### EMISSIVE_MAP_APPLY_MORPH_COLOR (see [ALBEDO_MAP_APPLY_MORPH_COLOR](#ALBEDO_MAP_APPLY_MORPH_COLOR))

* ##### EMISSIVE_MAP_APPLY_MORPH_INTENSITY
   Texture colors to multiply with intensity from morph controller (Intensity+/-).
* ##### EMISSIVE_MAP_APPLY_BLINK
   You can set the blink using following `code`.

   `0` . None
   `1` . colors to multiply with frequency from `emissiveBlink`. like : const float3 emissiveBlink = float3(1.0, 2.0, 3.0);
   `2` . colors to multiply with frequency from `Blink` morph controller, see `PointLight.pmx` for more information

* ##### EMISSIVE_MAP_FILE ([ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float3 emissive = 0.0 ~ 1.0;
* ##### const float3 emissiveBlink = 0.0 ~ 10.0;
* ##### const float  emissiveIntensity = 0 ~ 100 and above
* ##### const float2 emissiveMapLoopNum = 0.0 ~ inf; (see [albedoMapLoopNum](#albedoMapLoopNum))

Shading Model ID
-------------
* ##### CUSTOM_ENABLE
    | ID | Material          | CustomA   | CustomB |
    | :- |:------------------|:----------|:--------|
    | 0  | Default           | Invalid    | Invalid |
    | 1  | PreIntegrated Skin| Curvature  | Transmittance color |
    | 2  | Unlit placeholder | Invalid    | Invalid |
    | 3  | Anisotropy        | Anisotropic| Shift tangent |
    | 4  | Glass             | Curvature  | Transmittance color |
    | 5  | Cloth             | Sheen      | Fuzz Color |
    | 6  | Clear Coat        | Smoothness | Invalid |
    | 7  | Subsurface        | Curvature  | Transmittance color |
    | 8  | Cel Shading       | Threshold  | Shadow color |
    | 9  | ToonBased Shading | Haredness  | Shadow color |

    ##### Tips:
    `Subsurface` : The `curvature` also called `opacity`, defines the overall scattering intensity affects all the surface, see the UE4 [docs](https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/LightingModels/SubSurfaceProfile/index.html) for more information
	[![link text](../Shader/screenshots/curvature_small.png)](https://raw.githubusercontent.com/ray-cast/ray-mmd/developing/Shader/screenshots/curvature.png)
    `Glass` : In order to make refraction work, you must set alpha value of the pmx to less then `0.999`
    `Cloth` : `Sheen` is interpolation between `GGX` and `InvGGX`, see [paper](http://blog.selfshadow.com/publications/s2017-shading-course/imageworks/s2017_pbs_imageworks_sheen.pdf) for cloth information
    `Cloth` : `Fuzz Color` is f0 of fresnel params in sRGB color-space, defines the overall color of the specular
    `Toon`  : see [paper](https://zhuanlan.zhihu.com/p/26409746) for more information, but chinese

* ##### CUSTOM_A_MAP_FROM  (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### CUSTOM_A_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### CUSTOM_A_MAP_COLOR_FLIP
* ##### CUSTOM_A_MAP_SWIZZLE (see [ALPHA_MAP_SWIZZLE](#ALPHA_MAP_SWIZZLE))
* ##### CUSTOM_A_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### CUSTOM_A_MAP_FILE "custom.png" (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float customA = 0.0 ~ 1.0; (linear-space)
* ##### const float2 customAMapLoopNum = 1.0; (see [albedoMapLoopNum](#albedoMapLoopNum))

* ##### CUSTOM_B_MAP_FROM (see [ALBEDO_MAP_FROM](#ALBEDO_MAP_FROM))
* ##### CUSTOM_B_MAP_UV_FLIP (see [ALBEDO_MAP_UV_FLIP](#ALBEDO_MAP_UV_FLIP))
* ##### CUSTOM_B_MAP_COLOR_FLIP
* ##### CUSTOM_B_MAP_APPLY_SCALE (see [ALBEDO_MAP_APPLY_SCALE](#ALBEDO_MAP_APPLY_SCALE))
* ##### CUSTOM_B_MAP_FILE "custom.png" (see [ALBEDO_MAP_FILE](#ALBEDO_MAP_FILE))

* ##### const float3 customB = 0.0 ~ 1.0; (sRGB color-space)
* ##### const float2 customBMapLoopNum = 1.0; (see [albedoMapLoopNum](#albedoMapLoopNum))

FAQ:
--------------------
* What is sRGB-color and Gamma
    * The Gamma is near 2.2 used most of time, About sRGB and Gamma, You can see docs for more information
    * `https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch24.html`
    * `https://en.wikipedia.org/wiki/SRGB`

* What is gloss map
    * Gloss map is a `smoothness map`

* How to use roughness map
    * Enter the path to the `SMOOTHNESS_MAP_FILE` and set `SMOOTHNESS_MAP_TYPE` to 1

* Where melanin
    * It has moved into `ALBEDO_SUB_ENABLE`, see `ALBEDO_SUB_ENABLE` for more information

* Why increase number of parallaxMapLoopNum will increase the loop number of albedo, normals, etc
    * Bacause parallax coordinates can be calculated from `height map`,that are then used to access textures with `albedo`, `normals`, `smoothness`, `metalness`, etc, In other words like fetched data (`albedo`, `normals`, etc) from parallax coordinates * `parallaxMapLoopNum` * `albedo`/`normal`/MapLoopNum

repository: ray-cast/ray-mmd

gha_language: HLSL

gha_license_id: NOASSERTION

repo_name: ray-cast/ray-mmd


text:
# depth-estimation
## [High Quality Monocular Depth Estimation via Transfer Learning (arXiv 2018)](https://arxiv.org/abs/1812.11941)
**[Ibraheem Alhashim](https://ialhashim.github.io/)** and **Peter Wonka**

The original code is at the repo [Dense Depth Original Code](https://github.com/ialhashim/DenseDepth)
This version has been modified by me to
* Add support for different encoders
* Test the effect of decreasing decoder length
* Create cool visualizations by testing the model on self collected images and videos

## Requirements
* This code is tested with Keras 2.2.4, Tensorflow 1.13, CUDA 9.0, on a machine with an NVIDIA Titan V and 16GB+ RAM running on Windows 10 or Ubuntu 16.
* Other packages needed `keras pillow matplotlib scikit-learn scikit-image opencv-python pydot` and `GraphViz`.

## Data
* [NYU Depth V2 (50K)](https://s3-eu-west-1.amazonaws.com/densedepth/nyu_data.zip) (4.1 GB): You don't need to extract the dataset since the code loads the entire zip file into memory when training.

## Download a pretrained model from the original paper
* [NYU Depth V2](https://s3-eu-west-1.amazonaws.com/densedepth/nyu.h5) (165 MB)

We Above paper used the pre-trained keras DenseNet201, DenseNet121, DenseNet169 and ResNet50 models, We used [unet](https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) [Model](https://github.com/zhixuhao/unet) encodered and decoder model with the same optimizer and the loss function to replicate the result

## Training with UNet encoder
* Train the model:
```
python depth_estimation.py 5 5 0.001

# First argument : batch_size
# Second argument: epochs
# Third argument: : learing rate
```

Once the model is created you can check the depth measured image result using webcam with following command
* webcam result verification
```
python video_display.py model.h5

# model.h5 is the created model
```

## Result Comparison

* The original paper model response:

* U-Net Model response:

## Reference
Corresponding paper to cite:
```
@article{Alhashim2018,
  author    = {Ibraheem Alhashim and Peter Wonka},
  title     = {High Quality Monocular Depth Estimation via Transfer Learning},
  journal   = {arXiv e-prints},
  volume    = {abs/1812.11941},
  year      = {2018},
  url       = {https://arxiv.org/abs/1812.11941},
  eid       = {arXiv:1812.11941},
  eprint    = {1812.11941}
}
```

repository: MahinKannan/depth-estimation

repo_name: MahinKannan/depth-estimation


text:
# firewall_scraper

Scrapes the latest release notes off a Palo Alto Networks firewall.

Written in Python 3.7. For this to work, you'll need a firewall to scrape.

## Setup
Download [Google Chrome](https://www.google.com/chrome/) and a corresponding [Chrome driver](https://sites.google.com/a/chromium.org/chromedriver/downloads).

Create the file `~/.panrc` that looks like this:
```
FW_IP=<IP_ADDRESS>
BINARY_LOCATION=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
DRIVER=${HOME}/_dev/pandorica/vanilladriver
DOWNLOAD_DIR=${HOME}/_dev/versiondocs
LOGGING_LEVEL=INFO
```
* `FW_IP`: IP of your firewall
* `BINARY_LOCATION`: path to your Chrome executable
* `DRIVER`: path to your Chrome driver
* `DOWNLOAD_DIR`: path to the directory where you'd like to download files
* `LOGGING_LEVEL`: defines what log messages to print to console; one of `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`

Install the `requirements.txt` file, preferably in a virtual environment. From this repository:
```
python -m venv .env
source .env/bin/activate
pip install -r requirements.txt
```

## Use
Source your virtual environment: `source .env/bin/activate`

Run the script: `python support_scraper.py`

Files will be downloaded to the directory specified in your `.panrc`.


text:
# Graphcool Server
The Graphcool Server is composed of a GraphQL database, the Realtime Subscriptions API, and an engine enabling asynchronous, event-driven flows using serverless functions.

## Running the tests
1. Make sure to have docker up and running
1. Make sure to make the env variables in `env_example` available, for example with `source env_example`.
1. Start all required dependencies with `docker-compose -f docker-compose/backend-dev.yml up -d --remove-orphans`
1. Run `sbt test`. This will take a few minutes.

If you want to clean up afterwards, make sure to run `docker-compose -f docker-compose/backend-dev.yml down -v --remove-orphans`

repository: robertjchristian/graphcool-framework

repo_name: robertjchristian/graphcool-framework


text:
---
title: Nikola
repo: getnikola/nikola
homepage: http://www.getnikola.com
language:
  - Python
license:
  - MIT
templates:
  - Jinja2
  - Mako
twitter: GetNikola
description: A static website and blog generator
---

Nikola is a Static Site and Blog Generator

In goes content, out comes a website, ready to deploy.

### What Can Nikola Do?

It has many features, but here are some of the nicer ones:

- Blogs, with tags, feeds, archives, comments, etc.
- Themable
- Fast builds, thanks to doit
- Flexible, extensible via plugins
- Small codebase (programmers can understand all of Nikola core in a day)
- reStructuredText [Cheatsheet] or Markdown as input language (also Wiki, BBCode, Textile, AsciiDoc, Python Notebooks, Misaka, Pandoc, txt2tags, orgmode, and HTML)
- Easy image galleries (just drop files in a folder!)
- Syntax highlighting for almost any programming language or markup
- Multilingual sites, translated to 18 languages.
- Doesn't reinvent wheels, leverages existing tools.
- Python 2 and 3 compatible.


text:
# Tweets Delete

Delete old tweets.

This is a php script meant to be run on a UNIX-compatible system as a cronjob where once a day tweets older than one year will be deleted.

 I accept no responsibility.**

## Install

1. [Request your Twitter archive](https://twitter.com/settings/account#tweet-export).
2. [Create an app on Twitter](https://apps.twitter.com) to get your keys.
3. Download or clone this repository.
4. Rename `config.example.php` to `config.php` and fill with your keys.
5. Copy `tweet.js` to the same folder as Tweets Delete and rename to `tweets.json`.
6. Remove `window.YTD.tweet.part0 = ` from beginning of `tweets.json`.
7. `composer install`.
8. `./tweets-delete.php`.

[MIT License](https://opensource.org/licenses/MIT)

repository: dustinwilson/tweets-delete

repo_name: dustinwilson/tweets-delete


text:
# google-doc-annotation-help
Google Doc Add-On Script to help manage manual, inline annotations

## Installation
1. Open a Google Doc on which you would like to enable this Add-On.
2. Click Tools > Script Editor...
3. Copy the contents of AnnotationHelper.gs into the Code.gs panel replacing all the contents of the stubbed out Code.gs.
4. Click Save.
5. Enter AnnotationHelper as the project name.
6. Reload the Doc.

## Usage
Convert annotations in the form [a] [ab] [a1] [ab12] to sequentially numeric annotations in the form [1].

To run click Add-ons > Annotation Helper > Convert Annotations.

Example:
```
  Here is some text with an annotation [a]. And some more text with another annotation [b].
   [a] - some note.
   [b] - another note.

   Here is even more text with some more complex annotations [ab][c1][ab2][c1234]
   [ab] - note ab
   [c1] - note c1
   [ab2] - note ab2
   [c1234] - note c1234
 ```

 Result after conversion:
 ```
   Here is some text with an annotation [1]. And some more text with another annotation [2].
   [1] - some note.
   [2] - another note.

   Here is even more text with some more complex annotations [3][4][5][6]
   [3] - note ab
   [4] - note c1
   [5] - note ab2
   [6] - note c1234
 ```

repository: msmeeks/google-doc-annotation-help

repo_name: msmeeks/google-doc-annotation-help


text:
The HTTP interface for this application is a [JSONAPI](http://jsonapi.org/)-compliant RESTful API. You should use a compliant client, such as [json_api_client](https://github.com/chingor13/json_api_client) to make integration as simple as possible.

Currently, there is no security and only a two resources exist: webhooks and deliveries.

### Webhooks

Note: webhooks are read-only and cannot be modified once created, hence why no update / delete endpoints exist.

#### Index

`GET /api/v1/webhooks`

This index provides a list of all webhooks in the application.

#### Show

`GET /api/v1/webhooks/:id`

This view will return the attributes of a single webhook.

Webhooks have a useful `status` attribute that will be one of the following:

* `queued` - awaiting processing
* `delivered` - successfully transmitted
* `retrying` - failed transmission but queued for another attempt
* `failed` - failed transmission and retry attempts have been exhausted

#### Create

`POST /api/v1/webhooks`

This endpoint allows you to submit a webhook to be queued for delivery. The endpoint is idempotent so if you submit a second webhook with the same `identifier` attribute, it will treat this as a duplicate record and return the original, without queuing a second delivery.

Deliveries are attempted 3 times by default before the background queue will give up, though you can customise this by setting `attempt_limit`. You can also disable this by passing `0` as the value.

##### Attributes

* `identifier` – uuid, required, unique
* `url` – string, required
* `headers` – object (key-value pairs), optional
* `body` – string, required
* `auth` – object (`username` / `password` keys), optional
* `attempt_limit` – integer, optional, min: 0, max: 10, default: 3
* `timeout` – integer, optional, min: 1, max: 60, default: 5
* `proxy` – boolean / url, optional, default: false
* `signatures` - array of objects, optional, default: empty
* `event_source` - object, optional

Note: if `proxy` is set to true, it will use the system HTTP Proxy, which will be loaded from `ENV['QUOTAGUARDSTATIC_URL']`, `ENV['FIXIE_URL']` (allowing you to simply install either addon via Heroku), or `ENV['HTTP_PROXY_URL']` if you wish to use something custom.

Note 2: for the algorithm in `signatures`, the supported values are `hmac-sha1`, `hmac-sha256`, `hmac-sha384`, `hmac-sha512`, `hmac-md5`.

##### Example Body

```json
{
  "data": {
    "type": "webhooks",
    "attributes": {
      "identifier": "6cb9b8eb-9d57-4541-9522-80838cd1a019",
      "url": "http://example.com/path/to/endpoint",
      "headers": {
        "Content-Type": "application/json"
      },
      "body": "{\"key\":\"value\"}",
      "auth": {
        "username": "testuser001",
        "password": "arandompassword"
      },
      "attempt_limit": 5,
      "timeout": 5,
      "signatures": [
        {
          "header": "X-Signature",
          "shared_secret": "secret",
          "algorithm": "hmac-sha1"
        }
      ],
      "event_source": {
        "event": "object_created",
        "object_type": "record",
        "object_id": 123
      }
    }
  }
}
```

### Deliveries

Note: deliveries are read-only, you cannot even create them as the resource is designed to simply report on the status of webhook delivery attempts made from the background queues.

#### Example Object

```json
{
  "id": 1,
  "type": "deliveries",
  "attributes": {
    "status": "delivered",
    "response_time_ms": 103,
    "response_status_code": 200,
    "response_headers": {
      "Content-Type": "text/plain"
    },
    "response_body": "Success!"
  }
}
```

#### Index

`GET /api/v1/webhooks/:webhook_id/deliveries`

This index provides a list of all deliveries for the given webhook's id.

#### Show

`GET /api/v1/webhooks/:webhook_id/deliveries/:id`

This view will return the attributes of a single delivery.

repository: okvv/satellite-webhooks

repo_name: okvv/satellite-webhooks


text:
# Title

[2016-03-11] Challenge #257 [Hard] Word Squares Part 2

# Difficulty

Hard

# Tags

words

# Description

Back to word squares, a type of acrostic, a word puzzle. A word square is formed using a grid with letters arranged that spell valid English language words when you read from left to right or from top to bottom. The challenge is that in arranging the words that you spell valid words.

Today's challenge is to input a set of dimensions (n*m) and work with the [enable1.txt](https://github.com/dolph/dictionary/blob/master/enable1.txt) dictionary file and produce a valid word square.

# Input Description

You'll be given pair of integers telling you how many rows and columns to solve for. Example:

    4 4

# Output Description

Your program should emit a valid word square with the letters placed to form valid English language words. Example:

    rose
    oven
    send
    ends

# Challenge Input

    6 6
    5 7

# Challenge Output

Multiple valid ones may be produced, but here's a few:

    glasses
    relapse
    imitate
    smeared
    tannery

    garter
    averse
    recite
    tribal
    estate
    reeled

repository: paralax/ColossalOpera

gha_language: Python

repo_name: paralax/ColossalOpera


text:
 Caesar-cipher

Desavio: "Criptografia de Júlio César";  Aceleração  Codenation

## :checkered_flag: To run the project:
1. Make a clone (`git clone`) of this repository;
2. Enter the `cd caesar-cipher-codenation.dev` folder;
3. Copy `.env.example` to `.env` and add your token;
4. Run `yarn` to install the dependencies;
5. Run `yarn start` to start the server.

## 📝License

This project is under license from MIT. See the [LICENSE](LICENSE) file for more details.

---

[Stay in touch!](https://www.linkedin.com/in/jonas-castro-b4044111a/)


text:
# crlapply

* **Version:** 0.2-0
* **Status:** [![Build Status](https://travis-ci.org/wrathematics/crlapply.png)](https://travis-ci.org/wrathematics/crlapply)
* **License:** [BSD 2-Clause](http://opensource.org/licenses/BSD-2-Clause)
* **Author:** Drew Schmidt

Checkpoint/restart is a useful strategy for running expensive functions in constrained environments (non-reliable hardware, restricted time limits, etc). This package adds an `lapply()`-like function that automatically handles checkpointing.

## Installation

You can install the stable version from [the HPCRAN](https://hpcran.org) using the usual `install.packages()`:

```r
install.packages("crlapply", repos="https://hpcran.org")
```

The development version is maintained on GitHub:

```r
remotes::install_github("wrathematics/crlapply")
```

## Package Use

We'll take a very simple example with a fake "expensive" function:

```r
costly = function(x, waittime)
{
  Sys.sleep(waittime)
  print(paste("iteration:", x))

  sqrt(x)
}

ret = crlapply::crlapply(1:10, costly, checkpoint_file="/tmp/cr.rdata", waittime=0.5)

unlist(ret)
```

We can save this to the file `example.r`. We'll run it and kill it a few times:

```bash
$ Rscript example.r
[1] "iteration: 1"
[1] "iteration: 2"
[1] "iteration: 3"
[1] "iteration: 4"
^C
$ Rscript example.r
[1] "iteration: 5"
[1] "iteration: 6"
[1] "iteration: 7"
^C
$ Rscript example.r
[1] "iteration: 8"
[1] "iteration: 9"
[1] "iteration: 10"
```

The final line of the script, when executed, will produce the following:

```r
unlist(ret)
##  [1] 1.000000 1.414214 1.732051 2.000000 2.236068 2.449490 2.645751 2.828427
##  [9] 3.000000 3.162278
```

We could have run this in an interactive R session instead of with in batch with `Rscript`. However, this should make it more obvious what is going on. Each time, we not only halt function execution, but *destroy the entire running R environment*. In the end, we are still able to read the entire set of results. This is because the intermediary results are stored in the checkpoint (that's the whole point). And the results can be any native R object.

So, if your results are very large (say the returns of big, expensive modeling functions) then those large objects will have to be saved/loaded each time there is a save or a reload. Set the `checkpoint_freq` argument appropriately.

One final note, is that the checkpoint/restart won't work for R objects that don't serialize. Anything that is secretly an [external pointer](https://cran.r-project.org/doc/manuals/R-exts.html#External-pointers-and-weak-references) will not be able to properly save/reload, and it may not be obvious at runtime that this is so. The save/reload may "work" in that they execute, but the data will be worthless and could cause the R session to crash when trying to operate on those objects. A few examples of packages whose objects are external pointers are [ngram](https://cran.r-project.org/web/packages/ngram/index.html) and [fmlr](https://hpcran.org/packages/fmlr/index.html).

repository: wrathematics/crlapply

repo_name: wrathematics/crlapply